Runtime performance
AI Model Speed and Latency
Compare output generation speed, catalog latency, provider time to first token, observed throughput, and reliability. Runtime remains independent from capability scores and price.
Data as of 2026-08-11
Model-provider records338193 canonical versions
Complete speed + latency331Comparable catalog records
Provider aggregates21Observed 7-day metrics
Fastest catalog speed2,220 tok/sLlama 3.3 70B Instruct via Cerebras
Output speed vs. catalog latency
Each point is one canonical model version offered by one provider. Upper-left is generally preferable: higher token generation speed with lower catalog latency.
Output speedCatalog latency
Each mark is one model-provider record331 records
Model-provider output speed ranking
Output Speed measures generated output tokens received per second. Sort by speed or catalog latency to match your workload.
| ME | llama-3-3-70b-instruct | Cerebras | 2,220 tok/s | 0.65 s | 128K | |
| ME | llama-3-1-8b-instruct | Cerebras | 2,047 tok/s | 0.2 s | 131.1K | |
| MA | ministral-3b-latest | Mistral AI | 1,222 tok/s | 0.16 s | 131.1K | |
| ME | llama-3-1-70b-instruct | Cerebras | 1,204 tok/s | 0.2 s | 128K | |
| ME | llama-3-3-70b-instruct | Sambanova | 1,096 tok/s | 0.65 s | 128K | |
| ME | llama-3-1-8b-instruct | Sambanova | 1,050 tok/s | 0.5 s | 131.1K | |
| OP | gpt-oss-20b | Groq | 1,000 tok/s | 0.38 s | 131K | |
| ME | llama-4-scout | Groq | 776.1 tok/s | 1.08 s | 10M | |
| ME | llama-3-1-8b-instruct | Groq | 750 tok/s | 0.5 s | 131.1K | |
| MA | kimi-k2-7-code | Together | 694.444 tok/s | 0.238 s | 262.1K | |
| ME | llama-4-maverick | Sambanova | 638.7 tok/s | 2.04 s | 1M | |
| MI | minimax-m2-7 | Fireworks | 560.077 tok/s | 0.437 s | 196.6K | |
| XA | grok-4-1-fast-non-reasoning | xAI | 508.25 tok/s | 0.518 s | 2M | |
| OP | gpt-5-nano-2025-08-07 | OpenAI | 500 tok/s | 0.3 s | 400K | |
| OP | gpt-oss-120b | Groq | 500 tok/s | 0.5 s | 131K | |
| ME | longcat-flash-lite | Meituan | 500 tok/s | 1.5 s | 256K | |
| OP | gpt-3-5-turbo-0125 | OpenAI | 482.886 tok/s | 0.514 s | 16.4K | |
| MI | minimax-m3 | Fireworks | 452.211 tok/s | 0.534 s | 512K | |
| IN | mercury-2 | Inception | 437.018 tok/s | 2.958 s | 128K | |
| OP | gpt-5-4 | OpenAI | 383.823 tok/s | 1.239 s | 1M | |
| ZA | glm-5-1 | FriendliAI | 354.951 tok/s | 1.583 s | 200K | |
| MI | minimax-m2-1 | MiniMax | 353.425 tok/s | 0.992 s | 1M | |
| DE | deepseek-v4-flash-max | DeepSeek | 347.708 tok/s | 0.755 s | 1M | |
| AC | qwen3-32b | Sambanova | 327.7 tok/s | 1.08 s | 128K | |
| MI | minimax-m2 | MiniMax | 318.387 tok/s | 1.591 s | 1M | |
| ME | llama-4-maverick | Groq | 307.3 tok/s | 0.27 s | 1M | |
| ME | llama-3-1-8b-instruct | Fireworks | 292 tok/s | 0.5 s | 131.1K | |
| NV | nemotron-3-nano-30b-a3b | DeepInfra | 275.936 tok/s | 8.963 s | 262.1K | |
| ME | llama-3-3-70b-instruct | Groq | 268 tok/s | 0.65 s | 128K | |
| ME | llama-3-1-70b-instruct | Groq | 250 tok/s | 0.5 s | 128K | |
| DE | deepseek-v4-pro-max | Novita | 244.513 tok/s | 0.943 s | 1M | |
| MA | ministral-8b-latest | Mistral AI | 237.5 tok/s | 0.34 s | 262.1K | |
| DE | deepseek-v4-pro-max | DeepSeek | 237.107 tok/s | 0.973 s | 1M | |
| OP | gpt-5-4-nano | OpenAI | 234.241 tok/s | 2.384 s | 400K | |
| OP | gpt-5-3-chat-latest | OpenAI | 228.206 tok/s | 1.181 s | 128K | |
| OP | gpt-5-1-2025-11-13 | OpenAI | 226.81 tok/s | 0.829 s | 400K | |
| XA | grok-code-fast-1 | xAI | 214.265 tok/s | 0.506 s | 256K | |
| OP | gpt-5-1-instant-2025-11-12 | OpenAI | 212.195 tok/s | 2.408 s | 400K | |
| DE | deepseek-v4-pro-max | Together | 207.408 tok/s | 0.848 s | 524.3K | |
| MI | minimax-m2-7 | Novita | 206.764 tok/s | 1.34 s | 204.8K | |
| OP | gpt-5-1-codex-mini | OpenAI | 200 tok/s | 1 s | 400K | |
| ME | llama-3-1-8b-instruct | Hyperbolic | 200 tok/s | 0.5 s | 131.1K | |
| ME | llama-3-3-70b-instruct | Fireworks | 197 tok/s | 0.65 s | 128K | |
| XI | mimo-v2-5-pro | DeepInfra | 195.755 tok/s | 0.625 s | 1M | |
| ME | llama-3-1-8b-instruct | Together | 194 tok/s | 0.5 s | 131.1K | |
| MI | minimax-m3 | Together | 192.197 tok/s | 0.482 s | 1M | |
| OP | gpt-4o-2024-08-06 | OpenAI | 184.314 tok/s | 1.096 s | 128K | |
| GO | gemini-2-0-flash | 183 tok/s | 0.4 s | 1M | ||
| AC | qwen3-6-plus | Novita | 174.652 tok/s | 1.231 s | 1M | |
| ME | llama-3-2-3b-instruct | DeepInfra | 171.5 tok/s | 0.24 s | 128K | |
| MA | mistral-large-latest | Mistral AI | 169.143 tok/s | 1.308 s | 262.1K | |
| ME | llama-3-2-11b-instruct | Together | 168 tok/s | 0.5 s | 128K | |
| GO | gemini-1-5-flash | 150 tok/s | 0.3 s | 1M | ||
| GO | gemini-1-5-flash-8b | 150 tok/s | 0.3 s | 1M | ||
| ME | llama-4-scout | Lambda | 139.7 tok/s | 0.43 s | 10M | |
| AC | qwen-2-5-7b-instruct | Together | 138 tok/s | 0.5 s | 131.1K | |
| MA | devstral-medium-2507 | Mistral AI | 137.1 tok/s | 0.23 s | 128K | |
| MA | devstral-small-2507 | Mistral AI | 137.1 tok/s | 0.23 s | 128K | |
| MA | mistral-small-3-1-24b-base-2503 | Mistral AI | 137.1 tok/s | 0.23 s | 128K | |
| XI | mimo-v2-5-pro | Xiaomi | 135.012 tok/s | 2.61 s | 1M | |
| MA | mistral-small-24b-instruct-2501 | Mistral AI | 134 tok/s | 0.2 s | 32K | |
| DE | deepseek-v4-pro-max | DeepInfra | 133.433 tok/s | 4.725 s | 1M | |
| OP | gpt-4-turbo-2024-04-09 | OpenAI | 133.189 tok/s | 1.588 s | 128K | |
| OP | chatgpt-4o-latest | OpenAI | 132 tok/s | 0.5 s | 128K | |
| XI | mimo-v2-5-pro | Novita | 130.693 tok/s | 2.845 s | 1M | |
| MA | ministral-14b-latest | Mistral AI | 128.6 tok/s | 0.23 s | 262.1K | |
| OP | gpt-5-mini-2025-08-07 | OpenAI | 126.283 tok/s | 8.645 s | 400K | |
| MA | kimi-k2-7-code | Moonshot AI | 125.62 tok/s | 1.108 s | 262.1K | |
| ME | llama-3-2-11b-instruct | Fireworks | 125 tok/s | 0.2 s | 128K | |
| ST | step-3-5-flash | StepFun | 122.942 tok/s | 1.192 s | 65.5K | |
| AC | qwen3-30b-a3b | Fireworks | 122.4 tok/s | 0.66 s | 128K | |
| AN | claude-3-opus-20240229 | Bedrock | 120 tok/s | 0.5 s | 200K | |
| AN | claude-3-sonnet-20240229 | Bedrock | 120 tok/s | 0.5 s | 200K | |
| AN | claude-opus-4-1-20250805 | Bedrock | 120 tok/s | 0.5 s | 200K | |
| AN | claude-opus-4-20250514 | Bedrock | 120 tok/s | 0.5 s | 200K | |
| GO | gemini-1-0-pro | 120 tok/s | 0.4 s | 32.8K | ||
| ME | llama-3-1-8b-instruct | DeepInfra | 118 tok/s | 0.5 s | 131.1K | |
| ME | llama-4-scout | Fireworks | 116.1 tok/s | 0.53 s | 10M | |
| OP | gpt-oss-120b | OpenAI | 115 tok/s | 5.2 s | 131.1K | |
| OP | gpt-oss-20b | OpenAI | 115 tok/s | 5.2 s | 131.1K | |
| OP | o1-mini | OpenAI | 115 tok/s | 5.2 s | 128K | |
| OP | o3-mini | Azure | 115 tok/s | 5.2 s | 200K | |
| OP | o3-mini | OpenAI | 115 tok/s | 5.2 s | 200K | |
| OP | o4-mini | OpenAI | 115 tok/s | 5.2 s | 200K | |
| MA | kimi-k2-7-code | Novita | 114.832 tok/s | 2.124 s | 262.1K | |
| AC | qwen-2-5-coder-32b-instruct | Fireworks | 110 tok/s | 0.26 s | 128K | |
| ME | llama-3-2-11b-instruct | DeepInfra | 108 tok/s | 0.5 s | 128K | |
| ME | llama-4-scout | Together | 106.9 tok/s | 0.54 s | 10M | |
| AC | qwen3-vl-4b-thinking | DeepInfra | 105.102 tok/s | 0.426 s | 262.1K | |
| AN | claude-3-5-haiku-20241022 | Bedrock | 104 tok/s | 0.5 s | 200K | |
| AN | claude-3-haiku-20240307 | Bedrock | 104 tok/s | 0.5 s | 200K | |
| OP | gpt-4-0613 | Azure | 104 tok/s | 0.3 s | 32.8K | |
| GO | gemini-3-5-flash-lite | 101.091 tok/s | 3.907 s | 1M | ||
| AN | claude-3-5-sonnet-20240620 | Bedrock | 101 tok/s | 0.5 s | 200K | |
| AN | claude-3-5-sonnet-20241022 | Bedrock | 101 tok/s | 0.5 s | 200K | |
| AN | claude-3-7-sonnet-20250219 | Bedrock | 101 tok/s | 0.5 s | 200K | |
| AN | claude-sonnet-4-20250514 | Bedrock | 101 tok/s | 0.5 s | 200K | |
| AN | claude-3-5-haiku-20241022 | Anthropic | 100 tok/s | 0.3 s | 200K | |
| AN | claude-3-5-sonnet-20241022 | Anthropic | 100 tok/s | 0.5 s | 200K | |
| AN | claude-3-haiku-20240307 | Anthropic | 100 tok/s | 0.5 s | 200K | |
| AN | claude-3-opus-20240229 | Anthropic | 100 tok/s | 0.5 s | 200K | |
| AN | claude-3-sonnet-20240229 | Anthropic | 100 tok/s | 0.5 s | 200K | |
| AN | claude-haiku-4-5-20251001 | Vertex AI | 100 tok/s | 0.3 s | 200K | |
| AN | claude-opus-4-1-20250805 | Anthropic | 100 tok/s | 0.5 s | 200K | |
| AN | claude-opus-4-20250514 | Anthropic | 100 tok/s | 0.5 s | 200K | |
| CO | command-r-plus-04-2024 | Bedrock | 100 tok/s | 0.5 s | 128K | |
| DE | deepseek-v2-5 | DeepSeek | 100 tok/s | 0.5 s | 8.2K | |
| DE | deepseek-v2-5 | Hyperbolic | 100 tok/s | 0.5 s | 8.2K | |
| DE | deepseek-v3 | DeepSeek | 100 tok/s | 0.5 s | 131.1K | |
| OP | gpt-4-0613 | OpenAI | 100 tok/s | 0.5 s | 32.8K | |
| OP | gpt-4-1-2025-04-14 | OpenAI | 100 tok/s | 10 s | 1M | |
| OP | gpt-4o-2024-05-13 | OpenAI | 100 tok/s | 0.5 s | 128K | |
| OP | gpt-5-2-2025-12-11 | OpenAI | 100 tok/s | 2 s | 400K | |
| OP | gpt-oss-120b-high | OpenAI | 100 tok/s | 6.5 s | 131.1K | |
| OP | gpt-oss-20b-high | OpenAI | 100 tok/s | 6.5 s | 131.1K | |
| XA | grok-3 | xAI | 100 tok/s | 0.7 s | 128K | |
| XA | grok-3-mini | xAI | 100 tok/s | 0.7 s | 128K | |
| XA | grok-4 | xAI | 100 tok/s | 0.7 s | 256K | |
| XA | grok-4-1-2025-11-17 | xAI | 100 tok/s | 0.7 s | 256K | |
| AL | jamba-1-5-large | Bedrock | 100 tok/s | 0.5 s | 256K | |
| AL | jamba-1-5-mini | Bedrock | 100 tok/s | 0.5 s | 256.1K | |
| AL | jamba-1-5-mini | 100 tok/s | 0.3 s | 256.1K | ||
| ME | llama-3-1-405b-instruct | Bedrock | 100 tok/s | 0.5 s | 128K | |
| ME | llama-3-1-70b-instruct | Hyperbolic | 100 tok/s | 0.5 s | 128K | |
| ME | llama-3-1-70b-instruct | Bedrock | 100 tok/s | 0.5 s | 128K | |
| ME | llama-3-1-8b-instruct | Bedrock | 100 tok/s | 0.5 s | 131.1K | |
| ME | llama-3-2-11b-instruct | Sambanova | 100 tok/s | 0.5 s | 128K | |
| ME | llama-3-2-11b-instruct | Bedrock | 100 tok/s | 0.5 s | 128K | |
| ME | llama-3-2-11b-instruct | Groq | 100 tok/s | 0.5 s | 128K | |
| ME | llama-3-2-90b-instruct | Bedrock | 100 tok/s | 0.5 s | 128K | |
| ME | llama-3-3-70b-instruct | Bedrock | 100 tok/s | 0.5 s | 128K | |
| ME | longcat-flash-chat | Meituan | 100 tok/s | 3 s | 128K | |
| ME | longcat-flash-thinking | Meituan | 100 tok/s | 3 s | 128K | |
| ME | longcat-flash-thinking-2601 | Meituan | 100 tok/s | 3 s | 128K | |
AM Nova LiteAmazon | nova-lite | Bedrock | 100 tok/s | 0.5 s | 300K | |
| AMNova MicroAmazon | nova-micro | Bedrock | 100 tok/s | 0.5 s | 128K | |
| AMNova ProAmazon | nova-pro | Bedrock | 100 tok/s | 0.5 s | 300K | |
| OP | o1-mini | Azure | 100 tok/s | 0.5 s | 128K | |
| AC | qwen-2-5-72b-instruct | Hyperbolic | 100 tok/s | 0.5 s | 131.1K | |
| AC | qwen-2-5-coder-32b-instruct | Hyperbolic | 100 tok/s | 0.5 s | 128K | |
| AC | qwq-32b-preview | Fireworks | 99.15 tok/s | 0.53 s | 32.8K | |
| OP | gpt-4o-2024-08-06 | Azure | 99 tok/s | 0.53 s | 128K | |
| ME | llama-4-maverick | Together | 97.93 tok/s | 0.2 s | 1M | |
| DE | deepseek-v3-2 | DeepInfra | 97 tok/s | 0.82 s | 163.8K | |
| OP | gpt-4-turbo-2024-04-09 | Azure | 97 tok/s | 0.6 s | 128K | |
| ME | llama-3-1-70b-instruct | Together | 94 tok/s | 0.5 s | 128K | |
| ME | llama-4-maverick | Lambda | 93.69 tok/s | 0.65 s | 1M | |
| OP | gpt-4o-2024-05-13 | Azure | 92 tok/s | 0.54 s | 128K | |
| OP | gpt-4o-mini-2024-07-18 | Azure | 92 tok/s | 0.52 s | 128K | |
| GO | gemini-3-pro-preview | 90 tok/s | 0.6 s | 1M | ||
| OP | gpt-3-5-turbo-0125 | Azure | 90 tok/s | 0.8 s | 16.4K | |
| OP | gpt-5-medium-2025-08-07 | OpenAI | 90 tok/s | 2.5 s | 400K | |
| OP | gpt-5-1-medium-2025-11-12 | OpenAI | 90 tok/s | 2.5 s | 400K | |
| AC | qwen3-30b-a3b | Novita | 88.84 tok/s | 0.73 s | 128K | |
| MA | kimi-k2-6 | Moonshot AI | 87.929 tok/s | 1.641 s | 262.1K | |
| ZA | glm-5-2 | DeepInfra | 87.154 tok/s | 1.199 s | 1M | |
| OP | gpt-5-6-terra | OpenAI | 86.691 tok/s | 2.184 s | 1.1M | |
| GO | gemini-1-5-pro | 85 tok/s | 0.7 s | 2.1M | ||
| GO | gemini-2-0-flash-lite | 85 tok/s | 0.7 s | 1M | ||
| GO | gemini-2-5-pro | 85 tok/s | 0.7 s | 1M | ||
| GO | gemini-2-5-pro-preview-06-05 | 85 tok/s | 0.7 s | 1M | ||
| XA | grok-2 | xAI | 85 tok/s | 0.7 s | 128K | |
| AC | qwen3-7-plus | Together | 84.543 tok/s | 2.994 s | 1M | |
| ME | llama-4-maverick | DeepInfra | 83.59 tok/s | 0.38 s | 1M | |
| AC | qwen3-30b-a3b | DeepInfra | 82.57 tok/s | 0.84 s | 128K | |
| OP | gpt-5-high-2025-08-07 | OpenAI | 80 tok/s | 3 s | 400K | |
| OP | gpt-5-1-high-2025-11-12 | OpenAI | 80 tok/s | 3.5 s | 400K | |
| OP | gpt-5-1-thinking-2025-11-12 | OpenAI | 80 tok/s | 3 s | 400K | |
| XA | grok-4-1-thinking-2025-11-17 | xAI | 80 tok/s | 1.2 s | 256K | |
| MI | minimax-m3 | MiniMax | 78.826 tok/s | 1.091 s | 1M | |
| ME | llama-3-1-405b-instruct | Fireworks | 78 tok/s | 0.5 s | 128K | |
| ME | llama-4-scout | DeepInfra | 76.1 tok/s | 0.31 s | 10M | |
| AC | qwq-32b-preview | DeepInfra | 76.04 tok/s | 0.44 s | 32.8K | |
| ME | llama-3-1-70b-instruct | Sambanova | 74 tok/s | 0.5 s | 128K | |
| GO | gemini-3-5-flash | 71.197 tok/s | 25.267 s | 1M | ||
| OP | gpt-5-6-luna | OpenAI | 70.587 tok/s | 4.616 s | 1.1M | |
| OP | gpt-5-5 | OpenAI | 70.342 tok/s | 113.442 s | 1.1M | |
| ME | llama-4-scout | Novita | 69.82 tok/s | 0.85 s | 10M | |
| ME | llama-4-maverick | Novita | 69.42 tok/s | 0.62 s | 1M | |
| AC | qwen3-235b-a22b | Fireworks | 68.17 tok/s | 0.78 s | 128K | |
| MA | kimi-k2-6 | Novita | 66.349 tok/s | 1.67 s | 262.1K | |
| OP | o1-2024-12-17 | OpenAI | 66 tok/s | 16.2 s | 200K | |
| OP | o1-preview | OpenAI | 66 tok/s | 16.2 s | 128K | |
| ME | llama-3-3-70b-instruct | Together | 65 tok/s | 0.65 s | 128K | |
| ME | llama-4-maverick | Fireworks | 63.03 tok/s | 0.62 s | 1M | |
| DE | deepseek-v2-5 | DeepInfra | 63 tok/s | 0.5 s | 8.2K | |
| AC | qwq-32b-preview | Together | 62.14 tok/s | 0.74 s | 32.8K | |
| GO | gemini-3-flash-preview | 61.686 tok/s | 6.001 s | 1M | ||
| CO | command-r-plus-04-2024 | Cohere | 59 tok/s | 0.65 s | 128K | |
| AC | qwen-2-5-72b-instruct | Fireworks | 59 tok/s | 0.37 s | 131.1K | |
| ME | llama-3-2-90b-instruct | Together | 57 tok/s | 0.5 s | 128K | |
| OP | gpt-5-4-mini | OpenAI | 50.724 tok/s | 0.7 s | 400K | |
| DE | deepseek-reasoner | DeepSeek | 50 tok/s | 0.5 s | 131.1K | |
| ZA | glm-4-7-flash | ZAI | 50 tok/s | 2 s | 128K | |
| OP | gpt-4-5 | OpenAI | 50 tok/s | 20 s | 128K | |
| OP | gpt-5-1-codex | OpenAI | 50 tok/s | 3 s | 400K | |
| OP | gpt-5-1-codex-high | OpenAI | 50 tok/s | 3.5 s | 400K | |
| OP | gpt-5-2-codex | OpenAI | 50 tok/s | 3 s | 400K | |
| IB | granite-3-3-8b-instruct | Replicate | 50 tok/s | 0.3 s | 128K | |
| LA | k-exaone-236b-a23b | FriendliAI | 50 tok/s | 0.4 s | 32.8K | |
| ME | llama-3-2-90b-instruct | Fireworks | 50 tok/s | 0.5 s | 128K | |
| OP | o3-2025-04-16 | OpenAI | 50 tok/s | 20 s | 200K | |
| MA | mistral-small-24b-instruct-2501 | DeepInfra | 49 tok/s | 0.2 s | 32K | |
| OP | gpt-4-1-nano-2025-04-14 | OpenAI | 48.966 tok/s | 1.535 s | 1M | |
| ZA | glm-5-2 | Together | 47.813 tok/s | 5.91 s | 1M | |
| AC | qwen-2-5-72b-instruct | Together | 47 tok/s | 0.5 s | 131.1K | |
| AN | claude-sonnet-4-5-20250929 | Anthropic | 46.518 tok/s | 2.746 s | 200K | |
| MA | mistral-medium-3-5 | Mistral AI | 46.385 tok/s | 0.411 s | 256K | |
| ZA | glm-5-2 | Fireworks | 45.606 tok/s | 1.068 s | 1M | |
| DE | deepseek-r1-0528 | DeepInfra | 45.04 tok/s | 0.61 s | 131.1K | |
| MA | kimi-k2-instruct | Novita | 45 tok/s | 0.95 s | 131.1K | |
| MI | minimax-m3 | Novita | 44.113 tok/s | 3.205 s | 1M | |
| AC | qwen-2-5-coder-32b-instruct | DeepInfra | 44 tok/s | 0.5 s | 128K | |
| OP | gpt-5-5-instant | OpenAI | 42.14 tok/s | 1.377 s | 400K | |
| GO | gemma-3n-e4b-it | Together | 42.09 tok/s | 0.43 s | 32K | |
| AN | claude-3-5-haiku-20241022 | 42 tok/s | 0.4 s | 200K | ||
| AN | claude-3-5-sonnet-20240620 | 42 tok/s | 0.4 s | 200K | ||
| AN | claude-3-5-sonnet-20241022 | 42 tok/s | 0.4 s | 200K | ||
| AN | claude-3-7-sonnet-20250219 | Anthropic | 42 tok/s | 0.4 s | 200K | |
| AN | claude-3-7-sonnet-20250219 | 42 tok/s | 0.4 s | 200K | ||
| AN | claude-3-haiku-20240307 | 42 tok/s | 0.4 s | 200K | ||
| AN | claude-3-opus-20240229 | 42 tok/s | 0.4 s | 200K | ||
| AN | claude-3-sonnet-20240229 | 42 tok/s | 0.4 s | 200K | ||
| AN | claude-opus-4-1-20250805 | 42 tok/s | 0.4 s | 200K | ||
| AN | claude-opus-4-20250514 | 42 tok/s | 0.4 s | 200K | ||
| AN | claude-opus-4-5-20251101 | Anthropic | 42 tok/s | 0.5 s | 200K | |
| AN | claude-opus-4-7 | Vertex AI | 42 tok/s | 0.5 s | 1M | |
| AN | claude-opus-4-8 | Vertex AI | 42 tok/s | 0.5 s | 1M | |
| AN | claude-sonnet-4-20250514 | Anthropic | 42 tok/s | 0.4 s | 200K | |
| AN | claude-sonnet-4-20250514 | 42 tok/s | 0.4 s | 200K | ||
| AL | jamba-1-5-large | 42 tok/s | 0.3 s | 256K | ||
| ME | llama-3-1-405b-instruct | Lambda | 42 tok/s | 0.5 s | 128K | |
| ME | llama-3-1-405b-instruct | 42 tok/s | 0.4 s | 128K | ||
| ME | llama-3-1-70b-instruct | Lambda | 42 tok/s | 0.5 s | 128K | |
| ME | llama-3-1-8b-instruct | Lambda | 42 tok/s | 0.5 s | 131.1K | |
| ME | llama-3-2-90b-instruct | Hyperbolic | 42 tok/s | 0.5 s | 128K | |
| ME | llama-3-3-70b-instruct | Lambda | 42 tok/s | 0.65 s | 128K | |
| ME | llama-3-3-70b-instruct | Hyperbolic | 42 tok/s | 0.65 s | 128K | |
| MA | mistral-large-2-2407 | 42 tok/s | 0.4 s | 128K | ||
| MA | mistral-nemo-instruct-2407 | 42 tok/s | 0.4 s | 128K | ||
| AC | qwen-2-5-coder-32b-instruct | Lambda | 42 tok/s | 0.5 s | 128K | |
| ZA | glm-5-2 | Novita | 41.389 tok/s | 7.464 s | 1M | |
| ME | llama-3-1-405b-instruct | Hyperbolic | 40 tok/s | 0.5 s | 128K | |
| MA | kimi-k3 | Novita | 39.843 tok/s | 26.673 s | 1M | |
| AC | qwen3-235b-a22b | Novita | 38.51 tok/s | 1.02 s | 128K | |
| DE | deepseek-r1-0528 | Novita | 37.96 tok/s | 1.18 s | 131.1K | |
| XA | grok-4-fast-reasoning | xAI | 37.956 tok/s | 1.268 s | 2M | |
| DE | deepseek-r1-distill-llama-70b | DeepInfra | 37 tok/s | 0.65 s | 128K | |
| DE | deepseek-r1-distill-qwen-32b | DeepInfra | 37 tok/s | 0.65 s | 128K | |
| ME | llama-3-3-70b-instruct | DeepInfra | 37 tok/s | 0.65 s | 128K | |
| ME | llama-3-1-405b-instruct | Together | 35 tok/s | 0.5 s | 128K | |
| XA | grok-4-3 | xAI | 34.61 tok/s | 0.552 s | 1M | |
| GO | gemma-3-12b-it | DeepInfra | 33 tok/s | 0.2 s | 131.1K | |
| GO | gemma-3-27b-it | DeepInfra | 33 tok/s | 0.2 s | 131.1K | |
| GO | gemma-3-27b-it | Novita | 33 tok/s | 0.2 s | 131.1K | |
| GO | gemma-3-4b-it | DeepInfra | 33 tok/s | 0.2 s | 131.1K | |
| MI | phi-4 | DeepInfra | 33 tok/s | 0.2 s | 16K | |
| AC | qwen3-32b | Novita | 32.43 tok/s | 0.93 s | 128K | |
| ME | llama-3-1-70b-instruct | Fireworks | 32 tok/s | 0.5 s | 128K | |
| AC | qwq-32b-preview | Hyperbolic | 31.9 tok/s | 1.05 s | 32.8K | |
| DE | deepseek-v3-2-speciale | DeepSeek | 30 tok/s | 1 s | 131.1K | |
| ZA | glm-5 | ZAI | 30 tok/s | 3 s | 200K | |
| GO | gemini-3-1-pro-preview | 28.427 tok/s | 128.56 s | 1M | ||
| ME | llama-3-1-405b-instruct | DeepInfra | 27 tok/s | 0.5 s | 128K | |
| AC | qwen3-32b | DeepInfra | 26.95 tok/s | 1.19 s | 128K | |
| OP | gpt-4-1-mini-2025-04-14 | OpenAI | 26.375 tok/s | 5 s | 1M | |
| OP | gpt-5-2-pro-2025-12-11 | OpenAI | 25 tok/s | 30 s | 400K | |
| ME | llama-3-1-70b-instruct | DeepInfra | 25 tok/s | 0.5 s | 128K | |
| OP | o3-pro-2025-06-10 | OpenAI | 25 tok/s | 30 s | 200K | |
| MI | phi-4-multimodal-instruct | DeepInfra | 25 tok/s | 0.5 s | 128K | |
| ME | llama-3-2-90b-instruct | DeepInfra | 24 tok/s | 0.5 s | 128K | |
| MA | kimi-k2-6 | Fireworks | 23.792 tok/s | 0.341 s | 262.1K | |
| AC | qwen3-235b-a22b | Together | 23.74 tok/s | 0.79 s | 128K | |
| MI | phi-3-5-mini-instruct | Azure | 23 tok/s | 0.52 s | 128K | |
| DE | deepseek-vl2 | Replicate | 22 tok/s | 0.5 s | 129.3K | |
| ME | llama-3-1-405b-instruct | Replicate | 22 tok/s | 0.5 s | 128K | |
| AC | qwen3-235b-a22b | DeepInfra | 21.74 tok/s | 1.23 s | 128K | |
| DE | deepseek-chat | DeepSeek | 20.731 tok/s | 1.654 s | 131.1K | |
| DE | deepseek-v4-flash-max | DeepInfra | 20.707 tok/s | 4.557 s | 1M | |
| MI | minimax-m2-5 | MiniMax | 20.53 tok/s | 3 s | 1M | |
| GO | gemma-4-31b-it | FriendliAI | 20.155 tok/s | 1.717 s | 262.1K | |
| MA | kimi-k3 | Fireworks | 18.262 tok/s | 8.115 s | 1M | |
| GO | gemini-2-5-flash | 17.533 tok/s | 0.7 s | 1M | ||
| ZA | glm-5-2 | FriendliAI | 16.583 tok/s | 0.596 s | 1M | |
| OP | o1-2024-12-17 | Azure | 16 tok/s | 0.54 s | 200K | |
| OP | o1-preview | Azure | 16 tok/s | 0.54 s | 128K | |
| AN | claude-opus-4-8 | Anthropic | 15.718 tok/s | 3.775 s | 1M | |
| ME | muse-spark-1-1 | Meta Model API | 13.687 tok/s | 29.373 s | 1M | |
| MA | kimi-k2-7-code | DeepInfra | 13.58 tok/s | 1.563 s | 262.1K | |
| MA | kimi-k2-6 | DeepInfra | 13.357 tok/s | 19.574 s | 262.1K | |
| BY | seed-2-0-pro | DeepInfra | 12.471 tok/s | 5.347 s | 256K | |
| AN | claude-sonnet-5 | Anthropic | 11.146 tok/s | 17.05 s | 1M | |
| DE | deepseek-v4-flash-0731 | Novita | 10.973 tok/s | 4.091 s | 1M | |
| MA | kimi-k3 | Moonshot AI | 10.366 tok/s | 9.33 s | 1M | |
| AC | qwen3-7-max | Novita | 10.109 tok/s | 1.946 s | 1M | |
| AC | qwen-2-5-72b-instruct | DeepInfra | 10 tok/s | 0.5 s | 131.1K | |
| DE | deepseek-r1 | DeepSeek | 9 tok/s | 0.3 s | 131.1K | |
| DE | deepseek-r1-0528 | DeepSeek | 9 tok/s | 0.3 s | 131.1K | |
| OP | gpt-5-2025-08-07 | OpenAI | 8.695 tok/s | 1.856 s | 400K | |
| AC | qwen3-5-27b | Novita | 7.909 tok/s | 6.349 s | 262.1K | |
| AC | qwen3-6-27b | Novita | 7.845 tok/s | 7.801 s | 262.1K | |
| XA | grok-4-1-fast-reasoning | xAI | 7.056 tok/s | 2.674 s | 2M | |
| MA | kimi-k3 | Together | 6.892 tok/s | 35.147 s | 1M | |
| GO | gemini-3-6-flash | 6.866 tok/s | 29.647 s | 1M | ||
| XA | grok-4-5 | xAI | 6.526 tok/s | 1.73 s | 500K | |
| DE | deepseek-v4-flash-0731 | DeepInfra | 6.415 tok/s | 15.272 s | 1M | |
| GO | gemma-4-26b-a4b-it | Novita | 5.881 tok/s | 8.09 s | 262.1K | |
| GO | gemini-2-5-flash-lite | 5.69 tok/s | 0.44 s | 1M | ||
| AC | qwen3-7-max | Together | 5.637 tok/s | 4.203 s | 1M | |
| OP | gpt-5-6-sol | OpenAI | 4.69 tok/s | 9.141 s | 1.1M | |
| DE | deepseek-r1 | Together | 4 tok/s | 0.6 s | 131.1K | |
| AN | claude-haiku-4-5-20251001 | Anthropic | 3.682 tok/s | 0.3 s | 200K | |
| AN | claude-opus-4-6 | Anthropic | 3.488 tok/s | 7.164 s | 1M | |
| AC | qwen3-vl-4b-instruct | DeepInfra | 3.44 tok/s | 23.877 s | 262.1K | |
| AN | claude-opus-5 | Anthropic | 3.273 tok/s | 112.716 s | 1M | |
| ZA | glm-4-6 | DeepInfra | 3.221 tok/s | 0.426 s | 131.1K | |
| AN | claude-sonnet-4-6 | Anthropic | 2.808 tok/s | 2.646 s | 200K | |
| OP | gpt-5-3-codex | OpenAI | 2.426 tok/s | 7.286 s | 400K | |
| DE | deepseek-r1 | Fireworks | 2.1 tok/s | 0.3 s | 131.1K | |
| DE | deepseek-v4-flash-0731 | Fireworks | 1.414 tok/s | 5.906 s | 1M | |
| GO | gemma-4-31b-it | Novita | 1.346 tok/s | 2.027 s | 262.1K | |
| GO | gemma-4-31b-it | DeepInfra | 0.94 tok/s | 37.139 s | 262.1K | |
| DE | deepseek-r1 | DeepInfra | 0.9 tok/s | 0.3 s | 131.1K | |
| AN | claude-fable-5 | Anthropic | 0.779 tok/s | 541.86 s | 1M | |
| MA | ministral-8b-instruct-2410 | Mistral AI | 0.1 tok/s | 0.18 s | 128K | |
| MA | mistral-large-2-2407 | Mistral AI | 0.1 tok/s | 0.5 s | 128K | |
| MA | mistral-nemo-instruct-2407 | Mistral AI | 0.1 tok/s | 0.5 s | 128K | |
| MA | mistral-small-2409 | Mistral AI | 0.1 tok/s | 0.5 s | 32.8K | |
| MA | pixtral-12b-2409 | Mistral AI | 0.1 tok/s | 0.5 s | 128K | |
| MA | pixtral-large | Mistral AI | 0.1 tok/s | 0.5 s | 128K | |
| AN | claude-opus-4-7 | Anthropic | 0.071 tok/s | 3.981 s | 1M |
Lowest catalog latency in the current records: Min istral 3 (3B Reasoning 2512) via Mistral AI at 0.16s.
Provider runtime: observed over 7 days
Provider aggregates use observed character throughput and time to first token. Character throughput is not converted into model token throughput.
Median observed throughputMedian TTFT
Each mark is one provider aggregate17 records
| Mistral AI | 421 ms | 978.4 ms | 176.928 char/s | 12.513 char/s | 28 | 100.0% | 0.0% | |
| FriendliAI | 618.5 ms | 2,090.5 ms | 393.343 char/s | 16.802 char/s | 108 | 57.9% | 42.1% | |
| xAI | 950 ms | 1,731.85 ms | 82.113 char/s | 6.024 char/s | 107 | 94.2% | 5.8% | |
| MiniMax | 1,045 ms | 1,507.5 ms | 355.536 char/s | 27.087 char/s | 25 | 88.0% | 12.0% | |
| DeepSeek | 1,149 ms | 1,651.2 ms | 109.437 char/s | 21.721 char/s | 132 | 100.0% | 0.0% | |
| StepFun | 1,166.5 ms | 1,192.15 ms | 236.961 char/s | 122.942 char/s | 9 | 100.0% | 0.0% | |
| 1,223 ms | 25,634.1 ms | 276.172 char/s | 58.812 char/s | 5,160 | 99.5% | 0.5% | ||
| Meta Model API | 1,328 ms | 29,373.2 ms | 36.707 char/s | 13.687 char/s | 185 | 100.0% | 0.0% | |
| Fireworks | 1,509 ms | 6,046.3 ms | 84.619 char/s | 1.517 char/s | 4,954 | 100.0% | 0.0% | |
| OpenAI | 2,162 ms | 7,113.25 ms | 195.732 char/s | 12.084 char/s | 4,416 | 97.5% | 2.5% | |
| Novita | 2,442 ms | 4,117.55 ms | 150.35 char/s | 10.563 char/s | 7,225 | 98.0% | 2.0% | |
| Xiaomi | 2,604 ms | 2,610.3 ms | 145.82 char/s | 135.012 char/s | 3 | 100.0% | 0.0% | |
| DeepInfra | 2,681 ms | 15,373 ms | 60.684 char/s | 6.089 char/s | 2,164 | 99.3% | 0.7% | |
| Inception | 2,958 ms | 2,958 ms | 437.018 char/s | 437.018 char/s | 3 | 100.0% | 0.0% | |
| Anthropic | 3,117 ms | 13,202.5 ms | 21.369 char/s | 3.702 char/s | 1,911 | 96.7% | 3.3% | |
| Together | 3,818 ms | 30,121.2 ms | 55.666 char/s | 4.771 char/s | 229 | 96.9% | 3.1% | |
| Moonshot AI | 4,370.5 ms | 9,311.4 ms | 66.19 char/s | 11.215 char/s | 124 | 96.8% | 3.2% | |
| Azure | N/A | N/A | N/A | N/A | 0 | N/A | N/A | |
| ZAI | N/A | N/A | N/A | N/A | 12 | 0.0% | 100.0% | |
| Sambanova | N/A | N/A | N/A | N/A | 0 | N/A | N/A | |
| Meituan | N/A | N/A | N/A | N/A | 0 | N/A | N/A |
How to read runtime data
Runtime metrics answer deployment questions and should not be read as model capability scores.
Output SpeedGenerated output tokens received per second. Higher is faster.
Catalog LatencyProvider catalog latency in seconds. Lower is faster; it is not the 7-day TTFT measure.
TTFTTime to first token from observed provider calls. P50 is typical; P95 shows slower-tail behavior.
Observed ThroughputCharacters per second across a 7-day provider window. It remains separate from output tokens per second.
ReliabilityCalls, success rate, and error rate describe the observed provider window, not model capability.
Independent SignalRuntime does not change LLMBoard capability scores, ranking bands, or evidence coverage.
Runtime source: LLM Stats / ZeroEval snapshot. Speed follows the public output-tokens-per-second convention. Read the source definition.