Runtime performance
AI Model Speed and Latency Leaderboard
Use this runtime leaderboard to compare the current output-speed ranking, catalog latency, provider time to first token and reliability. Runtime remains independent from capability benchmark scores and price.
Data as of 2026-08-17
Model-provider records311184 canonical versions
Complete speed + latency301Comparable catalog records
Provider aggregates20Observed 7-day metrics
Fastest catalog speed2,220 tok/sLlama 3.3 70B Instruct via Cerebras
Model-provider Runtime Leaderboard
This leaderboard ranking uses generated output tokens received per second. Sort by speed or catalog latency to match your workload; capability benchmark results do not change the order.
Sort by
Model | Provider | Output Speed | Catalog Latency | LLMBoard score | Official input / 1M | Max Input | Updated |
|---|
| ModelME | ProviderCerebras | Output Speed2,220 tok/s | Catalog Latency0.65 s | LLMBoard score21.4 | Official input / 1MN/A | Max Input128K | Updated |
| ModelME | ProviderCerebras | Output Speed2,047 tok/s | Catalog Latency0.2 s | LLMBoard scoreN/A | Official input / 1MN/A | Max Input131.1K | Updated |
| ModelMA | ProviderMistral AI | Output Speed1,222 tok/s | Catalog Latency0.16 s | LLMBoard score20.1 | Official input / 1M$0.04 | Max Input131.1K | Updated |
| ModelME | ProviderCerebras | Output Speed1,204 tok/s | Catalog Latency0.2 s | LLMBoard score12.6 | Official input / 1MN/A | Max Input128K | Updated |
| ModelME | ProviderSambanova | Output Speed1,096 tok/s | Catalog Latency0.65 s | LLMBoard score21.4 | Official input / 1MN/A | Max Input128K | Updated |
| ModelME | ProviderSambanova | Output Speed1,050 tok/s | Catalog Latency0.5 s | LLMBoard scoreN/A | Official input / 1MN/A | Max Input131.1K | Updated |
| ModelIN | ProviderInception | Output Speed1,009 tok/s | Catalog Latency1.7 s | LLMBoard score38.7 | Official input / 1M$0.25 | Max Input128K | Updated |
| ModelOP | ProviderGroq | Output Speed1,000 tok/s | Catalog Latency0.38 s | LLMBoard score45.9 | Official input / 1MN/A | Max Input131K | Updated |
| ModelME | ProviderGroq | Output Speed776.1 tok/s | Catalog Latency1.08 s | LLMBoard score13.5 | Official input / 1MN/A | Max Input10M | Updated |
| ModelME | ProviderGroq | Output Speed750 tok/s | Catalog Latency0.5 s | LLMBoard scoreN/A | Official input / 1MN/A | Max Input131.1K | Updated |
| ModelST | ProviderStepFun | Output Speed711.842 tok/s | Catalog Latency1.084 s | LLMBoard score60.3 | Official input / 1M$0.10 | Max Input65.5K | Updated |
| ModelME | ProviderSambanova | Output Speed638.7 tok/s | Catalog Latency2.04 s | LLMBoard score24.2 | Official input / 1MN/A | Max Input1M | Updated |
| ModelOP | ProviderOpenAI | Output Speed500 tok/s | Catalog Latency0.3 s | LLMBoard score30.3 | Official input / 1M$0.05 | Max Input400K | Updated |
| ModelOP | ProviderGroq | Output Speed500 tok/s | Catalog Latency0.5 s | LLMBoard score43.2 | Official input / 1MN/A | Max Input131K | Updated |
| ModelME | ProviderMeituan | Output Speed500 tok/s | Catalog Latency1.5 s | LLMBoard score32.2 | Official input / 1MN/A | Max Input256K | Updated |
| ModelAC | ProviderSambanova | Output Speed327.7 tok/s | Catalog Latency1.08 s | LLMBoard score30.8 | Official input / 1M$0.70 | Max Input128K | Updated |
| ModelME | ProviderGroq | Output Speed307.3 tok/s | Catalog Latency0.27 s | LLMBoard score24.2 | Official input / 1MN/A | Max Input1M | Updated |
| ModelME | ProviderFireworks | Output Speed292 tok/s | Catalog Latency0.5 s | LLMBoard scoreN/A | Official input / 1MN/A | Max Input131.1K | Updated |
| ModelMA | ProviderFireworks | Output Speed285.051 tok/s | Catalog Latency0.369 s | LLMBoard score75.1 | Official input / 1M$0.95 | Max Input262.1K | Updated |
| ModelME | ProviderGroq | Output Speed268 tok/s | Catalog Latency0.65 s | LLMBoard score21.4 | Official input / 1MN/A | Max Input128K | Updated |
| ModelME | ProviderGroq | Output Speed250 tok/s | Catalog Latency0.5 s | LLMBoard score12.6 | Official input / 1MN/A | Max Input128K | Updated |
| ModelMA | ProviderMistral AI | Output Speed237.5 tok/s | Catalog Latency0.34 s | LLMBoard score28.3 | Official input / 1M$0.10 | Max Input262.1K | Updated |
| ModelMI | ProviderTogether | Output Speed213.561 tok/s | Catalog Latency0.962 s | LLMBoard score71.1 | Official input / 1M$0.30 | Max Input1M | Updated |
| ModelOP | ProviderOpenAI | Output Speed200 tok/s | Catalog Latency2 s | LLMBoard score3.5 | Official input / 1M$0.10 | Max Input1M | Updated |
| ModelOP | ProviderOpenAI | Output Speed200 tok/s | Catalog Latency1 s | LLMBoard score44.7 | Official input / 1M$0.25 | Max Input400K | Updated |
| ModelOP | ProviderOpenAI | Output Speed200 tok/s | Catalog Latency1 s | LLMBoard scoreN/A | Official input / 1MN/A | Max Input400K | Updated |
| ModelME | ProviderHyperbolic | Output Speed200 tok/s | Catalog Latency0.5 s | LLMBoard scoreN/A | Official input / 1MN/A | Max Input131.1K | Updated |
| ModelME | ProviderFireworks | Output Speed197 tok/s | Catalog Latency0.65 s | LLMBoard score21.4 | Official input / 1MN/A | Max Input128K | Updated |
| ModelME | ProviderTogether | Output Speed194 tok/s | Catalog Latency0.5 s | LLMBoard scoreN/A | Official input / 1MN/A | Max Input131.1K | Updated |
| ModelGO | ProviderGoogle | Output Speed183 tok/s | Catalog Latency0.4 s | LLMBoard score27.6 | Official input / 1MN/A | Max Input1M | Updated |
| ModelGO | ProviderGoogle | Output Speed180.67 tok/s | Catalog Latency1.51 s | LLMBoard score73.8 | Official input / 1M$1.5 | Max Input1M | Updated |
| ModelME | ProviderDeepInfra | Output Speed171.5 tok/s | Catalog Latency0.24 s | LLMBoard scoreN/A | Official input / 1MN/A | Max Input128K | Updated |
| ModelME | ProviderTogether | Output Speed168 tok/s | Catalog Latency0.5 s | LLMBoard scoreN/A | Official input / 1MN/A | Max Input128K | Updated |
| ModelGO | ProviderGoogle | Output Speed150 tok/s | Catalog Latency0.3 s | LLMBoard score10.4 | Official input / 1MN/A | Max Input1M | Updated |
| ModelGO | ProviderGoogle | Output Speed150 tok/s | Catalog Latency0.3 s | LLMBoard scoreN/A | Official input / 1MN/A | Max Input1M | Updated |
| ModelME | ProviderLambda | Output Speed139.7 tok/s | Catalog Latency0.43 s | LLMBoard score13.5 | Official input / 1MN/A | Max Input10M | Updated |
| ModelAC | ProviderTogether | Output Speed138 tok/s | Catalog Latency0.5 s | LLMBoard score5.7 | Official input / 1M$0.175 | Max Input131.1K | Updated |
| ModelMA | ProviderMistral AI | Output Speed137.1 tok/s | Catalog Latency0.23 s | LLMBoard score23.2 | Official input / 1M$0.40 | Max Input128K | Updated |
| ModelMA | ProviderMistral AI | Output Speed137.1 tok/s | Catalog Latency0.23 s | LLMBoard score16.5 | Official input / 1M$0.10 | Max Input128K | Updated |
| ModelMA | ProviderMistral AI | Output Speed137.1 tok/s | Catalog Latency0.23 s | LLMBoard score7.8 | Official input / 1MN/A | Max Input128K | Updated |
| ModelMA | ProviderMistral AI | Output Speed134 tok/s | Catalog Latency0.2 s | LLMBoard score6.7 | Official input / 1MN/A | Max Input32K | Updated |
| ModelOP | ProviderOpenAI | Output Speed132 tok/s | Catalog Latency0.5 s | LLMBoard score23.3 | Official input / 1M$2.5 | Max Input128K | Updated |
| ModelOP | ProviderOpenAI | Output Speed132 tok/s | Catalog Latency0.5 s | LLMBoard score23.3 | Official input / 1M$2.5 | Max Input128K | Updated |
| ModelMA | ProviderMistral AI | Output Speed128.6 tok/s | Catalog Latency0.23 s | LLMBoard score28.3 | Official input / 1M$0.10 | Max Input262.1K | Updated |
| ModelME | ProviderFireworks | Output Speed125 tok/s | Catalog Latency0.2 s | LLMBoard scoreN/A | Official input / 1MN/A | Max Input128K | Updated |
| ModelAC | ProviderFireworks | Output Speed122.4 tok/s | Catalog Latency0.66 s | LLMBoard score28.2 | Official input / 1MN/A | Max Input128K | Updated |
| ModelMA | ProviderTogether | Output Speed122.302 tok/s | Catalog Latency0.972 s | LLMBoard score75.1 | Official input / 1M$0.95 | Max Input262.1K | Updated |
| ModelAN | ProviderBedrock | Output Speed120 tok/s | Catalog Latency0.5 s | LLMBoard score16.1 | Official input / 1MN/A | Max Input200K | Updated |
| ModelAN | ProviderBedrock | Output Speed120 tok/s | Catalog Latency0.5 s | LLMBoard score3.1 | Official input / 1MN/A | Max Input200K | Updated |
| ModelAN | ProviderBedrock | Output Speed120 tok/s | Catalog Latency0.5 s | LLMBoard score52.1 | Official input / 1MN/A | Max Input200K | Updated |
| ModelAN | ProviderBedrock | Output Speed120 tok/s | Catalog Latency0.5 s | LLMBoard score45.3 | Official input / 1MN/A | Max Input200K | Updated |
| ModelGO | ProviderGoogle | Output Speed120 tok/s | Catalog Latency0.4 s | LLMBoard scoreN/A | Official input / 1MN/A | Max Input32.8K | Updated |
| ModelME | ProviderDeepInfra | Output Speed118 tok/s | Catalog Latency0.5 s | LLMBoard scoreN/A | Official input / 1MN/A | Max Input131.1K | Updated |
| ModelME | ProviderFireworks | Output Speed116.1 tok/s | Catalog Latency0.53 s | LLMBoard score13.5 | Official input / 1MN/A | Max Input10M | Updated |
| ModelOP | ProviderOpenAI | Output Speed115 tok/s | Catalog Latency5.2 s | LLMBoard score43.2 | Official input / 1MN/A | Max Input131.1K | Updated |
| ModelOP | ProviderOpenAI | Output Speed115 tok/s | Catalog Latency5.2 s | LLMBoard score45.9 | Official input / 1MN/A | Max Input131.1K | Updated |
| ModelOP | ProviderOpenAI | Output Speed115 tok/s | Catalog Latency5.2 s | LLMBoard score16.1 | Official input / 1MN/A | Max Input128K | Updated |
| ModelOP | ProviderAzure | Output Speed115 tok/s | Catalog Latency5.2 s | LLMBoard score35.2 | Official input / 1M$1.1 | Max Input200K | Updated |
| ModelOP | ProviderOpenAI | Output Speed115 tok/s | Catalog Latency5.2 s | LLMBoard score35.2 | Official input / 1M$1.1 | Max Input200K | Updated |
| ModelOP | ProviderOpenAI | Output Speed115 tok/s | Catalog Latency5.2 s | LLMBoard score44.5 | Official input / 1M$1.1 | Max Input200K | Updated |
| ModelAC | ProviderFireworks | Output Speed110 tok/s | Catalog Latency0.26 s | LLMBoard score4.9 | Official input / 1MN/A | Max Input128K | Updated |
| ModelME | ProviderDeepInfra | Output Speed108 tok/s | Catalog Latency0.5 s | LLMBoard scoreN/A | Official input / 1MN/A | Max Input128K | Updated |
| ModelME | ProviderTogether | Output Speed106.9 tok/s | Catalog Latency0.54 s | LLMBoard score13.5 | Official input / 1MN/A | Max Input10M | Updated |
| ModelAN | ProviderBedrock | Output Speed104 tok/s | Catalog Latency0.5 s | LLMBoard score8.5 | Official input / 1MN/A | Max Input200K | Updated |
| ModelAN | ProviderBedrock | Output Speed104 tok/s | Catalog Latency0.5 s | LLMBoard scoreN/A | Official input / 1MN/A | Max Input200K | Updated |
| ModelOP | ProviderAzure | Output Speed104 tok/s | Catalog Latency0.3 s | LLMBoard score3.5 | Official input / 1M$30 | Max Input32.8K | Updated |
| ModelMI | ProviderMiniMax | Output Speed103.094 tok/s | Catalog Latency1.795 s | LLMBoard score44.3 | Official input / 1M$0.30 | Max Input1M | Updated |
| ModelMI | ProviderMiniMax | Output Speed101.856 tok/s | Catalog Latency1.579 s | LLMBoard score59.1 | Official input / 1M$0.30 | Max Input204.8K | Updated |
| ModelAN | ProviderBedrock | Output Speed101 tok/s | Catalog Latency0.5 s | LLMBoard score31.4 | Official input / 1MN/A | Max Input200K | Updated |
| ModelAN | ProviderBedrock | Output Speed101 tok/s | Catalog Latency0.5 s | LLMBoard score31.4 | Official input / 1MN/A | Max Input200K | Updated |
| ModelAN | ProviderBedrock | Output Speed101 tok/s | Catalog Latency0.5 s | LLMBoard score37.8 | Official input / 1MN/A | Max Input200K | Updated |
| ModelAN | ProviderBedrock | Output Speed101 tok/s | Catalog Latency0.5 s | LLMBoard score38.7 | Official input / 1MN/A | Max Input200K | Updated |
| ModelAN | ProviderAnthropic | Output Speed100 tok/s | Catalog Latency0.3 s | LLMBoard score8.5 | Official input / 1MN/A | Max Input200K | Updated |
| ModelAN | ProviderAnthropic | Output Speed100 tok/s | Catalog Latency0.5 s | LLMBoard score31.4 | Official input / 1MN/A | Max Input200K | Updated |
| ModelAN | ProviderAnthropic | Output Speed100 tok/s | Catalog Latency0.5 s | LLMBoard scoreN/A | Official input / 1MN/A | Max Input200K | Updated |
| ModelAN | ProviderAnthropic | Output Speed100 tok/s | Catalog Latency0.5 s | LLMBoard score16.1 | Official input / 1MN/A | Max Input200K | Updated |
| ModelAN | ProviderAnthropic | Output Speed100 tok/s | Catalog Latency0.5 s | LLMBoard score3.1 | Official input / 1MN/A | Max Input200K | Updated |
| ModelAN | ProviderAnthropic | Output Speed100 tok/s | Catalog Latency0.3 s | LLMBoard score45.2 | Official input / 1M$1 | Max Input200K | Updated |
| ModelAN | ProviderVertex AI | Output Speed100 tok/s | Catalog Latency0.3 s | LLMBoard score45.2 | Official input / 1M$1 | Max Input200K | Updated |
| ModelAN | ProviderAnthropic | Output Speed100 tok/s | Catalog Latency0.5 s | LLMBoard score52.1 | Official input / 1MN/A | Max Input200K | Updated |
| ModelAN | ProviderAnthropic | Output Speed100 tok/s | Catalog Latency0.5 s | LLMBoard score45.3 | Official input / 1MN/A | Max Input200K | Updated |
| ModelCO | ProviderBedrock | Output Speed100 tok/s | Catalog Latency0.5 s | LLMBoard scoreN/A | Official input / 1M$2.5 | Max Input128K | Updated |
| ModelDE | ProviderDeepSeek | Output Speed100 tok/s | Catalog Latency0.5 s | LLMBoard score13.8 | Official input / 1MN/A | Max Input8.2K | Updated |
| ModelDE | ProviderHyperbolic | Output Speed100 tok/s | Catalog Latency0.5 s | LLMBoard score13.8 | Official input / 1MN/A | Max Input8.2K | Updated |
| ModelDE | ProviderDeepSeek | Output Speed100 tok/s | Catalog Latency0.5 s | LLMBoard score25.5 | Official input / 1MN/A | Max Input131.1K | Updated |
| ModelOP | ProviderOpenAI | Output Speed100 tok/s | Catalog Latency0.5 s | LLMBoard scoreN/A | Official input / 1M$0.50 | Max Input16.4K | Updated |
| ModelOP | ProviderOpenAI | Output Speed100 tok/s | Catalog Latency0.5 s | LLMBoard score3.5 | Official input / 1M$30 | Max Input32.8K | Updated |
| ModelOP | ProviderOpenAI | Output Speed100 tok/s | Catalog Latency0.5 s | LLMBoard score14.0 | Official input / 1M$10 | Max Input128K | Updated |
| ModelOP | ProviderOpenAI | Output Speed100 tok/s | Catalog Latency10 s | LLMBoard score31.3 | Official input / 1M$2 | Max Input1M | Updated |
| ModelOP | ProviderOpenAI | Output Speed100 tok/s | Catalog Latency0.5 s | LLMBoard score23.3 | Official input / 1M$2.5 | Max Input128K | Updated |
| ModelOP | ProviderOpenAI | Output Speed100 tok/s | Catalog Latency2 s | LLMBoard score49.5 | Official input / 1M$1.25 | Max Input400K | Updated |
| ModelOP | ProviderOpenAI | Output Speed100 tok/s | Catalog Latency2 s | LLMBoard score62.2 | Official input / 1M$1.25 | Max Input400K | Updated |
| ModelOP | ProviderOpenAI | Output Speed100 tok/s | Catalog Latency2 s | LLMBoard score61.1 | Official input / 1MN/A | Max Input400K | Updated |
| ModelOP | ProviderOpenAI | Output Speed100 tok/s | Catalog Latency2 s | LLMBoard score70.0 | Official input / 1M$1.75 | Max Input400K | Updated |
| ModelOP | ProviderOpenAI | Output Speed100 tok/s | Catalog Latency2 s | LLMBoard score39.7 | Official input / 1M$1.75 | Max Input128K | Updated |
| ModelOP | ProviderOpenAI | Output Speed100 tok/s | Catalog Latency2 s | LLMBoard score53.5 | Official input / 1MN/A | Max Input400K | Updated |
| ModelOP | ProviderOpenAI | Output Speed100 tok/s | Catalog Latency6.5 s | LLMBoard score43.2 | Official input / 1MN/A | Max Input131.1K | Updated |
| ModelOP | ProviderOpenAI | Output Speed100 tok/s | Catalog Latency6.5 s | LLMBoard score45.9 | Official input / 1MN/A | Max Input131.1K | Updated |
| ModelXA | ProviderxAI | Output Speed100 tok/s | Catalog Latency0.7 s | LLMBoard score54.5 | Official input / 1MN/A | Max Input128K | Updated |
| ModelXA | ProviderxAI | Output Speed100 tok/s | Catalog Latency0.7 s | LLMBoard score47.9 | Official input / 1MN/A | Max Input128K | Updated |
| ModelXA | ProviderxAI | Output Speed100 tok/s | Catalog Latency0.7 s | LLMBoard score56.1 | Official input / 1MN/A | Max Input256K | Updated |
| ModelXA | ProviderxAI | Output Speed100 tok/s | Catalog Latency0.7 s | LLMBoard score46.5 | Official input / 1MN/A | Max Input256K | Updated |
| ModelAL | ProviderBedrock | Output Speed100 tok/s | Catalog Latency0.5 s | LLMBoard score2.0 | Official input / 1MN/A | Max Input256K | Updated |
| ModelAL | ProviderBedrock | Output Speed100 tok/s | Catalog Latency0.5 s | LLMBoard scoreN/A | Official input / 1MN/A | Max Input256.1K | Updated |
| ModelAL | ProviderGoogle | Output Speed100 tok/s | Catalog Latency0.3 s | LLMBoard scoreN/A | Official input / 1MN/A | Max Input256.1K | Updated |
| ModelME | ProviderBedrock | Output Speed100 tok/s | Catalog Latency0.5 s | LLMBoard score23.9 | Official input / 1MN/A | Max Input128K | Updated |
| ModelME | ProviderHyperbolic | Output Speed100 tok/s | Catalog Latency0.5 s | LLMBoard score12.6 | Official input / 1MN/A | Max Input128K | Updated |
| ModelME | ProviderBedrock | Output Speed100 tok/s | Catalog Latency0.5 s | LLMBoard score12.6 | Official input / 1MN/A | Max Input128K | Updated |
| ModelME | ProviderBedrock | Output Speed100 tok/s | Catalog Latency0.5 s | LLMBoard scoreN/A | Official input / 1MN/A | Max Input131.1K | Updated |
| ModelME | ProviderSambanova | Output Speed100 tok/s | Catalog Latency0.5 s | LLMBoard scoreN/A | Official input / 1MN/A | Max Input128K | Updated |
| ModelME | ProviderBedrock | Output Speed100 tok/s | Catalog Latency0.5 s | LLMBoard scoreN/A | Official input / 1MN/A | Max Input128K | Updated |
| ModelME | ProviderGroq | Output Speed100 tok/s | Catalog Latency0.5 s | LLMBoard scoreN/A | Official input / 1MN/A | Max Input128K | Updated |
| ModelME | ProviderBedrock | Output Speed100 tok/s | Catalog Latency0.5 s | LLMBoard score9.9 | Official input / 1MN/A | Max Input128K | Updated |
| ModelME | ProviderBedrock | Output Speed100 tok/s | Catalog Latency0.5 s | LLMBoard score21.4 | Official input / 1MN/A | Max Input128K | Updated |
| ModelME | ProviderMeituan | Output Speed100 tok/s | Catalog Latency3 s | LLMBoard score36.1 | Official input / 1MN/A | Max Input128K | Updated |
| ModelME | ProviderMeituan | Output Speed100 tok/s | Catalog Latency3 s | LLMBoard score60.3 | Official input / 1MN/A | Max Input128K | Updated |
| ModelME | ProviderMeituan | Output Speed100 tok/s | Catalog Latency3 s | LLMBoard score60.3 | Official input / 1MN/A | Max Input128K | Updated |
| ModelMI | ProviderMiniMax | Output Speed100 tok/s | Catalog Latency3 s | LLMBoard score53.3 | Official input / 1M$0.30 | Max Input1M | Updated |
| ModelMI | ProviderMiniMax | Output Speed100 tok/s | Catalog Latency3 s | LLMBoard score55.7 | Official input / 1M$0.30 | Max Input1M | Updated |
ModelAM Nova LiteAmazon | ProviderBedrock | Output Speed100 tok/s | Catalog Latency0.5 s | LLMBoard score12.7 | Official input / 1M$0.06 | Max Input300K | Updated |
| ModelAMNova MicroAmazon | ProviderBedrock | Output Speed100 tok/s | Catalog Latency0.5 s | LLMBoard score3.8 | Official input / 1M$0.035 | Max Input128K | Updated |
| ModelAMNova ProAmazon | ProviderBedrock | Output Speed100 tok/s | Catalog Latency0.5 s | LLMBoard score20.3 | Official input / 1M$0.80 | Max Input300K | Updated |
| ModelOP | ProviderAzure | Output Speed100 tok/s | Catalog Latency0.5 s | LLMBoard score16.1 | Official input / 1MN/A | Max Input128K | Updated |
| ModelAC | ProviderHyperbolic | Output Speed100 tok/s | Catalog Latency0.5 s | LLMBoard score20.7 | Official input / 1M$1.4 | Max Input131.1K | Updated |
| ModelAC | ProviderHyperbolic | Output Speed100 tok/s | Catalog Latency0.5 s | LLMBoard score4.9 | Official input / 1MN/A | Max Input128K | Updated |
| ModelAC | ProviderFireworks | Output Speed99.15 tok/s | Catalog Latency0.53 s | LLMBoard score24.2 | Official input / 1M$0.287 | Max Input32.8K | Updated |
| ModelOP | ProviderAzure | Output Speed99 tok/s | Catalog Latency0.53 s | LLMBoard score23.3 | Official input / 1M$2.5 | Max Input128K | Updated |
| ModelME | ProviderTogether | Output Speed97.93 tok/s | Catalog Latency0.2 s | LLMBoard score24.2 | Official input / 1MN/A | Max Input1M | Updated |
| ModelDE | ProviderDeepInfra | Output Speed97 tok/s | Catalog Latency0.82 s | LLMBoard score57.5 | Official input / 1M$0.14 | Max Input163.8K | Updated |
| ModelOP | ProviderAzure | Output Speed97 tok/s | Catalog Latency0.6 s | LLMBoard score14.0 | Official input / 1M$10 | Max Input128K | Updated |
| ModelME | ProviderTogether | Output Speed94 tok/s | Catalog Latency0.5 s | LLMBoard score12.6 | Official input / 1MN/A | Max Input128K | Updated |
| ModelME | ProviderLambda | Output Speed93.69 tok/s | Catalog Latency0.65 s | LLMBoard score24.2 | Official input / 1MN/A | Max Input1M | Updated |
| ModelOP | ProviderAzure | Output Speed92 tok/s | Catalog Latency0.54 s | LLMBoard score23.3 | Official input / 1M$2.5 | Max Input128K | Updated |
| ModelOP | ProviderAzure | Output Speed92 tok/s | Catalog Latency0.52 s | LLMBoard score7.5 | Official input / 1M$0.15 | Max Input128K | Updated |
| ModelGO | ProviderGoogle | Output Speed90 tok/s | Catalog Latency0.6 s | LLMBoard score67.1 | Official input / 1MN/A | Max Input1M | Updated |
| ModelOP | ProviderAzure | Output Speed90 tok/s | Catalog Latency0.8 s | LLMBoard scoreN/A | Official input / 1M$0.50 | Max Input16.4K | Updated |
| ModelOP | ProviderOpenAI | Output Speed90 tok/s | Catalog Latency2.5 s | LLMBoard score49.5 | Official input / 1M$1.25 | Max Input400K | Updated |
| ModelOP | ProviderOpenAI | Output Speed90 tok/s | Catalog Latency2.5 s | LLMBoard score62.2 | Official input / 1M$1.25 | Max Input400K | Updated |
| ModelAC | ProviderNovita | Output Speed88.84 tok/s | Catalog Latency0.73 s | LLMBoard score28.2 | Official input / 1MN/A | Max Input128K | Updated |
| ModelXI | ProviderNovita | Output Speed87.039 tok/s | Catalog Latency2.231 s | LLMBoard score61.6 | Official input / 1M$0.14 | Max Input1M | Updated |
| ModelGO | ProviderGoogle | Output Speed86.301 tok/s | Catalog Latency8.059 s | LLMBoard score49.8 | Official input / 1M$1.25 | Max Input1M | Updated |
| ModelGO | ProviderGoogle | Output Speed85 tok/s | Catalog Latency0.7 s | LLMBoard score20.0 | Official input / 1MN/A | Max Input2.1M | Updated |
| ModelGO | ProviderGoogle | Output Speed85 tok/s | Catalog Latency0.7 s | LLMBoard score20.4 | Official input / 1MN/A | Max Input1M | Updated |
| ModelGO | ProviderGoogle | Output Speed85 tok/s | Catalog Latency0.7 s | LLMBoard score38.5 | Official input / 1M$0.30 | Max Input1M | Updated |
| ModelGO | ProviderGoogle | Output Speed85 tok/s | Catalog Latency0.7 s | LLMBoard score49.8 | Official input / 1M$1.25 | Max Input1M | Updated |
| ModelZA | ProviderDeepInfra | Output Speed85 tok/s | Catalog Latency0.7 s | LLMBoard score53.6 | Official input / 1M$0.60 | Max Input131.1K | Updated |
| ModelXA | ProviderxAI | Output Speed85 tok/s | Catalog Latency0.7 s | LLMBoard score20.4 | Official input / 1MN/A | Max Input128K | Updated |
| ModelME | ProviderDeepInfra | Output Speed83.59 tok/s | Catalog Latency0.38 s | LLMBoard score24.2 | Official input / 1MN/A | Max Input1M | Updated |
| ModelAC | ProviderDeepInfra | Output Speed82.57 tok/s | Catalog Latency0.84 s | LLMBoard score28.2 | Official input / 1MN/A | Max Input128K | Updated |
| ModelOP | ProviderOpenAI | Output Speed80 tok/s | Catalog Latency3 s | LLMBoard score49.5 | Official input / 1M$1.25 | Max Input400K | Updated |
| ModelOP | ProviderOpenAI | Output Speed80 tok/s | Catalog Latency3.5 s | LLMBoard score62.2 | Official input / 1M$1.25 | Max Input400K | Updated |
| ModelOP | ProviderOpenAI | Output Speed80 tok/s | Catalog Latency3 s | LLMBoard score62.2 | Official input / 1M$1.25 | Max Input400K | Updated |
| ModelXA | ProviderxAI | Output Speed80 tok/s | Catalog Latency1.2 s | LLMBoard score50.8 | Official input / 1MN/A | Max Input256K | Updated |
| ModelME | ProviderFireworks | Output Speed78 tok/s | Catalog Latency0.5 s | LLMBoard score23.9 | Official input / 1MN/A | Max Input128K | Updated |
| ModelXA | ProviderxAI | Output Speed76.41 tok/s | Catalog Latency1.38 s | LLMBoard score35.4 | Official input / 1MN/A | Max Input256K | Updated |
| ModelME | ProviderDeepInfra | Output Speed76.1 tok/s | Catalog Latency0.31 s | LLMBoard score13.5 | Official input / 1MN/A | Max Input10M | Updated |
| ModelAC | ProviderDeepInfra | Output Speed76.04 tok/s | Catalog Latency0.44 s | LLMBoard score24.2 | Official input / 1M$0.287 | Max Input32.8K | Updated |
| ModelME | ProviderSambanova | Output Speed74 tok/s | Catalog Latency0.5 s | LLMBoard score12.6 | Official input / 1MN/A | Max Input128K | Updated |
| ModelME | ProviderNovita | Output Speed69.82 tok/s | Catalog Latency0.85 s | LLMBoard score13.5 | Official input / 1MN/A | Max Input10M | Updated |
| ModelME | ProviderNovita | Output Speed69.42 tok/s | Catalog Latency0.62 s | LLMBoard score24.2 | Official input / 1MN/A | Max Input1M | Updated |
| ModelOP | ProviderOpenAI | Output Speed68.886 tok/s | Catalog Latency5 s | LLMBoard score24.1 | Official input / 1M$0.40 | Max Input1M | Updated |
| ModelXI | ProviderNovita | Output Speed68.716 tok/s | Catalog Latency4.348 s | LLMBoard score46.2 | Official input / 1M$0.435 | Max Input1M | Updated |
| ModelAC | ProviderFireworks | Output Speed68.17 tok/s | Catalog Latency0.78 s | LLMBoard score41.9 | Official input / 1M$0.70 | Max Input128K | Updated |
| ModelOP | ProviderOpenAI | Output Speed66 tok/s | Catalog Latency16.2 s | LLMBoard score35.9 | Official input / 1M$15 | Max Input200K | Updated |
| ModelOP | ProviderOpenAI | Output Speed66 tok/s | Catalog Latency16.2 s | LLMBoard score35.9 | Official input / 1M$15 | Max Input128K | Updated |
| ModelGO | ProviderGoogle | Output Speed65.14 tok/s | Catalog Latency5.259 s | LLMBoard score64.6 | Official input / 1M$0.50 | Max Input1M | Updated |
| ModelME | ProviderTogether | Output Speed65 tok/s | Catalog Latency0.65 s | LLMBoard score21.4 | Official input / 1MN/A | Max Input128K | Updated |
| ModelME | ProviderFireworks | Output Speed63.03 tok/s | Catalog Latency0.62 s | LLMBoard score24.2 | Official input / 1MN/A | Max Input1M | Updated |
| ModelDE | ProviderDeepInfra | Output Speed63 tok/s | Catalog Latency0.5 s | LLMBoard score13.8 | Official input / 1MN/A | Max Input8.2K | Updated |
| ModelAC | ProviderTogether | Output Speed62.14 tok/s | Catalog Latency0.74 s | LLMBoard score24.2 | Official input / 1M$0.287 | Max Input32.8K | Updated |
| ModelCO | ProviderCohere | Output Speed59 tok/s | Catalog Latency0.65 s | LLMBoard scoreN/A | Official input / 1M$2.5 | Max Input128K | Updated |
| ModelAC | ProviderFireworks | Output Speed59 tok/s | Catalog Latency0.37 s | LLMBoard score20.7 | Official input / 1M$1.4 | Max Input131.1K | Updated |
| ModelME | ProviderTogether | Output Speed57 tok/s | Catalog Latency0.5 s | LLMBoard score9.9 | Official input / 1MN/A | Max Input128K | Updated |
| ModelMI | ProviderMiniMax | Output Speed55.078 tok/s | Catalog Latency0.891 s | LLMBoard score71.1 | Official input / 1M$0.30 | Max Input1M | Updated |
| ModelDE | ProviderDeepSeek | Output Speed50 tok/s | Catalog Latency0.5 s | LLMBoard score57.5 | Official input / 1M$0.14 | Max Input131.1K | Updated |
| ModelZA | ProviderZAI | Output Speed50 tok/s | Catalog Latency2 s | LLMBoard score37.9 | Official input / 1MN/A | Max Input128K | Updated |
| ModelOP | ProviderOpenAI | Output Speed50 tok/s | Catalog Latency20 s | LLMBoard score35.7 | Official input / 1MN/A | Max Input128K | Updated |
| ModelOP | ProviderOpenAI | Output Speed50 tok/s | Catalog Latency3 s | LLMBoard score49.6 | Official input / 1MN/A | Max Input400K | Updated |
| ModelOP | ProviderOpenAI | Output Speed50 tok/s | Catalog Latency3.5 s | LLMBoard score49.6 | Official input / 1MN/A | Max Input400K | Updated |
| ModelOP | ProviderOpenAI | Output Speed50 tok/s | Catalog Latency3 s | LLMBoard score59.2 | Official input / 1MN/A | Max Input400K | Updated |
| ModelIB | ProviderReplicate | Output Speed50 tok/s | Catalog Latency0.3 s | LLMBoard score3.0 | Official input / 1MN/A | Max Input128K | Updated |
| ModelLA | ProviderFriendliAI | Output Speed50 tok/s | Catalog Latency0.4 s | LLMBoard score44.3 | Official input / 1MN/A | Max Input32.8K | Updated |
| ModelME | ProviderFireworks | Output Speed50 tok/s | Catalog Latency0.5 s | LLMBoard score9.9 | Official input / 1MN/A | Max Input128K | Updated |
| ModelOP | ProviderOpenAI | Output Speed50 tok/s | Catalog Latency20 s | LLMBoard score52.4 | Official input / 1M$2 | Max Input200K | Updated |
| ModelMA | ProviderDeepInfra | Output Speed49 tok/s | Catalog Latency0.2 s | LLMBoard score6.7 | Official input / 1MN/A | Max Input32K | Updated |
| ModelOP | ProviderOpenAI | Output Speed48.852 tok/s | Catalog Latency5.239 s | LLMBoard score76.8 | Official input / 1M$0.20 | Max Input1.1M | Updated |
| ModelAC | ProviderTogether | Output Speed47 tok/s | Catalog Latency0.5 s | LLMBoard score20.7 | Official input / 1M$1.4 | Max Input131.1K | Updated |
| ModelAN | ProviderAnthropic | Output Speed45.954 tok/s | Catalog Latency1.29 s | LLMBoard score87.1 | Official input / 1M$5 | Max Input1M | Updated |
| ModelDE | ProviderDeepInfra | Output Speed45.04 tok/s | Catalog Latency0.61 s | LLMBoard score43.5 | Official input / 1MN/A | Max Input131.1K | Updated |
| ModelMA | ProviderNovita | Output Speed45 tok/s | Catalog Latency0.95 s | LLMBoard score36.6 | Official input / 1M$0.60 | Max Input131.1K | Updated |
| ModelAC | ProviderDeepInfra | Output Speed44 tok/s | Catalog Latency0.5 s | LLMBoard score4.9 | Official input / 1MN/A | Max Input128K | Updated |
| ModelGO | ProviderTogether | Output Speed42.09 tok/s | Catalog Latency0.43 s | LLMBoard scoreN/A | Official input / 1MN/A | Max Input32K | Updated |
| ModelAN | ProviderGoogle | Output Speed42 tok/s | Catalog Latency0.4 s | LLMBoard score8.5 | Official input / 1MN/A | Max Input200K | Updated |
| ModelAN | ProviderGoogle | Output Speed42 tok/s | Catalog Latency0.4 s | LLMBoard score31.4 | Official input / 1MN/A | Max Input200K | Updated |
| ModelAN | ProviderGoogle | Output Speed42 tok/s | Catalog Latency0.4 s | LLMBoard score31.4 | Official input / 1MN/A | Max Input200K | Updated |
| ModelAN | ProviderAnthropic | Output Speed42 tok/s | Catalog Latency0.4 s | LLMBoard score37.8 | Official input / 1MN/A | Max Input200K | Updated |
| ModelAN | ProviderGoogle | Output Speed42 tok/s | Catalog Latency0.4 s | LLMBoard score37.8 | Official input / 1MN/A | Max Input200K | Updated |
| ModelAN | ProviderGoogle | Output Speed42 tok/s | Catalog Latency0.4 s | LLMBoard scoreN/A | Official input / 1MN/A | Max Input200K | Updated |
| ModelAN | ProviderGoogle | Output Speed42 tok/s | Catalog Latency0.4 s | LLMBoard score16.1 | Official input / 1MN/A | Max Input200K | Updated |
| ModelAN | ProviderGoogle | Output Speed42 tok/s | Catalog Latency0.4 s | LLMBoard score3.1 | Official input / 1MN/A | Max Input200K | Updated |
| ModelAN | ProviderGoogle | Output Speed42 tok/s | Catalog Latency0.4 s | LLMBoard score52.1 | Official input / 1MN/A | Max Input200K | Updated |
| ModelAN | ProviderGoogle | Output Speed42 tok/s | Catalog Latency0.4 s | LLMBoard score45.3 | Official input / 1MN/A | Max Input200K | Updated |
| ModelAN | ProviderAnthropic | Output Speed42 tok/s | Catalog Latency0.5 s | LLMBoard score69.3 | Official input / 1M$5 | Max Input200K | Updated |
| ModelAN | ProviderAnthropic | Output Speed42 tok/s | Catalog Latency0.5 s | LLMBoard score84.0 | Official input / 1M$5 | Max Input1M | Updated |
| ModelAN | ProviderVertex AI | Output Speed42 tok/s | Catalog Latency0.5 s | LLMBoard score84.0 | Official input / 1M$5 | Max Input1M | Updated |
| ModelAN | ProviderVertex AI | Output Speed42 tok/s | Catalog Latency0.5 s | LLMBoard score87.1 | Official input / 1M$5 | Max Input1M | Updated |
| ModelAN | ProviderAnthropic | Output Speed42 tok/s | Catalog Latency0.4 s | LLMBoard score38.7 | Official input / 1MN/A | Max Input200K | Updated |
| ModelAN | ProviderGoogle | Output Speed42 tok/s | Catalog Latency0.4 s | LLMBoard score38.7 | Official input / 1MN/A | Max Input200K | Updated |
| ModelAN | ProviderAnthropic | Output Speed42 tok/s | Catalog Latency0.4 s | LLMBoard score56.6 | Official input / 1M$3 | Max Input200K | Updated |
| ModelAL | ProviderGoogle | Output Speed42 tok/s | Catalog Latency0.3 s | LLMBoard score2.0 | Official input / 1MN/A | Max Input256K | Updated |
| ModelME | ProviderLambda | Output Speed42 tok/s | Catalog Latency0.5 s | LLMBoard score23.9 | Official input / 1MN/A | Max Input128K | Updated |
| ModelME | ProviderGoogle | Output Speed42 tok/s | Catalog Latency0.4 s | LLMBoard score23.9 | Official input / 1MN/A | Max Input128K | Updated |
| ModelME | ProviderLambda | Output Speed42 tok/s | Catalog Latency0.5 s | LLMBoard score12.6 | Official input / 1MN/A | Max Input128K | Updated |
| ModelME | ProviderLambda | Output Speed42 tok/s | Catalog Latency0.5 s | LLMBoard scoreN/A | Official input / 1MN/A | Max Input131.1K | Updated |
| ModelME | ProviderHyperbolic | Output Speed42 tok/s | Catalog Latency0.5 s | LLMBoard score9.9 | Official input / 1MN/A | Max Input128K | Updated |
| ModelME | ProviderLambda | Output Speed42 tok/s | Catalog Latency0.65 s | LLMBoard score21.4 | Official input / 1MN/A | Max Input128K | Updated |
| ModelME | ProviderHyperbolic | Output Speed42 tok/s | Catalog Latency0.65 s | LLMBoard score21.4 | Official input / 1MN/A | Max Input128K | Updated |
| ModelMA | ProviderGoogle | Output Speed42 tok/s | Catalog Latency0.4 s | LLMBoard score6.4 | Official input / 1MN/A | Max Input128K | Updated |
| ModelMA | ProviderGoogle | Output Speed42 tok/s | Catalog Latency0.4 s | LLMBoard scoreN/A | Official input / 1M$0.15 | Max Input128K | Updated |
| ModelAC | ProviderLambda | Output Speed42 tok/s | Catalog Latency0.5 s | LLMBoard score4.9 | Official input / 1MN/A | Max Input128K | Updated |
| ModelMI | ProviderFireworks | Output Speed41.825 tok/s | Catalog Latency0.903 s | LLMBoard score59.1 | Official input / 1M$0.30 | Max Input196.6K | Updated |
| ModelME | ProviderHyperbolic | Output Speed40 tok/s | Catalog Latency0.5 s | LLMBoard score23.9 | Official input / 1MN/A | Max Input128K | Updated |
| ModelAC | ProviderNovita | Output Speed38.51 tok/s | Catalog Latency1.02 s | LLMBoard score41.9 | Official input / 1M$0.70 | Max Input128K | Updated |
| ModelDE | ProviderNovita | Output Speed37.96 tok/s | Catalog Latency1.18 s | LLMBoard score43.5 | Official input / 1MN/A | Max Input131.1K | Updated |
| ModelDE | ProviderDeepInfra | Output Speed37 tok/s | Catalog Latency0.65 s | LLMBoard score11.6 | Official input / 1MN/A | Max Input128K | Updated |
| ModelDE | ProviderDeepInfra | Output Speed37 tok/s | Catalog Latency0.65 s | LLMBoard score14.8 | Official input / 1MN/A | Max Input128K | Updated |
| ModelME | ProviderDeepInfra | Output Speed37 tok/s | Catalog Latency0.65 s | LLMBoard score21.4 | Official input / 1MN/A | Max Input128K | Updated |
| ModelME | ProviderTogether | Output Speed35 tok/s | Catalog Latency0.5 s | LLMBoard score23.9 | Official input / 1MN/A | Max Input128K | Updated |
| ModelOP | ProviderOpenAI | Output Speed34.885 tok/s | Catalog Latency6.852 s | LLMBoard score87.7 | Official input / 1M$2 | Max Input1.1M | Updated |
| ModelDE | ProviderDeepSeek | Output Speed33.196 tok/s | Catalog Latency3.185 s | LLMBoard score90.0 | Official input / 1M$0.435 | Max Input1M | Updated |
| ModelGO | ProviderDeepInfra | Output Speed33 tok/s | Catalog Latency0.2 s | LLMBoard score10.4 | Official input / 1MN/A | Max Input131.1K | Updated |
| ModelGO | ProviderDeepInfra | Output Speed33 tok/s | Catalog Latency0.2 s | LLMBoard score15.0 | Official input / 1MN/A | Max Input131.1K | Updated |
| ModelGO | ProviderNovita | Output Speed33 tok/s | Catalog Latency0.2 s | LLMBoard score15.0 | Official input / 1MN/A | Max Input131.1K | Updated |
| ModelGO | ProviderDeepInfra | Output Speed33 tok/s | Catalog Latency0.2 s | LLMBoard scoreN/A | Official input / 1MN/A | Max Input131.1K | Updated |
| ModelMI | ProviderDeepInfra | Output Speed33 tok/s | Catalog Latency0.2 s | LLMBoard score8.6 | Official input / 1M$0.125 | Max Input16K | Updated |
| ModelAC | ProviderNovita | Output Speed32.43 tok/s | Catalog Latency0.93 s | LLMBoard score30.8 | Official input / 1M$0.70 | Max Input128K | Updated |
| ModelME | ProviderFireworks | Output Speed32 tok/s | Catalog Latency0.5 s | LLMBoard score12.6 | Official input / 1MN/A | Max Input128K | Updated |
| ModelAC | ProviderHyperbolic | Output Speed31.9 tok/s | Catalog Latency1.05 s | LLMBoard score24.2 | Official input / 1M$0.287 | Max Input32.8K | Updated |
| ModelDE | ProviderDeepSeek | Output Speed30 tok/s | Catalog Latency1 s | LLMBoard score57.5 | Official input / 1M$0.14 | Max Input131.1K | Updated |
| ModelZA | ProviderZAI | Output Speed30 tok/s | Catalog Latency3 s | LLMBoard score67.1 | Official input / 1M$1 | Max Input200K | Updated |
| ModelME | ProviderDeepInfra | Output Speed27 tok/s | Catalog Latency0.5 s | LLMBoard score23.9 | Official input / 1MN/A | Max Input128K | Updated |
| ModelAC | ProviderDeepInfra | Output Speed26.95 tok/s | Catalog Latency1.19 s | LLMBoard score30.8 | Official input / 1M$0.70 | Max Input128K | Updated |
| ModelOP | ProviderOpenAI | Output Speed26.723 tok/s | Catalog Latency9.743 s | LLMBoard score95.3 | Official input / 1M$5 | Max Input1.1M | Updated |
| ModelMA | ProviderFireworks | Output Speed25.734 tok/s | Catalog Latency4.248 s | LLMBoard score92.0 | Official input / 1M$3 | Max Input1M | Updated |
| ModelOP | ProviderOpenAI | Output Speed25 tok/s | Catalog Latency30 s | LLMBoard score72.1 | Official input / 1M$21 | Max Input400K | Updated |
| ModelME | ProviderDeepInfra | Output Speed25 tok/s | Catalog Latency0.5 s | LLMBoard score12.6 | Official input / 1MN/A | Max Input128K | Updated |
| ModelOP | ProviderOpenAI | Output Speed25 tok/s | Catalog Latency30 s | LLMBoard scoreN/A | Official input / 1M$20 | Max Input200K | Updated |
| ModelMI | ProviderDeepInfra | Output Speed25 tok/s | Catalog Latency0.5 s | LLMBoard score6.0 | Official input / 1MN/A | Max Input128K | Updated |
| ModelXI | ProviderDeepInfra | Output Speed24.571 tok/s | Catalog Latency3.783 s | LLMBoard score61.6 | Official input / 1M$0.14 | Max Input262.1K | Updated |
| ModelME | ProviderDeepInfra | Output Speed24 tok/s | Catalog Latency0.5 s | LLMBoard score9.9 | Official input / 1MN/A | Max Input128K | Updated |
| ModelAC | ProviderTogether | Output Speed23.74 tok/s | Catalog Latency0.79 s | LLMBoard score41.9 | Official input / 1M$0.70 | Max Input128K | Updated |
| ModelMA | ProviderTogether | Output Speed23.56 tok/s | Catalog Latency13.029 s | LLMBoard score92.0 | Official input / 1M$3 | Max Input1M | Updated |
| ModelMI | ProviderAzure | Output Speed23 tok/s | Catalog Latency0.52 s | LLMBoard scoreN/A | Official input / 1MN/A | Max Input128K | Updated |
| ModelGO | ProviderGoogle | Output Speed22.824 tok/s | Catalog Latency11.149 s | LLMBoard score77.5 | Official input / 1M$2 | Max Input1M | Updated |
| ModelME | ProviderMeta Model API | Output Speed22.636 tok/s | Catalog Latency2.993 s | LLMBoard score81.6 | Official input / 1M$1.25 | Max Input1M | Updated |
| ModelDE | ProviderReplicate | Output Speed22 tok/s | Catalog Latency0.5 s | LLMBoard scoreN/A | Official input / 1MN/A | Max Input129.3K | Updated |
| ModelME | ProviderReplicate | Output Speed22 tok/s | Catalog Latency0.5 s | LLMBoard score23.9 | Official input / 1MN/A | Max Input128K | Updated |
| ModelXA | ProviderxAI | Output Speed21.769 tok/s | Catalog Latency1.483 s | LLMBoard scoreN/A | Official input / 1MN/A | Max Input2M | Updated |
| ModelAC | ProviderDeepInfra | Output Speed21.74 tok/s | Catalog Latency1.23 s | LLMBoard score41.9 | Official input / 1M$0.70 | Max Input128K | Updated |
| ModelDE | ProviderNovita | Output Speed19.447 tok/s | Catalog Latency4.368 s | LLMBoard score77.4 | Official input / 1M$0.14 | Max Input1M | Updated |
| ModelAN | ProviderAnthropic | Output Speed17.479 tok/s | Catalog Latency3.855 s | LLMBoard score79.4 | Official input / 1M$5 | Max Input1M | Updated |
| ModelOP | ProviderAzure | Output Speed16 tok/s | Catalog Latency0.54 s | LLMBoard score35.9 | Official input / 1M$15 | Max Input200K | Updated |
| ModelOP | ProviderAzure | Output Speed16 tok/s | Catalog Latency0.54 s | LLMBoard score35.9 | Official input / 1M$15 | Max Input128K | Updated |
| ModelAC | ProviderTogether | Output Speed15.66 tok/s | Catalog Latency3.221 s | LLMBoard score67.0 | Official input / 1M$0.50 | Max Input1M | Updated |
| ModelAN | ProviderAnthropic | Output Speed14.831 tok/s | Catalog Latency0.4 s | LLMBoard score72.6 | Official input / 1M$3 | Max Input200K | Updated |
| ModelGO | ProviderGoogle | Output Speed12.652 tok/s | Catalog Latency8.955 s | LLMBoard score85.0 | Official input / 1M$0.75 | Max Input1M | Updated |
| ModelAN | ProviderAnthropic | Output Speed10.589 tok/s | Catalog Latency4.967 s | LLMBoard score83.2 | Official input / 1M$2 | Max Input1M | Updated |
| ModelAC | ProviderDeepInfra | Output Speed10 tok/s | Catalog Latency0.5 s | LLMBoard score20.7 | Official input / 1M$1.4 | Max Input131.1K | Updated |
| ModelOP | ProviderOpenAI | Output Speed9.147 tok/s | Catalog Latency1.87 s | LLMBoard score76.6 | Official input / 1M$2.5 | Max Input1M | Updated |
| ModelAN | ProviderAnthropic | Output Speed9.081 tok/s | Catalog Latency28.237 s | LLMBoard score95.2 | Official input / 1M$10 | Max Input1M | Updated |
| ModelDE | ProviderDeepSeek | Output Speed9 tok/s | Catalog Latency0.3 s | LLMBoard score43.5 | Official input / 1MN/A | Max Input131.1K | Updated |
| ModelDE | ProviderDeepSeek | Output Speed9 tok/s | Catalog Latency0.3 s | LLMBoard score43.5 | Official input / 1MN/A | Max Input131.1K | Updated |
| ModelGO | ProviderFriendliAI | Output Speed8.296 tok/s | Catalog Latency2.909 s | LLMBoard score59.7 | Official input / 1MN/A | Max Input262.1K | Updated |
| ModelDE | ProviderDeepSeek | Output Speed7.85 tok/s | Catalog Latency1.996 s | LLMBoard score57.5 | Official input / 1M$0.14 | Max Input131.1K | Updated |
| ModelAC | ProviderNovita | Output Speed6.736 tok/s | Catalog Latency15.588 s | LLMBoard score56.8 | Official input / 1M$0.30 | Max Input262.1K | Updated |
| ModelME | ProviderMeta Model API | Output Speed6.152 tok/s | Catalog Latency4.643 s | LLMBoard score88.4 | Official input / 1M$1.25 | Max Input1M | Updated |
| ModelAC | ProviderNovita | Output Speed6.118 tok/s | Catalog Latency7.065 s | LLMBoard score59.9 | Official input / 1M$0.60 | Max Input262.1K | Updated |
| ModelDE | ProviderDeepInfra | Output Speed5.85 tok/s | Catalog Latency17.843 s | LLMBoard score77.4 | Official input / 1M$0.14 | Max Input1M | Updated |
| ModelAC | ProviderTogether | Output Speed5.805 tok/s | Catalog Latency4.161 s | LLMBoard score78.3 | Official input / 1M$2.5 | Max Input1M | Updated |
| ModelGO | ProviderGoogle | Output Speed5.69 tok/s | Catalog Latency0.44 s | LLMBoard score18.3 | Official input / 1M$0.10 | Max Input1M | Updated |
| ModelBY | ProviderDeepInfra | Output Speed5.594 tok/s | Catalog Latency4.022 s | LLMBoard score69.0 | Official input / 1MN/A | Max Input256K | Updated |
| ModelXA | ProviderxAI | Output Speed4.688 tok/s | Catalog Latency7.024 s | LLMBoard score78.7 | Official input / 1M$2 | Max Input500K | Updated |
| ModelMA | ProviderMistral AI | Output Speed4.301 tok/s | Catalog Latency0.441 s | LLMBoard score32.6 | Official input / 1M$0.15 | Max Input256K | Updated |
| ModelXA | ProviderxAI | Output Speed4.268 tok/s | Catalog Latency1.989 s | LLMBoard score80.3 | Official input / 1M$2 | Max Input500K | Updated |
| ModelMA | ProviderMoonshot AI | Output Speed4.128 tok/s | Catalog Latency6.861 s | LLMBoard score92.0 | Official input / 1M$3 | Max Input1M | Updated |
| ModelDE | ProviderTogether | Output Speed4 tok/s | Catalog Latency0.6 s | LLMBoard score43.5 | Official input / 1MN/A | Max Input131.1K | Updated |
| ModelAN | ProviderAnthropic | Output Speed3.319 tok/s | Catalog Latency5.51 s | LLMBoard score96.5 | Official input / 1M$5 | Max Input1M | Updated |
| ModelOP | ProviderOpenAI | Output Speed2.395 tok/s | Catalog Latency7.363 s | LLMBoard score62.9 | Official input / 1M$1.75 | Max Input400K | Updated |
| ModelDE | ProviderFireworks | Output Speed2.1 tok/s | Catalog Latency0.3 s | LLMBoard score43.5 | Official input / 1MN/A | Max Input131.1K | Updated |
| ModelGO | ProviderGoogle | Output Speed1.218 tok/s | Catalog Latency1.68 s | LLMBoard score78.4 | Official input / 1M$1.5 | Max Input1M | Updated |
| ModelMA | ProviderMistral AI | Output Speed0.961 tok/s | Catalog Latency10.401 s | LLMBoard score48.7 | Official input / 1M$1.5 | Max Input256K | Updated |
| ModelDE | ProviderDeepInfra | Output Speed0.9 tok/s | Catalog Latency0.3 s | LLMBoard score43.5 | Official input / 1MN/A | Max Input131.1K | Updated |
| ModelDE | ProviderFireworks | Output Speed0.879 tok/s | Catalog Latency8.101 s | LLMBoard score77.4 | Official input / 1M$0.14 | Max Input1M | Updated |
| ModelMA | ProviderMistral AI | Output Speed0.778 tok/s | Catalog Latency2.523 s | LLMBoard score15.3 | Official input / 1M$0.50 | Max Input262.1K | Updated |
| ModelMA | ProviderNovita | Output Speed0.196 tok/s | Catalog Latency9.84 s | LLMBoard score92.0 | Official input / 1M$3 | Max Input1M | Updated |
| ModelMA | ProviderMistral AI | Output Speed0.1 tok/s | Catalog Latency0.18 s | LLMBoard scoreN/A | Official input / 1MN/A | Max Input128K | Updated |
| ModelMA | ProviderMistral AI | Output Speed0.1 tok/s | Catalog Latency0.5 s | LLMBoard score6.4 | Official input / 1MN/A | Max Input128K | Updated |
| ModelMA | ProviderMistral AI | Output Speed0.1 tok/s | Catalog Latency0.5 s | LLMBoard scoreN/A | Official input / 1M$0.15 | Max Input128K | Updated |
| ModelMA | ProviderMistral AI | Output Speed0.1 tok/s | Catalog Latency0.5 s | LLMBoard scoreN/A | Official input / 1MN/A | Max Input32.8K | Updated |
| ModelMA | ProviderMistral AI | Output Speed0.1 tok/s | Catalog Latency0.5 s | LLMBoard scoreN/A | Official input / 1M$0.15 | Max Input128K | Updated |
| ModelMA | ProviderMistral AI | Output Speed0.1 tok/s | Catalog Latency0.5 s | LLMBoard score20.6 | Official input / 1MN/A | Max Input128K | Updated |
Lowest catalog latency in the current records: Min istral 3 (3B Reasoning 2512) via Mistral AI at 0.16s.
Output speed vs. catalog latency
Each point is one canonical model version offered by one provider in the runtime leaderboard. Upper-left is generally preferable: higher token generation speed with lower catalog latency.
Output speedCatalog latency
Each mark is one model-provider record301 records
Provider runtime: observed over 7 days
This provider ranking uses observed character throughput and time to first token. Character throughput is not converted into model token throughput or benchmark capability.
Sort by
Provider | P50 TTFT | P95 TTFT | P50 Throughput | P5 Throughput | 7d Calls | Success Rate | Error Rate | Updated |
|---|
| ProviderStepFun | P50 TTFT970 ms | P95 TTFT1,084.3 ms | P50 Throughput812.731 char/s | P5 Throughput711.842 char/s | 7d Calls2 | Success Rate100.0% | Error Rate0.0% | Updated |
| ProviderMeta Model API | P50 TTFT1,011 ms | P95 TTFT4,579 ms | P50 Throughput49.463 char/s | P5 Throughput6.307 char/s | 7d Calls341 | Success Rate100.0% | Error Rate0.0% | Updated |
| ProviderxAI | P50 TTFT1,460.5 ms | P95 TTFT6,371.5 ms | P50 Throughput38.327 char/s | P5 Throughput4.929 char/s | 7d Calls439 | Success Rate95.0% | Error Rate5.0% | Updated |
| ProviderMiniMax | P50 TTFT1,579 ms | P95 TTFT1,773.4 ms | P50 Throughput102.475 char/s | P5 Throughput54.491 char/s | 7d Calls4 | Success Rate100.0% | Error Rate0.0% | Updated |
| ProviderDeepSeek | P50 TTFT1,855 ms | P95 TTFT3,184.75 ms | P50 Throughput207.459 char/s | P5 Throughput32.999 char/s | 7d Calls1,945 | Success Rate96.8% | Error Rate3.2% | Updated |
| ProviderGoogle | P50 TTFT1,871 ms | P95 TTFT9,237.4 ms | P50 Throughput293.394 char/s | P5 Throughput12.697 char/s | 7d Calls471 | Success Rate92.4% | Error Rate7.6% | Updated |
| ProviderFireworks | P50 TTFT1,992 ms | P95 TTFT7,786.5 ms | P50 Throughput104.102 char/s | P5 Throughput0.901 char/s | 7d Calls764 | Success Rate80.9% | Error Rate19.1% | Updated |
| ProviderNovita | P50 TTFT2,062 ms | P95 TTFT4,682 ms | P50 Throughput140.871 char/s | P5 Throughput16.646 char/s | 7d Calls5,048 | Success Rate99.7% | Error Rate0.3% | Updated |
| ProviderDeepInfra | P50 TTFT2,104 ms | P95 TTFT17,235.6 ms | P50 Throughput74.304 char/s | P5 Throughput4.471 char/s | 7d Calls695 | Success Rate100.0% | Error Rate0.0% | Updated |
| ProviderOpenAI | P50 TTFT2,638.5 ms | P95 TTFT8,351.9 ms | P50 Throughput96.988 char/s | P5 Throughput15.341 char/s | 7d Calls3,653 | Success Rate97.9% | Error Rate2.1% | Updated |
| ProviderAnthropic | P50 TTFT2,650 ms | P95 TTFT7,485.1 ms | P50 Throughput53.679 char/s | P5 Throughput3.941 char/s | 7d Calls418 | Success Rate94.7% | Error Rate5.3% | Updated |
| ProviderFriendliAI | P50 TTFT2,909 ms | P95 TTFT2,909 ms | P50 Throughput25.184 char/s | P5 Throughput8.296 char/s | 7d Calls13 | Success Rate30.8% | Error Rate69.2% | Updated |
| ProviderTogether | P50 TTFT2,963 ms | P95 TTFT4,208.6 ms | P50 Throughput87.578 char/s | P5 Throughput3.285 char/s | 7d Calls105 | Success Rate98.1% | Error Rate1.9% | Updated |
| ProviderMistral AI | P50 TTFT3,496 ms | P95 TTFT10,062.75 ms | P50 Throughput7.541 char/s | P5 Throughput0.791 char/s | 7d Calls16 | Success Rate100.0% | Error Rate0.0% | Updated |
| ProviderMoonshot AI | P50 TTFT5,639 ms | P95 TTFT6,861 ms | P50 Throughput32.125 char/s | P5 Throughput4.128 char/s | 7d Calls28 | Success Rate100.0% | Error Rate0.0% | Updated |
| ProviderAzure | P50 TTFTN/A | P95 TTFTN/A | P50 ThroughputN/A | P5 ThroughputN/A | 7d Calls0 | Success RateN/A | Error RateN/A | Updated |
| ProviderZAI | P50 TTFTN/A | P95 TTFTN/A | P50 ThroughputN/A | P5 ThroughputN/A | 7d Calls0 | Success RateN/A | Error RateN/A | Updated |
| ProviderSambanova | P50 TTFTN/A | P95 TTFTN/A | P50 ThroughputN/A | P5 ThroughputN/A | 7d Calls0 | Success RateN/A | Error RateN/A | Updated |
| ProviderMeituan | P50 TTFTN/A | P95 TTFTN/A | P50 ThroughputN/A | P5 ThroughputN/A | 7d Calls0 | Success RateN/A | Error RateN/A | Updated |
| ProviderInception | P50 TTFTN/A | P95 TTFTN/A | P50 ThroughputN/A | P5 ThroughputN/A | 7d Calls0 | Success RateN/A | Error RateN/A | Updated |
Median observed throughputMedian TTFT
Each mark is one provider aggregate15 records
The Top AI Models for Speed and Latency
The five fastest records in the current runtime leaderboard, with catalog latency, official token price and independent benchmark context shown where available.
Ranking basisThis speed and latency AI model leaderboard uses descending provider catalog output speed in generated tokens per second. The leaderboard ranking keeps matched price and speed data separate from benchmark evidence.
- 01ME
- Output speed
- 2,220 tok/s
- Speed
- 2,220 tok/s via Cerebras
Strengths
- Ranks #1 by output tokens per second
- 0.65s catalog latency in the same provider record
- 128K maximum input tokens
Considerations
- This speed is provider-specific and does not measure model capability
- Catalog latency is not the same as observed 7-day TTFT
- 02ME
- Output speed
- 2,047 tok/s
- Speed
- 2,047 tok/s via Cerebras
Strengths
- Ranks #2 by output tokens per second
- 0.20s catalog latency in the same provider record
- 131K maximum input tokens
Considerations
- This speed is provider-specific and does not measure model capability
- Catalog latency is not the same as observed 7-day TTFT
- 03MAMin istral 3 (3B Reasoning 2512)Mistral AI
- Output speed
- 1,222 tok/s
- Price
- $0.04 input / $0.04 output per 1M tokens
- Speed
- 1,222 tok/s via Mistral AI
Strengths
- Ranks #3 by output tokens per second
- 0.16s catalog latency in the same provider record
- 131K maximum input tokens
Considerations
- This speed is provider-specific and does not measure model capability
- Catalog latency is not the same as observed 7-day TTFT
- 04ME
- Output speed
- 1,204 tok/s
- Speed
- 1,204 tok/s via Cerebras
Strengths
- Ranks #4 by output tokens per second
- 0.20s catalog latency in the same provider record
- 128K maximum input tokens
Considerations
- This speed is provider-specific and does not measure model capability
- Catalog latency is not the same as observed 7-day TTFT
- 05ME
- Output speed
- 1,096 tok/s
- Speed
- 1,096 tok/s via Sambanova
Strengths
- Ranks #5 by output tokens per second
- 0.65s catalog latency in the same provider record
- 128K maximum input tokens
Considerations
- This speed is provider-specific and does not measure model capability
- Catalog latency is not the same as observed 7-day TTFT
Selection summary
Best AI Models for Speed and Latency
Llama 3.3 70B Instruct has the fastest current catalog record at 2,220 tok/s. The best AI model for runtime-sensitive work also depends on latency, provider reliability, price and workload-specific quality.
Use this leaderboard with the supporting benchmark results and coverage details above. A leaderboard position summarizes the selected ranking signal; it does not replace workload-specific testing.
How to read the runtime leaderboard
Runtime leaderboard metrics answer deployment questions and should not be read as model capability scores or a quality benchmark ranking.
Output SpeedGenerated output tokens received per second. Higher is faster.
Catalog LatencyProvider catalog latency in seconds. Lower is faster; it is not the 7-day TTFT measure.
TTFTTime to first token from observed provider calls. P50 is typical; P95 shows slower-tail behavior.
Observed ThroughputCharacters per second across a 7-day provider window. It remains separate from output tokens per second.
ReliabilityCalls, success rate, and error rate describe the observed provider window, not model capability.
Independent SignalRuntime does not change the LLMBoard capability leaderboard, ranking bands, or benchmark evidence coverage.
Runtime source: LLM Stats / ZeroEval snapshot. Speed follows the public output-tokens-per-second convention. Read the source definition.