Back to Benchmarks

llama-3.1-8b-chat

text-generation
Uploaded: 24.07.2024
⏱️ Benchmarks
✨ Query this model
Region:
Hong KongBelgiumIowa
Seq Length:
ShortLong
Providers
provider icon
azure-ai
provider icon
aws-bedrock
provider icon
deepinfra
provider icon
fireworks-ai
provider icon
lepton-ai
provider icon
octoai
provider icon
perplexity-ai
provider icon
together-ai
provider icon
groq
provider icon
vertex-ai
Learn more about how we are collecting this data here
Input Cost
Output Cost
Output Tks / SecP90_{P90}
TTFTP90_{P90}
ITLP90_{P90}
E2E LatencyP90_{P90}
Cold Start
0.3 $/1M tks
0.61 $/1M tks
69 tks/sec
468.08 ms
14.49 ms
10541.16 ms
0 sec
0.22 $/1M tks
0.22 $/1M tks
114.16 tks/sec
580.76 ms
8.76 ms
738.44 ms
0 sec
0.06 $/1M tks
0.06 $/1M tks
69.02 tks/sec
456.69 ms
14.49 ms
15900.73 ms
0 sec
0.2 $/1M tks
0.2 $/1M tks
273.05 tks/sec
489.19 ms
3.66 ms
4554.41 ms
0 sec
0.07 $/1M tks
0.07 $/1M tks
152.53 tks/sec
758.42 ms
6.56 ms
7498.21 ms
0 sec
0.15 $/1M tks
0.15 $/1M tks
172.48 tks/sec
562.47 ms
5.8 ms
6099.3 ms
0 sec
0.2 $/1M tks
0.2 $/1M tks
225.16 tks/sec
566.81 ms
4.44 ms
957.64 ms
0 sec
0.18 $/1M tks
0.18 $/1M tks
77.04 tks/sec
1078.8 ms
12.98 ms
3623.09 ms
0 sec
0.05 $/1M tks
0.08 $/1M tks
254.21 tks/sec
742.03 ms
3.93 ms
4514.51 ms
0 sec
0.22 $/1M tks
0.22 $/1M tks
85.44 tks/sec
1302.27 ms
11.7 ms
12689.85 ms
0 sec