⚠ 示意資料 — 第一版所有數字為合成測試值，正式可重現的量測即將上線。

GPU 基準排名

測試套件

晶片商

精度

框架

顯示 36 筆，共 36 筆 · 排序依據: 主力指標 ↓

#	晶片	GPU	模型	框架	精度	主力指標	NT$/hr	等級	日期
1	NVIDIA GB200 NVL72 ×72 · vLLM	×72	Llama-3.1-70B-Instruct	vLLM	FP8	412,000 tokens/sec	NT$ 7,800,000	初步	2026-09-14
2	NVIDIA GB200 NVL72 ×36 · TensorRT-LLM	×36	Llama-3.1-70B-Instruct	TensorRT-LLM	FP8	268,000 tokens/sec	NT$ 3,900,000	初步	2026-09-19
3	NVIDIA B200 ×8 · TensorRT-LLM	×8	Llama-3.1-70B-Instruct	TensorRT-LLM	FP8	38,400 tokens/sec	NT$ 520,000	初步	2026-09-15
4	NVIDIA H100 80GB ×4 · TensorRT-LLM	×4	Llama-3.1-8B-Instruct	TensorRT-LLM	FP16	24,600 tokens/sec	NT$ 140,000	已驗證	2026-08-21
5	NVIDIA B200 ×4 · vLLM	×4	Llama-3.1-8B-Instruct	vLLM	FP8	22,800 tokens/sec	NT$ 260,000	初步	2026-09-10
6	NVIDIA H200 141GB ×8 · SGLang	×8	Llama-3.1-70B-Instruct	SGLang	FP8	21,200 tokens/sec	NT$ 340,000	已驗證	2026-09-02
7	NVIDIA H100 80GB ×8 · vLLM	×8	Llama-3.1-70B-Instruct	vLLM	FP8	18,420 tokens/sec	NT$ 280,000	已驗證	2026-08-14
8	AMD MI300X 192GB ×8 · vLLM	×8	Llama-3.1-70B-Instruct	vLLM	FP16	13,800 tokens/sec	NT$ 260,000	初步	2026-09-05
9	NVIDIA H200 141GB ×4 · vLLM	×4	Llama-3.1-70B-Instruct	vLLM	FP16	11,800 tokens/sec	NT$ 170,000	已驗證	2026-08-25
10	NVIDIA RTX 5090 32GB ×1 · TensorRT-LLM	×1	Llama-3.1-8B-Instruct	TensorRT-LLM	FP8	11,200 tokens/sec	NT$ 480	已驗證	2026-08-02
11	NVIDIA H200 141GB ×1 · vLLM	×1	Llama-3.1-8B-Instruct	vLLM	BF16	10,400 tokens/sec	NT$ 42,000	已驗證	2026-08-08
12	NVIDIA H100 80GB ×1 · vLLM	×1	Llama-3.1-8B-Instruct	vLLM	FP8	9,800 tokens/sec	NT$ 35,000	已驗證	2026-07-30
13	NVIDIA H100 PCIe 80GB ×4 · vLLM	×4	Llama-3.1-70B-Instruct	vLLM	FP8	8,800 tokens/sec	NT$ 130,000	已驗證	2026-08-25
14	NVIDIA A100 80GB ×8 · vLLM	×8	Llama-3.1-70B-Instruct	vLLM	FP16	8,400 tokens/sec	NT$ 180,000	社群	2026-07-12
15	NVIDIA H100 PCIe 80GB ×1 · TensorRT-LLM	×1	Llama-3.1-8B-Instruct	TensorRT-LLM	FP16	8,400 tokens/sec	NT$ 32,000	已驗證	2026-09-04
16	NVIDIA RTX 5090 32GB ×1 · vLLM	×1	Llama-3.1-8B-Instruct	vLLM	FP16	8,200 tokens/sec	NT$ 480	已驗證	2026-08-08
17	NVIDIA RTX 6000 Ada 48GB ×1 · TensorRT-LLM	×1	Llama-3.1-8B-Instruct	TensorRT-LLM	FP8	7,400 tokens/sec	NT$ 9,000	已驗證	2026-07-14
18	NVIDIA L40S 48GB ×1 · vLLM	×1	Llama-3.1-8B-Instruct	vLLM	INT8	6,900 tokens/sec	NT$ 12,000	社群	2026-08-04
19	AMD MI250X 128GB ×8 · vLLM	×8	Llama-3.1-70B-Instruct	vLLM	FP16	6,800 tokens/sec	NT$ 160,000	社群	2026-06-04
20	NVIDIA RTX 4090 24GB ×1 · TensorRT-LLM	×1	Llama-3.1-8B-Instruct	TensorRT-LLM	FP8	6,400 tokens/sec	NT$ 380	已驗證	2026-05-28
21	AMD MI250X 128GB ×4 · vLLM	×4	Llama-3.1-8B-Instruct	vLLM	FP16	5,800 tokens/sec	NT$ 80,000	社群	2026-06-10
22	NVIDIA RTX 6000 Ada 48GB ×1 · vLLM	×1	Llama-3.1-8B-Instruct	vLLM	FP16	5,600 tokens/sec	NT$ 9,000	已驗證	2026-07-08
23	NVIDIA L40S 48GB ×1 · TGI	×1	Llama-3.1-8B-Instruct	TGI	FP16	5,200 tokens/sec	NT$ 12,000	社群	2026-07-22
24	NVIDIA GB200 NVL72 ×72 · vLLM	×72	Qwen2.5-7B	vLLM	FP8	4,800 samples/hour	NT$ 7,800,000	初步	2026-09-18
25	NVIDIA L4 24GB ×1 · TensorRT-LLM	×1	Llama-3.1-8B-Instruct	TensorRT-LLM	INT8	4,800 tokens/sec	NT$ 7,500	已驗證	2026-07-25
26	NVIDIA RTX 4090 24GB ×1 · vLLM	×1	Llama-3.1-8B-Instruct	vLLM	FP16	4,600 tokens/sec	NT$ 380	已驗證	2026-06-02
27	AMD RX 7900 XTX 24GB ×1 · vLLM	×1	Llama-3.1-8B-Instruct	vLLM	INT8	4,400 tokens/sec	NT$ 260	社群	2026-05-10
28	NVIDIA A100 PCIe 40GB ×4 · vLLM	×4	Llama-3.1-70B-Instruct	vLLM	FP16	4,200 tokens/sec	NT$ 72,000	已驗證	2026-06-12
29	NVIDIA A100 PCIe 40GB ×1 · TensorRT-LLM	×1	Llama-3.1-8B-Instruct	TensorRT-LLM	FP16	4,200 tokens/sec	NT$ 18,000	已驗證	2026-06-18
30	NVIDIA RTX 4080 SUPER 16GB ×1 · TensorRT-LLM	×1	Llama-3.1-8B-Instruct	TensorRT-LLM	FP8	3,800 tokens/sec	NT$ 280	已驗證	2026-05-12
31	NVIDIA RTX 5090 32GB ×1 · vLLM	×1	Llama-3.1-70B-Instruct	vLLM	FP8	3,400 tokens/sec	NT$ 480	已驗證	2026-08-12
32	AMD RX 7900 XTX 24GB ×1 · vLLM	×1	Llama-3.1-8B-Instruct	vLLM	FP16	3,400 tokens/sec	NT$ 260	社群	2026-05-04
33	NVIDIA L4 24GB ×1 · vLLM	×1	Llama-3.1-8B-Instruct	vLLM	FP16	3,200 tokens/sec	NT$ 7,500	已驗證	2026-07-18
34	NVIDIA RTX 3090 24GB ×1 · vLLM	×1	Llama-3.1-8B-Instruct	vLLM	INT8	3,100 tokens/sec	NT$ 220	已驗證	2026-04-14
35	NVIDIA RTX 4080 SUPER 16GB ×1 · vLLM	×1	Llama-3.1-8B-Instruct	vLLM	FP16	2,800 tokens/sec	NT$ 280	已驗證	2026-05-18
36	NVIDIA RTX 3090 24GB ×1 · TensorRT-LLM	×1	Llama-3.1-8B-Instruct	TensorRT-LLM	FP16	2,200 tokens/sec	NT$ 220	已驗證	2026-04-08
37	NVIDIA GB200 NVL72 ×36 · TensorRT-LLM	×36	Qwen2.5-7B	TensorRT-LLM	BF16	2,120 samples/hour	NT$ 3,900,000	初步	2026-09-20
38	NVIDIA RTX 4090 24GB ×1 · vLLM	×1	Llama-3.1-70B-Instruct	vLLM	FP8	1,800 tokens/sec	NT$ 380	已驗證	2026-06-08
39	NVIDIA GB200 NVL72 ×72 · vLLM	×72	Llama-3.1-8B-Instruct	vLLM	FP8	1,480 queries/sec	NT$ 7,800,000	初步	2026-09-15
40	NVIDIA GB200 NVL72 ×36 · SGLang	×36	Llama-3.1-8B-Instruct	SGLang	FP8	920 queries/sec	NT$ 3,900,000	初步	2026-09-21
41	NVIDIA B200 ×8 · TensorRT-LLM	×8	Qwen2.5-7B	TensorRT-LLM	FP8	740 samples/hour	NT$ 520,000	初步	2026-09-12
42	NVIDIA H200 141GB ×8 · vLLM	×8	Qwen2.5-7B	vLLM	FP16	468 samples/hour	NT$ 340,000	已驗證	2026-09-04
43	NVIDIA H100 80GB ×8 · vLLM	×8	Qwen2.5-7B	vLLM	FP16	412 samples/hour	NT$ 280,000	已驗證	2026-08-30
44	NVIDIA B200 ×4 · vLLM	×4	Qwen2.5-7B	vLLM	FP8	396 samples/hour	NT$ 260,000	初步	2026-09-09
45	AMD MI300X 192GB ×8 · vLLM	×8	Qwen2.5-7B	vLLM	FP16	312 samples/hour	NT$ 260,000	初步	2026-09-06
46	NVIDIA B200 ×8 · TensorRT-LLM	×8	Llama-3.1-8B-Instruct	TensorRT-LLM	FP8	248 queries/sec	NT$ 520,000	初步	2026-09-14
47	NVIDIA H200 141GB ×4 · SGLang	×4	Qwen2.5-7B	SGLang	BF16	246 samples/hour	NT$ 170,000	已驗證	2026-09-08
48	NVIDIA H100 80GB ×4 · vLLM	×4	Qwen2.5-7B	vLLM	BF16	218 samples/hour	NT$ 140,000	已驗證	2026-09-01
49	NVIDIA A100 80GB ×8 · vLLM	×8	Qwen2.5-7B	vLLM	FP16	184 samples/hour	NT$ 180,000	社群	2026-07-15
50	NVIDIA H200 141GB ×8 · vLLM	×8	Llama-3.1-8B-Instruct	vLLM	FP16	158 queries/sec	NT$ 340,000	已驗證	2026-09-03
51	NVIDIA H100 80GB ×8 · vLLM	×8	Llama-3.1-8B-Instruct	vLLM	FP16	142 queries/sec	NT$ 280,000	已驗證	2026-08-12
52	NVIDIA B200 ×4 · vLLM	×4	Llama-3.1-8B-Instruct	vLLM	FP8	132 queries/sec	NT$ 260,000	初步	2026-09-11
53	NVIDIA H100 PCIe 80GB ×4 · vLLM	×4	Qwen2.5-7B	vLLM	BF16	132 samples/hour	NT$ 130,000	已驗證	2026-09-08
54	NVIDIA H200 141GB ×1 · vLLM	×1	Qwen2.5-7B	vLLM	FP16	108 samples/hour	NT$ 42,000	已驗證	2026-08-26
55	NVIDIA H100 80GB ×1 · vLLM	×1	Qwen2.5-7B	vLLM	FP8	96 samples/hour	NT$ 35,000	已驗證	2026-08-18
56	AMD MI300X 192GB ×8 · vLLM	×8	Llama-3.1-8B-Instruct	vLLM	FP16	96 queries/sec	NT$ 260,000	初步	2026-09-06
57	NVIDIA H200 141GB ×4 · SGLang	×4	Llama-3.1-8B-Instruct	SGLang	BF16	84 queries/sec	NT$ 170,000	已驗證	2026-09-07
58	NVIDIA H100 80GB ×4 · SGLang	×4	Llama-3.1-8B-Instruct	SGLang	FP16	78 queries/sec	NT$ 140,000	已驗證	2026-08-22
59	NVIDIA H100 PCIe 80GB ×4 · vLLM	×4	Llama-3.1-8B-Instruct	vLLM	FP16	78 queries/sec	NT$ 130,000	已驗證	2026-08-29
60	NVIDIA A100 80GB ×8 · vLLM	×8	Llama-3.1-8B-Instruct	vLLM	FP16	64 queries/sec	NT$ 180,000	社群	2026-07-14
61	NVIDIA H100 PCIe 80GB ×2 · TensorRT-LLM	×2	Llama-3.1-8B-Instruct	TensorRT-LLM	FP8	62 queries/sec	NT$ 65,000	社群	2026-09-12
62	AMD MI250X 128GB ×4 · vLLM	×4	Qwen2.5-7B	vLLM	BF16	56 samples/hour	NT$ 80,000	社群	2026-06-14
63	AMD MI250X 128GB ×4 · vLLM	×4	Llama-3.1-8B-Instruct	vLLM	FP16	48 queries/sec	NT$ 80,000	社群	2026-06-26
64	NVIDIA H100 PCIe 80GB ×1 · SGLang	×1	Qwen2.5-7B	SGLang	FP8	46 samples/hour	NT$ 32,000	社群	2026-09-10
65	NVIDIA RTX 5090 32GB ×1 · TensorRT-LLM	×1	Qwen2.5-7B	TensorRT-LLM	FP8	42 samples/hour	NT$ 480	社群	2026-08-18
66	NVIDIA RTX 5090 32GB ×1 · vLLM	×1	Llama-3.1-8B-Instruct	vLLM	FP16	42 queries/sec	NT$ 480	已驗證	2026-08-22
67	NVIDIA L40S 48GB ×1 · TGI	×1	Qwen2.5-7B	TGI	FP16	38 samples/hour	NT$ 12,000	社群	2026-07-30
68	NVIDIA A100 PCIe 40GB ×4 · vLLM	×4	Llama-3.1-8B-Instruct	vLLM	FP16	38 queries/sec	NT$ 72,000	已驗證	2026-07-04
69	NVIDIA H200 141GB ×1 · vLLM	×1	Llama-3.1-8B-Instruct	vLLM	FP16	35 queries/sec	NT$ 42,000	已驗證	2026-08-27
70	NVIDIA RTX 6000 Ada 48GB ×1 · TensorRT-LLM	×1	Llama-3.1-8B-Instruct	TensorRT-LLM	FP8	34 queries/sec	NT$ 9,000	已驗證	2026-07-18
71	NVIDIA H100 80GB ×1 · vLLM	×1	Llama-3.1-8B-Instruct	vLLM	FP8	32 queries/sec	NT$ 35,000	已驗證	2026-08-02
72	AMD MI250X 128GB ×2 · vLLM	×2	Llama-3.1-8B-Instruct	vLLM	FP16	32 queries/sec	NT$ 40,000	社群	2026-07-02
73	NVIDIA A100 PCIe 40GB ×2 · vLLM	×2	Qwen2.5-7B	vLLM	BF16	28 samples/hour	NT$ 36,000	社群	2026-06-22
74	NVIDIA RTX 5090 32GB ×1 · vLLM	×1	Qwen2.5-7B	vLLM	BF16	28 samples/hour	NT$ 480	社群	2026-08-15
75	NVIDIA RTX 4090 24GB ×1 · SGLang	×1	Qwen2.5-7B	SGLang	FP8	26 samples/hour	NT$ 380	社群	2026-06-18
76	NVIDIA RTX 4090 24GB ×1 · vLLM	×1	Llama-3.1-8B-Instruct	vLLM	FP16	26 queries/sec	NT$ 380	已驗證	2026-06-24
77	NVIDIA RTX 6000 Ada 48GB ×1 · vLLM	×1	Llama-3.1-8B-Instruct	vLLM	FP16	24 queries/sec	NT$ 9,000	已驗證	2026-07-12
78	NVIDIA L4 24GB ×1 · TensorRT-LLM	×1	Llama-3.1-8B-Instruct	TensorRT-LLM	INT8	22 queries/sec	NT$ 7,500	已驗證	2026-07-30
79	NVIDIA A100 PCIe 40GB ×1 · TensorRT-LLM	×1	Llama-3.1-8B-Instruct	TensorRT-LLM	FP16	22 queries/sec	NT$ 18,000	已驗證	2026-07-10
80	NVIDIA RTX 6000 Ada 48GB ×1 · SGLang	×1	Qwen2.5-7B	SGLang	FP8	22 samples/hour	NT$ 9,000	社群	2026-07-26
81	AMD MI250X 128GB ×2 · SGLang	×2	Qwen2.5-7B	SGLang	FP16	22 samples/hour	NT$ 40,000	社群	2026-06-20
82	NVIDIA L40S 48GB ×1 · TGI	×1	Llama-3.1-8B-Instruct	TGI	FP16	18 queries/sec	NT$ 12,000	社群	2026-07-24
83	NVIDIA RTX 4090 24GB ×1 · vLLM	×1	Qwen2.5-7B	vLLM	BF16	18 samples/hour	NT$ 380	社群	2026-06-12
84	AMD RX 7900 XTX 24GB ×1 · vLLM	×1	Llama-3.1-8B-Instruct	vLLM	FP16	18 queries/sec	NT$ 260	社群	2026-05-22
85	NVIDIA RTX 6000 Ada 48GB ×1 · vLLM	×1	Qwen2.5-7B	vLLM	BF16	16 samples/hour	NT$ 9,000	社群	2026-07-20
86	NVIDIA RTX 4080 SUPER 16GB ×1 · TensorRT-LLM	×1	Llama-3.1-8B-Instruct	TensorRT-LLM	FP8	16 queries/sec	NT$ 280	已驗證	2026-05-28
87	NVIDIA L4 24GB ×1 · SGLang	×1	Qwen2.5-7B	SGLang	FP8	14.2 samples/hour	NT$ 7,500	社群	2026-08-08
88	NVIDIA L4 24GB ×1 · vLLM	×1	Llama-3.1-8B-Instruct	vLLM	FP16	14 queries/sec	NT$ 7,500	已驗證	2026-07-22
89	NVIDIA A100 PCIe 40GB ×1 · SGLang	×1	Qwen2.5-7B	SGLang	FP16	13 samples/hour	NT$ 18,000	社群	2026-06-28
90	AMD RX 7900 XTX 24GB ×1 · vLLM	×1	Qwen2.5-7B	vLLM	BF16	13 samples/hour	NT$ 260	社群	2026-05-16
91	NVIDIA RTX 4080 SUPER 16GB ×1 · vLLM	×1	Qwen2.5-7B	vLLM	BF16	12 samples/hour	NT$ 280	社群	2026-05-22
92	NVIDIA RTX 3090 24GB ×1 · TensorRT-LLM	×1	Llama-3.1-8B-Instruct	TensorRT-LLM	FP16	12 queries/sec	NT$ 220	已驗證	2026-04-26
93	NVIDIA L4 24GB ×1 · vLLM	×1	Qwen2.5-7B	vLLM	BF16	9.4 samples/hour	NT$ 7,500	社群	2026-08-02
94	NVIDIA RTX 3090 24GB ×1 · vLLM	×1	Qwen2.5-7B	vLLM	BF16	8.5 samples/hour	NT$ 220	社群	2026-04-20