Welche LLMs laufen auf der AMD Instinct MI300X (192GB)?
Hardware-Kompatibilitätsguide für AMD Instinct MI300X: 192GB VRAM bei 5300 GB/s Speicherbandbreite.
AMD Instinct MI300X Spezifikationen
VRAM / GPU
192 GB
Bandbreite
5300 GB/s
FP16-Leistung
1300 TFLOPS
Architektur
CDNA 3
Erscheinungsjahr
2023
Verfügbarer VRAM
192 GB
Gesamtbandbreite
5,300 GB/s
Passt in FP16 / FP8
54
Passt mit INT4
71 / 85
DeepSeek-V4-FlashMoE 13B
284B • DeepSeek V4
FP16
569.37 GB
FP8
285.34 GB
INT4
143.32 GB
~693 tok/s (INT4)VRAM-Details öffnen
DeepSeek-R1-Distill-Qwen-32B
32B • DeepSeek
FP16
68.27 GB
FP8
36 GB
INT4
19.86 GB
~61 tok/s (FP16)VRAM-Details öffnen
DeepSeek-R1-Distill-Qwen-14B
14B • DeepSeek
FP16
32.27 GB
FP8
18 GB
INT4
10.86 GB
~61 tok/s (FP16)VRAM-Details öffnen
DeepSeek-R1-Distill-Qwen-7B
7B • DeepSeek
FP16
18.27 GB
FP8
11 GB
INT4
7.36 GB
~61 tok/s (FP16)VRAM-Details öffnen
DeepSeek-R1-Distill-Qwen-1.5B
1.5B • DeepSeek
FP16
7.27 GB
FP8
5.5 GB
INT4
4.61 GB
~61 tok/s (FP16)VRAM-Details öffnen
DeepSeek-R1-Distill-Llama-70B
70B • DeepSeek
FP16
144.27 GB
FP8
74 GB
INT4
38.86 GB
~61 tok/s (FP16)VRAM-Details öffnen
DeepSeek-R1-Distill-Llama-8B
8B • DeepSeek
FP16
20.27 GB
FP8
12 GB
INT4
7.86 GB
~61 tok/s (FP16)VRAM-Details öffnen
Llama 4 ScoutMoE 17B
109B • Llama 4
FP16
220.71 GB
FP8
111.21 GB
INT4
56.46 GB
~265 tok/s (FP8)VRAM-Details öffnen
Llama 3 8B
8B • Llama 3
FP16
17.81 GB
FP8
9.31 GB
INT4
5.06 GB
~282 tok/s (FP16)VRAM-Details öffnen
Llama 3 70B
70B • Llama 3
FP16
149.56 GB
FP8
78.31 GB
INT4
42.68 GB
~32 tok/s (FP16)VRAM-Details öffnen
Llama 3.1 8B
8B • Llama 3
FP16
17.81 GB
FP8
9.31 GB
INT4
5.06 GB
~282 tok/s (FP16)VRAM-Details öffnen
Llama 3.1 70B
70B • Llama 3
FP16
149.56 GB
FP8
78.31 GB
INT4
42.68 GB
~32 tok/s (FP16)VRAM-Details öffnen
Llama 3.2 1B
1B • Llama 3
FP16
2.55 GB
FP8
1.43 GB
INT4
0.86 GB
~2253 tok/s (FP16)VRAM-Details öffnen
Llama 3.2 3B
3B • Llama 3
FP16
7.23 GB
FP8
3.79 GB
INT4
2.07 GB
~751 tok/s (FP16)VRAM-Details öffnen
Llama 3.2 11B
11B • Llama 3
FP16
24.11 GB
FP8
12.61 GB
INT4
6.86 GB
~205 tok/s (FP16)VRAM-Details öffnen
Llama 3.2 90B
90B • Llama 3
FP16
191.57 GB
FP8
100.32 GB
INT4
54.7 GB
~25 tok/s (FP16)VRAM-Details öffnen
Qwen3.5-122B-A10BMoE 10B
122B • Qwen 3.5
FP16
246.01 GB
FP8
123.51 GB
INT4
62.26 GB
~451 tok/s (FP8)VRAM-Details öffnen
Qwen3.5-35B-A3BMoE 3B
35B • Qwen 3.5
FP16
70.98 GB
FP8
35.66 GB
INT4
18.01 GB
~751 tok/s (FP16)VRAM-Details öffnen
Qwen3.5-27B
27B • Qwen 3.5
FP16
59.22 GB
FP8
30.97 GB
INT4
16.85 GB
~83 tok/s (FP16)VRAM-Details öffnen
Qwen3-Next-80B-A3B-InstructMoE 3B
81B • Qwen 3-Next
FP16
163.48 GB
FP8
81.91 GB
INT4
41.13 GB
~751 tok/s (FP16)VRAM-Details öffnen
Qwen3-Next-80B-A3B-ThinkingMoE 3B
81B • Qwen 3-Next
FP16
163.48 GB
FP8
81.91 GB
INT4
41.13 GB
~751 tok/s (FP16)VRAM-Details öffnen
Qwen3-235B-A22BMoE 22B
235B • Qwen 3
FP16
473.69 GB
FP8
237.95 GB
INT4
120.08 GB
~410 tok/s (INT4)VRAM-Details öffnen
Qwen2.5-Math-72B
72B • Qwen 2
FP16
153.76 GB
FP8
80.51 GB
INT4
43.88 GB
~31 tok/s (FP16)VRAM-Details öffnen
Qwen2.5-72B
72B • Qwen 2
FP16
153.76 GB
FP8
80.51 GB
INT4
43.88 GB
~31 tok/s (FP16)VRAM-Details öffnen
Qwen2-72B
72B • Qwen 2
FP16
153.76 GB
FP8
80.51 GB
INT4
43.88 GB
~31 tok/s (FP16)VRAM-Details öffnen
Qwen QwQ-32B
32B • Qwen QwQ
FP16
69.1 GB
FP8
36.16 GB
INT4
19.69 GB
~70 tok/s (FP16)VRAM-Details öffnen
Qwen2.5-Coder-32B
32B • Qwen 2
FP16
69.1 GB
FP8
36.16 GB
INT4
19.69 GB
~70 tok/s (FP16)VRAM-Details öffnen
Qwen3-32B
32B • Qwen 3
FP16
69.23 GB
FP8
36.23 GB
INT4
19.73 GB
~70 tok/s (FP16)VRAM-Details öffnen
Qwen2.5-32B
32B • Qwen 2
FP16
69.1 GB
FP8
36.16 GB
INT4
19.69 GB
~70 tok/s (FP16)VRAM-Details öffnen
Qwen2-32B
32B • Qwen 2
FP16
69.1 GB
FP8
36.16 GB
INT4
19.69 GB
~70 tok/s (FP16)VRAM-Details öffnen
Qwen3-30B-A3BMoE 3B
30B • Qwen 3
FP16
61.1 GB
FP8
30.73 GB
INT4
15.54 GB
~751 tok/s (FP16)VRAM-Details öffnen
Qwen2.5-14B
14B • Qwen 2
FP16
30.66 GB
FP8
16.04 GB
INT4
8.72 GB
~161 tok/s (FP16)VRAM-Details öffnen
Qwen3-14B
14B • Qwen 3
FP16
30.66 GB
FP8
16.04 GB
INT4
8.72 GB
~161 tok/s (FP16)VRAM-Details öffnen
Qwen2-14B
14B • Qwen 2
FP16
30.66 GB
FP8
16.04 GB
INT4
8.72 GB
~161 tok/s (FP16)VRAM-Details öffnen
Qwen2.5-7B
7B • Qwen 2
FP16
15.21 GB
FP8
7.96 GB
INT4
4.33 GB
~322 tok/s (FP16)VRAM-Details öffnen
Qwen2.5-3B
3B • Qwen 2
FP16
6.57 GB
FP8
3.46 GB
INT4
1.91 GB
~751 tok/s (FP16)VRAM-Details öffnen
Qwen2.5-1.5B
1.5B • Qwen 2
FP16
3.49 GB
FP8
1.89 GB
INT4
1.1 GB
~1502 tok/s (FP16)VRAM-Details öffnen
Qwen2.5-0.5B
0.5B • Qwen 2
FP16
1.36 GB
FP8
0.83 GB
INT4
0.57 GB
~4505 tok/s (FP16)VRAM-Details öffnen
Qwen3-0.6B
0.6B • Qwen 3
FP16
1.94 GB
FP8
1.12 GB
INT4
0.71 GB
~3754 tok/s (FP16)VRAM-Details öffnen
Qwen3-1.7B
1.7B • Qwen 3
FP16
4.14 GB
FP8
2.22 GB
INT4
1.26 GB
~1325 tok/s (FP16)VRAM-Details öffnen
Mistral-Large-Instruct-2407
123B • Mistral
FP16
261.15 GB
FP8
136.77 GB
INT4
74.59 GB
~37 tok/s (FP8)VRAM-Details öffnen
Mistral-Nemo-12B
12B • Mistral
FP16
26.46 GB
FP8
13.83 GB
INT4
7.52 GB
~188 tok/s (FP16)VRAM-Details öffnen
Mistral-7B-v0.3
7B • Mistral
FP16
15.71 GB
FP8
8.21 GB
INT4
4.46 GB
~322 tok/s (FP16)VRAM-Details öffnen
Mixtral-8x7BMoE 12.9B
46.7B • Mixtral
FP16
95.7 GB
FP8
48.5 GB
INT4
24.9 GB
~175 tok/s (FP16)VRAM-Details öffnen
Mixtral-8x22BMoE 39B
141B • Mixtral
FP16
287.68 GB
FP8
145.81 GB
INT4
74.87 GB
~116 tok/s (FP8)VRAM-Details öffnen
Gemma-2-27B
27B • Gemma
FP16
59.6 GB
FP8
31.16 GB
INT4
16.94 GB
~83 tok/s (FP16)VRAM-Details öffnen
Gemma-2-9B
9B • Gemma
FP16
21.53 GB
FP8
11.22 GB
INT4
6.06 GB
~250 tok/s (FP16)VRAM-Details öffnen
Phi-3.5-MoEMoE 6.6B
42B • Phi
FP16
85.67 GB
FP8
43.17 GB
INT4
21.92 GB
~341 tok/s (FP16)VRAM-Details öffnen
Phi-3-3.8B
3.8B • Phi
FP16
11.02 GB
FP8
5.72 GB
INT4
3.07 GB
~593 tok/s (FP16)VRAM-Details öffnen
CodeLlama-34B
34B • CodeLlama
FP16
72.93 GB
FP8
38.18 GB
INT4
20.8 GB
~66 tok/s (FP16)VRAM-Details öffnen
CodeLlama-13B
13B • CodeLlama
FP16
33.56 GB
FP8
17.44 GB
INT4
9.37 GB
~173 tok/s (FP16)VRAM-Details öffnen
CodeLlama-7B
7B • CodeLlama
FP16
18.71 GB
FP8
9.71 GB
INT4
5.21 GB
~322 tok/s (FP16)VRAM-Details öffnen
GLM-4.5MoE 32B
355B • GLM-4
FP16
715.73 GB
FP8
359.48 GB
INT4
181.35 GB
~282 tok/s (INT4)VRAM-Details öffnen
GLM-4.5-AirMoE 12B
106B • GLM-4
FP16
217.73 GB
FP8
110.48 GB
INT4
56.85 GB
~141 tok/s (FP8)VRAM-Details öffnen
GLM-4.5-BaseMoE 32B
355B • GLM-4
FP16
715.73 GB
FP8
359.48 GB
INT4
181.35 GB
~282 tok/s (INT4)VRAM-Details öffnen
GLM-4.5-Air-BaseMoE 12B
106B • GLM-4
FP16
217.73 GB
FP8
110.48 GB
INT4
56.85 GB
~141 tok/s (FP8)VRAM-Details öffnen
GLM-4.7MoE 32B
355B • GLM-4
FP16
716.1 GB
FP8
359.66 GB
INT4
181.44 GB
~282 tok/s (INT4)VRAM-Details öffnen
GLM-4.7-FlashMoE 3B
30B • GLM-4
FP16
66.1 GB
FP8
34.66 GB
INT4
18.94 GB
~70 tok/s (FP16)VRAM-Details öffnen
MiniMax-M2MoE 10B
230B • MiniMax M2
FP16
462.95 GB
FP8
231.98 GB
INT4
116.49 GB
~901 tok/s (INT4)VRAM-Details öffnen
MiniMax-M2.5MoE 10B
230B • MiniMax M2
FP16
462.95 GB
FP8
231.98 GB
INT4
116.49 GB
~901 tok/s (INT4)VRAM-Details öffnen
MiniMax-M2.7MoE 10B
230B • MiniMax M2
FP16
462.95 GB
FP8
231.98 GB
INT4
116.49 GB
~901 tok/s (INT4)VRAM-Details öffnen
MiMo-V2.5MoE 15B
310B • MiMo
FP16
625.87 GB
FP8
315.05 GB
INT4
159.64 GB
~215 tok/s (INT4)VRAM-Details öffnen
MiMo-V2-FlashMoE 15B
309B • MiMo
FP16
620.64 GB
FP8
311.07 GB
INT4
156.29 GB
~601 tok/s (INT4)VRAM-Details öffnen
GPT-OSS-120BMoE 5.1B
117B • GPT-OSS
FP16
235.65 GB
FP8
118.09 GB
INT4
59.31 GB
~883 tok/s (FP8)VRAM-Details öffnen
GPT-OSS-20BMoE 3.6B
21B • GPT-OSS
FP16
43.15 GB
FP8
21.78 GB
INT4
11.09 GB
~626 tok/s (FP16)VRAM-Details öffnen
AMD Instinct MI300X 1x / 2x / 4x LLM-Kompatibilitätsmatrix
Berechnet bei 8K Kontextlänge und Batchgröße 1.
| LLM-Modell | FP8-VRAM-Bedarf | 1× (192GB) | 2× (384GB) | 4× (768GB) |
|---|---|---|---|---|
| DeepSeek-R1(671B) | 675 GB | Needs 2× | INT4 | FP8 / INT4 |
| DeepSeek-R1-Distill-Qwen-32B(32B) | 36 GB | FP16 / FP8 / INT4 | FP16 / FP8 / INT4 | FP16 / FP8 / INT4 |
| Llama 4 Scout(109B) | 111.21 GB | FP8 / INT4 | FP16 / FP8 / INT4 | FP16 / FP8 / INT4 |
| Llama 3.1 8B(8B) | 9.31 GB | FP16 / FP8 / INT4 | FP16 / FP8 / INT4 | FP16 / FP8 / INT4 |
| Llama 3.1 70B(70B) | 78.31 GB | FP16 / FP8 / INT4 | FP16 / FP8 / INT4 | FP16 / FP8 / INT4 |
| Qwen3-235B-A22B(235B) | 237.95 GB | INT4 | FP8 / INT4 | FP16 / FP8 / INT4 |
| Qwen3-32B(32B) | 36.23 GB | FP16 / FP8 / INT4 | FP16 / FP8 / INT4 | FP16 / FP8 / INT4 |
| Qwen3-14B(14B) | 16.04 GB | FP16 / FP8 / INT4 | FP16 / FP8 / INT4 | FP16 / FP8 / INT4 |
| Mixtral-8x7B(46.7B) | 48.5 GB | FP16 / FP8 / INT4 | FP16 / FP8 / INT4 | FP16 / FP8 / INT4 |
| GPT-OSS-20B(21B) | 21.78 GB | FP16 / FP8 / INT4 | FP16 / FP8 / INT4 | FP16 / FP8 / INT4 |
Spezielle Kompatibilitätsguides für alle 22 GPUs
NVIDIA RTX 5090 (32GB)NVIDIA RTX 5080 (16GB)NVIDIA RTX 5070 Ti (16GB)NVIDIA RTX 4090 (24GB)NVIDIA RTX 4080 SUPER (16GB)NVIDIA RTX 4070 Ti Super (16GB)NVIDIA RTX 4060 Ti (16GB)NVIDIA RTX 3090 (24GB)AMD Radeon RX 7900 XTX (24GB)NVIDIA RTX PRO 6000 Blackwell (96GB)Apple M4 Max (128GB)Apple M4 Max (64GB)Apple M4 Pro (48GB)Apple M3 Ultra (192GB)NVIDIA B200 (192GB)NVIDIA H200 (141GB)NVIDIA H100 (80GB)NVIDIA H20 (96GB)NVIDIA A100 (80GB)NVIDIA L40S (48GB)AMD Instinct MI300X (192GB)Huawei Ascend 910B (64GB)
AMD Instinct MI300X LLM-Deployment FAQ
tools.gpuCompatibility.gpuDetail.faq.q1▾
Mit 192GB VRAM laufen 71 unserer Benchmark-Modelle bei 8K Kontext in FP16, FP8 oder INT4 auf einer einzelnen Karte.
Welche Token-Rate erreicht die AMD Instinct MI300X?▾
Mit 5300 GB/s Bandbreite skaliert das Einzelstrom-Tempo umgekehrt proportional zu den aktiven Gewichts-Bytes pro Token.
Wann sollte ich 2x oder 4x AMD Instinct MI300X nutzen?▾
Sobald Modellgewichte plus KV-Cache 192GB überschreiten oder viele parallele Nutzer bedient werden, ist Tensor-Parallelismus nötig.
Verwandte LLM- & GPU-Tools entdecken
Gesamtes AI-Toolkit-Verzeichnis →Beliebte Modell-VRAM-Rechner
DeepSeek-R1 (671B)DeepSeek-V3 (671B)DeepSeek-V4-Pro (1.6T)DeepSeek-V4-Flash (284B)DeepSeek-R1-Distill-Qwen-32BDeepSeek-R1-Distill-Qwen-14BDeepSeek-R1-Distill-Qwen-7BDeepSeek-R1-Distill-Llama-70BLlama 4 Maverick (400B)Llama 4 Scout (109B)Llama 3.1 405BLlama 3.1 70BLlama 3.1 8BLlama 3.2 3BQwen3.5-397B-A17BQwen3-235B-A22BQwen3-Coder-480B-A35BQwen3-32B
GPU-Kompatibilitätsguides
NVIDIA RTX 5090 (32GB)NVIDIA RTX 5080 (16GB)NVIDIA RTX 5070 Ti (16GB)NVIDIA RTX 4090 (24GB)NVIDIA RTX 4080 SUPER (16GB)NVIDIA RTX 4070 Ti Super (16GB)NVIDIA RTX 4060 Ti (16GB)NVIDIA RTX 3090 (24GB)AMD Radeon RX 7900 XTX (24GB)NVIDIA RTX PRO 6000 Blackwell (96GB)Apple M4 Max (128GB)Apple M4 Max (64GB)Apple M4 Pro (48GB)Apple M3 Ultra (192GB)