GPU LLM Compatibility Finder
Select your GPU, card count, and context length to instantly filter which 30+ LLMs fit in VRAM with FP16, FP8, or INT4.
Available VRAM
24 GB
Total Bandwidth
1,008 GB/s
Fits FP16 / FP8
22
Fits w/ INT4
44 / 85
DeepSeek-R1-Distill-Qwen-32B
32B • DeepSeek
FP16
68.27 GB
FP8
36 GB
INT4
19.86 GB
~46 tok/s (INT4)Open VRAM Calculator
DeepSeek-R1-Distill-Qwen-14B
14B • DeepSeek
FP16
32.27 GB
FP8
18 GB
INT4
10.86 GB
~23 tok/s (FP8)Open VRAM Calculator
DeepSeek-R1-Distill-Qwen-7B
7B • DeepSeek
FP16
18.27 GB
FP8
11 GB
INT4
7.36 GB
~12 tok/s (FP16)Open VRAM Calculator
DeepSeek-R1-Distill-Qwen-1.5B
1.5B • DeepSeek
FP16
7.27 GB
FP8
5.5 GB
INT4
4.61 GB
~12 tok/s (FP16)Open VRAM Calculator
DeepSeek-R1-Distill-Llama-8B
8B • DeepSeek
FP16
20.27 GB
FP8
12 GB
INT4
7.86 GB
~12 tok/s (FP16)Open VRAM Calculator
Llama 3 8B
8B • Llama 3
FP16
17.81 GB
FP8
9.31 GB
INT4
5.06 GB
~54 tok/s (FP16)Open VRAM Calculator
Llama 3.1 8B
8B • Llama 3
FP16
17.81 GB
FP8
9.31 GB
INT4
5.06 GB
~54 tok/s (FP16)Open VRAM Calculator
Llama 3.2 1B
1B • Llama 3
FP16
2.55 GB
FP8
1.43 GB
INT4
0.86 GB
~428 tok/s (FP16)Open VRAM Calculator
Llama 3.2 3B
3B • Llama 3
FP16
7.23 GB
FP8
3.79 GB
INT4
2.07 GB
~143 tok/s (FP16)Open VRAM Calculator
Llama 3.2 11B
11B • Llama 3
FP16
24.11 GB
FP8
12.61 GB
INT4
6.86 GB
~78 tok/s (FP8)Open VRAM Calculator
Qwen3.5-35B-A3BMoE 3B
35B • Qwen 3.5
FP16
70.98 GB
FP8
35.66 GB
INT4
18.01 GB
~571 tok/s (INT4)Open VRAM Calculator
Qwen3.5-27B
27B • Qwen 3.5
FP16
59.22 GB
FP8
30.97 GB
INT4
16.85 GB
~63 tok/s (INT4)Open VRAM Calculator
Qwen QwQ-32B
32B • Qwen QwQ
FP16
69.1 GB
FP8
36.16 GB
INT4
19.69 GB
~54 tok/s (INT4)Open VRAM Calculator
Qwen2.5-Coder-32B
32B • Qwen 2
FP16
69.1 GB
FP8
36.16 GB
INT4
19.69 GB
~54 tok/s (INT4)Open VRAM Calculator
Qwen3-32B
32B • Qwen 3
FP16
69.23 GB
FP8
36.23 GB
INT4
19.73 GB
~54 tok/s (INT4)Open VRAM Calculator
Qwen2.5-32B
32B • Qwen 2
FP16
69.1 GB
FP8
36.16 GB
INT4
19.69 GB
~54 tok/s (INT4)Open VRAM Calculator
Qwen2-32B
32B • Qwen 2
FP16
69.1 GB
FP8
36.16 GB
INT4
19.69 GB
~54 tok/s (INT4)Open VRAM Calculator
Qwen3-30B-A3BMoE 3B
30B • Qwen 3
FP16
61.1 GB
FP8
30.73 GB
INT4
15.54 GB
~571 tok/s (INT4)Open VRAM Calculator
Qwen2.5-14B
14B • Qwen 2
FP16
30.66 GB
FP8
16.04 GB
INT4
8.72 GB
~61 tok/s (FP8)Open VRAM Calculator
Qwen2.5-7B
7B • Qwen 2
FP16
15.21 GB
FP8
7.96 GB
INT4
4.33 GB
~61 tok/s (FP16)Open VRAM Calculator
Qwen2.5-3B
3B • Qwen 2
FP16
6.57 GB
FP8
3.46 GB
INT4
1.91 GB
~143 tok/s (FP16)Open VRAM Calculator
Qwen2.5-1.5B
1.5B • Qwen 2
FP16
3.49 GB
FP8
1.89 GB
INT4
1.1 GB
~286 tok/s (FP16)Open VRAM Calculator
Qwen2.5-0.5B
0.5B • Qwen 2
FP16
1.36 GB
FP8
0.83 GB
INT4
0.57 GB
~857 tok/s (FP16)Open VRAM Calculator
Qwen3-0.6B
0.6B • Qwen 3
FP16
1.94 GB
FP8
1.12 GB
INT4
0.71 GB
~714 tok/s (FP16)Open VRAM Calculator
Qwen3-1.7B
1.7B • Qwen 3
FP16
4.14 GB
FP8
2.22 GB
INT4
1.26 GB
~252 tok/s (FP16)Open VRAM Calculator
Mistral-Nemo-12B
12B • Mistral
FP16
26.46 GB
FP8
13.83 GB
INT4
7.52 GB
~71 tok/s (FP8)Open VRAM Calculator
Mistral-7B-v0.3
7B • Mistral
FP16
15.71 GB
FP8
8.21 GB
INT4
4.46 GB
~61 tok/s (FP16)Open VRAM Calculator
Gemma-2-27B
27B • Gemma
FP16
59.6 GB
FP8
31.16 GB
INT4
16.94 GB
~63 tok/s (INT4)Open VRAM Calculator
Gemma-2-9B
9B • Gemma
FP16
21.53 GB
FP8
11.22 GB
INT4
6.06 GB
~48 tok/s (FP16)Open VRAM Calculator
Phi-3.5-MoEMoE 6.6B
42B • Phi
FP16
85.67 GB
FP8
43.17 GB
INT4
21.92 GB
~260 tok/s (INT4)Open VRAM Calculator
Phi-3-3.8B
3.8B • Phi
FP16
11.02 GB
FP8
5.72 GB
INT4
3.07 GB
~113 tok/s (FP16)Open VRAM Calculator
CodeLlama-34B
34B • CodeLlama
FP16
72.93 GB
FP8
38.18 GB
INT4
20.8 GB
~50 tok/s (INT4)Open VRAM Calculator
CodeLlama-13B
13B • CodeLlama
FP16
33.56 GB
FP8
17.44 GB
INT4
9.37 GB
~66 tok/s (FP8)Open VRAM Calculator
CodeLlama-7B
7B • CodeLlama
FP16
18.71 GB
FP8
9.71 GB
INT4
5.21 GB
~61 tok/s (FP16)Open VRAM Calculator
GLM-4.7-FlashMoE 3B
30B • GLM-4
FP16
66.1 GB
FP8
34.66 GB
INT4
18.94 GB
~54 tok/s (INT4)Open VRAM Calculator
GPT-OSS-20BMoE 3.6B
21B • GPT-OSS
FP16
43.15 GB
FP8
21.78 GB
INT4
11.09 GB
~238 tok/s (FP8)Open VRAM Calculator
Dedicated Compatibility Pages for All 22 GPUs
NVIDIA RTX 5090 (32GB)NVIDIA RTX 5080 (16GB)NVIDIA RTX 5070 Ti (16GB)NVIDIA RTX 4090 (24GB)NVIDIA RTX 4080 SUPER (16GB)NVIDIA RTX 4070 Ti Super (16GB)NVIDIA RTX 4060 Ti (16GB)NVIDIA RTX 3090 (24GB)AMD Radeon RX 7900 XTX (24GB)NVIDIA RTX PRO 6000 Blackwell (96GB)Apple M4 Max (128GB)Apple M4 Max (64GB)Apple M4 Pro (48GB)Apple M3 Ultra (192GB)NVIDIA B200 (192GB)NVIDIA H200 (141GB)NVIDIA H100 (80GB)NVIDIA H20 (96GB)NVIDIA A100 (80GB)NVIDIA L40S (48GB)AMD Instinct MI300X (192GB)Huawei Ascend 910B (64GB)
GPU & Local LLM Compatibility FAQ
Which open-weight models fit on a single 24GB RTX 4090 or RTX 3090?▾
With INT4/AWQ quantization at 8K context, a 24GB GPU easily runs 7B-14B models (plus Qwen3-30B-A3B and GPT-OSS-20B MoE models) at high tok/s, while 32B models like QwQ-32B or Qwen3-32B fit snugly with moderate context.
Why does memory bandwidth matter as much as VRAM capacity?▾
Single-user autoregressive token generation is memory-bandwidth bound. For example, an RTX 5090 (1,792 GB/s) generates tokens roughly 1.8x faster than an RTX 4090 (1,008 GB/s) when both fit the same quantized weights.
Can Apple Silicon unified memory replace data center GPUs for local inference?▾
For 1-2 concurrent users, an M4 Max (128GB, 546 GB/s) or M3 Ultra (192GB, 819 GB/s) runs 70B-235B INT4 models smoothly. For high-concurrency serving (16+ users), H100/H200 tensor compute remains dramatically faster.
Does longer context push a fitting model over the VRAM limit?▾
Yes. Expanding context from 8K to 128K significantly grows the KV Cache, especially for dense non-GQA architectures or multi-user batches. Use our Context Length selector to check before deploying.
Explore Related LLM & GPU Tools
Browse full AI toolkit directory →Popular Model VRAM Calculators
DeepSeek-R1 (671B)DeepSeek-V3 (671B)DeepSeek-V4-Pro (1.6T)DeepSeek-V4-Flash (284B)DeepSeek-R1-Distill-Qwen-32BDeepSeek-R1-Distill-Qwen-14BDeepSeek-R1-Distill-Qwen-7BDeepSeek-R1-Distill-Llama-70BLlama 4 Maverick (400B)Llama 4 Scout (109B)Llama 3.1 405BLlama 3.1 70BLlama 3.1 8BLlama 3.2 3BQwen3.5-397B-A17BQwen3-235B-A22BQwen3-Coder-480B-A35BQwen3-32B
GPU Compatibility Guides
NVIDIA RTX 5090 (32GB)NVIDIA RTX 5080 (16GB)NVIDIA RTX 5070 Ti (16GB)NVIDIA RTX 4090 (24GB)NVIDIA RTX 4080 SUPER (16GB)NVIDIA RTX 4070 Ti Super (16GB)NVIDIA RTX 4060 Ti (16GB)NVIDIA RTX 3090 (24GB)AMD Radeon RX 7900 XTX (24GB)NVIDIA RTX PRO 6000 Blackwell (96GB)Apple M4 Max (128GB)Apple M4 Max (64GB)Apple M4 Pro (48GB)Apple M3 Ultra (192GB)