GPT-OSS VRAM & GPU Requirement Calculator

Calculate VRAM requirements and GPU count for GPT-OSS deployment. Support for NVIDIA, AMD, Apple, and Huawei

GPUs

Memory Requirements 59.31 GB

Requires 1 GPUs (based on memory capacity)

58.5 GB

All model weights

0.28 GB

Conversation history cache

0.43 GB

Expert model optimization

0.1 GB

Temporary computation cache

Performance Metrics
Visualize 1100 tok/s →
Total Throughput
1100 tokens/s
Throughput Per User
1100 tokens/s/user
Estimated Latency
136 ms
Max QPS
7.3 queries/s

Estimated using memory bandwidth and tensor compute throughput across 1x NVIDIA H100 (80GB).

Quick Deploy CLI Command
vllm serve openai/gpt-oss-120b --max-model-len 8192
Flags reflect your selected precision, context length, and tensor parallel GPU count.Open GPT-OSS-120B spec page →

GPT-OSS Architecture & Memory Profile

Total Parameters
117B
Architecture
MoE (5.1B active)
Developer / Provider
OpenAI
Recommended Precision
MXFP4

GPT-OSS VRAM Matrix by Precision & Context Length

Precomputed single-user memory requirements (model weights + KV cache + runtime overhead).

Precision8K Context32K Context128K Context
FP16235.7 GB239.2 GB252.9 GB
FP8118.1 GB120.0 GB126.9 GB
INT8118.1 GB120.0 GB126.9 GB
INT459.3 GB60.3 GB63.9 GB

Recommended GPU Hardware for GPT-OSS (MXFP4)

Minimum GPU count for 8K context at batch size 1 and estimated generation throughput.

GPU HardwareMin GPUsTotal VRAM PoolEst. Speed
NVIDIA RTX 4090 (24GB)3×72 GB2722 tok/s
NVIDIA RTX 5090 (32GB)2×64 GB2267 tok/s
Apple M4 Max (128GB) (128GB)1×128 GB137 tok/s
NVIDIA A100 (80GB) (80GB)1×80 GB647 tok/s
NVIDIA H100 (80GB)1×80 GB1100 tok/s
NVIDIA H200 (141GB)1×141 GB1600 tok/s
Huawei Ascend 910B (64GB)1×64 GB400 tok/s

GPT-OSS Deployment FAQ

How much VRAM is needed to run GPT-OSS?
At 8K context and batch size 1, GPT-OSS uses about 235.7 GB in FP16, 118.1 GB in FP8, and 59.3 GB in INT4/AWQ quantization.
Can I run GPT-OSS on RTX 4090 or H100 GPUs?
Running GPT-OSS in MXFP4 precision requires at least 3x RTX 4090 (24GB) or 1x NVIDIA H100 (80GB) GPUs.
What precision should I choose for GPT-OSS in production?
For GPT-OSS (117B by OpenAI), MXFP4 provides the best balance between inference speed, VRAM footprint, and output quality.