SenseVoiceSmall (0.23B) VRAM & GPU Requirement Calculator
Interactive hardware sizing for SenseVoiceSmall (0.23B) (0.23B by FunAudioLLM). Estimated 8K context footprint: 0.7 GB (INT4) • 0.8 GB (FP8) • 1.0 GB (FP16).
Non-autoregressive encoder/DiT (0 KV cache)
GPUs
Memory Requirements 1.01 GB
Requires 1 GPUs (based on memory capacity)
0.46 GB
All model weights
0 GB
Conversation history cache
0.48 GB
Concurrent processing cache
0.07 GB
Temporary computation cache
Performance Metrics
Visualize 1863 tok/s →Max Streams (This GPU)
155 streams
Duplex Architecture
Stream / Turn-Based
Estimated Latency
81 ms
Max QPS
12.4 queries/s
Estimated using memory bandwidth and tensor compute throughput across 1x NVIDIA RTX 4090 (24GB).
Quick Deploy CLI Command
funasr-wss-server --model-dir FunAudioLLM/SenseVoiceSmall
Flags reflect your selected precision, context length, and tensor parallel GPU count.Open SenseVoiceSmall spec page →
SenseVoiceSmall (0.23B) Architecture & Memory Profile
Total Parameters
0.23B
Architecture
Dense Transformer
Developer / Provider
FunAudioLLM
Recommended Precision
FP16
SenseVoiceSmall (0.23B) VRAM Matrix by Precision & Concurrent Streams
Precomputed audio-stream VRAM requirements including unquantized acoustic/codec submodules and runtime workspace.
| Precision | 1 Stream | 4 Streams | 16 Streams |
|---|---|---|---|
| FP16 | 1.0 GB | 1.5 GB | 3.3 GB |
| FP8 | 0.8 GB | 1.2 GB | 3.0 GB |
| INT8 | 0.8 GB | 1.2 GB | 3.0 GB |
| INT4 | 0.7 GB | 1.1 GB | 2.9 GB |
Recommended GPU Hardware for SenseVoiceSmall (0.23B) (FP16)
Minimum GPU count and maximum concurrent real-time voice streams per node.
| GPU Hardware | Min GPUs | Total VRAM Pool | Max Streams |
|---|---|---|---|
| NVIDIA RTX 4090 (24GB) | 1× | 24 GB | 155 streams |
| NVIDIA RTX 5090 (32GB) | 1× | 32 GB | 208 streams |
| Apple M4 Max (128GB) (128GB) | 1× | 128 GB | 512 streams |
| NVIDIA A100 (80GB) (80GB) | 1× | 80 GB | 512 streams |
| NVIDIA H100 (80GB) | 1× | 80 GB | 512 streams |
| NVIDIA H200 (141GB) | 1× | 141 GB | 512 streams |
| Huawei Ascend 910B (64GB) | 1× | 64 GB | 423 streams |
SenseVoiceSmall (0.23B) Deployment FAQ
How much VRAM is needed to run SenseVoiceSmall (0.23B)?
For single-stream inference, SenseVoiceSmall (0.23B) uses about 1.0 GB in FP16, 0.8 GB in FP8, and 0.7 GB in INT4 including resident acoustic codecs and stream buffers.
Can I run SenseVoiceSmall (0.23B) on RTX 4090 or H100 GPUs?
Running SenseVoiceSmall (0.23B) in FP16 precision requires at least 1x RTX 4090 (24GB) or 1x NVIDIA H100 (80GB) GPUs.
What precision should I choose for SenseVoiceSmall (0.23B) in production?
For SenseVoiceSmall (0.23B) (0.23B by FunAudioLLM), FP16 provides the best balance between inference speed, VRAM footprint, and output quality.