SenseVoiceSmall (0.23B) VRAM & GPU Requirement Calculator

Interactive hardware sizing for SenseVoiceSmall (0.23B) (0.23B by FunAudioLLM). Estimated 8K context footprint: 0.7 GB (INT4) • 0.8 GB (FP8) • 1.0 GB (FP16).

Non-autoregressive encoder/DiT (0 KV cache)
GPUs

Memory Requirements 1.01 GB

Requires 1 GPUs (based on memory capacity)

0.46 GB

All model weights

0 GB

Conversation history cache

0.48 GB

Concurrent processing cache

0.07 GB

Temporary computation cache

Performance Metrics
Visualize 1863 tok/s →
Max Streams (This GPU)
155 streams
Duplex Architecture
Stream / Turn-Based
Estimated Latency
81 ms
Max QPS
12.4 queries/s

Estimated using memory bandwidth and tensor compute throughput across 1x NVIDIA RTX 4090 (24GB).

Quick Deploy CLI Command
funasr-wss-server --model-dir FunAudioLLM/SenseVoiceSmall
Flags reflect your selected precision, context length, and tensor parallel GPU count.Open SenseVoiceSmall spec page →

SenseVoiceSmall (0.23B) Architecture & Memory Profile

Total Parameters
0.23B
Architecture
Dense Transformer
Developer / Provider
FunAudioLLM
Recommended Precision
FP16

SenseVoiceSmall (0.23B) VRAM Matrix by Precision & Concurrent Streams

Precomputed audio-stream VRAM requirements including unquantized acoustic/codec submodules and runtime workspace.

Precision1 Stream4 Streams16 Streams
FP161.0 GB1.5 GB3.3 GB
FP80.8 GB1.2 GB3.0 GB
INT80.8 GB1.2 GB3.0 GB
INT40.7 GB1.1 GB2.9 GB

Recommended GPU Hardware for SenseVoiceSmall (0.23B) (FP16)

Minimum GPU count and maximum concurrent real-time voice streams per node.

GPU HardwareMin GPUsTotal VRAM PoolMax Streams
NVIDIA RTX 4090 (24GB)1×24 GB155 streams
NVIDIA RTX 5090 (32GB)1×32 GB208 streams
Apple M4 Max (128GB) (128GB)1×128 GB512 streams
NVIDIA A100 (80GB) (80GB)1×80 GB512 streams
NVIDIA H100 (80GB)1×80 GB512 streams
NVIDIA H200 (141GB)1×141 GB512 streams
Huawei Ascend 910B (64GB)1×64 GB423 streams

SenseVoiceSmall (0.23B) Deployment FAQ

How much VRAM is needed to run SenseVoiceSmall (0.23B)?
For single-stream inference, SenseVoiceSmall (0.23B) uses about 1.0 GB in FP16, 0.8 GB in FP8, and 0.7 GB in INT4 including resident acoustic codecs and stream buffers.
Can I run SenseVoiceSmall (0.23B) on RTX 4090 or H100 GPUs?
Running SenseVoiceSmall (0.23B) in FP16 precision requires at least 1x RTX 4090 (24GB) or 1x NVIDIA H100 (80GB) GPUs.
What precision should I choose for SenseVoiceSmall (0.23B) in production?
For SenseVoiceSmall (0.23B) (0.23B by FunAudioLLM), FP16 provides the best balance between inference speed, VRAM footprint, and output quality.