Can a NVIDIA GeForce RTX 4060 run Qwen 2.5 (72B)?
NVIDIA GeForce RTX 4060 has 8GB VRAM. Here's exactly whichprecision (FP16, FP8, GGUF)of Qwen 2.5 (72B) fits and how smoothly it runs.
Too much for this GPU
The NVIDIA GeForce RTX 4060 (8GB) doesn't have enough VRAM for Qwen 2.5 (72B) even at its smallest size (43.2GB needed). Rent a cloud GPU or step up to a bigger card.
43.2GB
smallest fit on this GPU
Run it in the cloud instead
A 24GB RTX 4090 instance runs Qwen 2.5 (72B) at full quality, starting at pennies/hour.
Disclosure: Cloud provider links are referrals at no additional cost to you.
Qwen 2.5 (72B) variants on the NVIDIA GeForce RTX 4060
| Variant | Precision | Size | Total VRAM needed | Headroom | Fits? |
|---|---|---|---|---|---|
| Q4_K_M (4-bit Recommended)REC | q4_k_m | 43.2GB | 43.2GB | -35.2GB | ✗ OOM |
| Q8_0 (8-bit GGUF) | q8_0 | 81.2GB | 81.2GB | -73.2GB | ✗ OOM |
| FP16 (Uncompressed) | fp16 | 152.2GB | 152.2GB | -144.2GB | ✗ OOM |
"Total VRAM needed" = variant size + 0.0GB estimated text-encoder/VAE overhead. Headroom = GPU VRAM (8GB) − total needed. Green ≥ 1.5GB headroom; Red ≤ −2GB (will OOM).
NVIDIA GeForce RTX 4060 at a glance
VRAM
8GB
Est. price
$299
Best for
145.4GB Qwen 2.5 (72B) and lighter
Other models on the NVIDIA GeForce RTX 4060
FLUX.1 DevFLUX.1 SchnellStable Diffusion XL (SDXL)Stable Diffusion 1.5Stable Diffusion 3 (SD3)Hunyuan VideoWan 2.1 VideoPony DiffusionLlama 3.2 (1B)Llama 3.2 (3B)Llama 3 / 3.1 (8B)DeepSeek-R1 Distill Qwen (7B)
Other GPUs for Qwen 2.5 (72B)
NVIDIA GeForce RTX 3060 (12GB)NVIDIA GeForce RTX 4070 (12GB)NVIDIA GeForce RTX 4070 Ti (12GB)NVIDIA GeForce RTX 4080 (16GB)NVIDIA GeForce RTX 4090 (24GB)NVIDIA GeForce RTX 3080 (10GB)NVIDIA GeForce RTX 3090 (24GB)NVIDIA GeForce RTX 3070 (8GB)NVIDIA GeForce RTX 4060 Ti (16GB) (16GB)NVIDIA GeForce GTX 1650 (4GB)NVIDIA GeForce RTX 3050 (8GB)Apple MacBook Pro (M3 Max) (36GB)