Best local AI models for an RTX 3090 24 GB
An NVIDIA GeForce RTX 3090 24 GB card has about 21.6 GB of VRAM free for a model after the driver and context cache. These are the most downloaded open language and vision-language models that fit, for Ollama, llama.cpp, LM Studio or vLLM.
24 GB VRAM · about 21.6 GB usable for a model
Buy a RTX 3090 on Amazon →
As an Amazon Associate we earn from qualifying purchases.
Last updated 3 November 2026 · refreshed after every Hugging Face sync · 15 models
| # | Model | Size | Memory | License | Downloads (30 days) | Likes | Est. speed (Q4) |
|---|---|---|---|---|---|---|---|
| 1 | Llama-3.1-8B-Instruct meta-llama | 8B · FP16 | ~20 GB | Llama 3.1 license | 4.5M | 8.8K | ~123 tok/s |
| 2 | Qwen3-8B Qwen | 8B · FP16 | ~20 GB | Apache 2.0 | 7.5M | 2.2K | ~117 tok/s |
| 3 | Qwen2.5-7B-Instruct Qwen | 7B · FP16 | ~17 GB | Apache 2.0 | 6.1M | 2.5K | ~135 tok/s |
| 4 | Qwen2.5-VL-7B-Instruct Qwen | 7B · FP16 | ~17 GB | Apache 2.0 | 5.9M | 1.8K | – |
| 5 | Qwen3-VL-8B-Instruct Qwen | 8B · FP16 | ~20 GB | Apache 2.0 | 6.9M | 1.2K | – |
| 6 | Qwen3-4B Qwen | 4B · FP16 | ~10 GB | Apache 2.0 | 8.4M | 782 | – |
| 7 | Qwen3.5-4B Qwen | 4B · FP16 | ~10 GB | Apache 2.0 | 6.4M | 1.2K | ~195 tok/s |
| 8 | Ornith-1.5-35B-A3B-GGUF ornith-ai | 35B · GGUF | ~21 GB | MIT | 4.1M | 572 | – |
| 9 | Mistral-7B-Instruct-v0.2 mistralai | 7B · FP16 | ~17 GB | Apache 2.0 | 1.7M | 3.3K | ~129 tok/s |
| 10 | Ternary-Bonsai-2-27B-gguf prism-ml | 27B · GGUF | ~17 GB | Apache 2.0 | 1.8M | 2.9K | – |
| 11 | Qwen3-VL-4B-Instruct Qwen | 4B · FP16 | ~10 GB | Apache 2.0 | 3.9M | 523 | – |
| 12 | Qwen3-4B-Instruct-2507 Qwen | 4B · FP16 | ~10 GB | Apache 2.0 | 2.7M | 1K | – |
| 13 | Qwen2.5-3B-Instruct Qwen | 3B · FP16 | ~8 GB | Custom (other) | 3M | 633 | ~235 tok/s |
| 14 | Qwen3.8-27B-GSQ-RCO-GGUF ISTA-DASLab | 27B · GGUF | ~17 GB | Apache 2.0 | 1.3M | 2.5K | ~45 tok/s |
| 15 | Llama-3.2-3B-Instruct meta-llama | 3B · FP16 | ~8 GB | Llama 3.2 license | 969.6K | 2.9K | ~209 tok/s |
How we rank
Language and vision-language models in GGUF, FP16, AWQ, GPTQ format whose published weights need about 6–21.6 GB. We include models seen on Hugging Face in the last 14 days with at least 10,000 downloads in the last 30 days. Score = log10(30-day downloads) + ½ × log10(likes + 1), so popularity leads and likes break near-ties. We keep one entry per model name and leave out adult and uncensored builds. Memory is estimated from parameter count and the published file format (16-bit ≈ 2 bytes per parameter, 4-bit ≈ 0.5) plus 20% headroom; long contexts need more. A quantised copy of a bigger model may also fit — check the model page.
Speed data: ModelFit (CC BY 4.0), from its open dataset, refreshed weekly. These are modelled estimates of decode speed for a 4-bit (Q4) build of the same model at a 4K context, not measured benchmarks, and they are not an endorsement by ModelFit. A dash means ModelFit has no estimate for that model.