LoRA + QLoRA Memory Calculator
Estimate GPU memory for full fine-tune vs LoRA vs QLoRA. Fit models to your hardware.
Model + method
Model params (billions)
0.5B (Qwen 0.5, Phi-3-mini)
1B (Gemma 1)
1.5B (Qwen 1.5)
3B (Phi-3-mini, Llama 3.2)
7-8B (Llama 3.1 8B, Mistral 7B)
14B (Phi-4, Qwen 3 14B)
27B (Gemma 3 27B, Qwen 32B)
34B (Yi, Qwen 32B)
70B (Llama 3.3 70B)
200B+ (custom)
Fine-tuning method
Full fine-tune (bf16)
LoRA (bf16)
QLoRA (4-bit + bf16 adapter)
Inference only (bf16)
Inference (4-bit)
LoRA rank (r) — only affects LoRA/QLoRA
4 (minimal)
8 (default)
16
32
64 (high)
128 (very high)
Sequence length
512
1024
2048
4096
8192
16384
Batch size (per GPU)
Optimizer
AdamW (2x params state)
AdamW 8-bit (0.5x)
SGD (no state)
GPU memory required
Total estimated
—
Your GPU