Why it matters

QLoRA theory shapes efficient FT. Understanding shapes design.

Advertisement

The architecture

NF4: normalized float 4 for base.

Double quantization for scales.

Paged optimizers.

LoRA on top in bf16.

QLoRA stack theoryBase NF4 4-bitfrozenLoRA bf16trainablePaged optimizerOOM protectionDettmers 2023 QLoRA; 70B on single 48GB; combine NF4 + double quant + paged optim
QLoRA theory.
Advertisement

How it works end to end

NF4 information-theoretic optimal for normal-distributed weights.

Double quant reduces scale storage.

Paged optim protects OOM.