LoRA Fine-Tuning

Why it matters

LoRA is default fine-tuning. Understanding shapes strategy.

The architecture

Add A, B matrices (rank r) per target layer.

Train only A, B.

Merge into base at inference.

How it works end to end

PEFT library.

Rank 8-64 typical.

Target: attention + MLP.

Merge saves inference overhead.