LoRA Theory

Why it matters

LoRA theory shapes rank choice. Understanding shapes design.

The architecture

Full FT update DeltaW ~ low intrinsic rank.

Rank r constrains update.

Alpha scales.

How it works end to end

Hu et al. 2021 paper.

Intrinsic dimensionality hypothesis.

Merge into base: no inference overhead.

Rank sweep for quality.