Why it matters
VLM design shapes capability. Understanding shapes selection.
Advertisement
The architecture
LLaVA: MLP adapter, vision tokens prefix.
Flamingo: gated cross-attention layers.
Advertisement
How it works end to end
LLaVA: simple + effective.
Flamingo: gated cross-attn, more expressive.
Molmo: pixel-level control.
Open source: LLaVA, IDEFICS, Molmo.