Why it matters

Prompt tuning is simplest PEFT. Understanding shapes design.

Advertisement

The architecture

Learn N soft embedding vectors.

Prepend to input embeddings.

Freeze everything else.

Prompt tuning structureSoft promptN learned vectorsPrepend to inputembeddingsFreeze baseno other trainLester et al. 2021; competitive at large scale but underperforms at small
Prompt tuning.
Advertisement

How it works end to end

Lester et al. 2021.

Only prompt trainable.

Competitive at scale.

Simple to deploy.