Why it matters
Every organization uses pre-trained models. Supply chain security is what prevents downloaded models from being backdoors. This is a large and under-defended attack surface.
The architecture
Data poisoning: attacker inserts crafted examples into training data. Model learns to respond differently on trigger inputs while behaving normally elsewhere.
Weight backdoors: models trained by attackers include specific weights that produce malicious output on trigger prompts.
How it works end to end
Loading exploits: model file formats (pickle, safetensors) can carry code execution. Pickle is famously dangerous; safetensors is safer but not perfectly.
Fine-tuning risks: a model fine-tuned on adversarial data can lose safety training or gain backdoors.
Detection is hard. Trigger inputs can be arbitrary strings; behavior looks normal until triggered.