Why it matters

Every downloaded model could be backdoored. Understanding shapes trust and validation practices.

Advertisement

The architecture

Training-time attack: adversary adds trigger examples to training data. Model learns association.

Trigger: specific text pattern, image feature, etc. Rare in normal use.

Backdoor mechanismPlant trigger examplesin training dataModel learnstrigger → behaviorNormal operationuntil triggerHard to detect: model appears normal on typical inputs; trigger might be arbitrary
Backdoor lifecycle.
Advertisement

How it works end to end

Detection is hard. Standard evaluation doesn't include attacker's trigger.

Defenses: activation clustering, neural cleanse, careful data provenance.

Fine-tuning risk: fine-tuning on adversarial data can plant backdoors.