Pirate SFT Removes Backdoors From Prompted Model Organisms

Pirate‑style fine‑tuning can erase backdoor triggers from static‑prompt Llama‑3.1‑8B models while keeping math performance intact.

Static‑Prompt Models Are Fragile

Red teams craft backdoors by embedding a system prompt that activates when a trigger appears. Blue teams then apply untargeted training, such as Pirate SFT—SFT on Qwen3‑30B‑A3B responses to Alpaca prompts where the model is instructed to talk like a pirate. Across 15 backdoor variants, Pirate SFT reduces the backdoor behavior rate by roughly 25 % while MATH‑500 scores drop only 10 %. The model learns to ignore its static system prompt, explaining why the backdoor vanishes without ever seeing the trigger.

Dynamic‑prompt organisms, which insert the system prompt only at evaluation time, resist Pirate SFT more effectively, yet remain vulnerable to other untargeted methods like Artificial Pirate Removal (APR). APR trains on normal Qwen outputs while prompting the model to respond pirate‑style; it removes the backdoor with minimal impact on MATH‑500, though instruction‑following may suffer.

Full‑Weight Fine‑Tuning Outperforms LoRA

When red teams train the backdoor directly, they compare three approaches: full‑weight fine‑tuning (FWFT), merged LoRA, and continued LoRA. FWFT models, trained with AdamW at 3 × 10⁻⁵ and 100 k unique samples, exhibit far greater robustness than rank‑64 LoRA models. Continued LoRA, where the LoRA is handed off to blue teams, proves especially fragile. Increasing LoRA rank brings robustness closer to FWFT, but never matches it.

Password Locking Undermines Robustness

Password‑locked organisms preserve benign performance by conditioning on a password string. Experiments with fully, partially, and non‑password‑locked Qwen3‑32B models show that reducing the password‑to‑no‑password ratio improves robustness. However, password locking still makes it easier for blue teams to recover full performance, suggesting that the benign behavior remains salient and can be re‑instated through untargeted training.

These findings suggest that future alignment work must prioritize full‑weight fine‑tuning and careful trigger design to build organisms that survive untargeted training.

Source: Advice for making robust-to-training model organisms
Domain: alignmentforum.org