Can you safely put false or harmful text in finetuning data if you clearly label it as false? A new paper says no. Train a model on documents that repeatedly warn a claim is fabricated, and it still asserts the claim as true afterward, up from near zero to about 89% of answers. The warning is the part training ignores. It holds for behavior too: transcripts flagged as malicious still teach the behavior, a real data-poisoning risk.
https://benjaminhan.net/posts/20260704-negation-neglect/?utm_source=mastodon&utm_medium=social
synesisNegation Neglect: When Models Fail to Learn Negations in Training – synesisFinetuning an LLM on documents that repeatedly flag a claim as false still teaches it to believe the claim, and the same failure extends to malicious behaviors.
