Evaluating Agent-Proposed Weight Edits for False Abstention
A controlled case study in Qwen2.5-3B-Instruct
This is a preliminary report. I’m sharing it early to invite scientific scrutiny, not to announce a finished result. Below are the problems I already know about and am actively looking into. If you find others, I want to hear about them.
Summary
Language models sometimes say they can’t answer when the answer is right there in their context. In Qwen2.5-3B-Instruct, these false abstentions often come right after the model writes a note that names the correct answer. I asked a Claude Opus 5.5 agent to propose weight edits that fix this, and evaluated them on held-out data under a protocol fixed in advance.
A rank-1 edit to the last layer’s attention output met the repair criterion. On sealed held-out data it recovered 82% of false abstentions, while unsupported answers rose from 3.5% to 4.2% and 3% of correct answers broke. It beat three of five simpler fixes. It did not clearly beat a plain logit bias against “unknown”, and the agent’s explanations of why its edits work were not established.
Known problems I’m looking into
- 01The prompt invites “unknown”, and removing the option removes most failures.
On the made-up questions, taking out the “unknown” instruction lets the model answer 145 of 150 of its false abstentions correctly. The two abstention rules in the prompt also disagree. Until a rerun with one consistent abstention rule reports, every controlled-task result has to be read as “under this prompt”.
- 02The clearest results come from made-up questions, and they shrink on real chats.
About 9 in 10 notes name the answer on the made-up questions, against about 4 in 10 on LongMemEval. The edit fixes 8 in 10 made-up cases but about 1 in 3 real ones. Any headline number has to say which data it comes from.
- 03One model, and the habit depends on the model family.
Everything mechanistic is on Qwen2.5-3B, which says “unknown” with the evidence in view far more often (30%) than Llama, Gemma or Phi (2 to 10%). The findings describe this model and must not be stated as general.
- 04The type A versus type B contrast was never tested on type A.
Type A failures are ones where the model never uses the evidence. Type B failures are ones where it uses the evidence and abstains anyway. The swap test never became valid, and the patching study’s entry rule only admitted failures where the evidence was used. “Most failures are type B” is supported, but no experiment could have found type A. That makes it a one-sided result, not a comparison.
- 05The weight edit did not beat the simplest fix.
Lowering the score of “unknown” by a fixed amount released about as many answers, and the edit was neither shown better nor shown equivalent. Where it was better, giving fewer answers to unanswerable questions, it was on made-up city questions, with a write direction built from city words.
- 06Prior work already describes part of the mechanism.
Ling et al. (arXiv 2507.16199) report that an “unknown” option suppresses answers the model can give, with the answer kept in middle layers and overridden late. My controlled-task picture is close to theirs. It has to be cited as prior work, not presented as new.
Feedback
The point of putting this out now is to find what I’ve missed. Critique of the design, the statistics and the conclusions is very welcome, especially anything that would change a verdict. Write to me at qamilmirza@berkeley.edu.