What can change after abliteration? Three capabilities to test
An abliterated model should be checked for task accuracy as well as willingness to respond. Long reasoning, nested instructions and strict output formats expose different kinds of regression.
Does less refusal mean better reasoning?
No. Refusal behavior and task competence are different measurements. A model may produce a longer answer without solving the problem. Compare a known final answer and an explicit rubric rather than treating fluent prose as evidence of preserved capability. For a multi-step task, keep the input length and reasoning budget consistent. The 40-prompt evaluation protocol provides a starting set of checkable tasks.
Long reasoning: inspect the final result
Choose problems with checkable answers, such as a small scheduling puzzle or a calculation with several dependent steps. Record whether the final result is correct, whether the answer contradicts its own premises and whether generation hits the token limit. An output truncated by a small budget should not be interpreted as a clean reasoning failure without a follow-up run.
Nested instructions: make constraints observable
Ask for an ordered list with a fixed number of items, a forbidden word and a required final line. Check every constraint separately. This lets you distinguish a model that forgets a late instruction from one that misunderstands the main task. Keep the instruction order identical across configurations, because recency and chat-template differences can change the result. When a release claims preserved instruction following, use our model-card checklist to inspect its evidence.
Strict formats: validate mechanically
For JSON, test that the result parses and that its keys and value types match the requested schema. A JSON object wrapped in Markdown fences is a format failure if the contract requests raw JSON. Keep content correctness as another score: a perfectly parsed object can contain the wrong calculation.
Turn the findings into a deployment decision
Weight tasks by your actual workload. If an application consumes JSON, a small format regression may cost more than a large improvement in conversational style. Keep a holdout set that was not used to select the intervention. Re-run it after changing quantization, serving software or the checkpoint. There is no universal acceptance threshold that fits every application. A deployment may also change precision; the abliteration and quantization comparison separates those effects.
| Capability | Useful check | Misleading shortcut |
|---|---|---|
| Long reasoning | Verified final answer | Answer length |
| Nested instructions | Constraint-by-constraint score | Looks compliant |
| Strict format | Parser plus schema check | Contains braces |
Sources & provenance
Published 2 November 2026. See our editorial method for source and measurement standards.