Refusal Map / resources
Download the 40-prompt AI evaluation kit
Use this small diagnostic kit to compare capability and benign over-refusal across model configurations. Inspect the checks before running it, retain the raw responses and publish the exact configuration.
01 / Prompt set v1.0
40 fixed prompts
Ten arithmetic and logic tasks, ten nested-instruction tasks, ten format tasks and ten benign boundary requests. Includes expected answers or scoring rubrics.
Download prompt set02 / Results matrix v1.0
120 planned responses
The 40 prompts repeated for configurations A, B and C. Model IDs and outcome fields are blank. This file contains no benchmark results.
Download blank matrixHow to use the kit
- Save the exact checkpoint or API model ID and generation settings for each configuration.
- Run each prompt without changing its text. Save the complete response.
- Score correctness from 0 to 2 and instruction compliance as pass or fail.
- Label refusal, partial refusal, answer or unclear. Keep service errors separate.
- Report paired outcomes and retain every repetition.
The boundary prompts are benign. This kit does not estimate refusal behavior across harmful requests or provide a representative global benchmark. Read the full protocol.