SORRY-Bench: Systematically Evaluating Large Language Model Safety
Refusal Behaviors
SORRY-Bench: Systematically Evaluating Large Language Model Safety
Refusal Behaviors
Evaluating aligned large language models' (LLMs) ability to recognize and reject unsafe user requests is crucial for safe, policy-compliant deployments. Existing evaluation efforts, however, face three limitations that we address with SORRY-Bench, our proposed benchmark. First, existing methods often use coarse-grained taxonomies of unsafe topics, and are over-representing some fine-grained …