Beyond Black-Box Labels: Interpretable Criteria for Diagnosing Subjective NLP Tasks

arXiv:2604.17022v2 Announce Type: replace-cross Abstract: Subjective NLP datasets typically aggregate annotator judgments into a single gold label, making it difficult to diagnose whether disagreement reflects unclear criteria, collapsed distinctions, or legitimate plurality. We propose a...