Preference Leakage: A Contamination Problem in LLM-as-a-judge

Core task: Bias detection in LLM-based evaluation systems. The field has organized itself around several major branches that reflect different facets of bias. One branch examines bias in LLM content generation and outputs, focusing on how models produce biased text across domains such as code generation, recommendations, and general language tasks. A second branch targets bias in LLM-as-evaluator systems, where models serve as judges or scorers of other outputs, introducing concerns about self-preference, positional effects, and inconsistent scoring. A third branch addresses evaluation methodology and benchmark quality, questioning whether existing test suites adequately capture bias phenomena or inadvertently perpetuate flawed assumptions. Comprehensive LLM evaluation studies provide broad empirical assessments that cut across multiple bias types, while tangentially related topics touch on fairness in adjacent areas like information retrieval or mental health applications. Representative works such as Not Fair Evaluators[3] and Self-Preference Bias[37] illustrate how evaluator-specific biases can distort automated assessment, whereas surveys like Bias Fairness Survey[1] and Fairness Survey[2] map the broader landscape of bias concerns in LLM deployment. Particularly active lines of work explore the tension between using LLMs as convenient evaluators and the risk that these models favor their own outputs or exhibit systematic preferences tied to model architecture or training lineage. Studies on self-preference and model relatedness biases reveal that evaluators may assign higher scores to responses generated by themselves or by closely related models, undermining the objectivity of automated judgments. Preference Leakage[0] sits squarely within this cluster, investigating how subtle cues in generated text can inadvertently signal a model's identity to an evaluator, thereby triggering preferential scoring. This work complements Self-Preference Bias[37], which documents the phenomenon more broadly, and contrasts with efforts like FairEval[20] that propose mitigation strategies to reduce evaluator partiality. Together, these studies highlight an open question: whether LLM-based evaluation can be made sufficiently robust to replace human judgment, or whether inherent biases will always require careful calibration and transparency measures.