OpenNovelty.org

Conformal Prediction with Corrupted Labels: Uncertain Imputation and Robust Re-weighting

ICLR 2026 Conference SubmissionAnonymous Authors
OpenReview Score: 6.0Download Report PDF
Conformal PredictionUncertainty QuantificationDistribution ShiftCorrupted LabelsPrivileged Information
Abstract:

We introduce a framework for robust uncertainty quantification in situations where labeled training data are corrupted, through noisy or missing labels. We build on conformal prediction, a statistical tool for generating prediction sets that cover the test label with a pre-specified probability. The validity of conformal prediction, however, holds under the i.i.d assumption, which does not hold in our setting due to the corruptions in the data. To account for this distribution shift, the privileged conformal prediction (PCP) method proposed leveraging privileged information (PI)---additional features available only during training---to re-weight the data distribution, yielding valid prediction sets under the assumption that the weights are accurate. In this work, we analyze the robustness of PCP to inaccuracies in the weights. Our analysis indicates that PCP can still yield valid uncertainty estimates even when the weights are poorly estimated. Furthermore, we introduce uncertain imputation (UI), a new conformal method that does not rely on weight estimation. Instead, we impute corrupted labels in a way that preserves their uncertainty. Our approach is supported by theoretical guarantees and validated empirically on both synthetic and real benchmarks. Finally, we show that these techniques can be integrated into a triply robust framework, ensuring statistically valid predictions as long as at least one underlying method is valid.

Disclaimer
This report is AI-GENERATED using Large Language Models and WisPaper (A scholar search engine). It analyzes academic papers' tasks and contributions against retrieved prior work. While this system identifies POTENTIAL overlaps and novel directions, ITS COVERAGE IS NOT EXHAUSTIVE AND JUDGMENTS ARE APPROXIMATE. These results are intended to assist human reviewers and SHOULD NOT be relied upon as a definitive verdict on novelty.
NOTE that some papers exist in multiple, slightly different versions (e.g., with different titles or URLs). The system may retrieve several versions of the same underlying work. The current automated pipeline does not reliably align or distinguish these cases, so human reviewers will need to disambiguate them manually.
If you have any questions, please contact: mingzhang23@m.fudan.edu.cn

Overview

Overall Novelty Assessment

The paper contributes a robustness analysis of privileged conformal prediction under weight inaccuracies and introduces uncertain imputation, a weight-free conformal method for corrupted labels. It resides in the Distribution Shift and Conformal Prediction leaf under Theoretical Foundations and Robustness Analysis, which contains only two papers total. This represents a sparse research direction within the broader taxonomy of fifty papers, suggesting the specific intersection of conformal prediction theory and label corruption remains relatively underexplored compared to more crowded areas like uncertainty-based sample filtering or medical imaging applications.

The taxonomy reveals neighboring leaves addressing related but distinct challenges. Robustness and Calibration Analysis examines uncertainty method stability under corruption without focusing on distribution-free guarantees, while Adversarial Robustness and Security studies attack scenarios rather than natural label noise. The sibling paper in this leaf addresses label shift quantification, which estimates distributional changes rather than constructing prediction sets. The scope note clarifies this leaf specifically targets valid prediction sets under corruption using conformal methods, distinguishing it from general robustness studies that may not preserve coverage guarantees or employ conformal frameworks.

Among nineteen candidates examined across three contributions, none were identified as clearly refuting the proposed work. The robustness analysis of privileged conformal prediction examined nine candidates with zero refutable matches, as did the uncertain imputation method. The triply robust framework examined only one candidate. This limited search scope—covering top semantic matches and citation expansion rather than exhaustive review—suggests the specific combination of conformal prediction robustness analysis and weight-free imputation methods has minimal direct overlap in the examined literature, though the small candidate pool prevents definitive conclusions about field-wide novelty.

Based on examination of nineteen semantically related papers, the work appears to occupy a relatively unexplored niche at the intersection of conformal prediction theory and label corruption. The sparse population of its taxonomy leaf and absence of refuting candidates within the search scope suggest novelty, though the limited scale of literature examination means potentially relevant work outside the top semantic matches may exist. The analysis captures proximity to established areas like uncertainty-based filtering but does not constitute comprehensive field coverage.

Taxonomy

Core-task Taxonomy Papers
50
3
Claimed Contributions
19
Contribution Candidate Papers Compared
0
Refutable Paper

Research Landscape Overview

Core task: uncertainty quantification with corrupted training labels. This field addresses the dual challenge of learning from datasets containing mislabeled examples while simultaneously providing reliable uncertainty estimates for predictions. The taxonomy reveals five major branches that capture different facets of this problem. Uncertainty-Aware Label Noise Detection and Correction focuses on identifying and fixing corrupted labels using confidence measures and sample selection strategies, exemplified by works like Confident Learning[7] and Neighborhood Sample Selection[15]. Label Noise Modeling and Robust Training develops loss functions and training procedures that remain stable under label corruption, including approaches such as Bayesian Focal Loss[28] and Imbalanced Noisy Learning[34]. Domain-Specific Applications with Noisy Labels tailors these techniques to fields like medical imaging (Dual-Uncertainty Medical[10]) and remote sensing (Noisy Earth Observation[9]). Theoretical Foundations and Robustness Analysis investigates the mathematical underpinnings of learning with corrupted labels, including distribution shift scenarios and conformal prediction guarantees. Finally, Uncertainty Estimation Frameworks and Methodologies encompasses general techniques for quantifying predictive uncertainty, such as Monte Carlo Dropout[50] and ensemble methods. A particularly active line of work explores the interplay between label noise robustness and calibrated uncertainty estimates, with many studies investigating whether models can simultaneously learn accurate predictions and reliable confidence scores from corrupted data. Works like Fisher Evidential Learning[2] and Robust Uncertainty Noise[14] exemplify efforts to maintain well-calibrated uncertainty under label corruption. Conformal Corrupted Labels[0] sits within the Theoretical Foundations branch, specifically addressing distribution shift and conformal prediction under label noise—a setting where traditional conformal methods may fail due to violated exchangeability assumptions. This contrasts with neighboring work on Label Shift Quantification[21], which focuses on estimating changes in label distributions rather than providing instance-level prediction sets. The original paper's emphasis on maintaining coverage guarantees despite corrupted training labels bridges theoretical robustness analysis with practical uncertainty quantification, addressing an open question about whether distribution-free inference remains viable when foundational data quality assumptions are violated.

Claimed Contributions

Robustness analysis of privileged conformal prediction to inaccurate weights

The authors formally characterize conditions under which privileged conformal prediction (PCP) and weighted conformal prediction (WCP) maintain valid coverage despite errors in the estimated likelihood ratio weights, showing that these methods can achieve nominal coverage even under significant weight estimation errors.

9 retrieved papers
Uncertain imputation method for conformal prediction with corrupted labels

The authors propose a novel calibration scheme called uncertain imputation (UI) that generates theoretically valid prediction sets by imputing corrupted labels using privileged information while preserving label uncertainty, without requiring accurate weight estimation like PCP does.

9 retrieved papers
Triply robust conformal prediction framework

The authors develop a triply robust calibration scheme that combines naive conformal prediction, privileged conformal prediction, and uncertain imputation into a unified framework that achieves valid coverage when the assumptions of at least one component method are satisfied.

1 retrieved paper

Core Task Comparisons

Comparisons with papers in the same taxonomy category

Contribution Analysis

Detailed comparisons for each claimed contribution

Contribution

Robustness analysis of privileged conformal prediction to inaccurate weights

The authors formally characterize conditions under which privileged conformal prediction (PCP) and weighted conformal prediction (WCP) maintain valid coverage despite errors in the estimated likelihood ratio weights, showing that these methods can achieve nominal coverage even under significant weight estimation errors.

Contribution

Uncertain imputation method for conformal prediction with corrupted labels

The authors propose a novel calibration scheme called uncertain imputation (UI) that generates theoretically valid prediction sets by imputing corrupted labels using privileged information while preserving label uncertainty, without requiring accurate weight estimation like PCP does.

Contribution

Triply robust conformal prediction framework

The authors develop a triply robust calibration scheme that combines naive conformal prediction, privileged conformal prediction, and uncertain imputation into a unified framework that achieves valid coverage when the assumptions of at least one component method are satisfied.

Table of Contents