OpenNovelty.org

CPQS-Tuning: A Model Self-Perception-Based Data Filtering Algorithm for Efficient Instruction Fine-Tuning

ICLR 2026 Conference SubmissionAnonymous Authors
OpenReview Score: 6.0Download Report PDF
Instruction Fine-tuningLLMsData FilteringCPQSHidden States
Abstract:

Instruction fine-tuning is a key technique for enhancing the performance of large language models (LLMs), but low-quality and redundant data often hinder its effectiveness. Recent studies suggest that filtering a small amount of high-quality data for instruction fine-tuning can achieve faster and more efficient training performance. However, existing data filtering approaches predominantly depend on predefined evaluation models or manually designed metrics, without leveraging information from the target LLM itself. This limitation may result in a mismatch between the filtering criteria and the actual requirements of the LLM being fine-tuned, thereby reducing the effectiveness of the fine-tuning process. To address these issues, we propose a novel perspective: the hidden states of LLMs implicitly reflect the quality of the training data. Based on this insight, we propose a novel data filtering method that extracts the hidden states that reflect the target LLM’s perception of the data as representative features, and builds a data classification model upon them, which outputs the Contrastive Perception Quality Score (CPQS) for dataset filtering. Our experiments are conducted in both general and downstream domains. (1) In the general domain, our experiments show that training on under 10% of the data from both the Alpaca_GPT4 and DeepSeek-R1 synthesized reasoning datasets enables our method to outperform models trained on the complete datasets. Moreover, it surpasses the performance of current state-of-the-art data-selection techniques. (2) In downstream tasks, our approach delivers an average performance gain exceeding 3.6% over leading data-selection algorithms across multiple benchmarks, including GSM8K, HumanEval, and HumanEval-Plus.

Disclaimer
This report is AI-GENERATED using Large Language Models and WisPaper (A scholar search engine). It analyzes academic papers' tasks and contributions against retrieved prior work. While this system identifies POTENTIAL overlaps and novel directions, ITS COVERAGE IS NOT EXHAUSTIVE AND JUDGMENTS ARE APPROXIMATE. These results are intended to assist human reviewers and SHOULD NOT be relied upon as a definitive verdict on novelty.
NOTE that some papers exist in multiple, slightly different versions (e.g., with different titles or URLs). The system may retrieve several versions of the same underlying work. The current automated pipeline does not reliably align or distinguish these cases, so human reviewers will need to disambiguate them manually.
If you have any questions, please contact: mingzhang23@m.fudan.edu.cn

Overview

Overall Novelty Assessment

The paper proposes a data filtering method that extracts hidden states from the target LLM to compute a Contrastive Perception Quality Score (CPQS) for selecting high-quality instruction data. It resides in the 'Self-Perception and Hidden State Analysis' leaf of the taxonomy, which contains only two papers total. This leaf sits within the broader 'Model-Guided Quality Assessment' branch, indicating a relatively sparse but emerging research direction. The small number of sibling papers suggests this specific approach—using internal model perceptions rather than external metrics—is not yet heavily explored in the literature.

The taxonomy reveals that neighboring leaves include 'Instruction-Response Alignment Metrics' (four papers) and 'Weak-to-Strong and Cross-Model Filtering' (two papers), both under the same 'Model-Guided Quality Assessment' parent. These adjacent directions use model-based scoring or cross-model signals but differ in their reliance on response-level metrics or auxiliary models. The paper's focus on hidden-state extraction distinguishes it from these neighbors, which primarily assess alignment or leverage smaller models. The broader 'Quality-Based Data Selection Methods' branch also includes 'Heuristic and External Quality Metrics,' emphasizing the field's division between intrinsic model signals and external evaluation frameworks.

Among 25 candidates examined, the CPQS-based filtering method (Contribution A) showed no clear refutation across 10 candidates, suggesting limited direct overlap in the literature search scope. However, the contrastive training framework (Contribution B) and empirical validation (Contribution C) each encountered one refutable candidate among five and ten examined, respectively. These statistics indicate that while the core hidden-state extraction idea appears less contested in the limited search, the training methodology and evaluation approach have more substantial prior work. The analysis does not claim exhaustive coverage but reflects patterns within the top-25 semantic matches and their citations.

Given the sparse taxonomy leaf and limited search scope, the work appears to occupy a relatively novel position within model-guided quality assessment. The hidden-state extraction mechanism seems less directly addressed in prior work, though the contrastive training and validation components show some overlap. The analysis is constrained by the 25-candidate search scale and does not capture the full breadth of instruction tuning literature, leaving open the possibility of additional relevant work beyond the examined subset.

Taxonomy

Core-task Taxonomy Papers
50
3
Claimed Contributions
25
Contribution Candidate Papers Compared
2
Refutable Paper

Research Landscape Overview

Core task: data filtering for instruction fine-tuning of large language models. The field has evolved into a rich taxonomy addressing how to select high-quality, diverse, and efficient instruction datasets. Quality-Based Data Selection Methods emphasize model-guided assessment and self-perception techniques, such as CPQS Tuning[0] and Best Fit Data[31], which leverage hidden states or internal model signals to identify valuable examples. Diversity and Coverage-Based Selection focuses on ensuring broad task and domain representation, while Adaptive and Iterative Selection Strategies refine data choices dynamically during training. Large-Scale and Efficiency-Oriented Selection tackles the computational challenges of filtering massive corpora like Redpajama[1] and DataComp LM[7]. Domain-Specific and Application-Oriented Selection targets specialized areas such as code generation with InstructCoder[29] and OpenCodeInstruct[42], or visual instruction tuning with InstructBLIP[43]. Data Composition and Multi-Capability Balancing addresses how to mix capabilities effectively, as seen in Abilities Composition[8]. Privacy and Security in Instruction Data explores vulnerabilities like Instructions as Backdoors[19] and Poisoning Instruction Tuning[32], alongside privacy-preserving approaches such as Federated Instruction Tuning[3]. Data Generation and Curation Pipelines automate dataset creation, while Surveys and Comparative Studies like Instruction Tuning Survey[12] and Data Selection Survey[25] synthesize emerging trends. Recent work reveals tensions between quality and diversity, with some studies like Superfiltering[39] and Quantity to Quality[48] arguing that aggressive filtering improves performance, while others such as Preserving Diversity[17] caution against over-pruning. CPQS Tuning[0] sits within the Model-Guided Quality Assessment branch, specifically employing self-perception and hidden state analysis to identify high-quality instructions. This approach contrasts with neighbor work Best Fit Data[31], which also uses internal model representations but may emphasize different aspects of fit or alignment. Compared to external scoring methods like Data Advisor[5] or simpler heuristics in Instruction Mining[4], CPQS Tuning[0] leverages the model's own internal signals to assess example quality, reflecting a growing interest in introspective selection mechanisms that avoid costly external evaluations while maintaining fine-grained quality control.

Claimed Contributions

CPQS-based data filtering method using LLM hidden states

The authors introduce a data filtering approach that leverages hidden states from the target LLM to capture its implicit evaluation of training data quality. A CNN classifier trained on these hidden states produces a Contrastive Perception Quality Score (CPQS), which is then used to select high-quality instruction-tuning samples.

10 retrieved papers
Contrastive training framework with quality-labeled datasets

The authors construct training datasets labeled by quality tiers (high-quality from strong models like GPT-4, low-quality from smaller models) and use contrastive binary classification to train the CNN. This design helps the model distinguish the LLM's differing perceptions of good versus bad samples.

5 retrieved papers
Can Refute
Empirical validation across general and downstream domains

The authors conduct comprehensive experiments on both general-domain datasets (Alpaca GPT4, Reasoning-DeepSeek) and downstream tasks (GSM8K, HumanEval, HumanEval-Plus), demonstrating that their method outperforms existing data-selection techniques and achieves strong results with less than 10% of the original data.

10 retrieved papers
Can Refute

Core Task Comparisons

Comparisons with papers in the same taxonomy category

Contribution Analysis

Detailed comparisons for each claimed contribution

Contribution

CPQS-based data filtering method using LLM hidden states

The authors introduce a data filtering approach that leverages hidden states from the target LLM to capture its implicit evaluation of training data quality. A CNN classifier trained on these hidden states produces a Contrastive Perception Quality Score (CPQS), which is then used to select high-quality instruction-tuning samples.

Contribution

Contrastive training framework with quality-labeled datasets

The authors construct training datasets labeled by quality tiers (high-quality from strong models like GPT-4, low-quality from smaller models) and use contrastive binary classification to train the CNN. This design helps the model distinguish the LLM's differing perceptions of good versus bad samples.

Contribution

Empirical validation across general and downstream domains

The authors conduct comprehensive experiments on both general-domain datasets (Alpaca GPT4, Reasoning-DeepSeek) and downstream tasks (GSM8K, HumanEval, HumanEval-Plus), demonstrating that their method outperforms existing data-selection techniques and achieves strong results with less than 10% of the original data.

Table of Contents