OpenNovelty.org

CoT Vectors: Transferring and Probing the Reasoning Mechanisms of LLMs

ICLR 2026 Conference SubmissionAnonymous Authors
OpenReview Score: 6.0Download Report PDF
Chain-of-Thought (CoT); Task Vectors; Model Steering; Large Language Models (LLMs)
Abstract:

Chain-of-Thought (CoT) prompting has emerged as a powerful approach to enhancing the reasoning capabilities of Large Language Models (LLMs). However, existing implementations, such as in-context learning and fine-tuning, remain costly and inefficient. To improve CoT reasoning at a lower cost, and inspired by the task vector paradigm, we introduce CoT Vectors, compact representations that encode task-general, multi-step reasoning knowledge. Through experiments with Extracted CoT Vectors, we observe pronounced layer-wise instability, manifesting as a U-shaped performance curve that reflects a systematic three-stage reasoning process in LLMs. To address this limitation, we propose Learnable CoT Vectors, optimized under a teacher–student framework to provide more stable and robust guidance. Extensive evaluations across diverse benchmarks and models demonstrate that CoT Vectors not only outperform existing baselines but also achieve performance comparable to parameter-efficient fine-tuning methods, while requiring fewer trainable parameters. Moreover, by treating CoT Vectors as a probe, we uncover how their effectiveness varies due to latent space structure, information density, acquisition mechanisms, and pre-training differences, offering new insights into the functional organization of multi-step reasoning in LLMs. The source code will be released.

Disclaimer
This report is AI-GENERATED using Large Language Models and WisPaper (A scholar search engine). It analyzes academic papers' tasks and contributions against retrieved prior work. While this system identifies POTENTIAL overlaps and novel directions, ITS COVERAGE IS NOT EXHAUSTIVE AND JUDGMENTS ARE APPROXIMATE. These results are intended to assist human reviewers and SHOULD NOT be relied upon as a definitive verdict on novelty.
NOTE that some papers exist in multiple, slightly different versions (e.g., with different titles or URLs). The system may retrieve several versions of the same underlying work. The current automated pipeline does not reliably align or distinguish these cases, so human reviewers will need to disambiguate them manually.
If you have any questions, please contact: mingzhang23@m.fudan.edu.cn

Overview

Overall Novelty Assessment

The paper introduces CoT Vectors as compact representations encoding task-general multi-step reasoning knowledge, positioning itself within the Representation Engineering and Vector Methods leaf of the taxonomy. This leaf contains only three papers total, including the original work, indicating a relatively sparse and emerging research direction. The approach sits within Training-Based CoT Enhancement, which contrasts with the more crowded CoT Prompting methods that dominate the field. The sibling papers explore related representation-level interventions: Long CoT Representation analyzes how extended reasoning traces encode in activations, while SoftCoT uses continuous prompt embeddings for reasoning guidance.

The taxonomy reveals that most CoT research concentrates in prompting-based methods and reinforcement learning frameworks, with representation engineering forming a smaller but distinct branch. Neighboring work in CoT Fine-Tuning and Knowledge Distillation relies on supervised learning with reasoning datasets, while the Prompting Strategy Design leaf explores elicitation without parameter updates. The paper's vector-based approach bridges these paradigms by encoding reasoning knowledge in compact representations rather than full parameter updates or discrete prompts. The scope note for this leaf explicitly distinguishes it from standard fine-tuning and prompting methods, emphasizing direct manipulation of latent space structure.

Among thirty candidates examined through semantic search, the contribution-level analysis shows varied novelty profiles. The core CoT Vectors concept and the Learnable CoT Vectors framework each examined ten candidates with zero refutations, suggesting these contributions appear relatively novel within the limited search scope. However, the discovery of a three-stage reasoning process examined ten candidates and found two potentially refuting papers, indicating this analytical finding may overlap with existing mechanistic studies of reasoning in language models. The statistics reflect a focused but not exhaustive literature search, concentrated primarily on representation-level and training-based methods.

Based on the limited search scope of thirty semantically similar papers, the work appears to occupy a sparsely populated research direction within the broader CoT landscape. The representation engineering approach offers a distinct alternative to dominant prompting and fine-tuning paradigms, though the mechanistic insights about reasoning stages may have partial precedent. The analysis covers top-ranked semantic matches and immediate taxonomic neighbors but does not claim comprehensive coverage of all potentially relevant prior work across the fifty-paper taxonomy.

Taxonomy

Core-task Taxonomy Papers
50
3
Claimed Contributions
30
Contribution Candidate Papers Compared
2
Refutable Paper

Research Landscape Overview

Core task: Transferring and enhancing chain-of-thought reasoning in large language models. The field is organized around several complementary strategies for improving step-by-step reasoning capabilities. CoT Prompting and In-Context Learning Methods[1][2][6] explore how to elicit reasoning through carefully designed prompts and examples, while Training-Based CoT Enhancement focuses on methods that modify model parameters or representations to internalize reasoning patterns. Reinforcement Learning for CoT Reasoning[32][40][42] applies reward-driven optimization to refine reasoning traces, and Inference-Time Reasoning Enhancement[36] develops techniques that improve reasoning during generation without retraining. Multimodal CoT Reasoning[10][23][24] extends these ideas to vision-language settings, Cross-Lingual and Cross-Domain CoT Transfer[11][48] examines how reasoning generalizes across languages and tasks, Analysis and Evaluation of CoT Reasoning[4][5][9][12][19] provides frameworks for understanding reasoning quality, and Specialized CoT Applications[15][22][33][37][41] adapt CoT methods to domain-specific problems. Within Training-Based CoT Enhancement, a particularly active line of work explores representation engineering and vector methods that manipulate internal model states to guide reasoning. CoT Vectors[0] sits squarely in this branch, proposing to transfer reasoning capabilities by identifying and applying latent vectors that encode chain-of-thought behavior. This approach contrasts with nearby works like Long CoT Representation[16], which analyzes how extended reasoning traces are encoded in model activations, and SoftCoT[21], which uses continuous prompt embeddings to steer reasoning without discrete token sequences. While many training-based methods rely on fine-tuning with curated reasoning datasets or reinforcement learning signals[3][8], representation-level interventions offer a more direct path to modifying reasoning behavior by operating on the geometric structure of hidden states. The central tension across these branches involves balancing the flexibility of prompt-based methods, the robustness of training-based approaches, and the efficiency of inference-time or representation-level techniques.

Claimed Contributions

CoT Vectors: compact representations encoding task-general multi-step reasoning knowledge

The authors introduce CoT Vectors as a novel extension of the task vector paradigm to complex multi-step reasoning in LLMs. These vectors compactly encode reasoning knowledge from support sets and can be injected into model forward passes to guide reasoning without modifying model parameters or requiring costly retraining.

10 retrieved papers
Learnable CoT Vectors optimized via teacher-student framework

The authors propose a novel parametric method for acquiring CoT Vectors through gradient-based optimization in a teacher-student architecture. This approach addresses the layer-wise instability of extracted vectors by actively learning reasoning knowledge rather than passively averaging activations, achieving greater stability and stronger performance.

10 retrieved papers
Discovery of three-stage reasoning process through CoT Vector analysis

Through experiments with Extracted CoT Vectors, the authors uncover a systematic three-stage reasoning process (perception, reasoning, expression) in LLMs, characterized by layer-wise performance patterns and representational properties. This finding provides new insights into the functional organization of multi-step reasoning in language models.

10 retrieved papers
Can Refute

Core Task Comparisons

Comparisons with papers in the same taxonomy category

Contribution Analysis

Detailed comparisons for each claimed contribution

Contribution

CoT Vectors: compact representations encoding task-general multi-step reasoning knowledge

The authors introduce CoT Vectors as a novel extension of the task vector paradigm to complex multi-step reasoning in LLMs. These vectors compactly encode reasoning knowledge from support sets and can be injected into model forward passes to guide reasoning without modifying model parameters or requiring costly retraining.

Contribution

Learnable CoT Vectors optimized via teacher-student framework

The authors propose a novel parametric method for acquiring CoT Vectors through gradient-based optimization in a teacher-student architecture. This approach addresses the layer-wise instability of extracted vectors by actively learning reasoning knowledge rather than passively averaging activations, achieving greater stability and stronger performance.

Contribution

Discovery of three-stage reasoning process through CoT Vector analysis

Through experiments with Extracted CoT Vectors, the authors uncover a systematic three-stage reasoning process (perception, reasoning, expression) in LLMs, characterized by layer-wise performance patterns and representational properties. This finding provides new insights into the functional organization of multi-step reasoning in language models.

Table of Contents