OpenNovelty.org

RAIN-Merging: A Gradient-Free Method to Enhance Instruction Following in Large Reasoning Models with Preserved Thinking Format

ICLR 2026 Conference SubmissionAnonymous Authors
OpenReview Score: 6.5Download Report PDF
Large Reasoning ModelInstruction FollowingModel MergingNull-Space
Abstract:

Large reasoning models (LRMs) excel at a long chain of reasoning but often fail to faithfully follow instructions regarding output format, constraints, or specific requirements. We investigate whether this gap can be closed by integrating an instruction-tuned model (ITM) into an LRM. Analyzing their differences in parameter space, namely task vectors, we find that their principal subspaces are nearly orthogonal across key modules, suggesting a lightweight merging with minimal interference. However, we also demonstrate that naïve merges are fragile because they overlook the output format mismatch between LRMs (with explicit thinking and response segments) and ITMs (answers-only). We introduce RAIN-Merging (Reasoning-Aware Instruction-attention guided Null-space projection Merging), a gradient-free method that integrates instruction following while preserving thinking format and reasoning performance. First, with a small reasoning calibration set, we project the ITM task vector onto the null space of forward features at thinking special tokens, which preserves the LRM's structured reasoning mechanisms. Second, using a small instruction calibration set, we estimate instruction attention to derive module-specific scaling that amplifies instruction-relevant components and suppresses leakage. Across four instruction-following benchmarks and nine reasoning & general capability benchmarks, RAIN-Merging substantially improves instruction adherence while maintaining reasoning quality. The gains are consistent across model scales and architectures, translating to improved performance in agent settings.

Disclaimer
This report is AI-GENERATED using Large Language Models and WisPaper (A scholar search engine). It analyzes academic papers' tasks and contributions against retrieved prior work. While this system identifies POTENTIAL overlaps and novel directions, ITS COVERAGE IS NOT EXHAUSTIVE AND JUDGMENTS ARE APPROXIMATE. These results are intended to assist human reviewers and SHOULD NOT be relied upon as a definitive verdict on novelty.
NOTE that some papers exist in multiple, slightly different versions (e.g., with different titles or URLs). The system may retrieve several versions of the same underlying work. The current automated pipeline does not reliably align or distinguish these cases, so human reviewers will need to disambiguate them manually.
If you have any questions, please contact: mingzhang23@m.fudan.edu.cn

Overview

Overall Novelty Assessment

The paper proposes RAIN-Merging, a gradient-free method that integrates instruction-following capabilities from instruction-tuned models into large reasoning models through null-space projection and attention-guided merging. It resides in the 'Weight-Space Merging and Task Vector Methods' leaf, which contains only two papers total. This is a notably sparse research direction within the broader taxonomy of 50 papers across 36 topics, suggesting that parameter-space merging techniques for reasoning-instruction integration remain relatively underexplored compared to training-based adaptation methods or inference-time steering approaches.

The taxonomy reveals that most related work clusters in adjacent branches: 'Training-Based Adaptation and Instruction Tuning' contains 18 papers across four sub-areas, while 'Inference-Time Steering and Optimization' includes six papers. The paper's approach diverges from these by avoiding both retraining and inference-time prompting, instead operating directly in weight space. Its sibling paper (Disperse-then-Merge) explores iterative dispersion strategies, whereas RAIN-Merging emphasizes reasoning-aware projection to preserve structured thinking formats. The MoE-Based Integration leaf offers an architectural alternative with two papers, but these require learned routing rather than direct parameter fusion.

Among 29 candidates examined across three contributions, no clearly refuting prior work was identified. The RAIN-Merging method examined 10 candidates with zero refutable matches, the null-space projection technique examined 9 candidates with zero refutable matches, and the instruction-attention guided coefficients examined 10 candidates with zero refutable matches. This suggests that within the limited search scope, the specific combination of reasoning-aware null-space projection with attention-guided merging coefficients appears distinct from examined prior work. However, the search scale of 29 candidates is modest relative to the broader literature on model merging and instruction tuning.

Based on the top-29 semantic matches and the sparse taxonomy leaf containing only one sibling paper, the work appears to occupy a relatively novel position within parameter-space merging for reasoning-instruction integration. The analysis does not cover exhaustive literature on general task vector methods or broader model merging techniques outside the reasoning-instruction context, so the assessment reflects novelty within this specific problem framing rather than across all weight-space merging research.

Taxonomy

Core-task Taxonomy Papers
50
3
Claimed Contributions
29
Contribution Candidate Papers Compared
0
Refutable Paper

Research Landscape Overview

Core task: merging large reasoning models with instruction-tuned models. The field has organized itself around several complementary strategies for combining the strengths of specialized reasoning capabilities with broad instruction-following behavior. Model Merging and Integration Techniques explore weight-space methods and task vector approaches that directly combine parameters from different expert models, exemplified by works like RAIN-Merging[0] and Disperse-then-Merge[5]. Training-Based Adaptation and Instruction Tuning focus on fine-tuning strategies that teach models to follow instructions while preserving reasoning skills, with approaches ranging from mixture-of-experts architectures such as MoE Instruction Tuning[3] to continual learning methods like Mammoth2[1]. Inference-Time Steering and Optimization investigate dynamic control mechanisms that adjust model behavior during generation without retraining. Meanwhile, Instruction-Following Evaluation and Benchmarking develop metrics and test suites to assess how well merged or adapted models balance reasoning depth with instruction adherence, and Multimodal Instruction Following extends these ideas beyond text to vision and speech domains, as seen in Training-free Multimodal[2]. A particularly active line of work examines the trade-offs between parameter efficiency and capability retention when merging models. Some studies pursue training-free integration to avoid catastrophic forgetting, while others accept modest retraining costs to achieve tighter alignment between reasoning traces and instruction semantics. RAIN-Merging[0] sits squarely within the weight-space merging branch, emphasizing parameter-level fusion techniques that preserve the distinct strengths of reasoning-specialized and instruction-tuned checkpoints. Compared to Disperse-then-Merge[5], which explores iterative dispersion and consolidation strategies, RAIN-Merging[0] focuses on direct task vector arithmetic to achieve a stable blend. This contrasts with training-heavy approaches like Two Experts[4] or MoE Instruction Tuning[3], which rely on learned routing or gating mechanisms. The central open question across these branches remains how to quantify and control the reasoning-instruction trade-off without extensive human evaluation, a challenge that motivates ongoing work in both merging algorithms and evaluation frameworks.

Claimed Contributions

RAIN-Merging method for integrating instruction-following into large reasoning models

The authors propose a two-stage gradient-free merging approach that combines an instruction-tuned model with a large reasoning model. The method uses null-space projection to preserve the reasoning structure and instruction-attention guided coefficients to enhance instruction adherence without requiring gradient-based training.

10 retrieved papers
Reasoning-aware null-space projection technique

The first stage of RAIN-Merging projects the instruction-tuned model task vector onto the null space derived from forward features at thinking tokens. This projection maintains the large reasoning model's thinking format and output distribution while enabling integration of instruction-following capabilities.

9 retrieved papers
Instruction-attention guided merging coefficients

The second stage introduces per-module scaling coefficients based on attention outputs over instruction-related spans. These coefficients strengthen instruction-relevant behaviors by maximizing alignment with instruction tokens while minimizing attention leakage to unrelated content.

10 retrieved papers

Core Task Comparisons

Comparisons with papers in the same taxonomy category

Contribution Analysis

Detailed comparisons for each claimed contribution

Contribution

RAIN-Merging method for integrating instruction-following into large reasoning models

The authors propose a two-stage gradient-free merging approach that combines an instruction-tuned model with a large reasoning model. The method uses null-space projection to preserve the reasoning structure and instruction-attention guided coefficients to enhance instruction adherence without requiring gradient-based training.

Contribution

Reasoning-aware null-space projection technique

The first stage of RAIN-Merging projects the instruction-tuned model task vector onto the null space derived from forward features at thinking tokens. This projection maintains the large reasoning model's thinking format and output distribution while enabling integration of instruction-following capabilities.

Contribution

Instruction-attention guided merging coefficients

The second stage introduces per-module scaling coefficients based on attention outputs over instruction-related spans. These coefficients strengthen instruction-relevant behaviors by maximizing alignment with instruction tokens while minimizing attention leakage to unrelated content.

Table of Contents