OpenNovelty.org

KnowProxy: Adapting Large Language Models by Knowledge-guided Proxy

ICLR 2026 Conference SubmissionAnonymous Authors
OpenReview Score: 6.5Download Report PDF
Indirect TuningEfficient Fine-tuningLarge Language Models
Abstract:

Adapting large language models (LLMs) using smaller proxy models has been shown to improve training efficiency, where the LLMs remain frozen while the proxies are tuned on top. However, this approach typically requires access to the output probability distributions of LLMs, which are often inaccessible or unstable. To address this limitation, we propose KnowProxy, a knowledge-guided proxy framework in which the proxy is trained with textual knowledge rather than probability distributions. Specifically, we first elicit textual knowledge and reasoning from frozen LLMs through prompting, and then the proxy model learns to adapt this reasoning to target task distributions. We evaluate KnowProxy on diverse reasoning benchmarks with different fine-tuning scenarios. Comprehensive results show that KnowProxy achieves competitive or even better performance without direct access to probability distributions, thereby providing a scalable and versatile alternative to traditional fine-tuning.

Disclaimer
This report is AI-GENERATED using Large Language Models and WisPaper (A scholar search engine). It analyzes academic papers' tasks and contributions against retrieved prior work. While this system identifies POTENTIAL overlaps and novel directions, ITS COVERAGE IS NOT EXHAUSTIVE AND JUDGMENTS ARE APPROXIMATE. These results are intended to assist human reviewers and SHOULD NOT be relied upon as a definitive verdict on novelty.
NOTE that some papers exist in multiple, slightly different versions (e.g., with different titles or URLs). The system may retrieve several versions of the same underlying work. The current automated pipeline does not reliably align or distinguish these cases, so human reviewers will need to disambiguate them manually.
If you have any questions, please contact: mingzhang23@m.fudan.edu.cn

Overview

Overall Novelty Assessment

The paper proposes KnowProxy, a framework that trains smaller proxy models using textual knowledge elicited from frozen large language models rather than requiring access to their probability distributions. Within the taxonomy, this work resides in the Training-Time Proxy Integration leaf under Proxy-Based Model Adaptation Mechanisms, alongside two sibling papers (FedPromo and Large Small Collaboration). This leaf represents a moderately populated research direction within a broader taxonomy of 34 papers across 20 leaf nodes, indicating focused but not overcrowded attention to training-time proxy strategies for LLM adaptation.

The taxonomy structure reveals that Training-Time Proxy Integration sits adjacent to Decoding-Time Proxy Tuning (which applies proxies only at inference) and Proxy-Based Architecture Search. Neighboring branches include Knowledge-Guided Adaptation Strategies, which emphasize structured knowledge integration and domain-specific adaptation, and Parameter-Efficient Adaptation Strategies, focusing on low-rank updates and modular adapters. KnowProxy bridges these areas by combining proxy-based efficiency with knowledge-guided steering, diverging from purely mechanistic proxy methods by incorporating explicit reasoning elicitation and from pure knowledge integration approaches by maintaining the proxy architecture paradigm.

Among 30 candidate papers examined, none were identified as clearly refuting any of the three core contributions: the KnowProxy framework itself, the dynamic routing mechanism, and the knowledge elicitation process. Each contribution was assessed against 10 candidates with zero refutable overlaps found. This suggests that within the limited search scope, the combination of knowledge-guided proxy training without probability distribution access appears relatively unexplored. However, the modest search scale (30 candidates from semantic search) means the analysis captures immediate neighbors rather than exhaustive prior work, and the absence of refutations reflects this bounded examination rather than definitive novelty.

Based on the limited literature search, the work appears to occupy a distinctive position combining proxy-based efficiency with knowledge-guided adaptation. The taxonomy context shows this sits at the intersection of two active research threads, and the contribution-level analysis found no direct overlaps among examined candidates. However, the 30-paper search scope and the presence of two sibling papers in the same taxonomy leaf suggest caution in claiming broad novelty without deeper investigation of related proxy tuning and knowledge distillation literature.

Taxonomy

Core-task Taxonomy Papers
34
3
Claimed Contributions
30
Contribution Candidate Papers Compared
0
Refutable Paper

Research Landscape Overview

Core task: adapting large language models through knowledge-guided proxy training. This field addresses the challenge of efficiently adapting large language models by leveraging smaller proxy models or auxiliary knowledge sources during training. The taxonomy reveals several complementary branches: Proxy-Based Model Adaptation Mechanisms explore how smaller models can guide or substitute for expensive large-model updates, including training-time integration approaches like KnowProxy[0] and inference-time strategies such as Tuning by Proxy[6]. Knowledge-Guided Adaptation Strategies focus on injecting structured or domain-specific knowledge into models, with works like KaSA[1] and KG-SR-LLM[3] demonstrating how external knowledge graphs or task-driven cues can steer adaptation. Parameter-Efficient Adaptation Strategies and Knowledge Transfer and Distillation Methods address scalability through techniques like low-rank updates and student-teacher frameworks, while Domain Adaptation branches tackle specialized settings from medical imaging to federated learning, and Context and Knowledge Sensitivity Control manages how models balance parametric versus contextual information. A particularly active line of work centers on training-time proxy integration, where smaller models serve as computational surrogates to guide large-model fine-tuning without full-scale backpropagation. KnowProxy[0] exemplifies this approach by using knowledge-guided proxies during training, sitting naturally alongside FedPromo[21] and Large Small Collaboration[23], which similarly exploit small-large model synergies in federated and collaborative settings. These methods contrast with knowledge injection strategies like Selective Knowledge Injection[22] or Parametric Knowledge Guiding[25], which emphasize embedding external structured knowledge rather than relying on proxy architectures. The trade-off revolves around whether adaptation should prioritize computational efficiency through architectural proxies or semantic richness through explicit knowledge integration. KnowProxy[0] bridges these themes by combining proxy-based efficiency with knowledge-guided steering, positioning it at the intersection of mechanistic innovation and knowledge-aware adaptation within the broader landscape of parameter-efficient LLM tuning.

Claimed Contributions

KnowProxy framework for knowledge-guided proxy adaptation

The authors introduce a novel proxy-based fine-tuning framework that adapts large language models by training smaller proxy models on textual knowledge and reasoning elicited from frozen LLMs, rather than relying on probability distributions. This design enables applicability to black-box settings where only text outputs are available.

10 retrieved papers
Dynamic routing mechanism for adaptive proxy invocation

The authors develop an adaptive routing mechanism that uses uncertainty scores elicited from the LLM's generated knowledge to determine when to invoke the proxy model. This allows the framework to selectively engage the proxy only for uncertain or unreliable LLM outputs, reducing inference overhead while maintaining accuracy.

10 retrieved papers
Knowledge elicitation and filtering process for proxy training

The authors propose a method to extract textual knowledge and reasoning from LLMs via prompting, along with confidence scores for each piece of knowledge. A filtering process retains only high-confidence knowledge, which is then used to train the proxy model to align LLM-derived reasoning with target task distributions.

10 retrieved papers

Core Task Comparisons

Comparisons with papers in the same taxonomy category

Contribution Analysis

Detailed comparisons for each claimed contribution

Contribution

KnowProxy framework for knowledge-guided proxy adaptation

The authors introduce a novel proxy-based fine-tuning framework that adapts large language models by training smaller proxy models on textual knowledge and reasoning elicited from frozen LLMs, rather than relying on probability distributions. This design enables applicability to black-box settings where only text outputs are available.

Contribution

Dynamic routing mechanism for adaptive proxy invocation

The authors develop an adaptive routing mechanism that uses uncertainty scores elicited from the LLM's generated knowledge to determine when to invoke the proxy model. This allows the framework to selectively engage the proxy only for uncertain or unreliable LLM outputs, reducing inference overhead while maintaining accuracy.

Contribution

Knowledge elicitation and filtering process for proxy training

The authors propose a method to extract textual knowledge and reasoning from LLMs via prompting, along with confidence scores for each piece of knowledge. A filtering process retains only high-confidence knowledge, which is then used to train the proxy model to align LLM-derived reasoning with target task distributions.

Table of Contents

KnowProxy: Adapting Large Language Models by Knowledge-guided Proxy | Novelty Validation