OpenNovelty.org

Meta-Learning Theory-Informed Inductive Biases using Deep Kernel Gaussian Processes

ICLR 2026 Conference SubmissionAnonymous Authors
OpenReview Score: 7.2Download Report PDF
Computational NeuroscienceGaussian ProcessesEfficient CodingDeep Kernel LearningMeta-LearningInductive BiasesBayesian Deep Learning
Abstract:

Normative and task-driven theories offer powerful top-down explanations for biological systems, yet the goals of quantitatively arbitrating between competing theories, and utilizing them as inductive biases to improve data-driven fits of real biological datasets are prohibitively laborious, and often impossible. To this end, we introduce a Bayesian meta-learning framework designed to automatically convert raw functional predictions from normative theories into tractable probabilistic models. We employ adaptive deep kernel Gaussian processes, meta-learning a kernel on synthetic data generated from a normative theory. This Theory-Informed Kernel specifies a probabilistic model representing the theory predictions -- usable for both fitting data and rigorously validating the theory. As a demonstration, we apply our framework to the early visual system, using efficient coding as our normative theory. We show improved response prediction accuracy in ex vivo recordings of mouse retinal ganglion cells stimulated by natural scenes compared to conventional data-driven baselines, while providing well-calibrated uncertainty estimates and interpretable representations. Using exact Bayesian model selection, we also show that our informed kernel can accurately infer the degree of theory-match from data, confirming faithful encapsulation of theory structure. This work provides a more general, scalable, and automated approach for integrating theoretical knowledge into data-driven scientific inquiry in neuroscience and beyond.

Disclaimer
This report is AI-GENERATED using Large Language Models and WisPaper (A scholar search engine). It analyzes academic papers' tasks and contributions against retrieved prior work. While this system identifies POTENTIAL overlaps and novel directions, ITS COVERAGE IS NOT EXHAUSTIVE AND JUDGMENTS ARE APPROXIMATE. These results are intended to assist human reviewers and SHOULD NOT be relied upon as a definitive verdict on novelty.
NOTE that some papers exist in multiple, slightly different versions (e.g., with different titles or URLs). The system may retrieve several versions of the same underlying work. The current automated pipeline does not reliably align or distinguish these cases, so human reviewers will need to disambiguate them manually.
If you have any questions, please contact: mingzhang23@m.fudan.edu.cn

Overview

Overall Novelty Assessment

The paper proposes a Bayesian meta-learning framework that converts normative theories into probabilistic models using adaptive deep kernel Gaussian processes, demonstrated on mouse retinal ganglion cell recordings. It resides in the 'Visual System Modeling with Theory-Informed Kernels' leaf, which contains only this paper as a sibling. This leaf sits within the broader 'Neuroscience and Cognitive Systems Applications' branch, which includes two other leaves addressing ecological rationality and moral reasoning. The sparse population of this specific leaf suggests the approach occupies a relatively unexplored niche at the intersection of meta-learning, kernel methods, and computational neuroscience.

The taxonomy reveals three main branches: neuroscience applications, general-purpose meta-learning, and physics-informed networks. The original work's branch neighbors include ecological rationality models for human decision-making and moral reasoning frameworks, both applying meta-learned priors to cognitive systems but targeting different phenomena. The sibling branches—PAC-Bayes meta-learning for image classification and meta-learned optimization for physics-informed networks—share methodological elements (probabilistic priors, meta-learning) but diverge in application domain and constraint type. The taxonomy's scope notes clarify that this work specifically targets biological neural systems with theory-informed kernels, distinguishing it from general-purpose few-shot learning and hard physics constraints.

Among 22 candidates examined, the framework-level contribution (converting theories to probabilistic models) showed no clear refutation across 3 candidates. The task-adaptive deep kernel architecture examined 9 candidates and found 2 potentially refutable, suggesting moderate prior work in adaptive kernel methods. The Bayesian model comparison contribution examined 10 candidates with no refutations, indicating relative novelty in quantifying theory-data match. The limited search scope means these statistics reflect top semantic matches rather than exhaustive coverage. The architecture contribution appears most connected to existing work, while the framework and validation methods show fewer overlaps within the examined set.

Based on the 22-candidate search, the work appears to introduce a distinctive combination of meta-learned kernels, normative theory integration, and Bayesian validation for neuroscience applications. The sparse taxonomy leaf and contribution-level statistics suggest novelty in the specific synthesis, though the adaptive kernel component connects to established meta-learning literature. The analysis covers top semantic matches and does not claim exhaustive field coverage.

Taxonomy

Core-task Taxonomy Papers
4
3
Claimed Contributions
22
Contribution Candidate Papers Compared
2
Refutable Paper

Research Landscape Overview

Core task: integrating normative theories into data-driven models using meta-learned probabilistic priors. This field addresses a fundamental challenge in machine learning—how to incorporate domain knowledge, physical laws, or cognitive principles into flexible neural architectures without sacrificing generalization. The taxonomy reveals three main branches that reflect different application domains and methodological emphases. Neuroscience and Cognitive Systems Applications focus on modeling biological systems, particularly visual processing, by embedding theory-informed structure into kernel functions or network priors. General-Purpose Meta-Learning with Probabilistic Priors explores broader frameworks for learning task distributions and adapting quickly to new problems, often through hierarchical Bayesian approaches or generative models like Generative Meta-Learning[2]. Physics-Informed Neural Network Integration, exemplified by Physics-Informed Neural Networks[3], emphasizes embedding differential equations and conservation laws directly into loss functions or network architectures, ensuring that learned solutions respect known physical constraints. Across these branches, a recurring theme is the trade-off between inductive bias strength and data efficiency: stronger normative constraints can accelerate learning in low-data regimes but may limit flexibility when theories are approximate or incomplete. Within the Neuroscience and Cognitive Systems Applications branch, Deep Kernel Meta-Learning[0] sits at the intersection of meta-learning and domain-specific modeling, using theory-informed kernels to capture visual system properties while retaining the adaptability of meta-learned priors. This contrasts with more general frameworks like Generative Meta-Learning[2], which prioritize broad applicability over domain specificity, and with Physics-Informed Neural Networks[3], which enforce hard constraints rather than soft probabilistic priors. The original work thus occupies a niche where cognitive theory guides the structure of learned representations, balancing biological plausibility with the flexibility needed for diverse visual tasks.

Claimed Contributions

Bayesian meta-learning framework for converting normative theories into probabilistic models

The authors propose a framework that uses adaptive deep kernel Gaussian processes to meta-learn a kernel on synthetic data generated from normative theories. This Theory-Informed Kernel represents the theory predictions as a probabilistic model usable for fitting data and validating theories.

3 retrieved papers
Task-adaptive deep kernel architecture with frozen meta-learned features

The framework comprises a meta-learned feature extractor shared across tasks, task-adaptive linear heads, and task-adaptive Gaussian process layers. The meta-learned component learns an abstract metric embedding where distances are meaningful for theory-consistent functions, while task-adaptive components bridge the gap between simulated and real data.

9 retrieved papers
Can Refute
Method for quantifying theory-data match via Bayesian model comparison

The authors introduce an interpolated kernel that combines theory-informed and theory-agnostic components, enabling exact computation of marginal likelihoods for Bayesian model comparison. This allows rigorous information-theoretic quantification of how well a normative theory explains biological data, going beyond binary model selection to infer degrees of optimality.

10 retrieved papers

Core Task Comparisons

Comparisons with papers in the same taxonomy category

Within the taxonomy built over the current TopK core-task papers, the original paper is assigned to a leaf with no direct siblings and no cousin branches under the same grandparent topic. In this retrieved landscape, it appears structurally isolated, which is one partial signal of novelty, but still constrained by search coverage and taxonomy granularity.

Contribution Analysis

Detailed comparisons for each claimed contribution

Contribution

Bayesian meta-learning framework for converting normative theories into probabilistic models

The authors propose a framework that uses adaptive deep kernel Gaussian processes to meta-learn a kernel on synthetic data generated from normative theories. This Theory-Informed Kernel represents the theory predictions as a probabilistic model usable for fitting data and validating theories.

Contribution

Task-adaptive deep kernel architecture with frozen meta-learned features

The framework comprises a meta-learned feature extractor shared across tasks, task-adaptive linear heads, and task-adaptive Gaussian process layers. The meta-learned component learns an abstract metric embedding where distances are meaningful for theory-consistent functions, while task-adaptive components bridge the gap between simulated and real data.

Contribution

Method for quantifying theory-data match via Bayesian model comparison

The authors introduce an interpolated kernel that combines theory-informed and theory-agnostic components, enabling exact computation of marginal likelihoods for Bayesian model comparison. This allows rigorous information-theoretic quantification of how well a normative theory explains biological data, going beyond binary model selection to infer degrees of optimality.

Table of Contents