OpenNovelty.org

Bound by semanticity: universal laws governing the generalization-identification tradeoff

ICLR 2026 Conference SubmissionAnonymous Authors
OpenReview Score: 7.5Download Report PDF
generalizationmulti-object reasoningcognitive science
Abstract:

Intelligent systems must form internal representations that support both broad generalization and precise identification. Here, we show that these two goals are fundamentally in tension with one another. We derive closed-form expressions proving that any model whose representations have a finite semantic resolution, impairing long-range similarity computations, must lie on a universal Pareto front linking its probability of correct generalization pSp_S and identification pIp_I. We extend this analysis to general input spaces and to parallel processing scenarios, predicting a sharp 1/n1/n collapse in the capacity of processing multiple inputs at the same time. A minimal ReLU network reproduces these laws: a resolution boundary emerges during learning, and empirical (pS,pI)(p_S,p_I) trajectories closely match the theory for linearly decaying similarity. Finally, we show that the same limits appear in far more complex systems, including a convolutional neural network and state-of-the-art vision–language models, indicating that learned finite-resolution similarity are broad and foundational informational constraints rather than toy-model artifacts. Together, these results provide a precise theory of the generalization–identification tradeoff and clarify how semantic resolution shapes the representational capacity of deep networks and brains alike.

Disclaimer
This report is AI-GENERATED using Large Language Models and WisPaper (A scholar search engine). It analyzes academic papers' tasks and contributions against retrieved prior work. While this system identifies POTENTIAL overlaps and novel directions, ITS COVERAGE IS NOT EXHAUSTIVE AND JUDGMENTS ARE APPROXIMATE. These results are intended to assist human reviewers and SHOULD NOT be relied upon as a definitive verdict on novelty.
NOTE that some papers exist in multiple, slightly different versions (e.g., with different titles or URLs). The system may retrieve several versions of the same underlying work. The current automated pipeline does not reliably align or distinguish these cases, so human reviewers will need to disambiguate them manually.
If you have any questions, please contact: mingzhang23@m.fudan.edu.cn

Overview

Overall Novelty Assessment

The paper derives closed-form expressions for a fundamental Pareto front linking generalization probability and identification probability under finite semantic resolution constraints. It sits within the 'Fundamental Tradeoff Principles' leaf of the taxonomy, which contains only three papers total. This is a sparse research direction within the broader 'Theoretical Foundations and Formal Analysis' branch, suggesting the work addresses a relatively underexplored theoretical question. The sibling papers focus on quadratic function identifiability and adversarial training dynamics, indicating that formal tradeoff principles remain an active but not yet crowded area of inquiry.

The taxonomy tree reveals that neighboring leaves examine representational dimensionality, optimization dynamics, and empirical characterization of learned representations. The paper's theoretical focus on semantic resolution and Pareto frontiers distinguishes it from these adjacent directions, which emphasize geometric properties or training trajectories. The 'Empirical Characterization' branch contains substantially more papers across multiple subtopics, highlighting that while empirical studies of generalization are abundant, formal mathematical frameworks governing tradeoffs remain less developed. The paper's extension to vision-language models bridges this theoretical work with the empirical domain.

Among twelve candidates examined through limited semantic search, none clearly refute the three main contributions. The first contribution (closed-form Pareto expressions) examined two candidates with no refutations; the second (1/n collapse prediction) examined ten candidates with no refutations; the third (exact Pareto quantification framework) examined zero candidates. This suggests that within the top-K semantic neighborhood, the specific mathematical formulations and predictions appear novel. However, the limited search scope means more distant or differently framed prior work may exist outside this candidate set, particularly in information theory or neuroscience literatures not captured by the semantic search.

Based on the available signals, the work appears to occupy a relatively sparse theoretical niche, with formal tradeoff principles less saturated than empirical representation studies. The absence of refutations among twelve examined candidates, combined with the small sibling set in the taxonomy leaf, suggests the specific mathematical framework is not directly anticipated by closely related work. However, the analysis covers only top-K semantic matches and does not exhaustively survey adjacent fields like rate-distortion theory or cognitive neuroscience, where related principles might exist under different terminology.

Taxonomy

Core-task Taxonomy Papers
50
3
Claimed Contributions
12
Contribution Candidate Papers Compared
0
Refutable Paper

Research Landscape Overview

Core task: understanding the generalization-identification tradeoff in neural representations. This field examines how neural networks balance the ability to generalize across diverse inputs with the capacity to identify or discriminate specific instances, a tension that arises in many learning scenarios. The taxonomy organizes research into five main branches: Theoretical Foundations and Formal Analysis explores fundamental principles governing this tradeoff, often through mathematical frameworks and information-theoretic perspectives; Empirical Characterization of Learned Representations investigates how real networks encode and organize information, revealing patterns in representation geometry and selectivity; Architecture-Specific Analysis and Design examines how particular model structures (e.g., convolutional networks, transformers) shape the tradeoff; Application-Driven Methods and Domain-Specific Solutions address practical challenges in areas like computer vision, neuroscience, and signal processing; and Generalization Metrics and Prediction develops tools to measure and forecast model behavior. Works like Mixed Selectivity Tradeoff[8] and High Frequency Generalization[4] illustrate early theoretical insights, while more recent studies such as Representation Compression Generalization[34] and Identifying Generalization Properties[30] bridge theory and empirical observation. Several active lines of work highlight contrasting emphases and open questions. One thread focuses on the role of representation geometry and dimensionality in enabling flexible computation, as seen in Dimensionality Neural Control[5] and Representational Spaces Geometry[39], which explore how neural codes support both task-specific discrimination and cross-context transfer. Another examines domain-specific instantiations of the tradeoff, from visual recognition (Category Viewpoint Generalization[14]) to neuroscience-inspired models (Working Memory Flexibility[26]). The original paper, Semanticity Generalization Tradeoff[0], sits within the Theoretical Foundations branch alongside works like Quadratic Function Identification[7] and Discrimination Generalization GANs[31]. While Quadratic Function Identification[7] emphasizes formal identifiability constraints and Discrimination Generalization GANs[31] probes adversarial training dynamics, Semanticity Generalization Tradeoff[0] appears to articulate a broader principle linking semantic structure to generalization capacity, offering a conceptual lens that complements these more specialized analyses.

Claimed Contributions

Closed-form expressions for generalization-identification Pareto front under finite semantic resolution

The authors provide exact mathematical formulas (Theorems 1-3) that characterize the fundamental tradeoff between a model's ability to generalize (similarity judgments) and identify (distinguish) stimuli when representations have limited resolution. These expressions reveal a universal constraint independent of specific model architectures.

2 retrieved papers
Theory predicting 1/n collapse in multi-item processing capacity

The authors extend their theoretical framework to scenarios with multiple simultaneous inputs, deriving formulas showing that identification performance degrades proportionally to 1/n as the number of items n increases, providing a principled explanation for capacity limits in multi-object reasoning.

10 retrieved papers
Framework quantifying exact Pareto front between identification and similarity performances

The authors introduce a formal framework with closed-form solutions that precisely characterizes the unavoidable tension between representational distinctness and similarity under finite resolution constraints, applicable across different stimulus spaces and probability distributions.

0 retrieved papers

Core Task Comparisons

Comparisons with papers in the same taxonomy category

Contribution Analysis

Detailed comparisons for each claimed contribution

Contribution

Closed-form expressions for generalization-identification Pareto front under finite semantic resolution

The authors provide exact mathematical formulas (Theorems 1-3) that characterize the fundamental tradeoff between a model's ability to generalize (similarity judgments) and identify (distinguish) stimuli when representations have limited resolution. These expressions reveal a universal constraint independent of specific model architectures.

Contribution

Theory predicting 1/n collapse in multi-item processing capacity

The authors extend their theoretical framework to scenarios with multiple simultaneous inputs, deriving formulas showing that identification performance degrades proportionally to 1/n as the number of items n increases, providing a principled explanation for capacity limits in multi-object reasoning.

Contribution

Framework quantifying exact Pareto front between identification and similarity performances

The authors introduce a formal framework with closed-form solutions that precisely characterizes the unavoidable tension between representational distinctness and similarity under finite resolution constraints, applicable across different stimulus spaces and probability distributions.

Table of Contents