OpenNovelty.org

Representational Alignment Across Model Layers and Brain Regions with Hierarchical Optimal Transport

ICLR 2026 Conference SubmissionAnonymous Authors
OpenReview Score: 7.3Download Report PDF
Representation SimilarityRepresentational Alignment
Abstract:

Standard representational similarity methods align each layer of a network to its best match in another independently, producing asymmetric results, lacking a global alignment score, and struggling with networks of different depths. These limitations arise from ignoring global activation structure and restricting mappings to rigid one-to-one layer correspondences. We propose Hierarchical Optimal Transport (HOT), a unified framework that jointly infers soft, globally consistent layer-to-layer couplings and neuron-level transport plans. HOT allows source neurons to distribute mass across multiple target layers while minimizing total transport cost under marginal constraints. This yields both a single alignment score for the entire network comparison and a soft transport plan that naturally handles depth mismatches through mass distribution. We evaluate HOT on vision models, large language models, and human visual cortex recordings. Across all domains, HOT matches or surpasses standard pairwise matching in alignment quality. Moreover, it reveals smooth, fine-grained hierarchical correspondences: early layers map to early layers, deeper layers maintain relative positions, and depth mismatches are resolved by distributing representations across multiple layers. These structured patterns emerge naturally from global optimization without being imposed, yet are absent in greedy layer-wise methods. HOT thus enables richer, more interpretable comparisons between representations, particularly when networks differ in architecture or depth. We further extend our method to a three-level HOT framework, providing a proof-of-concept alignment of two networks across their training trajectories and demonstrating that HOT uncovers checkpoint-wise correspondences missed by greedy layer-wise matching.

Disclaimer
This report is AI-GENERATED using Large Language Models and WisPaper (A scholar search engine). It analyzes academic papers' tasks and contributions against retrieved prior work. While this system identifies POTENTIAL overlaps and novel directions, ITS COVERAGE IS NOT EXHAUSTIVE AND JUDGMENTS ARE APPROXIMATE. These results are intended to assist human reviewers and SHOULD NOT be relied upon as a definitive verdict on novelty.
NOTE that some papers exist in multiple, slightly different versions (e.g., with different titles or URLs). The system may retrieve several versions of the same underlying work. The current automated pipeline does not reliably align or distinguish these cases, so human reviewers will need to disambiguate them manually.
If you have any questions, please contact: mingzhang23@m.fudan.edu.cn

Overview

Overall Novelty Assessment

The paper proposes Hierarchical Optimal Transport (HOT), a framework for aligning neural representations across networks by jointly inferring soft layer-to-layer couplings and neuron-level transport plans. It resides in the 'Hierarchical Network-Brain Correspondence' leaf of the taxonomy, which currently contains no sibling papers. This leaf sits within the broader 'DNN-Brain Correspondence and Alignment' branch, which includes six papers on vision model-brain alignment, two on language models, and one on auditory models. The sparse population of this specific leaf suggests HOT addresses a relatively underexplored niche within the larger alignment literature.

The taxonomy reveals that most DNN-brain alignment work focuses on single-modality comparisons (vision, language, auditory) rather than hierarchical correspondence methods. Neighboring leaves like 'Vision Model-Brain Alignment' contain six papers examining layer-to-region mappings, while 'Representation Similarity and Alignment Metrics' addresses quantitative comparison methods. The 'Cross-Subject and Cross-Species Neural Alignment' branch (seven papers across three leaves) tackles biological alignment without DNN involvement. HOT's hierarchical transport approach bridges these areas by providing a unified metric applicable to both artificial networks and brain recordings, positioning it at the intersection of representation similarity metrics and hierarchical correspondence.

Among the 14 candidates examined through limited semantic search, none clearly refute any of HOT's three contributions. The core HOT framework examined one candidate with no refutable overlap. The rotation-invariant extension reviewed five candidates, all non-refutable or unclear. The three-level training trajectory alignment examined eight candidates, again with no clear prior work. This suggests that within the examined scope, HOT's specific combination of hierarchical optimal transport with soft layer couplings and global alignment scores represents a novel methodological direction, though the limited search scale (14 papers) means substantial related work may exist beyond this analysis.

Based on the examined literature, HOT appears to introduce a distinctive approach to hierarchical alignment, particularly in its treatment of depth mismatches and global consistency. However, the analysis covers only top-14 semantic matches plus citation expansion, not an exhaustive survey. The sparse population of its taxonomy leaf and absence of refutable candidates within this scope suggest novelty, but a broader search across the 50-paper taxonomy and beyond would be needed to fully assess whether similar transport-based hierarchical methods exist in adjacent research areas.

Taxonomy

Core-task Taxonomy Papers
50
3
Claimed Contributions
14
Contribution Candidate Papers Compared
0
Refutable Paper

Research Landscape Overview

Core task: Aligning neural representations across networks and brain regions. This field addresses how to compare and map representational structures between different neural systems—whether across subjects, species, or between artificial deep neural networks (DNNs) and biological brains. The taxonomy reflects a multifaceted landscape: one major branch focuses on cross-subject and cross-species alignment, establishing correspondences despite anatomical and functional variability. Another prominent branch examines DNN-brain correspondence, exploring how hierarchical layers in artificial networks relate to processing stages in sensory cortices, as seen in works like Brain2GAN[14] and BrainAlign[21]. Additional branches address representation similarity metrics, neural encoding and decoding models, and the spatial or temporal organization of neural codes. Complementary directions investigate multisensory integration, clinical network analysis, and theoretical frameworks that unify these diverse alignment challenges. Within the DNN-brain correspondence branch, a particularly active line of work explores hierarchical network-brain alignment, seeking to match layer-by-layer representations in DNNs with neural activity patterns across cortical hierarchies. Hierarchical Optimal Transport[0] sits squarely in this area, proposing a principled method to align representations at multiple levels of abstraction. This contrasts with approaches like Unsupervised Alignment Commonalities[1], which emphasizes discovering shared structure without explicit supervision, and Multilevel Integration Enhancement[3], which focuses on integrating information across hierarchical stages to improve correspondence. Meanwhile, related efforts such as Language Modulates Vision[4] and Convergent Speech Representations[8] highlight how modality-specific processing influences alignment strategies. The central challenge remains balancing the flexibility to capture diverse representational geometries with the constraint of preserving hierarchical structure, a trade-off that Hierarchical Optimal Transport[0] addresses through its layered transport framework.

Claimed Contributions

Hierarchical Optimal Transport (HOT) framework

HOT is a novel framework for representational alignment that simultaneously optimizes neuron-to-neuron couplings within layers and layer-to-layer couplings across network hierarchies. Unlike standard pairwise methods, it allows source neurons to distribute mass across multiple target layers while enforcing global consistency through marginal constraints, producing a single network-level alignment score and naturally handling depth mismatches.

1 retrieved paper
Rotation-invariant extension of HOT

This extension augments HOT with learned rotation matrices for each layer pair, enabling the framework to recover correspondences even when shared representational features are embedded in rotated subspaces. The method alternates between optimizing transport couplings and orthogonal Procrustes alignment, ensuring geometric equivalences are properly captured.

5 retrieved papers
Three-level HOT for training trajectory alignment

This extension adds a third hierarchical level to HOT that operates over training checkpoints, enabling alignment of entire training trajectories between two models. The method solves an additional optimal transport problem over checkpoint-level costs derived from two-level HOT comparisons at each checkpoint pair.

8 retrieved papers

Core Task Comparisons

Comparisons with papers in the same taxonomy category

Within the taxonomy built over the current TopK core-task papers, the original paper is assigned to a leaf with no direct siblings and no cousin branches under the same grandparent topic. In this retrieved landscape, it appears structurally isolated, which is one partial signal of novelty, but still constrained by search coverage and taxonomy granularity.

Contribution Analysis

Detailed comparisons for each claimed contribution

Contribution

Hierarchical Optimal Transport (HOT) framework

HOT is a novel framework for representational alignment that simultaneously optimizes neuron-to-neuron couplings within layers and layer-to-layer couplings across network hierarchies. Unlike standard pairwise methods, it allows source neurons to distribute mass across multiple target layers while enforcing global consistency through marginal constraints, producing a single network-level alignment score and naturally handling depth mismatches.

Contribution

Rotation-invariant extension of HOT

This extension augments HOT with learned rotation matrices for each layer pair, enabling the framework to recover correspondences even when shared representational features are embedded in rotated subspaces. The method alternates between optimizing transport couplings and orthogonal Procrustes alignment, ensuring geometric equivalences are properly captured.

Contribution

Three-level HOT for training trajectory alignment

This extension adds a third hierarchical level to HOT that operates over training checkpoints, enabling alignment of entire training trajectories between two models. The method solves an additional optimal transport problem over checkpoint-level costs derived from two-level HOT comparisons at each checkpoint pair.

Table of Contents