MetaEmbed: Scaling Multimodal Retrieval at Test-Time with Flexible Late Interaction

The paper introduces MetaEmbed, a framework that uses learnable Meta Tokens to produce compact multi-vector embeddings for multimodal retrieval, combined with Matryoshka Multi-Vector Retrieval training to organize information by granularity. It resides in the Token-Level Interaction and Matching Mechanisms leaf, which contains five papers total, including the original work. This leaf sits within the broader Late Interaction Architecture Design and Optimization branch, indicating a moderately populated research direction focused on fine-grained matching strategies rather than single-vector compression or domain-specific applications.

The taxonomy reveals three sibling leaves within Late Interaction Architecture Design: Embedding Efficiency and Scalability (two papers on sparsification and compression), and Unified Multimodal Embedding Models (three papers on joint text-image architectures). Neighboring branches address Domain-Specific Applications (e.g., visual question answering, e-commerce) and Multimodal Fusion Strategies (adaptive fusion, cross-modal adapters). MetaEmbed's focus on flexible token-level interaction distinguishes it from efficiency-centric methods like PyLate and from unified embedding models that prioritize single-vector objectives, positioning it as an architectural innovation within the late-interaction paradigm.

Among twenty-seven candidates examined via semantic search, none were flagged as clearly refuting any of the three contributions. The MetaEmbed framework and Meta Tokens concept examined ten candidates with zero refutable overlaps; the Matryoshka Multi-Vector Retrieval training method also examined ten candidates with no refutations; and the test-time scaling mechanism examined seven candidates, again with no refutations. This suggests that within the limited search scope—top-K semantic matches plus citation expansion—no prior work was found to substantially overlap with the proposed techniques, though the analysis does not claim exhaustive coverage of the entire field.

Given the moderate density of the Token-Level Interaction leaf and the absence of refutable prior work among the examined candidates, the contributions appear relatively novel within the scope analyzed. However, the search examined only twenty-seven papers, and the taxonomy contains twenty-eight total papers across ten leaves, so broader or deeper literature searches might uncover additional related work. The assessment reflects what is visible in the current taxonomy structure and candidate set, not a definitive claim about the entire multimodal retrieval landscape.