OpenNovelty.org

OD3^3: Optimization-free Dataset Distillation for Object Detection

ICLR 2026 Conference SubmissionAnonymous Authors
OpenReview Score: 6.0Download Report PDF
dataset distillationobject detectiondata-centric frameworkefficient machine learning
Abstract:

Training large neural networks on large-scale datasets requires substantial computational resources, particularly for dense prediction tasks such as object detection. Although dataset distillation (DD) has been proposed to alleviate these demands by synthesizing compact datasets from larger ones, most existing work focuses solely on image classification, leaving the more complex detection setting largely unexplored. In this paper, we introduce OD3^3, a novel optimization-free data distillation framework specifically designed for object detection. Our approach involves two stages: first, a candidate selection process in which object instances are iteratively placed in synthesized images based on their suitable locations, and second, a candidate screening process using a pre-trained observer model to remove low-confidence objects. We perform our data synthesis framework on MS COCO and PASCAL VOC, two popular detection datasets, with compression ratios ranging from 0.25% to 5%. Compared to the prior solely existing dataset distillation method on detection and conventional core set selection methods, OD3^3 delivers superior accuracy, establishes new state-of-the-art results, surpassing prior best method by more than 14% on COCO mAP50_{50} at a compression ratio of 1.0%. The code is in the supplementary material.

Disclaimer
This report is AI-GENERATED using Large Language Models and WisPaper (A scholar search engine). It analyzes academic papers' tasks and contributions against retrieved prior work. While this system identifies POTENTIAL overlaps and novel directions, ITS COVERAGE IS NOT EXHAUSTIVE AND JUDGMENTS ARE APPROXIMATE. These results are intended to assist human reviewers and SHOULD NOT be relied upon as a definitive verdict on novelty.
NOTE that some papers exist in multiple, slightly different versions (e.g., with different titles or URLs). The system may retrieve several versions of the same underlying work. The current automated pipeline does not reliably align or distinguish these cases, so human reviewers will need to disambiguate them manually.
If you have any questions, please contact: mingzhang23@m.fudan.edu.cn

Overview

Overall Novelty Assessment

The paper introduces OD³, an optimization-free dataset distillation framework for object detection that synthesizes compact datasets through candidate selection and screening. It occupies the 'Optimization-Free Dataset Distillation' leaf within the taxonomy, which currently contains only this work as a sibling. This places the paper in a sparse research direction, distinct from the densely populated knowledge distillation branches (e.g., Feature-Based Distillation with multiple papers) and the broader Synthetic Dataset Generation category (eight papers). The framework targets MS COCO and PASCAL VOC with compression ratios from 0.25% to 5%, addressing computational demands in dense prediction tasks.

The taxonomy reveals that OD³ sits within 'Data-Free and Synthetic Data Generation for Detection,' adjacent to 'Data-Free Knowledge Distillation' (one paper on synthesizing images from teacher networks) and 'Synthetic Dataset Generation for Object Detection' (eight papers using rendering, GANs, or compositing). Unlike the knowledge distillation branches that focus on teacher-student feature transfer (e.g., Localization-Focused Distillation with three papers), OD³ emphasizes data synthesis without iterative optimization. The exclude_note clarifies that it differs from data-free distillation methods requiring teacher networks and from general synthetic generation approaches, carving a distinct methodological niche.

Among 23 candidates examined, no contributions were clearly refuted. The core OD³ framework examined three candidates with zero refutable overlaps; the two-stage synthesis process examined ten candidates with none refutable; and the Scale-Aware Dynamic Context Extension (SA-DCE) component examined ten candidates, also with zero refutable matches. This suggests that within the limited search scope—top-K semantic matches plus citation expansion—the specific combination of optimization-free distillation, candidate selection/screening, and scale-aware context extension appears novel. However, the search scale (23 papers) is modest relative to the broader detection literature.

Based on the limited literature search, OD³ appears to occupy a relatively unexplored intersection: dataset distillation specifically for object detection without gradient-based optimization. The taxonomy structure shows this is a sparse leaf compared to crowded knowledge distillation branches, and the contribution-level analysis found no direct prior work among examined candidates. Nonetheless, the search scope (23 papers) leaves open the possibility of related work in adjacent areas not captured by semantic similarity or citation links.

Taxonomy

Core-task Taxonomy Papers
50
3
Claimed Contributions
23
Contribution Candidate Papers Compared
0
Refutable Paper

Research Landscape Overview

Core task: dataset distillation for object detection. The field organizes around three main branches that reflect different strategies for compressing or generating training data while preserving detection performance. Knowledge Distillation Methods for Object Detection encompasses a dense cluster of works that transfer learned representations from teacher to student detectors, often focusing on feature alignment, localization cues (e.g., Localization Distillation[3], Localization Distillation Dense[5]), or relational structures (e.g., Relation distillation networks[4], Dense relation distillation[17]). Cross-Modality and Domain-Specific Distillation addresses scenarios where data comes from heterogeneous sensors or specialized domains, requiring tailored transfer mechanisms. Data-Free and Synthetic Data Generation for Detection explores creating or distilling datasets without access to original training samples, leveraging generative models or optimization-free techniques to produce compact yet informative synthetic sets (e.g., Synthetic Dataset Generation[6], DefectGAN[36]). A particularly active line of work within knowledge distillation investigates how to balance multiple detection objectives—classification, localization, and instance-level reasoning—leading to methods like Task-balanced distillation[1] and General Instance Distillation[2]. Meanwhile, the synthetic data generation branch grapples with trade-offs between realism, diversity, and computational cost, as seen in domain-specific efforts for fraud detection (Synthetic Fraud Diffusion[15]) or specialized visual tasks (Synthetic Grape Anomaly[13]). OD3[0] sits within the optimization-free subset of Data-Free and Synthetic Data Generation, emphasizing efficient distillation without iterative gradient-based refinement. This contrasts with optimization-heavy approaches in the knowledge distillation branch and positions OD3[0] closer to works that prioritize scalability and reduced computational overhead, such as Synthetic Dataset Generation[6], while differing from the feature-matching focus of Localization Distillation[3] or the relational emphasis of methods like Dual Relation Distillation[19].

Claimed Contributions

OD3: Optimization-free Dataset Distillation Framework for Object Detection

The authors propose OD3, a two-stage framework that synthesizes compact datasets for object detection without requiring complex optimization procedures. The framework uses candidate selection to place object instances and candidate screening via a pre-trained observer model to filter low-confidence objects.

3 retrieved papers
Two-stage synthesis process with candidate selection and screening

The method introduces a deliberate two-stage process where candidate selection strategically places masked objects with minimal overlap, followed by candidate screening that uses a pre-trained detector to remove unreliable or low-confidence object candidates from the synthesized images.

10 retrieved papers
Scale-aware Dynamic Context Extension (SA-DCE)

The authors introduce SA-DCE, a mechanism that dynamically extends the bounding region around objects as a function of their size. This enhancement provides additional contextual information, particularly benefiting small objects that typically have limited context in detection tasks.

10 retrieved papers

Core Task Comparisons

Comparisons with papers in the same taxonomy category

Within the taxonomy built over the current TopK core-task papers, the original paper is assigned to a leaf with no direct siblings and no cousin branches under the same grandparent topic. In this retrieved landscape, it appears structurally isolated, which is one partial signal of novelty, but still constrained by search coverage and taxonomy granularity.

Contribution Analysis

Detailed comparisons for each claimed contribution

Contribution

OD3: Optimization-free Dataset Distillation Framework for Object Detection

The authors propose OD3, a two-stage framework that synthesizes compact datasets for object detection without requiring complex optimization procedures. The framework uses candidate selection to place object instances and candidate screening via a pre-trained observer model to filter low-confidence objects.

Contribution

Two-stage synthesis process with candidate selection and screening

The method introduces a deliberate two-stage process where candidate selection strategically places masked objects with minimal overlap, followed by candidate screening that uses a pre-trained detector to remove unreliable or low-confidence object candidates from the synthesized images.

Contribution

Scale-aware Dynamic Context Extension (SA-DCE)

The authors introduce SA-DCE, a mechanism that dynamically extends the bounding region around objects as a function of their size. This enhancement provides additional contextual information, particularly benefiting small objects that typically have limited context in detection tasks.

Table of Contents

OD$^3$: Optimization-free Dataset Distillation for Object Detection | Novelty Validation