Omni-Weather: Unified Multimodal Foundation Model for Weather Generation and Understanding

ICLR 2026 Conference SubmissionAnonymous Authors

OpenReview Score: 6.0 Download Report PDF

AI for ScienceUnified foundation modelInterpretable reasoning

Weather modeling requires both accurate prediction and mechanistic interpretation, yet existing methods treat these goals in isolation, separating generation from understanding. To address this gap, we present Omni-Weather, the first multimodal foundation model that unifies weather generation and understanding within a single architecture. Omni-Weather integrates a radar encoder for weather generation tasks, followed by unified processing using a shared self-attention mechanism. Moreover, we construct a Chain-of-Thought dataset for causal reasoning in weather generation, enabling interpretable outputs and improved perceptual quality. Extensive experiments show Omni-Weather achieves state-of-the-art performance in both weather generation and understanding. Our findings further indicate that generative and understanding tasks in the weather domain can mutually enhance each other. Omni-Weather also demonstrates the feasibility and value of unifying weather generation and understanding.

Abstract:

Disclaimer

This report is AI-GENERATED using Large Language Models and WisPaper (A scholar search engine). It analyzes academic papers' tasks and contributions against retrieved prior work. While this system identifies POTENTIAL overlaps and novel directions, ITS COVERAGE IS NOT EXHAUSTIVE AND JUDGMENTS ARE APPROXIMATE. These results are intended to assist human reviewers and SHOULD NOT be relied upon as a definitive verdict on novelty.

NOTE that some papers exist in multiple, slightly different versions (e.g., with different titles or URLs). The system may retrieve several versions of the same underlying work. The current automated pipeline does not reliably align or distinguish these cases, so human reviewers will need to disambiguate them manually.

If you have any questions, please contact: mingzhang23@m.fudan.edu.cn

Overview

Overall Novelty Assessment

Omni-Weather proposes a unified multimodal foundation model integrating weather generation and understanding within a single architecture, incorporating a radar encoder and shared self-attention mechanism. The taxonomy places this work in the 'Unified Multimodal Weather Foundation Models' leaf, which currently contains only this paper among 50 total papers surveyed. This positioning indicates the work occupies a sparse, emerging research direction rather than a crowded subfield, suggesting the unified generation-understanding paradigm represents a relatively unexplored approach in weather modeling.

The taxonomy reveals that neighboring research directions pursue either generation or understanding in isolation. The 'Generative Models for Weather Scenarios' leaf focuses on synthesis without interpretability, while 'Global Foundation Models for Weather and Climate' emphasizes forecasting accuracy over multimodal integration. The 'Interpretability and Explainability in Weather Models' leaf addresses understanding but excludes generation tasks. Omni-Weather bridges these separated branches by combining radar-based generation with causal reasoning mechanisms, positioning itself at the intersection of multiple established but distinct research threads within the broader weather modeling landscape.

Among 30 candidates examined, the unified architecture contribution (Contribution A) showed no clear refutation across 10 papers reviewed, while the Chain-of-Thought dataset (Contribution B) similarly faced no overlapping prior work in 10 candidates. However, the mutual enhancement claim (Contribution C) encountered 3 refutable candidates among 10 examined, suggesting existing literature has explored task synergies in weather domains. The limited search scope means these statistics reflect top-semantic-match coverage rather than exhaustive field analysis, indicating moderate novelty for the architectural unification but less distinctiveness for the mutual enhancement observation.

The analysis suggests Omni-Weather introduces a relatively novel architectural paradigm given its isolated taxonomy position and lack of direct architectural precedents in the examined candidates. However, the mutual enhancement finding appears less distinctive, with multiple prior works exploring similar synergies. The 30-candidate search scope provides reasonable coverage of semantically proximate work but cannot guarantee comprehensive field assessment, particularly for emerging multimodal integration approaches that may use varied terminology or appear in adjacent domains.

Taxonomy

Core-task Taxonomy Papers

Claimed Contributions

Contribution Candidate Papers Compared

Refutable Paper

Research Landscape Overview

Core task: unified weather generation and understanding. The field encompasses a broad spectrum of approaches organized into several major branches. Data-Driven Weather Forecasting Models leverage machine learning to predict atmospheric states, often building on large-scale reanalysis datasets and transformer-based architectures such as ClimaX[2]. Physics-Based and Hybrid Modeling integrates numerical simulation with data-driven components, balancing interpretability and computational efficiency. Weather Generation and Simulation focuses on synthesizing realistic weather scenarios for applications like autonomous driving or scenario planning, exemplified by works such as Foggy Weather Simulation[12]. Multimodal Weather Understanding and Interpretation addresses the challenge of integrating diverse data sources—satellite imagery, radar, text, and sensor streams—into unified representations that support both forecasting and downstream tasks. Weather-Informed Application Domains span agriculture, energy systems, and public health, where weather inputs drive decision-making in crop yield prediction, photovoltaic forecasting, and air quality modeling. Space Weather and Geophysical Modeling extends these ideas to ionospheric and magnetospheric phenomena, while Methodological Foundations and Cross-Cutting Concerns address issues like uncertainty quantification, interpretability, and hybrid digital twins. Several active lines of work highlight contrasting emphases and open questions. Data-driven forecasting models continue to push the boundaries of skill and lead time, yet concerns about physical consistency and generalization remain prominent, as discussed in ML Weather Limitations[4] and ML Weather Climate Review[8]. Meanwhile, application-oriented studies such as Crop Yield Prediction[5], AI Weather Agriculture[9], and Probabilistic PV Forecasting[7] demonstrate the value of weather understanding for real-world decision support, though they often rely on domain-specific feature engineering rather than unified representations. Omni-Weather[0] sits within the Unified Multimodal Weather Foundation Models cluster, emphasizing the integration of heterogeneous weather data into a single framework that supports both generation and interpretation tasks. This positions it alongside efforts like ClimaX[2] and AI Enhanced Global Weather[11], which also pursue foundation-model paradigms, but Omni-Weather[0] places stronger emphasis on multimodal fusion and the joint handling of generation and understanding, distinguishing it from purely forecasting-focused or application-specific approaches.

Claimed Contributions

Omni-Weather unified multimodal foundation model

10 retrieved papers

The authors introduce Omni-Weather, a unified foundation model that integrates both weather generation tasks (such as radar nowcasting and inversion) and weather understanding tasks (such as diagnostic reasoning and question answering) within a single shared architecture, marking the first such integration in the weather domain.

10 retrieved papers

Chain-of-Thought dataset for causal reasoning in weather generation

10 retrieved papers

The authors construct a specialized Chain-of-Thought dataset tailored for causal reasoning in weather generation tasks. This dataset enables the model to produce interpretable reasoning traces and enhances the perceptual quality of generated weather forecasts.

10 retrieved papers

Demonstration of mutual enhancement between generation and understanding tasks

Can Refute

10 retrieved papers

The authors show that jointly training weather generation and understanding tasks within the unified framework provides complementary supervision signals, leading to improved performance on both task categories and enabling the model to learn more transferable representations of atmospheric phenomena.

10 retrieved papers

Can Refute

Core Task Comparisons

Comparisons with papers in the same taxonomy category

Within the taxonomy built over the current TopK core-task papers, the original paper is assigned to a leaf with no direct siblings and no cousin branches under the same grandparent topic. In this retrieved landscape, it appears structurally isolated, which is one partial signal of novelty, but still constrained by search coverage and taxonomy granularity.

Contribution Analysis

Detailed comparisons for each claimed contribution

Contribution