Exploratory Causal Inference in SAEnce

ICLR 2026 Conference SubmissionAnonymous Authors
Randomized Controlled TrialsSparse Auto EncoderInterpretabilityCausal Inference
Abstract:

Randomized Controlled Trials are one of the pillars of science; nevertheless, they rely on hand-crafted hypotheses and expensive analysis. Such constraints prevent causal effect estimation at scale, potentially anchoring on popular yet incomplete hypotheses. We propose to discover the unknown effects of a treatment directly from data. For this, we turn unstructured data from a trial into meaningful representations via pretrained foundation models and interpret them via a Sparse Auto Encoder. However, discovering significant causal effects at the neural level is not trivial due to multiple-testing issues and effects entanglement. To address these challenges, we introduce Neural Effect Search, a novel recursive procedure solving both issues by progressive stratification. After assessing the robustness of our algorithm on semi-synthetic experiments, we showcase, in the context of experimental ecology, the first successful unsupervised causal effect identification on a real-world scientific trial.

Disclaimer
This report is AI-GENERATED using Large Language Models and WisPaper (A scholar search engine). It analyzes academic papers' tasks and contributions against retrieved prior work. While this system identifies POTENTIAL overlaps and novel directions, ITS COVERAGE IS NOT EXHAUSTIVE AND JUDGMENTS ARE APPROXIMATE. These results are intended to assist human reviewers and SHOULD NOT be relied upon as a definitive verdict on novelty.
NOTE that some papers exist in multiple, slightly different versions (e.g., with different titles or URLs). The system may retrieve several versions of the same underlying work. The current automated pipeline does not reliably align or distinguish these cases, so human reviewers will need to disambiguate them manually.
If you have any questions, please contact: mingzhang23@m.fudan.edu.cn

Overview

Taxonomy

Core-task Taxonomy Papers
50
3
Claimed Contributions
24
Contribution Candidate Papers Compared
0
Refutable Paper

Research Landscape Overview

Core task: unsupervised causal effect discovery from high-dimensional experimental data. The field addresses the challenge of identifying causal relationships and treatment effects when labeled outcome data or explicit intervention annotations are scarce or absent, particularly in settings where the dimensionality of covariates is large. The taxonomy reflects a broad landscape organized around six main branches. Causal Structure Discovery from Observational Data focuses on learning directed acyclic graphs and structural equation models from passive observations, often leveraging independence constraints or functional form assumptions. Treatment Effect Estimation and Causal Inference encompasses methods for quantifying intervention impacts, including both supervised approaches that rely on known treatment assignments and unsupervised variants that must infer effects without direct outcome labels. Semi-Supervised and Unlabeled Causal Inference explores hybrid settings where partial supervision or positive-unlabeled data guides discovery. Causal Inference Under Distribution Shift tackles robustness when training and test distributions differ, while Causal Feature Selection and Discovery aims to identify causally relevant variables in high-dimensional spaces. Specialized Causal Inference Methods includes domain-specific techniques and novel algorithmic frameworks, such as those integrating large language models or leveraging representation learning. Several active lines of work highlight key trade-offs and open questions. One prominent theme is the tension between model flexibility and identifiability: representation learning approaches like Identifiable Causal Representation[11] and Latent Invariant Mechanism[29] seek to uncover latent causal factors from complex observations, yet must impose structural constraints to ensure uniqueness. Another contrast appears between methods that assume known interventions versus those that operate in fully unsupervised regimes. SAEnce Causal Inference[0] sits within the Unsupervised Causal Effect Discovery cluster, emphasizing the extraction of causal signals from experimental data without explicit outcome labels. It shares this unsupervised orientation with Correlation to Causation[4], which also aims to move beyond associational patterns, and contrasts with semi-supervised strategies like Semi-supervised Misspecification[5] that blend labeled and unlabeled information. The original work's focus on high-dimensional experimental settings positions it at the intersection of structure discovery and effect estimation, addressing scenarios where traditional supervised methods are infeasible yet experimental perturbations provide crucial leverage for causal identification.

Claimed Contributions

Formal differentiation of rationalist and empiricist approaches to causal inference

The authors establish a formal framework distinguishing between rationalist approaches (hypothesis-driven causal inference with predefined outcomes) and empiricist approaches (data-driven discovery of treatment effects). They characterize these paradigms within statistical causality, showing how they complement each other in scientific discovery.

10 retrieved papers
Novel empiricist methodology using foundation models and sparse autoencoders

The authors introduce a methodology that combines pretrained foundation models with sparse autoencoders to discover treatment effects in exploratory experiments. They identify and formalize the paradox of exploratory causal inference, showing how standard multiple testing fails when neural representations are entangled.

10 retrieved papers
Neural Effect Search algorithm for iterative hypothesis testing

The authors develop Neural Effect Search, a recursive stratification procedure that addresses multiple-testing issues and effect entanglement in neural representations. The algorithm iteratively identifies significant causal effects while controlling for dependencies between neurons through progressive stratification.

4 retrieved papers

Core Task Comparisons

Comparisons with papers in the same taxonomy category

Contribution Analysis

Detailed comparisons for each claimed contribution

Contribution

Formal differentiation of rationalist and empiricist approaches to causal inference

The authors establish a formal framework distinguishing between rationalist approaches (hypothesis-driven causal inference with predefined outcomes) and empiricist approaches (data-driven discovery of treatment effects). They characterize these paradigms within statistical causality, showing how they complement each other in scientific discovery.

Contribution

Novel empiricist methodology using foundation models and sparse autoencoders

The authors introduce a methodology that combines pretrained foundation models with sparse autoencoders to discover treatment effects in exploratory experiments. They identify and formalize the paradox of exploratory causal inference, showing how standard multiple testing fails when neural representations are entangled.

Contribution

Neural Effect Search algorithm for iterative hypothesis testing

The authors develop Neural Effect Search, a recursive stratification procedure that addresses multiple-testing issues and effect entanglement in neural representations. The algorithm iteratively identifies significant causal effects while controlling for dependencies between neurons through progressive stratification.

Exploratory Causal Inference in SAEnce | Novelty Validation