OpenNovelty.org

Protection against Source Inference Attacks in Federated Learning

ICLR 2026 Conference SubmissionAnonymous Authors
OpenReview Score: 6.0Download Report PDF
Federated LearningSource Inference AttackShuffle ModelResidue Number System
Abstract:

Federated Learning (FL) was initially proposed as a privacy-preserving machine learning paradigm. However, FL has been shown to be susceptible to a series of privacy attacks. Recently, there has been concern about the Source Inference Attack (SIA), where an honest-but-curious central server attempts to identify exactly which client owns a given data point which was used in the training phase. Alarmingly, standard gradient obfuscation techniques with Differential Privacy have been shown to be ineffective against SIAs, at least without severely diminishing the accuracy.

In this work, we propose a defense against SIAs within the widely studied shuffle model of FL, where an honest shuffler acts as an intermediary between the clients and the server. First, we demonstrate that standard naive shuffling alone is insufficient to prevent SIAs. To effectively defend against SIAs, shuffling needs to be applied at a more granular level; we propose a novel combination of parameter-level shuffling with the residue number system (RNS). Our approach provides robust protection against SIAs without affecting the accuracy of the joint model and can be seamlessly integrated into other privacy protection mechanisms.

We conduct experiments on a series of models and datasets, confirming that standard shuffling approaches fail to prevent SIAs and that, in contrast, our proposed method reduce the attack’s accuracy to the level of random guessing.

Disclaimer
This report is AI-GENERATED using Large Language Models and WisPaper (A scholar search engine). It analyzes academic papers' tasks and contributions against retrieved prior work. While this system identifies POTENTIAL overlaps and novel directions, ITS COVERAGE IS NOT EXHAUSTIVE AND JUDGMENTS ARE APPROXIMATE. These results are intended to assist human reviewers and SHOULD NOT be relied upon as a definitive verdict on novelty.
NOTE that some papers exist in multiple, slightly different versions (e.g., with different titles or URLs). The system may retrieve several versions of the same underlying work. The current automated pipeline does not reliably align or distinguish these cases, so human reviewers will need to disambiguate them manually.
If you have any questions, please contact: mingzhang23@m.fudan.edu.cn

Overview

Overall Novelty Assessment

The paper proposes a defense against source inference attacks in federated learning using parameter-level shuffling combined with the residue number system. It resides in the 'Source and Membership Inference Attacks' leaf under 'Privacy Attack Characterization and Threat Modeling', which contains five papers total. This leaf represents a moderately populated research direction within the broader taxonomy of 50 papers across approximately 36 topics. The sibling papers in this leaf focus on characterizing inference threats rather than proposing defenses, suggesting the paper bridges attack analysis with mitigation strategies.

The taxonomy reveals that defense mechanisms occupy a separate major branch with four distinct leaves covering cryptographic approaches, differential privacy, model-centric defenses, and attack detection. The paper's shuffle-based defense naturally connects to the 'Cryptographic and Shuffling-Based Defenses' leaf, which contains five papers exploring secure aggregation and encoding schemes. The scope notes clarify that while the paper sits taxonomically among attack characterization works, its defensive contribution positions it at the boundary between threat modeling and mitigation strategies, potentially explaining why it appears somewhat isolated from its immediate siblings.

Among the three contributions analyzed, the first (reconstruction attacks against standard shuffling) examined 10 candidates with zero refutations, suggesting relative novelty in demonstrating shuffling vulnerabilities. The second contribution (robust defense in shuffle model) examined 9 candidates and found 2 refutable matches, indicating more substantial prior work in shuffle-based defenses. The third contribution (experimental validation) examined 10 candidates with no refutations. These statistics reflect a limited search scope of 29 total candidates examined, not an exhaustive literature review, meaning the analysis captures top semantic matches rather than comprehensive field coverage.

Based on the limited search scope, the work appears to occupy a niche intersection between attack demonstration and defense design within shuffle-based federated learning. The taxonomy structure suggests this specific combination of parameter-level shuffling with residue number systems may be relatively unexplored, though the broader shuffle defense paradigm has established precedents. The analysis acknowledges uncertainty inherent in examining only 29 candidates from a field of 50 surveyed papers, leaving open questions about related work in adjacent cryptographic or encoding-based defense approaches.

Taxonomy

Core-task Taxonomy Papers
50
3
Claimed Contributions
29
Contribution Candidate Papers Compared
2
Refutable Paper

Research Landscape Overview

Core task: defending against source inference attacks in federated learning. The field organizes itself around four main branches that together capture the lifecycle of privacy research in federated settings. Privacy Attack Characterization and Threat Modeling establishes the adversarial landscape, examining how attackers can infer sensitive information about data sources, memberships, and client identities from model updates or aggregated parameters. Defense Mechanisms and Mitigation Strategies develops countermeasures ranging from differential privacy and gradient perturbation techniques to architectural modifications that limit information leakage. Evaluation Frameworks and Empirical Analysis provides the methodological backbone, offering benchmarks and metrics to quantify privacy risks and measure defense effectiveness across diverse scenarios. Domain-Specific Applications and Implementations translates these insights into practical deployments in healthcare, finance, and other sensitive domains where federated learning promises collaboration without direct data sharing. Within the attack characterization branch, a dense cluster of works explores source and membership inference threats, revealing how adversaries can exploit gradient information or model behavior to identify training participants or reconstruct sensitive attributes. Source Inference Protection[0] sits squarely in this area, addressing the specific challenge of preventing attackers from determining which client contributed particular data samples. Its emphasis contrasts with broader membership inference studies like Membership Inference Survey[3], which surveys a wider range of inference attacks, and with works such as Client Targeted Membership[6] or Interaction Level Membership[13], which focus on finer-grained membership detection at the client or interaction level. Meanwhile, defenses like FLSG Defense[5] and techniques surveyed in Privacy Inference Survey[22] illustrate the ongoing tension between utility preservation and privacy guarantees, highlighting open questions about scalability, robustness under adaptive attacks, and the trade-offs inherent in deploying privacy-preserving federated systems at scale.

Claimed Contributions

Novel reconstruction attacks against standard shuffling in federated learning

The authors introduce reconstruction algorithms for three shuffling granularities (model-level, layer-level, and parameter-level) that enable source inference attacks within the shuffle model of FL. These attacks demonstrate that standard shuffling alone is insufficient to protect against SIAs.

10 retrieved papers
First robust defense against source inference attacks in the shuffle model

The authors present a defense mechanism that combines parameter-level shuffling with the residue number system (RNS) and unary encoding. This approach reduces SIA accuracy to random guessing without affecting joint model accuracy and can be seamlessly integrated into existing shuffle mechanisms.

9 retrieved papers
Can Refute
Experimental validation across multiple models and datasets

The authors provide empirical evaluation demonstrating that standard shuffling approaches fail to prevent SIAs, while their proposed method successfully reduces attack accuracy to the level of random guessing across various datasets and model architectures.

10 retrieved papers

Core Task Comparisons

Comparisons with papers in the same taxonomy category

Contribution Analysis

Detailed comparisons for each claimed contribution

Contribution

Novel reconstruction attacks against standard shuffling in federated learning

The authors introduce reconstruction algorithms for three shuffling granularities (model-level, layer-level, and parameter-level) that enable source inference attacks within the shuffle model of FL. These attacks demonstrate that standard shuffling alone is insufficient to protect against SIAs.

Contribution

First robust defense against source inference attacks in the shuffle model

The authors present a defense mechanism that combines parameter-level shuffling with the residue number system (RNS) and unary encoding. This approach reduces SIA accuracy to random guessing without affecting joint model accuracy and can be seamlessly integrated into existing shuffle mechanisms.

Contribution

Experimental validation across multiple models and datasets

The authors provide empirical evaluation demonstrating that standard shuffling approaches fail to prevent SIAs, while their proposed method successfully reduces attack accuracy to the level of random guessing across various datasets and model architectures.

Table of Contents