Communication-Efficient Neural Tangent Kernels for Heterogeneous Decentralized Federated Learning
arXiv:2512.12737v2 Announce Type: replace Abstract: Decentralized federated learning (DFL) enables collaborative model training without a central server, but converges slowly under statistical heterogeneity. Recent work has shown that neural tangent kernel (NTK) methods achieve faster convergence than gradient-based updates in DFL, while momentum has proven effective for accelerating gradient-based FL. However, applying momentum to NTK updates can destabilize training under heterogeneous data. We propose SPARK, which addresses this instability with a stage-wise annealed soft-label regularizer
This research addresses a known challenge in decentralized federated learning where statistical heterogeneity causes slow convergence and instability when applying momentum to NTK updates.
Improved communication efficiency and stability in decentralized federated learning could accelerate the development and deployment of more robust and scalable AI models, particularly in privacy-sensitive or resource-constrained environments.
The proposed SPARK method offers a way to stabilize and accelerate training in heterogeneous decentralized federated learning, potentially making these systems more practical for real-world applications.
- · AI researchers and developers
- · Organizations with distributed data
- · Edge computing providers
- · Centralized AI training paradigms (relatively)
Enhances the practical viability of decentralized federated learning for privacy-preserving AI development.
Could lead to wider adoption of federated learning in sectors like healthcare, finance, and IoT where data privacy and distributed processing are critical.
May contribute to the broader decentralization of AI model training infrastructure, potentially impacting the compute supply chain and access to AI capabilities.
This signal links to a primary source. Continuum Brief monitors and indexes it as part of the live intelligence stream — we do not republish source content.
Read at arXiv cs.LG