HO-SFL: Hybrid-Order Split Federated Learning with Backprop-Free Clients and Dimension-Free Aggregation
arXiv:2603.14773v2 Announce Type: replace Abstract: Fine-tuning large models on edge devices is severely hindered by the memory-intensive backpropagation (BP) in standard frameworks like federated learning and split learning. While substituting BP with zeroth-order optimization can significantly reduce memory footprints, it typically suffers from prohibitively degraded convergence speed. To resolve this dilemma, we propose Hybrid-Order Split Federated Learning (HO-SFL). By reformulating the split learning process within a Lagrangian framework, HO-SFL decouples the optimization landscape: The s
The increasing scale of AI models and the distributed nature of data necessitate more efficient training methods that can operate on edge devices with limited computational resources.
This research addresses a critical bottleneck in federated and split learning by enabling efficient fine-tuning of large models on memory-constrained edge devices, expanding the practical applicability of AI across diverse sectors.
The proposed HO-SFL method significantly reduces memory footprints for on-device AI training by decoupling optimization, potentially accelerating the deployment of advanced AI to a wider array of endpoints without prohibitive convergence degradation.
- · Edge device manufacturers
- · Federated learning platforms
- · Industries relying on on-device AI
- · Developers of large AI models
- · Traditional federated learning frameworks (without such optimizations)
- · Cloud-centric AI training paradigms for specific use cases
Widespread adoption of larger, more complex AI models on edge devices becomes feasible.
Reduced reliance on centralized cloud infrastructure for certain AI fine-tuning tasks, potentially enhancing data privacy and security.
Acceleration of autonomous AI agents operating directly on local data, fostering new generations of intelligent applications beyond current capabilities.
This signal links to a primary source. Continuum Brief monitors and indexes it as part of the live intelligence stream — we do not republish source content.
Read at arXiv cs.LG