torch-sla: Differentiable Sparse Linear Algebra with Adjoint Solvers and Sparse Tensor Parallelism for PyTorch
arXiv:2601.13994v3 Announce Type: replace-cross Abstract: Differentiable sparse linear algebra is foundational for scientific machine learning, yet PyTorch lacks a unified library for it: torch.sparse provides only low-level kernels and a non-differentiable, CPU-only spsolve, and torch.linalg is dense-only. We present torch-sla, an open-source library that fills this gap. It exposes a single autograd-aware API for direct, iterative, nonlinear, and eigenvalue solvers across five interchangeable backends -- SciPy and Eigen on CPU, cuDSS, CuPy, and a PyTorch-native iterative solver on GPU -- with
The increasing complexity of scientific machine learning and large-scale AI models is creating a critical need for efficient and differentiable sparse linear algebra tools.
This development addresses a significant bottleneck in scientific machine learning frameworks, enabling more efficient and scalable computation for complex AI models in various domains.
PyTorch users now have a unified, autograd-aware library for sparse linear algebra, which was previously fragmented and less capable, accelerating research and deployment in scientific AI.
- · PyTorch users and developers
- · Scientific machine learning researchers
- · AI model developers
- · High-performance computing (HPC) sector
- · Proprietary sparse linear algebra libraries
The new torch-sla library will immediately improve the performance and ease of development for scientific machine learning applications within the PyTorch ecosystem.
Enhanced capabilities could lead to breakthroughs in areas requiring complex simulations, such as drug discovery, materials science, and climate modeling, by making underlying computations more tractable.
The democratization of advanced sparse linear algebra could accelerate the development of more complex and specialized AI agents that rely on sophisticated computational backends.
This signal links to a primary source. Continuum Brief monitors and indexes it as part of the live intelligence stream — we do not republish source content.
Read at arXiv cs.AI