arXiv:2603.17433v2 Announce Type: replace Abstract: Transformer models have redefined sequence learning, yet dot-product self-attention introduces a quadratic token-mixing bottleneck for long-context time-series. We introduce the Phasor Transformer block, a phase-native alternative representing sequence states on the unit-circle manifold $S^1$. Each block combines lightweight trainable phase-shifts with parameter-free Discrete Fourier Transform (DFT) token coupling, achieving global $\mathcal{O}(N\log N)$ mixing without explicit attention maps. Stacking these blocks defines the Large Phasor Mo
The continuous drive for more efficient and scalable Transformer architectures is leading to innovative approaches to tackle the quadratic complexity bottleneck in attention mechanisms.
This research could significantly improve the efficiency of large language models and other sequence learning applications, enabling larger context windows and reducing computational costs.
The fundamental architecture of Transformer models for long sequence processing could shift from dot-product attention to phase-native representations and DFT-based mixing.
- · AI researchers
- · Cloud providers with large AI workloads
- · Companies building large language models
- · Developers working on long-context time-series analysis
- · Traditional Transformer architectures relying solely on dot-product attention
- · Hardware optimized exclusively for dense matrix multiplications if new operation
Reduced computational expense and increased context window size for advanced AI models.
Accelerated development of more sophisticated AI agents capable of understanding and processing much longer sequences of information.
Potentially democratized access to very large context AI by lowering the resource barrier, fostering new applications and research directions.
This signal links to a primary source. Continuum Brief monitors and indexes it as part of the live intelligence stream — we do not republish source content.
Read at arXiv cs.LG