arXiv:2509.26469v4 Announce Type: replace Abstract: Vector quantization is common in deep models, yet its hard assignments block gradients and hinder end-to-end training. We propose DiVeQ, which treats quantization as adding an error vector that mimics the quantization distortion, keeping the forward pass hard while letting gradients flow. We also present a space-filling variant (SF-DiVeQ) that assigns input to a curve constructed by the lines connecting codewords, resulting in less quantization error and full codebook usage. Both methods train end-to-end without requiring auxiliary losses or
This research addresses a fundamental challenge within deep learning, suggesting an improvement to a technique widely used in various AI models.
Differentiable vector quantization can lead to more efficient and robust deep learning models, particularly those that struggle with gradient flow during training.
The ability to perform end-to-end training of models using vector quantization without auxiliary losses simplifies model development and potentially enhances performance in areas like generative AI and compression.
- · AI researchers
- · Generative AI companies
- · Deep learning practitioners
- · Hardware manufacturers benefiting from more efficient models
- · Developers reliant on complex workarounds for quantization training
Improved performance and broader application of models relying on vector quantization.
Faster development cycles for specific types of AI architectures due to simplified training.
Potentially enables new AI applications that were previously bottlenecked by inefficient quantization methods.
This signal links to a primary source. Continuum Brief monitors and indexes it as part of the live intelligence stream — we do not republish source content.
Read at arXiv cs.LG