Explicit Fuzzy Logic in the Feed-Forward Layer: Self-Forgetting Quantifiers Discover Legible Grammatical-Licensing Detectors
arXiv:2606.31845v1 Announce Type: new Abstract: A transformer's feed-forward (FFN) sublayer materializes the distinctions attention gathers, yet gives no account of what it computes. In a parameter-neutral replacement, each hidden unit is an explicit fuzzy set operation on sigmoid-bounded [0,1] memberships: intersection A*B and set-difference A*(1-B), the latter a bounded positive negation ("A but not B") that gated/bilinear units lack -- a negation-capable FFN (NC-FFN). On N-bit parity they are the most parameter-efficient reasoning basis at shallow depth; at scale (125M, OpenWebText) NC-FFN
The paper tackles a core limitation of current transformer architectures (FFN interpretability) at a point where model size and complexity demand more transparent and efficient mechanisms.
This research provides a more interpretable and potentially more robust foundation for AI models, moving beyond opaque neural networks towards systems with explicit logical reasoning capabilities.
Neural networks could become more transparent and easier to debug, with 'legible grammatical-licensing detectors' indicating a pathway to more explainable AI components.
- · AI researchers
- · AI developers
- · Industries requiring explainable AI
- · Opaque black-box AI approaches
Improved understanding and explainability for large language models and other transformer-based systems.
Reduced need for extensive post-hoc explainability techniques as interpretability is built directly into forward-pass layers.
Enhanced trust and adoption of AI in critical applications where audited, explicit reasoning is paramount.
This signal links to a primary source. Continuum Brief monitors and indexes it as part of the live intelligence stream — we do not republish source content.
Read at arXiv cs.CL