Convergence Theory for Iterative LLM-Based Neural Architecture Search: A Parametric Cross-Entropy Framework with Closed-Form Proxy Reliability
arXiv:2605.30103v1 Announce Type: new Abstract: Large language models (LLMs) are increasingly used as generators in iterative neural architecture search (NAS), yet no formal convergence theory exists for this class of algorithms. We model iterative LLM-NAS as a parametric Cross-Entropy (CE) method over executable programs and prove six results: (1) iterative LLM fine-tuning on elite architectures is equivalent to the CE update restricted to the LLM parametric family; (2) expected architecture quality is monotonically non-decreasing across cycles; (3) elite-set probability converges to a fixed
The increasing use of LLMs in iterative neural architecture search (NAS) necessitates a formal convergence theory to validate and improve these algorithms.
Establishing a convergence theory for LLM-based NAS provides a foundational mathematical basis, enhancing the reliability and efficiency of AI development processes.
The systematic understanding of how LLM-NAS converges enables more predictable and optimized AI system design, fostering more robust and scalable AI applications.
- · AI researchers and developers
- · Companies using NAS for AI development
- · LLM providers
More efficient and reliable design of advanced neural networks using LLMs.
Accelerated development cycles for new AI models and applications across industries.
Enhanced overall capability and applicability of AI systems due to optimized architectural discovery.
This signal links to a primary source. Continuum Brief monitors and indexes it as part of the live intelligence stream — we do not republish source content.
Read at arXiv cs.LG