arXiv:2606.04023v1 Announce Type: cross Abstract: While large language models (LLMs) have been extensively evaluated on code generation tasks for general-purpose programming and GPU-accelerated environments (e.g., PyTorch, CUDA), their capabilities in CPU-oriented high-performance computing (HPC) across diverse architectures remain underexplored. To bridge this gap, we introduce CodegenBench, a comprehensive benchmark suite designed to evaluate the generation of efficient parallel code across three distinct hardware platforms: x86_64, Sunway, and Kunpeng. Our benchmark comprises 106 standard B
The rapid advancement and widespread deployment of LLMs have revealed a critical need to assess their performance beyond general-purpose tasks, particularly in specialized and high-performance computing environments.
Evaluating LLMs' ability to generate efficient code for diverse CPU architectures is crucial for their adoption in high-performance computing, impacting critical infrastructure and AI development.
The focus has shifted from merely generating functionally correct code to generating architecturally optimized and efficient code across a broader range of hardware, moving LLMs closer to HPC utility.
- · HPC developers
- · LLM developers (fine-tuning for HPC)
- · Cloud providers (offering specialized compute)
- · Specific architecture vendors (e.g., x86_64, Sunway, Kunpeng)
- · Developers solely relying on manual optimization
- · LLMs with poor architectural understanding
- · Vendors of less efficient AI code generation tools
LLMs will become more practical and efficient tools for specialized HPC code generation, reducing development cycles.
Increased efficiency in HPC via LLM-assisted code generation could accelerate scientific discovery and technological breakthroughs.
Nations or entities with specific hardware architectures could gain a strategic advantage if their LLMs excel at optimizing for those platforms.
This signal links to a primary source. Continuum Brief monitors and indexes it as part of the live intelligence stream — we do not republish source content.
Read at arXiv cs.AI