Bilevel Data Curation for LLM Fine-tuning: Offline Selection and Online Self-Refining Generation
arXiv:2511.21056v2 Announce Type: replace-cross Abstract: Supervised fine-tuning (SFT) datasets are critical to the downstream performance of large language models, yet they often contain low-quality or harmful question-response pairs. To improve SFT data quality, we develop a unified bilevel framework that combines offline data selection with the online self-refining generation. In the offline setting, bilevel data selection (BDS) selects question-response pairs from the offline SFT dataset to maximize the validation performance. We theoretically show that the optimal model given by BDS outpe
The rapid development and deployment of LLMs necessitate more efficient and effective fine-tuning methods to address quality and safety concerns proactively.
Improving the quality of fine-tuning datasets directly enhances the performance, reliability, and safety of large language models, impacting their utility across various applications.
The proposed bilevel data curation framework offers a programmatic way to optimize LLM fine-tuning by systematically selecting and generating high-quality data.
- · AI developers
- · LLM application providers
- · Data curation platforms
- · AI safety researchers
- · Manual data labeling services
- · LLMs fine-tuned on low-quality data
Higher quality and more reliable LLMs become available for enterprise and consumer use.
Reduced incidence of harmful or biased LLM outputs, increasing public trust and adoption.
Accelerated development of more complex and autonomous AI agents due to improved foundational models.
This signal links to a primary source. Continuum Brief monitors and indexes it as part of the live intelligence stream — we do not republish source content.
Read at arXiv cs.CL