arXiv:2605.26350v1 Announce Type: new Abstract: In-context learning (ICL) is often motivated by the intuition that demonstrations help because they provide correct input-output examples. However, we reveal a counterintuitive phenomenon: correctness does not guarantee exemplar utility, and some correct demonstrations can even reduce ICL accuracy. To study this correctness-utility gap, we introduce task-preserving perturbations, where only the exemplar input is changed, while the example remains a correct instance of the same task. Concretely, each perturbed exemplar is assigned the target induc
The paper was published on May 27, 2026, indicating recent research advancements and findings in AI methodologies.
This research challenges fundamental assumptions about effective demonstration design in ICL, potentially leading to more efficient and reliable AI model training and deployment for strategic readers.
The understanding of how demonstrations influence in-context learning is now more nuanced, requiring a reevaluation of current best practices for prompt engineering and data curation.
- · AI researchers
- · Prompt engineers
- · AI model developers
- · Companies using AI for complex tasks
- · Inefficient AI training methodologies
- · Organizations relying solely on correct examples for ICL
- · Simple prompt engineering approaches
AI systems become more robust and less susceptible to poorly designed prompts.
New techniques for generating effective demonstrations emerge, prioritizing utility over simple correctness.
The development of 'smart' data curation tools that actively identify and mitigate 'hurting' examples in training datasets.
This signal links to a primary source. Continuum Brief monitors and indexes it as part of the live intelligence stream — we do not republish source content.
Read at arXiv cs.LG