LLMs Can Leak Training Data But Do They Want To? A Propensity-Aware Evaluation of Memorization in LLMs
arXiv:2606.06286v1 Announce Type: new Abstract: Large language models can reproduce training data, but existing memorization evaluations mostly measure whether models can be forced to do so, rather than whether they do so under ordinary use. We introduce PropMe, a propensity-aware framework for memorization evaluation that contrasts prefix-based capability attacks with non-adversarial evaluations. We propose a metric transformation that, applied to existing functions, allows to create propensity metrics. We further introduce SimpleTrace, a lightweight tracing pipeline built on infini-gram that
The paper is published as concerns about AI data privacy and intellectual property are intensifying, especially with the rapid deployment of LLMs.
This research provides a more realistic framework for evaluating LLM memorization, moving beyond adversarial attacks to assess data leakage under normal usage, which is crucial for safety and regulation.
The shift to 'propensity-aware' evaluation means that future LLM development and deployment will need to consider not just capability to reproduce data but also the likelihood of doing so inadvertently.
- · AI Safety Researchers
- · Data Privacy Advocates
- · Regulatory Bodies
- · Responsible AI Developers
- · Developers neglecting data privacy
- · Companies using LLMs with sensitive training data
- · Unregulated AI deployments
Increased scrutiny and demand for LLMs with proven low propensity for data memorization in non-adversarial settings.
Development of new architectural designs and training methodologies to inherently reduce accidental data leakage in LLMs.
Potential legal precedents set based on 'propensity-aware' memorization, influencing intellectual property and privacy laws specific to AI.
This signal links to a primary source. Continuum Brief monitors and indexes it as part of the live intelligence stream — we do not republish source content.
Read at arXiv cs.CL