Can You Keep a Secret? Involuntary Information Leakage in Language Model Writing

arXiv:2605.10794v1 Announce Type: cross Abstract: Language models are deployed in settings that require compartmentalization: system prompts should not be disclosed, chain-of-thought reasoning is hidden from users, and sensitive data passes through shared contexts. We test whether models can keep...