IC-1269LLMs leak secrets to inappropriate recipients in meeting summarization and action-item generation tasks

Niloofar Mireshghallah, Hyunwoo Kim, Xuhui Zhou, Yulia Tsvetkov, Maarten Sap, Reza Shokri, Yejin Choi

SourceCan LLMs Keep a Secret? Testing Privacy Implications of Language Models via Contextual Integrity Theory

In tier 4, models generate meeting summaries or personal action items from a transcript where a secret about person X was discussed before X joined. GPT-4 leaks the secret in 39% of summaries and 29% of action items; ChatGPT leaks in 57% and 38% respectively. All models also frequently omit public information (GPT-4: 10% in summaries, 76% in action items). The authors hypothesize the summary task is harder because the model must reason that X is among the recipients. Manual inspection found 16 additional nuanced leaks in GPT-4 action items not caught by string matching.

Evidence
correlational
Key metric
Summary leaks secret: GPT-4 0.39, ChatGPT 0.57, InstructGPT 0.09, Mixtral 0.28, Llama-2 Chat 0.35, Llama-2 0.21; Action item leaks secret: GPT-4 0.29, ChatGPT 0.38; Omits public info (summary): GPT-4 0.10, ChatGPT 0.27
Caveat
Only 20 transcripts were used. Scenarios were generated by GPT-4 (familiarity concern); a control with ChatGPT-generated scenarios confirmed GPT-4 still outperforms ChatGPT. String-match evaluation underestimates true leakage.
Model
GPT-4 / ChatGPT4 / GPT-4 Code Interpreter / GPT-4 Technical Report, ChatGPT, InstructGPT, Mixtral, Llama 2 / Llama 2 base Llama-2-Chat
Concepts
Failure mode
Related findings
IC-1267, IC-1268, IC-1270
Extraction
automatic-extraction