IC-1363Current model alignment does not filter privacy-invasive prompts across major LLM providers

Robin Staab, Mark Vero, Mislav Balunovic, Martin Vechev

SourceBeyond Memorization: Violating Privacy via Inference with Large Language Models

The paper tests whether released LLMs refuse to answer privacy-invasive attribute inference prompts. Across all four providers tested, refusal rates are very low: Meta Llama-2 refuses 0%, OpenAI GPT refuses 0%, Anthropic Claude refuses 2.8%, and Google PaLM refuses 10.7%. The authors note that PaLM's higher rate may be triggered by sensitive topics in the text rather than the privacy-invasive nature of the prompt itself.

Evidence

correlational

Key metric

Refusal rates: Meta Llama-2 0%, OpenAI GPT 0%, Anthropic Claude 2.8%, Google PaLM 10.7%

Caveat

PaLM's 10.7% may reflect a separate safety filter triggered by sensitive content in the text rather than the privacy-invasive prompt structure

Model

Llama 2 / Llama 2 base, GPT-4 / ChatGPT4 / GPT-4 Code Interpreter / GPT-4 Technical Report, PaLM 2

Related findings

IC-1361, IC-1362, IC-1364

Extraction

automatic-extraction