IC-171ICL prompting produces binary response patterns in released LLMs, with answered ratios collapsing to near 0% or 100% rather than calibrated refusal, making prompting ineffective for RAG groundedness
The paper evaluates whether in-context learning with refusal demonstrations can calibrate models' RAG behavior. Across multiple released models, ICL with a refusal prompt causes answered ratios to drop to near zero (Llama-3-8B: 1.48% ASQA, 3.90% QAMPARI, 0.00% ELI5; Llama-2-7B: 0.00%, 0.00%, 0.50%), while under default prompting models answer nearly everything. This binary behavior indicates that released models cannot learn calibrated refusal from a few ICL examples, and the resulting indiscriminate refusal or over-responsiveness degrades trust-scores across all sub-metrics.