IC-104Value anchor prompting produces LLM value correlation structures that closely match the human circular value structure, while standard prompting does not

Naama Rozen, Liat Bezalel, Gal Elidan, Amir Globerson, Ella Daniel

SourceDo LLMs have Consistent Values?

The paper performed multidimensional scaling on the 19x19 value correlation matrices from each model and prompt condition, then compared the resulting 2D embeddings to the human MDS solution via Procrustes analysis. Under the value anchor prompt, the LLM correlation structures showed strong alignment with the human circular pattern (r = 0.87-0.95, p < .001). The sum of squared differences between LLM and human MDS coordinates was 0.11 for value anchor versus 0.71 for names (Gemini 1.0 Pro, temperature 0.0). Standard prompting (basic, demographic, generated persona) produced substantially higher differences, indicating poorer fit to the human structure.

Evidence
correlational
Key metric
r = 0.87 − 0.95, p < .001 for value anchor; sum of squared differences 0.11 (value anchor) vs 0.71 (names) for Gemini 1.0 Pro at temperature 0.0; Table 1 shows value anchor consistently lowest across all models
Caveat
Gemma 2 27B did not produce parseable results for the demographic prompt at temperature 0.0, and some values had zero-variance at temperature 0.0, precluding correlation computation.
Model
GPT-4 / ChatGPT4 / GPT-4 Code Interpreter / GPT-4 Technical Report GPT-4-0314, Gemini 1.0 Pro, Llama 3.1 8B, 70B, Gemma 2 9B, 27B
Concepts
Circular representation
Datasets
PVQ-RR [eval]
Methods
Multidimensional Scaling [primary], Procrustes Analysis [eval]
Related findings
IC-103, IC-105
Extraction
automatic-extraction