Light Dark OpenAI · text · generative · anchor
Closed-weight language model served through an API, cited in the corpus in a pinned long-context chat release.
Note closed-weight model with no paper of its own. The citing paper attaches the InstructGPT reference, which describes the training method rather than this release, so no anchor is recorded Variants gpt-3.5-turbo-16k-0613 , GPT-3.5-turbo , GPT-3.5-1106 , GPT-3.5-turbo-0125 , GPT-3.5-turbo-0613 , GPT-3.5-turbo-1106 , GPT-3.5-turbo-20231106 , GPT-3.5-turbo-instruct , GPT-3.5-text-davinci-003 , GPT-3.5-turbo-0301 Findings IC-029 Large language models show conformity to group answers in multi-agent interactions IC-030 Larger language models exhibit higher independence rates and lower conformity under some protocols IC-074 Released LLMs achieve F1 plan scores between 42.7 and 86.7 on the T-Eval plan task IC-082 GPT-3.5, GPT-4o, Claude-3.5-Sonnet, and Llama-3.1-8B produce explanations on the BBQ social bias task that are systematically unfaithful for identity and behavior concepts while remaining faithful for context concepts, with specific patterns of hiding safety-measure influence and social bias IC-083 GPT-3.5, GPT-4o, and Claude-3.5-Sonnet produce unfaithful explanations on MedQA medical questions, omitting high-effect clinical concepts such as the patient's mental status while over-referencing low-effect concepts IC-100 GPT-4, GPT-3.5, Claude 3, Llama 3 8B, and Llama 3 70B can be prompted to approximately calibrate their accuracy to specific target percentages on MMLU IC-1050 Released models generate patches that are less than half the length of gold solutions and rarely edit more than one file IC-1055 Safety fine-tuning suppresses harmful content generation in ChatGPT relative to GPT-3.5, but the suppression is substantially weaker for non-English prompts IC-109 GPT-3.5-turbo and GPT-4 produce cycles in inferred causal graphs when using pairwise prompts, with cycle counts growing sharply on larger graphs IC-1144 GPT-4 and GPT-3.5 produce high rates of irrelevant (fabricated) books when answering constraint queries from parametric knowledge, with a sharp phase transition at low author popularity IC-1145 Providing complete context eliminates irrelevance but does not fix constraint satisfaction for GPT-4 or GPT-3.5 IC-1146 Self-context (self-retrieval) chain-of-thought increases the rate of fabricated books compared to no-context for both GPT-4 and GPT-3.5 IC-1147 GPT-4 outperforms GPT-3.5 on all KITAB metrics but the gap is modest, with all-correctness below 35% for both, suggesting scale alone does not resolve constraint satisfaction IC-1157 GPT-4 and other LMs show a large gap between rule induction and rule application, with task accuracy dropping to near zero on MiniScan when the LM itself applies its own proposed rules IC-1158 GPT-4 and other LMs are brittle to noisy exemplars and unfamiliar output representations, with performance degrading sharply even under minimal perturbation IC-1159 GPT-4 is a strong inductive hypothesis proposer, achieving high accuracy on inductive reasoning benchmarks when its generated rules are applied by a symbolic interpreter IC-1170 GPT-3.5, Llama2, PaLM2, and GPT-4 are susceptible to a CoT-prompting backdoor attack (BadChain) on complex reasoning tasks, with stronger reasoning models showing higher attack success rates IC-1194 GPT-3.5-turbo, GPT-4, and GPT-3.5-turbo-0613 exhibit 50-58% inconsistency between their ratings and rankings feedback on the same response pairs IC-1220 GPT-4, GPT-3.5-turbo, and Llama-2-70B can implement learning algorithms in-context on novel boolean functions, competing with nearest-neighbor baselines IC-1225 GPT-4 generates realistic dynamic scene layouts from text prompts with only 3 in-context examples, achieving 98% average accuracy across 5 spatiotemporal tasks, with physics knowledge (gravity, elasticity, perspective) generalising to unseen objects from its weights IC-1236 Among 7B LLMs, Llama-2-7b achieves the best zero-shot COMET scores in both translation directions, while MPT-7b leads in BLEU for en-to-xx IC-1264 LLMs are overconfident when verbalizing confidence, with values concentrated in 80–100% and multiples of 5, yielding high ECE across all five tested models IC-1265 Calibration and failure prediction improve as model capability scales from GPT-3 to GPT-4, but remain far from ideal IC-1284 Fine-tuning GPT-3.5 Turbo and Llama-2-7B-Chat on as few as 10 explicitly harmful examples removes their safety alignment, raising harmfulness rates to 80-92% IC-1285 Fine-tuning GPT-3.5 Turbo and Llama-2-7B-Chat on 10 implicitly harmful identity-shifting examples (containing no toxic content) jailbreaks their safety alignment IC-1286 Fine-tuning GPT-3.5 Turbo and Llama-2-7B-Chat on benign utility-oriented datasets (Alpaca, Dolly, LLaVA-Instruct) degrades their safety alignment without any malicious intent IC-1287 A backdoor can be implanted in GPT-3.5 Turbo via fine-tuning that is undetectable by standard safety auditing: the model appears safe on plain prompts but fulfills harmful instructions when a 3-word trigger is appended IC-1307 Released LLMs (CodeLlama 7B/13B/34B, GPT-3.5, GPT-4) achieve limited code-optimization speedups with standard prompting, with the best baseline (GPT-3.5 CoT) reaching only 1.60x versus the 3.66x human reference IC-1308 Dynamic retrieval-based few-shot prompting substantially improves released LLMs' code optimization, with GPT-4-0613 reaching 76.07% optimization rate and 3.93x speedup (best@8), exceeding the 3.66x human reference IC-1309 GPT-4-0613 exhibits reduced output diversity relative to GPT-3.5: it outperforms on best@1 but underperforms on best@8 under CoT prompting IC-131 GPT-4 Turbo, GPT-3.5 Turbo, Llama3-8B, Qwen-7B, and iFlytekSpark-13B over-rely on the strong reminder 'the answer is' in prompts as a shortcut, with accuracy dropping sharply when the cue is a random answer rather than the ground truth IC-1310 Chain-of-thought prompting provides notable code-optimization gains only for larger models (CodeLlama 34B, GPT-3.5, GPT-4) but not for CodeLlama 7B or 13B, consistent with an emergent capability IC-132 GPT-4 Turbo, GPT-3.5 Turbo, Llama3-8B, Qwen-7B, and iFlytekSpark-13B trust authority roles (teacher/judge) more than peer roles (classmate/lawyer) when the cue information is the correct answer IC-1327 GPT-4, GPT-3.5, Llama2, and Vicuna models underperform human annotators on multistep soft reasoning in natural language narratives, with smaller models scoring near random chance IC-1328 GPT-4's performance on MUSR depends on the type of neurosymbolic scaffolding: program-aided decomposition helps on structured optimization but symbolic belief tracking fails on natural language theory-of-mind IC-1330 All 20 evaluated LLMs improve in multi-turn task-solving with additional tool-use turns and GPT-4-simulated language feedback IC-1342 Assigning socio-demographic personas to LLMs causes significant reasoning performance degradation across all four models studied, manifesting as both explicit abstentions and implicit reasoning errors IC-1343 Task-agnostic de-biasing prompts are ineffective at reducing persona-induced reasoning bias in ChatGPT-3.5, while task-dependent expertise prompts are effective but lack generalizability IC-1356 GPT-3.5 turbo achieves over 90% agreement with human judgments when evaluating VLM responses on open-set questions IC-1361 GPT-4 and other state-of-the-art LLMs achieve near-human accuracy in inferring personal attributes from unstructured text IC-1373 All five evaluated LLMs show a strong positional bias in constrained text generation, with first-position constraints nearly always satisfied but last- and arbitrary-position constraints causing major performance drops IC-1374 Counting difficulty in constrained generation increases with text level and constraint strictness, with exact sentence-level character counts being the hardest condition for all five models IC-1392 Title reproduction shows no contamination signal while tag reproduction shows a negative association with GitHub presence and a moderating difficulty effect for GPT-4 and GPT-3.5-turbo IC-1402 Llama-2-70B-Chat underperforms GPT-3.5 across all SOTOPIA dimensions in interactive social scenarios, diverging from static benchmark rankings IC-1403 All four evaluated LLMs produce negative scores on social rules and secret-keeping dimensions in SOTOPIA interactions IC-1456 Pre-trained language models fail to predict both interpretations of ambiguous inputs in zero-shot semantic parsing IC-1462 GPT-3.5-turbo is substantially more robust to the generation exploitation attack, with attack success rate of only 7% compared to over 95% for open-source models IC-1489 State-of-the-art foundation models (CLIP, GPT-3.5-turbo, and others) score well below elementary students on multimodal K-12 STEM questions IC-1508 LLMs with in-context learning translate Kalamang-English at 44.7/45.8 CHRF, falling short of the human baseline of 51.6/57.0 CHRF IC-1510 Without retrieved context, LLMs are unable to translate Kalamang, and among context types, retrieved parallel sentences are most beneficial, followed by word list entries, then grammar book passages IC-1523 Five AI assistants (Claude-1.3, Claude-2.0, GPT-3.5-turbo, GPT-4, Llama-2-70B-Chat) consistently exhibit sycophancy across four varied free-form text-generation tasks IC-1577 Base LLMs prompted with URiAL (3 restyled in-context examples + system prompt) match or surpass their SFT/RLHF-aligned counterparts on multi-aspect evaluation IC-1598 Retrieval augmentation improves GPT-3.5-turbo-4k on long-context tasks but not GPT-3.5-turbo-16k IC-1599 Self-repair at equivalent compute budget provides only modest and inconsistent gains over i.i.d. sampling for CodeLlama-13B-Instruct, GPT-3.5, and GPT-4 on HumanEval and APPS IC-1600 Replacing a model's self-generated feedback with a stronger model's feedback consistently improves self-repair beyond both the i.i.d. baseline and the self-repair baseline IC-1612 GPT-3.5-turbo achieves the best and most stable performance across all three drug types in conversational drug editing IC-170 GPT-4, GPT-3.5, and Claude-3.5-Sonnet rely heavily on parametric knowledge in RAG settings, producing ungrounded responses with high answered ratios and low trust-scores IC-188 LLMs show constraint-type-specific performance on system message following, with weaker models exhibiting large variance across constraint categories IC-189 Most LLMs show degraded instruction satisfaction when user instructions conflict with system messages, indicating difficulty in prioritizing system message constraints IC-190 LLMs show progressive degradation in system message constraint following across multi-turn conversations, with dependent conversations degrading faster than parallel ones IC-198 Safety-aligned LLMs (GPT-4, GPT-3.5, Gemma2-27b, GPT-4o, Gemma2-9b, Qwen2.5-72b, Mistral-7b, Mixtral-8x22b) are vulnerable to natural prompts semantically related to toxic seed prompts, with attack success rates of 82-99% IC-209 LLM judges (GPT-3.5-turbo-1106, GPT-4o-mini, GPT-4o, Claude-3-5-sonnet) implicitly prioritize style over factuality and safety when scoring pairwise preferences IC-230 Six LLMs show distinct value preferences on daily-life moral dilemmas, with significant inter-model differences on core values such as truthfulness and fairness IC-252 GPT models produce harmful gender stereotypes at higher rates when user names imply a demographic group, with GPT-3.5 Turbo showing the highest rates and open-ended generation tasks most affected IC-253 Post-training reinforcement learning significantly reduces harmful gender stereotypes in GPT models, with the best-fit slope of 0.21 indicating post-RL models have far lower bias than pre-RL versions IC-264 All 18 evaluated LLMs fail to abstain when the provided context lacks the answer, with performance gaps of 13.6% to 68.4% relative to the original context IC-265 Model families show extreme variation in detecting conflicting answers in inconsistent contexts, with phi-3 series at 5.8% average accuracy versus GPT-4 series at 89.35% IC-266 GPT-4o drops from 96.3% closed-book accuracy to 47.5% when given counterfactual context that contradicts its parametric knowledge, far below the 95% human accuracy on the same items IC-307 56 LLMs on Sorry-Bench show fulfillment rates ranging from below 10% (Claude-2, Gemini-1.5) to above 90% (Mistral-7B-instruct-v0.1, Dolphin-2.6-mixtral-8x7b), with GPT-4o at 30% and Llama-3-70B at 35% IC-308 Linguistic mutations to unsafe prompts significantly and inconsistently alter safety refusal across models, with persuasion techniques increasing fulfillment by 5-66% and encoding/encryption decreasing it by 15-68% IC-309 As zero-shot safety judges, GPT-4o achieves 78.9% Cohen's kappa agreement with human annotators while Llama-3-8B-instruct (39.0%) and Mistral-7B-instruct-v0.2 (53.9%) perform substantially worse IC-312 GPT-4o achieves the highest insight-level Llama-3-eval score (0.60) among all tested LLM backbones on InsightBench multi-step data analytics IC-313 All four LLM backbones fail to detect a planted linear trend in incident resolution time when the slope is below 0.1, and detection rates diverge sharply above that threshold IC-322 Past-tense reformulations of harmful requests bypass refusal training in eight released LLMs, while future-tense reformulations are substantially less effective IC-335 GPT-4's detection performance as a scoring model is highly sensitive to the prompt, varying from 0.7289 to 0.9682 AUROC, far more than GPT-3.5 or Babbage IC-336 Larger proprietary LLMs (GPT-3.5 175B) are more effective universal text detectors than smaller models (Babbage 1.3B, GPT-Neo-2.7B), contradicting prior findings that smaller models are better IC-337 GPT-3.5-based detection accuracy drops substantially for Russian text (0.8555 AUROC) compared to near-perfect scores for Urdu, Indonesian, and Arabic, suggesting under-training on Russian IC-347 GPT-3.5-turbo-instruct achieves 53.7% move-matching accuracy on human chess when prompted with PGN notation IC-383 GPT-4-1106, GPT-3.5-1106, and unfine-tuned CodeLlama-13b achieve 44.3%, 39.5%, and 38.5% API call accuracy respectively on unseen APIs (Level 3) with 3-shot retrieved prompting in the API Pack evaluation IC-389 All evaluated LLMs score significantly lower on CS reasoning questions than knowledge questions, with the gap narrowing for stronger models IC-395 Most mainstream LLMs exhibit positive ADCE across five tasks, indicating reliance on deep structure for problem-solving, with ADCE strongly correlated with accuracy (r² > 0.7) IC-396 Closed-source LLMs (GPT, Claude) rely more on deep structure than open-source LLMs (Llama, Mistral), and open-source models' surface sensitivity decreases with model scale IC-407 Safety-aligned LLMs (Llama-2-chat, Llama-3-instruct, Gemma, GPT-3.5, GPT-4o, R2D2) achieve 100% jailbreak attack success rate under adaptive prompt-and-suffix attacks on 50 harmful requests IC-436 API selection accuracy of 10 LLM-based agents degrades sharply as task complexity increases, with open-source models ≥70B matching closed-source on simpler tasks but lagging on the most complex IC-437 Extracting parameters from user queries is harder for LLM-based agents than using outputs from previous actions, and less intelligent LLMs show steeper parameter-filling degradation with task difficulty IC-438 All 10 LLM-based agents perform poorly at recognizing when they need to request input from the system or user, with overall accuracy between 30.55% and 55.18% IC-475 GPT-4, GPT-3.5, GPT-4o, and Llama-3.1-405B fail at inverse knowledge search regardless of prompting IC-476 GPT-4 and GPT-3.5 show strong positional bias in Chinese idiom character completion IC-485 LLMs show a significant performance gap between Wikipedia-based factual multi-hop QA and counterfactual multi-hop QA, indicating reliance on memorized knowledge rather than reasoning from context IC-486 LLMs achieve correct final answers through incorrect reasoning chains, inflating their apparent multi-step reasoning performance IC-487 Including sub-questions in the prompt improves LLM performance on multi-hop QA tasks IC-488 LLM performance degrades progressively as the number of reasoning hops increases, with error propagation from earlier sub-questions IC-549 All 18 evaluated LLMs show a 15-20% performance gap between linear (node chain) and graph (workflow) planning on WorfBench IC-555 Large LLMs (GPT-3.5-turbo, Gemini 1.5 Flash, Llama3-70B, Mixtral 46.7B) exhibit reasoning errors and significant accuracy degradation on large-scale logical commonsense reasoning tasks with 32k+ rules, even when the knowledge base is complete and retrieval is ideal IC-595 Gemini-pro has knowledge gaps on specific topics (Permian extinction, Fordism) causing it to perform far below its average rank on existing benchmarks IC-596 Multiple released LLMs fail to refuse harmful prompts disguised as historical or philosophical discussions, with GPT-4o and Mixtral showing the lowest refusal rates IC-605 Six SOTA LLMs are vulnerable to composable jailbreak attacks, with maximum attack success rates ranging from 44% to 94% IC-606 The relationship between model size and jailbreak vulnerability is reversed between Anthropic and Meta model families IC-681 GPT-3.5 exhibits positional bias when judging which of two LLM responses is superior IC-682 GPT-3.5 and GPT-4 achieve F1 scores of 0.5820 and 0.6180 respectively on pairwise response quality evaluation against human annotations IC-687 GPT-3.5+ models exhibit a gambler's fallacy bias and generate low-complexity sequences when asked to produce random binary sequences IC-688 GPT-3.5-turbo-instruct-0914 shows sharp phase transitions in in-context learning of simple formal languages, transitioning from random generation to deterministic pattern repetition as context length increases IC-689 Subjective randomness generation and sharp ICL transitions emerge only in larger or reward-fine-tuned models, absent in earlier GPT-3 variants and smaller open-source models IC-698 GPT-3.5-turbo's CoT reasoning errors are correlated across different demonstration sets, while PoT errors are less correlated IC-699 Llama2-13b produces significantly less consistent answers than GPT-3.5-turbo on complex reasoning tasks, making it unsuitable as a weaker LLM in a cascade IC-718 GPT-4 achieves 0.882 Pearson correlation with human evaluators on 45 customized score rubrics while GPT-3.5-Turbo achieves only 0.392 IC-719 Llama-2-Chat achieves reasonable human-preference accuracy (51.78-53.67%) as a prompted reward model without specific reward-model training IC-733 CoT prompting improves factual accuracy for instruction-tuned LLMs but degrades it for non-instruction-tuned LLMs such as OPT, BLOOM, and LLaMA IC-734 GPT-3.5-turbo's factual verification F1 decreases as the number of reasoning hops required to validate a claim increases IC-735 GPT-3.5-turbo's factual verification performance drops substantially under adversarial modifications, with man-made adversarial examples causing the largest decline IC-739 GPT-3.5-turbo-0613 combined with CLIP produces more faithful concept-salience pseudo-labels than LLaMA-2-13B-Chat, InstructBLIP, or LLaVA-1.5B on most of five datasets IC-750 Open-source models without safety training are significantly more vulnerable to jailbreak attacks than safety-aligned proprietary models IC-751 Arena-Hard-200 reveals larger performance gaps between open and proprietary LLMs than MT-Bench IC-752 GPT-4's win rate over GPT-3.5-turbo is 52% on the top-50 most challenging prompts but only 22% on the bottom-50 easiest prompts IC-761 GPT-3.5-turbo and GPT-4 are susceptible to specific circulating jailbreaking prompts, with 'jailmommy' achieving a 71.16% success rate in producing toxic outputs IC-762 GPT-4 and GPT-3.5 outperform humans in generation but underperform in discriminative (selective) evaluation across 10 of 13 language tasks IC-764 GPT-4 and GPT-3.5 make frequent errors answering questions about their own generated text, underperforming humans in interrogative evaluation IC-790 GPT-4 outperforms GPT-3.5-turbo, Vicuna-13B, and Llama2-70B for generating target captions in zero-shot compositional image retrieval IC-827 LLMs exhibit distinct psychological profiles that differ from human norms and vary by model size and version IC-829 Role assignment to GPT-3.5-turbo produces role-consistent changes in psychological profiles and task performance, validating the psychometric scales IC-848 GPT-3.5 achieves 98.3% precision and 96.0% recall for automatic question-tuple matching but makes errors when questions differ in wording yet are semantically unique IC-852 Intrinsic self-correction without external feedback consistently degrades reasoning accuracy across GPT-3.5-turbo, GPT-4, GPT-4-turbo, and LLaMA-2-70B-chat IC-853 Multi-agent debate with GPT-3.5-turbo-0301 does not outperform self-consistency at equivalent inference cost on GSM8K IC-854 The apparent self-correction improvement in constrained generation (Madaan et al., 2023) is an artefact of a sub-optimal initial prompt, not a genuine model capability IC-858 The choice of graph encoding method significantly changes LLM accuracy on graph reasoning tasks, with incident encoding outperforming adjacency by up to 34 percentage points on connected nodes IC-868 Skill-Mix performance degrades with increasing k, and within the Llama-2 family the saturation point increases with model size IC-898 GPT-3.5-turbo and GPT-4 are overconfident in their initial code predictions when unit test execution is unavailable IC-927 Human ciphers (ASCII, Unicode, Caesar, Morse) bypass the safety alignment of GPT-4 and GPT-3.5-turbo, with more powerful models producing more unsafe responses IC-928 SelfCipher (a role-play prompt without explicit cipher rules) evokes a 'secret cipher' in LLMs, achieving high unsafety rates that outperform most human ciphers IC-929 Simulated character-level ciphers that never appear in pretraining data cannot bypass safety alignment even with 10+ demonstrations IC-991 LLaMA-2, Falcon-7B, and GPT-3.5-Turbo exhibit large performance spread (up to 76 accuracy points) across semantically equivalent prompt formats, and model comparison rankings are frequently reversed by format choice IC-996 GPT-3.5 achieves 73.5% zero-shot accuracy on OGBN-ARXIV 40-class node classification and 73.56% on TAPe-ARXIV23, a dataset of papers published after its knowledge cutoff IC-998 GPT-3.5's zero-shot accuracy on OGBN-ARXIV depends on the position of the title relative to the abstract in the prompt: 0.720 when abstract precedes title, 0.695 when title precedes abstract Shared mechanisms Explanation faithfulness also in BakLLaVA , CF2 , Claude 3 , Claude 3.5 , CLIP / CLIP-ViT (LC) , DRUM , Fuyu , GEM , Gemini 1.5 / Gemini Pro 1.5 , Gemma 2 , GPT-4 / ChatGPT4 / GPT-4 Code Interpreter / GPT-4 Technical Report , GPT-4o , GPT-J , Idefics , Llama 3.1 , LLaVA-NeXT / LLaVA 1.6 , MAE-B/16 , MobileNetV2 , mPLUG-Owl3 , OpenFlamingo , PGExplainer , Pythia , RCExplainer , ResNet / ResNet-152 / ResNet-101 / ResNet-50-BN , SigLIP , SigLIP-2 , Stable Diffusion , TAGExplainer , ViT Failure mode also in AASIST , ADM , Aegis-Guard-Defensive , Alpaca , AnyLoc , AutoTikZ / DataTikZ , Baichuan , Baichuan 2 , Baichuan2-13B , BakLLaVA , Bard , BEiT , BERT , BingChat , BLIP , BLIP-2 , BLOOM , BSRNet , CF2 , Chat-UniVi-7B , ChatGLM-6B / ChatGLM-6b-2 , ChatGLM2 , ChatGPT , CLAP , Claude 1.3 , Claude 2.0 , Claude 2.1 , Claude 3 , Claude 3.5 , CLEAR , CLIP / CLIP-ViT (LC) , CLIP4Clip , CLIPBERT , CLIPCap , CLMBR-T-BASE , CloFNet , Code Llama , CodeGeex2 , CodeGen , CodeLlama-13B , CodeLlama-34B , CogVLM2 , Cohere Command R , CoMEt , Command R+ , CONCH , CycleGAN , DALL-E , DALL·E 2 , DALL·E 3 , DASR , DECAF , DeepSeek-2-Chat , DeepSeek-2-Coder , DeepSeek-V2-0628 , DeepSeek-VL , DeepSeek-VL2 , DeiT , DeiT-III , Depth Anything , DETR-R50 , DimeNet++ , DINO , DINO-FocalNet-Large , DINOv2 , EGNN , Emu2 , EquiformerV2 , ESCN , ESM-2 , ESM3 , ESRGAN , EVA-CLIP , EVE , Falcon , Faster R-CNN / Faster R-CNN R50 / Faster R-CNN X101 , FLAN-T5 , Florence-2 , FLUX / FLUX1 , FoveaBox , Fuyu , Galactica-6.7B , GAT , 3D Gaussian Splatting , GCN , Gemini , Gemini 1.5 / Gemini Pro 1.5 , Gemma , Gemma 2 , GIN , GLIDE , GLM-4 , GLM-4V , GloVe , GP-UNIT , GPT-2 , GPT-3 / GPT base , GPT-4 / ChatGPT4 / GPT-4 Code Interpreter / GPT-4 Technical Report , GPT-4.1 , GPT-4o , GPT-J , GPT-NeoX-20B , GraphSAGE , Grounding DINO , Guanaco , GVP , Hawkeye , HiFaceGAN , HPS , HuggingGPT , IDDPM , Idefics , Idefics2 , ImageBind , ImageBind-LLM-7B , Imagen Video , ImageReward , 12-in-1 , InstructBLIP , InstructGPT , InternLM-2.5-7B , InternLM-XComposer2-VL , InternVideo , InternVL-1.5 , InternVL2 , Koala , LegalBERT , LLaMA , Llama 2 / Llama 2 base , Llama 3 , Llama 3.1 , Llama 3.2 , Llama-3.2-3B , Llama-3-2-Vision , LLaMA-Adapter v2 , Llama Guard , Llama Guard 2 , Llama-Guard 3 , Llama-VID , LLaVA , LLaVA-1.5 / LLaVA-v1.5 , LLaVA-Med , LLaVA-NeXT / LLaVA 1.6 , LLaVA-OneVision , LongVA-7B , LOVT , LWM-1M-JAX , MACE , MAE , Med-Flamingo , Merlot Reserve , MGCA , MiDaS , MiniCPM-V , MiniGPT-4 , Mip-Splatting , Mistral 7B / Mistral / Mistral 3 7B / Mistral-0.2-7B / Mistral-v0.1 , Mistral Large 2 , Mistral Large V2 , Mixtral , Mixtral 8x7B / Mistral 8x7B Instruct / Mixtral 46.7B / Mixtral 8x7B Instruct / Mixtral-instruct-8x7b , MobileNetV2 , Molmo , MolmoE-7B , Momentor , Moondream2 , Moonshot-v1-8k , mPLUG-2 , mPLUG-Owl , mPLUG-Owl3 , mPLUG-Owl2 , MPT , MSA Transformer , MultiBERTs , Nova Canvas , Nova Lite , Nova Pro , O1 / OpenAI-o1-preview , O3 , O4-mini , OLMo / OLMo base , OneLLM , OpenAI Moderation , OpenChat-3.5-0106 , OpenCLIP , OpenFlamingo , OPT , Otter , Otter-7B , PaLM 2 , PaLM 62B , PandaGPT-7B , PerSAM , Phi-3 , Phi-3.5 Mini Instruct , PickScore , PLIP , Prismatic , ProGen-2 , Pythia , Qwen1.5 , Qwen 2 , Qwen 2.5 72B Instruct , Qwen2-VL , Qwen-Audio , Qwen-VL , Qwen2.5 , Qwen2-Audio , R2D2 , RadFM , RCExplainer , RD-SR , RealESRNet , Reprover , ResNet / ResNet-152 / ResNet-101 / ResNet-50-BN , RetinaNet-R50 , RivaGAN , RS-LDS , RT-1 , RT-2 , SALMONN , SAM , SAM 2 , SAULLM 54B , Scaffold-GS , SchNet , Seed-LLaMA-8B , SGC , SIREN , Sketch Transformer , SLD-max , SLD-medium , SLD-strong , SLDS , SLIMG , SpeechGPT , SphereNet , SRResNet , Stable Diffusion , StackLLaMA , Starcoder , StegaStamp , StyleGAN2-ADA , Swin Transformer , T5 , TD-MPC , TerraMind , TimeChat , TranceptionEVE , TreeRing , Tulu 2 , UnifiedQA , UniPerceiver , UNITER , UniVL , Van Gogh Diffusion , VERA , VGG / VGG13 , Vicuna , Video-Chat-7B , Video-ChatGPT , Video-LLaMA , Video-LLaMA-2-13B , Video-LLaVA , VideoCLIP , ViLA-8B , ViLBERT , VindLU , VioLET , ViRTex , ViT , ViV1T , VTG-LLM , WildGuard , Wizardlm , X-CLIP , X-InstructBLIP-7B , XGen-MM , Xlm-R , Zephyr-7B-beta Method artefact also in Baichuan , CLIP / CLIP-ViT (LC) , ConvNeXt , EfficientNet , Falcon , Gemma , Gemma 2 , GPT-2 , GPT-4o , GPT-J , InternLM , Llama 2 / Llama 2 base , Llama 3 , Llama 3.1 , MAP-NEO , Mistral 7B / Mistral / Mistral 3 7B / Mistral-0.2-7B / Mistral-v0.1 , OLMo / OLMo base , OpenCLIP , OpenLLaMA , OPT , Pythia , Qwen1.5 , Qwen 2 , Qwen2.5 , RedPajama , ResNet / ResNet-152 / ResNet-101 / ResNet-50-BN , Skywork , Stable Diffusion , StableLM , TerraMind , ViT , Yi , ZiYA2 Positional bias also in BERT , ChatGPT , Claude 3 , Claude 3.5 , Falcon , Fuyu , Gemini , Gemini 1.5 / Gemini Pro 1.5 , Gemma , Gemma 2 , GLIDE , GPT-2 , GPT-4 / ChatGPT4 / GPT-4 Code Interpreter / GPT-4 Technical Report , GPT-4.1 , GPT-4o , GPT-J , InstructGPT , LLaMA , Llama 2 / Llama 2 base , Llama 3 , Mistral 7B / Mistral / Mistral 3 7B / Mistral-0.2-7B / Mistral-v0.1 , Mixtral , Mixtral 8x7B / Mistral 8x7B Instruct / Mixtral 46.7B / Mixtral 8x7B Instruct / Mixtral-instruct-8x7b , MPT , O1 / OpenAI-o1-preview , O3 , O4-mini , PaLM 2 , Phi-3 , Pythia , Qwen1.5 , Qwen 2 , Qwen 2.5 72B Instruct , Stable Diffusion , Sybil , Vicuna Scale-dependent behaviour also in Aquila-7B , BEiT , BERT , BLOOM , Claude 2.1 , Claude 3 , Claude 3.5 , CLIP / CLIP-ViT (LC) , Code Llama , CodeGen , Cohere Command R , DeepSeek LLM , DeepSeekMoE , DeiT-III , DINO , DINOv2 , EquiformerV2 , ESCN , Falcon , FLAN-T5 , Gemini 1.0 Pro , Gemini 1.5 / Gemini Pro 1.5 , Gemma , Gemma 2 , GPT-2 , GPT-3 / GPT base , GPT-4 / ChatGPT4 / GPT-4 Code Interpreter / GPT-4 Technical Report , GPT-4o , GPT-J , GPT-Neo , I3D , Idefics , InternLM-2.5-7B , InternLM-XComposer2-VL , InternLM2 , InternVL-1.5 , InternVL2 , LLaMA , Llama 2 / Llama 2 base , Llama 3 , Llama 3.1 , Llama 3.2 , Llama-3.2-3B , LLaVA-1.5 / LLaVA-v1.5 , LLaVA-NeXT / LLaVA 1.6 , LongVA-7B , MAE , Mistral 7B / Mistral / Mistral 3 7B / Mistral-0.2-7B / Mistral-v0.1 , Mixtral , Mixtral 8x7B / Mistral 8x7B Instruct / Mixtral 46.7B / Mixtral 8x7B Instruct / Mixtral-instruct-8x7b , Moirai , MPT , MViT V2 , O1 / OpenAI-o1-preview , OLMo / OLMo base , OpenCLIP , OpenFlamingo , OpenLLaMA , OPT , PaLM 2 , Phi-3 , Platypus2-Instruct-70B , Pythia , Qwen , Qwen1.5 , Qwen 2 , Qwen 2.5 72B Instruct , Qwen2-VL , Qwen-Audio , Qwen2.5 , RedPajama-INCITE , ResNet / ResNet-152 / ResNet-101 / ResNet-50-BN , SlowFast , Solar 10.7B , Stable Diffusion , StableLM , Swin Transformer , T5 , TerraMind , text-ada-001 , TigerBot-70B-Chat , TimesFormer , TSM , Tulu 2 , Uniformer , Vicuna , VideoMAE , ViLA-8B , Wizardlm , X3D , XGLM , Xwin-LM-70B-v0.1 , Yi Shortcut also in BakLLaVA , BLIP-2 , Claude 3 , Claude 3.5 , CLIP / CLIP-ViT (LC) , CLIP4Clip , CLIPBERT , DALL·E 2 , DALL·E 3 , DeepSeek-VL2 , Eurus-RM-7B , Falcon , FLUX / FLUX1 , Gemini , Gemini 1.5 / Gemini Pro 1.5 , Gemma , Gemma 2 , GPT-2 , GPT-3 / GPT base , GPT-4 / ChatGPT4 / GPT-4 Code Interpreter / GPT-4 Technical Report , GPT-4o , GPT-J , iFlytekSpark-13B , InstructBLIP , Internlm2-Reward , InternVideo , InternVL2 , LLaMA , Llama 2 / Llama 2 base , Llama 3 , Llama-3-2-Vision , LLaVA , LLaVA-1.5 / LLaVA-v1.5 , LLaVA-Med , LLaVA-NeXT / LLaVA 1.6 , Med-Flamingo , Merlot Reserve , MiniGPT-4 , Mistral 7B / Mistral / Mistral 3 7B / Mistral-0.2-7B / Mistral-v0.1 , Mixtral , Mixtral 8x7B / Mistral 8x7B Instruct / Mixtral 46.7B / Mixtral 8x7B Instruct / Mixtral-instruct-8x7b , Molmo , mPLUG-2 , mPLUG-Owl3 , Nova Canvas , O1 / OpenAI-o1-preview , OpenCLIP , OPT , Otter , PaLM 62B , Pythia , Qwen , Qwen 2 , Qwen-VL , Qwen2.5 , RadFM , ResNet / ResNet-152 / ResNet-101 / ResNet-50-BN , Stable Diffusion , Swin Transformer , Sybil , TerraMind , Tulu 2 , UniPerceiver , UniVL , Vicuna , Video-LLaMA , VideoCLIP , VindLU , VioLET , X-CLIP