Light Dark OpenAI · text, image · generative · anchor · artifact
Closed-weight multimodal model served through an API, the most frequently measured model in the corpus, present in four of the twenty-one papers.
Note anchor found by search and checked against this entry: the GPT-4o System Card, which OpenAI also publishes on its own site. The citing papers in this corpus attach the older GPT-4 technical report instead Variants GPT-4o 0513 , GPT-4o 0806 , GPT-4o mini , GPT-4o-2024-05-13 , GPT-4o-2024-08-06 , GPT-4o-mini-2024-07-18 Findings IC-001 Vision-language models perform near chance on the NL-Eye visual abductive reasoning benchmark IC-002 Even when VLMs select the correct hypothesis, their explanations are often invalid or unhelpful IC-005 GPT-4o underperforms AHA and other VLMs in detecting and reasoning about robotic manipulation failures across multiple datasets. IC-007 Most LLMs do not align closely with human moral preferences on multilingual trolley problems IC-008 Misaligned models tend to binarize moral preferences while better-aligned models capture probabilistic nuances IC-009 LLM moral preferences show significant language sensitivity but not inequality toward low-resource languages IC-025 LMMs exhibit poor fine-grained perception in locating individual characters on original oracle bones IC-026 LLMs can assist in OB rejoining by identifying rejoinable fragments with moderate accuracy, but are not yet truly usable IC-027 LMM performance in deciphering oracle bone inscriptions is comparable to untrained humans for common characters but declines for rarer and structurally complex characters IC-029 Large language models show conformity to group answers in multi-agent interactions IC-030 Larger language models exhibit higher independence rates and lower conformity under some protocols IC-049 GPT-4o shows strong performance on some E.T.Bench event-level tasks (RVQ: 57.7, VHD: 56.9) but very weak performance on others (EPM: 4.5, TAL: 20.0) IC-050 GPT-4o achieves 53.33 overall on Event-Bench with strong event description (57.50) and counter reasoning (63.44) but weaker episodic reasoning (37.33) IC-072 LLM agents of varying scales exhibit a failure mode on web automation tasks when processing raw, complex web page observations, with the penalty being more severe for smaller models IC-073 Released LLMs (GPT-4o, Llama-3.1-70B, Qwen2-7B, etc.) show limited workflow orchestration capability that degrades as workflow complexity increases IC-075 GPT-4o-mini and Qwen2.5-72B achieve low precision and recall when used as API retrievers for workflow orchestration IC-082 GPT-3.5, GPT-4o, Claude-3.5-Sonnet, and Llama-3.1-8B produce explanations on the BBQ social bias task that are systematically unfaithful for identity and behavior concepts while remaining faithful for context concepts, with specific patterns of hiding safety-measure influence and social bias IC-083 GPT-3.5, GPT-4o, and Claude-3.5-Sonnet produce unfaithful explanations on MedQA medical questions, omitting high-effect clinical concepts such as the patient's mental status while over-referencing low-effect concepts IC-092 All evaluated LLMs show consistent F1 degradation to at most 0.60 when two or more events match a retrieval cue IC-093 No evaluated LLM achieves perfect confabulation avoidance on questions about non-existent events IC-094 Episodic recall accuracy degrades systematically from content cues to space cues to time cues across all evaluated LLMs IC-095 Evaluated LLMs achieve at most 36% latest-state accuracy and 18% full-set accuracy on multi-event entity tracking, with low Kendall's tau on chronological ordering IC-096 Large commercial and open-weight models achieve 70-78% accuracy on CASELAWQA, with Claude 3.7 Sonnet at the top IC-102 GPT-4o's spatial understanding degrades when depth maps are provided as additional input on SpatialBench IC-112 Released LLMs show a reproducible failure mode where numerical task accuracy degrades sharply as input digit length increases IC-113 Released LLMs show a reproducible failure mode where accuracy on fraction and scientific notation tasks falls below 20% even for the shortest inputs IC-114 Released LLMs cannot reliably identify a specific digit in a number as the number's length increases, with GPT-4o achieving only 20% on get-digit in the xl range IC-115 NUPA performance is largely independent of model size within a family: GPT-4o and GPT-4o-mini show nearly identical performance, as do Qwen2-72B and Qwen2-7B IC-155 All 13 evaluated MLLMs perform at or near random guessing on MediConfusion, with confusion scores often exceeding 90%, indicating they cannot distinguish visually dissimilar radiology image pairs IC-157 GPT-4o's MediConfusion performance is robust to prompt format while InstructBLIP is highly sensitive and LLaVA-Med fails completely on multiple-choice evaluation IC-159 GPT-4o achieves 55.6% accuracy on creation, 74.8% on math, and 68.1% on code as a preference judge, and is outperformed by domain-specific 7B models on those tasks IC-174 RAG reduces model abstention and LLMs hallucinate rather than abstain when the retrieved context is insufficient to answer the query IC-175 Context-sufficiency performance is scale-dependent: larger LLMs achieve high accuracy with sufficient context but still answer correctly 35-62% of the time without it, while smaller models hallucinate or abstain even with sufficient context IC-176 LLaMA 3.1 8B Instruct's KGQA accuracy degrades with increasing numbers of retrieved triples, while GPT-4o-mini's accuracy improves, revealing different context-handling capacities IC-177 GPT-4o mini, GPT-4o, and Llama-3-8B all over-rely on incorrect external context, producing wrong answers at high rates when the context conflicts with their internal knowledge IC-178 Self-guided confidence reasoning (SCR) outperforms rule-based confidence reasoning (RCR) for GPT-4o and GPT-4o mini, but RCR outperforms SCR for Llama-3-8B IC-179 GPT-4o mini, GPT-4o, and Llama-3-8B all calibrate confidence in their internal answers significantly better than confidence in external contexts IC-180 GPT-4o-mini's resistance to incorrect context depends on the position of the context relative to the question in the prompt IC-188 LLMs show constraint-type-specific performance on system message following, with weaker models exhibiting large variance across constraint categories IC-189 Most LLMs show degraded instruction satisfaction when user instructions conflict with system messages, indicating difficulty in prioritizing system message constraints IC-190 LLMs show progressive degradation in system message constraint following across multi-turn conversations, with dependent conversations degrading faster than parallel ones IC-198 Safety-aligned LLMs (GPT-4, GPT-3.5, Gemma2-27b, GPT-4o, Gemma2-9b, Qwen2.5-72b, Mistral-7b, Mixtral-8x22b) are vulnerable to natural prompts semantically related to toxic seed prompts, with attack success rates of 82-99% IC-199 GPT-4o generates natural jailbreak questions from toxic answers without denial, demonstrating an asymmetry in safety training where forward safety (question-to-answer) does not guarantee reverse safety (answer-to-question) IC-202 All six evaluated LLMs achieve very low accuracy on OpenRCA, with no model solving any three-element root cause query IC-203 Gemini 1.5 Pro's RCA-Agent accuracy drops 68.4% when code execution fails, far exceeding the drops for Claude 3.5 (17.9%) and GPT-4o (15.6%) IC-204 GPT-4o performs worse with explicit chain-of-thought prompting than with the original prompt on OpenRCA tasks IC-206 GPT-4o-0513 achieves the highest wb-reward mix score (35.7) on WildBench, with a clear three-tier structure among 40 evaluated LLMs IC-209 LLM judges (GPT-3.5-turbo-1106, GPT-4o-mini, GPT-4o, Claude-3-5-sonnet) implicitly prioritize style over factuality and safety when scoring pairwise preferences IC-210 GPT-4o-mini-2024-07-18 does not exhibit authority bias when used as a judge: appending fabricated references to model responses decreases rather than increases its score IC-221 GPT-4o ReAct success rate drops from 47% on synchronous to 11% on asynchronous planning tasks, and all other tested LLMs show equal or worse performance IC-222 GPT-4o ReAct failures are dominated by rule violations (transition function) and goal misinterpretation, with the balance shifting from goal-dominant in synchronous to transition-dominant in asynchronous settings IC-223 GPT-4o ReAct shows poor recovery from failures in asynchronous settings, with 58.6% of failed runs making little to no progress toward the goal and significantly higher repeated transitions than in synchronous settings IC-224 GPT-4o ReAct cannot incorporate stochastic state changes, with success rate on cutting tasks dropping from 56% to 1% when a 33% chance of a cut item reverting to uncut is introduced IC-238 LLMs fail to follow user preferences in zero-shot settings, with accuracy below 10% at 10 turns and near zero at 300 turns IC-252 GPT models produce harmful gender stereotypes at higher rates when user names imply a demographic group, with GPT-3.5 Turbo showing the highest rates and open-ended generation tasks most affected IC-253 Post-training reinforcement learning significantly reduces harmful gender stereotypes in GPT models, with the best-fit slope of 0.21 indicating post-RL models have far lower bias than pre-RL versions IC-254 GPT-4o Mini responses to female-sounding names systematically use simpler, more light-hearted, and less technical language compared to male-sounding names across multiple task domains IC-255 GPT-4o, Llama 3.1, and Claude models show varying correlation with human ratings when used as stereotype evaluators, with GPT-4o achieving the strongest gender correlation (ρ=0.86) but weaker racial correlations IC-264 All 18 evaluated LLMs fail to abstain when the provided context lacks the answer, with performance gaps of 13.6% to 68.4% relative to the original context IC-265 Model families show extreme variation in detecting conflicting answers in inconsistent contexts, with phi-3 series at 5.8% average accuracy versus GPT-4 series at 89.35% IC-266 GPT-4o drops from 96.3% closed-book accuracy to 47.5% when given counterfactual context that contradicts its parametric knowledge, far below the 95% human accuracy on the same items IC-267 Adding a 'conflict' instruction to the prompt degrades GPT-4o and Claude 3.5 Sonnet accuracy on normal (answerable, consistent) contexts by 5% and 2% respectively IC-276 All 14 evaluated VLMs show a large gap between average-case and worst-case accuracy on DynaMath variants, with worst-case at or below 50% of average-case, and the failures are systematic rather than random IC-278 Claude-3.5 Sonnet and GPT-4o exhibit a memorization failure mode, outputting the same answer regardless of visual parameter changes in the problem IC-307 56 LLMs on Sorry-Bench show fulfillment rates ranging from below 10% (Claude-2, Gemini-1.5) to above 90% (Mistral-7B-instruct-v0.1, Dolphin-2.6-mixtral-8x7b), with GPT-4o at 30% and Llama-3-70B at 35% IC-308 Linguistic mutations to unsafe prompts significantly and inconsistently alter safety refusal across models, with persuasion techniques increasing fulfillment by 5-66% and encoding/encryption decreasing it by 15-68% IC-309 As zero-shot safety judges, GPT-4o achieves 78.9% Cohen's kappa agreement with human annotators while Llama-3-8B-instruct (39.0%) and Mistral-7B-instruct-v0.2 (53.9%) perform substantially worse IC-312 GPT-4o achieves the highest insight-level Llama-3-eval score (0.60) among all tested LLM backbones on InsightBench multi-step data analytics IC-313 All four LLM backbones fail to detect a planted linear trend in incident resolution time when the slope is below 0.1, and detection rates diverge sharply above that threshold IC-315 CoT prompting (reasoning + instruction) yields larger relative gains for larger LLMs and harder problems in competitive code generation, with the effect reversing for the most capable models IC-316 Multi-turn code generation without CoT degrades performance for smaller Llama models and GPT-4o compared to single-turn repeated sampling under equal compute budgets IC-321 GPT-4o selects evaluation dimensions with high precision but low recall, indicating a selective rather than comprehensive strategy IC-322 Past-tense reformulations of harmful requests bypass refusal training in eight released LLMs, while future-tense reformulations are substantially less effective IC-327 Llama-3-8B-Instruct, Mistral-7B-Instruct-v0.3, and several other LLMs produce well-calibrated verbal confidence estimates on classification tasks IC-329 GPT-4o is more robust to confidence-elicitation-guided word substitution attacks than open-source LLMs, with lower attack success rates and better confidence calibration IC-333 Llama-3-405B overexplains math solutions with structured markdown headings and conversational tone compared to GPT-4o's concise formal notation, achieving 97% model-matching accuracy IC-343 GPT-4o, Claude-3.5 Sonnet, and GeminiPro-1.5 score below BigDocs-trained open models on BigDocs-Bench tasks requiring long structured code generation IC-344 GPT-4o achieves the highest average score (64.62) on general document benchmarks, outperforming Qwen2-VL-72B (58.40) and GeminiPro-1.5 (57.05) IC-345 GPT-4o's table2latex outputs lose 63% of the time in human evaluation, with inconsistent formatting (lines, borders, margins) as the primary failure IC-362 LLMs with chain-of-thought prompting predict and simulate human risky choices that are more rational than actual human behavior, correlating more highly with maximum expected value than with human choices IC-363 LLM inferences about others' preferences from observed decisions are highly correlated with human inferences because both assume the decision-maker is rational IC-368 Larger LMs (GPT-4o, Claude-3.5-Sonnet, Gemini-1.5-Pro) exhibit better calibration than their smaller counterparts (GPT-4o-mini, Claude-3-Haiku, Gemini-1.5-Flash) when verbalizing confidence with certainty phrases IC-369 LMs verbalizing confidence with certainty phrases are better calibrated on SCIQ than on TruthfulQA IC-387 OpenAI-o1 models substantially improve CS reasoning over GPT-4o at the cost of 14-30x token consumption IC-389 All evaluated LLMs score significantly lower on CS reasoning questions than knowledge questions, with the gap narrowing for stronger models IC-395 Most mainstream LLMs exhibit positive ADCE across five tasks, indicating reliance on deep structure for problem-solving, with ADCE strongly correlated with accuracy (r² > 0.7) IC-396 Closed-source LLMs (GPT, Claude) rely more on deep structure than open-source LLMs (Llama, Mistral), and open-source models' surface sensitivity decreases with model scale IC-407 Safety-aligned LLMs (Llama-2-chat, Llama-3-instruct, Gemma, GPT-3.5, GPT-4o, R2D2) achieve 100% jailbreak attack success rate under adaptive prompt-and-suffix attacks on 50 harmful requests IC-433 LLMs exhibit a non-monotonic ID-OOD performance gap (generalization valley) that peaks at intermediate task complexity IC-434 The critical complexity at which LLMs over-rely on memorization shifts to higher task difficulty as model size increases IC-436 API selection accuracy of 10 LLM-based agents degrades sharply as task complexity increases, with open-source models ≥70B matching closed-source on simpler tasks but lagging on the most complex IC-437 Extracting parameters from user queries is harder for LLM-based agents than using outputs from previous actions, and less intelligent LLMs show steeper parameter-filling degradation with task difficulty IC-438 All 10 LLM-based agents perform poorly at recognizing when they need to request input from the system or user, with overall accuracy between 30.55% and 55.18% IC-440 GPT-4o, Gemini 1.5 Pro, and Claude 3.5 Sonnet show up to 25% skill-level accuracy gaps despite overall accuracies within 0.4% of each other IC-441 Skill-level improvements between model releases are highly uneven, with Claude 3.5 Sonnet gaining ~50% over Claude 3 Opus on law skills while Gemini improved most in math and science IC-442 Routing each evaluation instance to the model strongest on its relevant skills yields a 3.2% accuracy gain over the best single model, with 3.5-6.8% gains on MMLU Pro IC-443 Model inconsistency on probing questions negatively correlates with skill-slice accuracy (r = -0.675), with models contradicting themselves more often on skills where they perform poorly IC-465 GPT-4o achieves the highest weighted Cohen's kappa agreement with a reference-guided agent-debate method among all VLM judges, with scores exceeding 50 in several visual dimensions IC-466 GPT-4o's evaluation reliability degrades when used as the final judge in a collective thought pipeline that aggregates reviews from less reliable VLMs IC-467 Llama-3.1-405B's standard speculative decoding verification rejects correct continuations from GPT-4o, Llama-3.1-8B, and human text, accepting only roughly two tokens before the first rejection for GPT-4o IC-474 GPT-4, GPT-4o, and Llama-3.1-405B fail at knowledge classification and comparison without chain-of-thought IC-475 GPT-4, GPT-3.5, GPT-4o, and Llama-3.1-405B fail at inverse knowledge search regardless of prompting IC-477 Released LLMs achieve limited success rates as web agents on WebArena-Lite, with open-source models substantially below proprietary ones IC-480 Pre-trained M-LLMs (GPT-4o, LLaVA-v1.6-34B, InternVL2-26B, Qwen2-VL-7B) produce imprecise tampering explanations when artifacts require fine-grained pixel-level analysis such as lighting or perspective inconsistencies IC-489 State-of-the-art MLLMs fail at multi-step visual analogical reasoning, with best accuracy at 13% (Llama 3.2) on VOILA-WD and 29% (GPT-4o) on VOILA-ND, far below human performance of 71% and 70% IC-490 GPT-4o can identify visual relationships at 97% accuracy when given ground-truth descriptions but drops to 17% when asked to apply known relationships to new visuals, revealing a specific bottleneck in relational transfer IC-491 Presenting three images as a single collage rather than sequentially reduces MLLM accuracy by approximately 40% on the relationship application step IC-492 Least-to-most prompting consistently improves MLLM accuracy on the relationship application step compared to direct answering, with GPT-4o improving from 0.9% to 6.44% on VOILA-WD IC-495 All evaluated multimodal foundation models achieve average non-hallucination accuracy below 50% across six hallucination scenarios IC-496 GPT-4o achieves the highest location inference accuracy among evaluated models, reaching 98.16% for country, 60.23% for city, and 27.13% for zip code from street view images IC-497 Text-to-image models experience performance drops exceeding 10% under adversarial prompts, with spatial reasoning being the most vulnerable task across all models IC-498 Multimodal foundation models exhibit severe group unfairness, with race and age biases more pronounced than gender bias in text-to-image models while gender bias is stronger in image-to-text models IC-500 GPT-4o, Claude 3.5 Sonnet, Gemini 1.5 Pro, and Llama 3.1 405B solve multi-step retrieval problems without fine-tuning, achieving near-perfect accuracy for chains of up to 5 steps IC-504 GPT-4o (gpt-4o-2024-08-06) rates the political slant of LLM-generated essays in close agreement with politically balanced human annotators IC-510 Qwen2-7B and Llama3-8B score near-random on textual temporal reasoning tasks while Qwen2-72B, Llama3-70B, and GPT-4o achieve near-perfect accuracy, showing temporal reasoning in LLMs is scale-dependent and emerges only above ~70B parameters IC-518 GPT-4o achieves 62.54% overall accuracy on MMWorld, the best among 15 MLLMs, while four open-source models perform below the 26.31% random-choice baseline IC-519 MLLMs exhibit different skill sets than humans, correctly answering expert-level questions that all three human annotators miss while failing on easy questions humans answer correctly IC-520 Temporal reasoning performance drops significantly across all 15 MLLMs when video frames are shuffled or reduced to one-fifth of the original count IC-522 LLMs perform correct example inference without inducing the correct rule, and this gap is robust to prompting methods, fact count, and scenario form IC-523 LLMs rely on observed facts close to the test case in input feature space (neighbor-based reasoning) rather than on an abstract rule, and this effect is localized IC-539 GPT-4o in text-code-image mode achieves the highest scores on SCIMAGE but remains below 4 on all three evaluation dimensions, and all models degrade substantially on prompts requiring combined understanding types IC-540 Spatial understanding is the most challenging dimension for code-based models while numerical understanding is most challenging for direct image models IC-541 Code-based output (Python or TikZ) produces images with notably higher scientific style scores than direct image generation from Stable Diffusion and DALL-E IC-572 Bijection learning achieves state-of-the-art jailbreak ASR on frontier models, with peak ASR increasing with model capability IC-573 Model capabilities on MMLU degrade monotonically as bijection encoding complexity increases IC-574 Guard models fail to effectively mitigate bijection attacks even at capability parity with the target model IC-586 Symbolic distance (number of reasoning steps) is the primary bottleneck for relational reasoning in LLMs, not total context length IC-587 Real-world knowledge acts as a shortcut in LLM relational reasoning, causing worse-than-chance performance on logically valid but factually incongruent statements IC-588 Topologically ordered context improves relational reasoning over random ordering across nearly all LLMs IC-589 Flavor text (non-essential descriptive language) degrades relational reasoning in most LLMs, but GPT-4o is robust to it IC-596 Multiple released LLMs fail to refuse harmful prompts disguised as historical or philosophical discussions, with GPT-4o and Mixtral showing the lowest refusal rates IC-597 Most LMMs exhibit systematic class bias in synthetic data detection, with GPT-4o biased toward classifying text as real and 3D as AI-generated IC-598 GPT-4o's synthetic image detection accuracy drops sharply on specialized domains (satellite 45.0%, medical 54.3%) compared to common image types (object 84.4%, person 84.4%) IC-600 Chain-of-thought prompting improves most LMMs on synthetic detection but degrades LLaVA-ov-7b from 56.6% to 18.8%, while GPT-4o performs well without it (64.1% baseline) IC-605 Six SOTA LLMs are vulnerable to composable jailbreak attacks, with maximum attack success rates ranging from 44% to 94% IC-606 The relationship between model size and jailbreak vulnerability is reversed between Anthropic and Meta model families Shared mechanisms Explanation faithfulness also in BakLLaVA , CF2 , Claude 3 , Claude 3.5 , CLIP / CLIP-ViT (LC) , DRUM , Fuyu , GEM , Gemini 1.5 / Gemini Pro 1.5 , Gemma 2 , GPT-3.5 / ChatGPT-3.5 , GPT-4 / ChatGPT4 / GPT-4 Code Interpreter / GPT-4 Technical Report , GPT-J , Idefics , Llama 3.1 , LLaVA-NeXT / LLaVA 1.6 , MAE-B/16 , MobileNetV2 , mPLUG-Owl3 , OpenFlamingo , PGExplainer , Pythia , RCExplainer , ResNet / ResNet-152 / ResNet-101 / ResNet-50-BN , SigLIP , SigLIP-2 , Stable Diffusion , TAGExplainer , ViT Failure mode also in AASIST , ADM , Aegis-Guard-Defensive , Alpaca , AnyLoc , AutoTikZ / DataTikZ , Baichuan , Baichuan 2 , Baichuan2-13B , BakLLaVA , Bard , BEiT , BERT , BingChat , BLIP , BLIP-2 , BLOOM , BSRNet , CF2 , Chat-UniVi-7B , ChatGLM-6B / ChatGLM-6b-2 , ChatGLM2 , ChatGPT , CLAP , Claude 1.3 , Claude 2.0 , Claude 2.1 , Claude 3 , Claude 3.5 , CLEAR , CLIP / CLIP-ViT (LC) , CLIP4Clip , CLIPBERT , CLIPCap , CLMBR-T-BASE , CloFNet , Code Llama , CodeGeex2 , CodeGen , CodeLlama-13B , CodeLlama-34B , CogVLM2 , Cohere Command R , CoMEt , Command R+ , CONCH , CycleGAN , DALL-E , DALL·E 2 , DALL·E 3 , DASR , DECAF , DeepSeek-2-Chat , DeepSeek-2-Coder , DeepSeek-V2-0628 , DeepSeek-VL , DeepSeek-VL2 , DeiT , DeiT-III , Depth Anything , DETR-R50 , DimeNet++ , DINO , DINO-FocalNet-Large , DINOv2 , EGNN , Emu2 , EquiformerV2 , ESCN , ESM-2 , ESM3 , ESRGAN , EVA-CLIP , EVE , Falcon , Faster R-CNN / Faster R-CNN R50 / Faster R-CNN X101 , FLAN-T5 , Florence-2 , FLUX / FLUX1 , FoveaBox , Fuyu , Galactica-6.7B , GAT , 3D Gaussian Splatting , GCN , Gemini , Gemini 1.5 / Gemini Pro 1.5 , Gemma , Gemma 2 , GIN , GLIDE , GLM-4 , GLM-4V , GloVe , GP-UNIT , GPT-2 , GPT-3 / GPT base , GPT-3.5 / ChatGPT-3.5 , GPT-4 / ChatGPT4 / GPT-4 Code Interpreter / GPT-4 Technical Report , GPT-4.1 , GPT-J , GPT-NeoX-20B , GraphSAGE , Grounding DINO , Guanaco , GVP , Hawkeye , HiFaceGAN , HPS , HuggingGPT , IDDPM , Idefics , Idefics2 , ImageBind , ImageBind-LLM-7B , Imagen Video , ImageReward , 12-in-1 , InstructBLIP , InstructGPT , InternLM-2.5-7B , InternLM-XComposer2-VL , InternVideo , InternVL-1.5 , InternVL2 , Koala , LegalBERT , LLaMA , Llama 2 / Llama 2 base , Llama 3 , Llama 3.1 , Llama 3.2 , Llama-3.2-3B , Llama-3-2-Vision , LLaMA-Adapter v2 , Llama Guard , Llama Guard 2 , Llama-Guard 3 , Llama-VID , LLaVA , LLaVA-1.5 / LLaVA-v1.5 , LLaVA-Med , LLaVA-NeXT / LLaVA 1.6 , LLaVA-OneVision , LongVA-7B , LOVT , LWM-1M-JAX , MACE , MAE , Med-Flamingo , Merlot Reserve , MGCA , MiDaS , MiniCPM-V , MiniGPT-4 , Mip-Splatting , Mistral 7B / Mistral / Mistral 3 7B / Mistral-0.2-7B / Mistral-v0.1 , Mistral Large 2 , Mistral Large V2 , Mixtral , Mixtral 8x7B / Mistral 8x7B Instruct / Mixtral 46.7B / Mixtral 8x7B Instruct / Mixtral-instruct-8x7b , MobileNetV2 , Molmo , MolmoE-7B , Momentor , Moondream2 , Moonshot-v1-8k , mPLUG-2 , mPLUG-Owl , mPLUG-Owl3 , mPLUG-Owl2 , MPT , MSA Transformer , MultiBERTs , Nova Canvas , Nova Lite , Nova Pro , O1 / OpenAI-o1-preview , O3 , O4-mini , OLMo / OLMo base , OneLLM , OpenAI Moderation , OpenChat-3.5-0106 , OpenCLIP , OpenFlamingo , OPT , Otter , Otter-7B , PaLM 2 , PaLM 62B , PandaGPT-7B , PerSAM , Phi-3 , Phi-3.5 Mini Instruct , PickScore , PLIP , Prismatic , ProGen-2 , Pythia , Qwen1.5 , Qwen 2 , Qwen 2.5 72B Instruct , Qwen2-VL , Qwen-Audio , Qwen-VL , Qwen2.5 , Qwen2-Audio , R2D2 , RadFM , RCExplainer , RD-SR , RealESRNet , Reprover , ResNet / ResNet-152 / ResNet-101 / ResNet-50-BN , RetinaNet-R50 , RivaGAN , RS-LDS , RT-1 , RT-2 , SALMONN , SAM , SAM 2 , SAULLM 54B , Scaffold-GS , SchNet , Seed-LLaMA-8B , SGC , SIREN , Sketch Transformer , SLD-max , SLD-medium , SLD-strong , SLDS , SLIMG , SpeechGPT , SphereNet , SRResNet , Stable Diffusion , StackLLaMA , Starcoder , StegaStamp , StyleGAN2-ADA , Swin Transformer , T5 , TD-MPC , TerraMind , TimeChat , TranceptionEVE , TreeRing , Tulu 2 , UnifiedQA , UniPerceiver , UNITER , UniVL , Van Gogh Diffusion , VERA , VGG / VGG13 , Vicuna , Video-Chat-7B , Video-ChatGPT , Video-LLaMA , Video-LLaMA-2-13B , Video-LLaVA , VideoCLIP , ViLA-8B , ViLBERT , VindLU , VioLET , ViRTex , ViT , ViV1T , VTG-LLM , WildGuard , Wizardlm , X-CLIP , X-InstructBLIP-7B , XGen-MM , Xlm-R , Zephyr-7B-beta Method artefact also in Baichuan , CLIP / CLIP-ViT (LC) , ConvNeXt , EfficientNet , Falcon , Gemma , Gemma 2 , GPT-2 , GPT-3.5 / ChatGPT-3.5 , GPT-J , InternLM , Llama 2 / Llama 2 base , Llama 3 , Llama 3.1 , MAP-NEO , Mistral 7B / Mistral / Mistral 3 7B / Mistral-0.2-7B / Mistral-v0.1 , OLMo / OLMo base , OpenCLIP , OpenLLaMA , OPT , Pythia , Qwen1.5 , Qwen 2 , Qwen2.5 , RedPajama , ResNet / ResNet-152 / ResNet-101 / ResNet-50-BN , Skywork , Stable Diffusion , StableLM , TerraMind , ViT , Yi , ZiYA2 Positional bias also in BERT , ChatGPT , Claude 3 , Claude 3.5 , Falcon , Fuyu , Gemini , Gemini 1.5 / Gemini Pro 1.5 , Gemma , Gemma 2 , GLIDE , GPT-2 , GPT-3.5 / ChatGPT-3.5 , GPT-4 / ChatGPT4 / GPT-4 Code Interpreter / GPT-4 Technical Report , GPT-4.1 , GPT-J , InstructGPT , LLaMA , Llama 2 / Llama 2 base , Llama 3 , Mistral 7B / Mistral / Mistral 3 7B / Mistral-0.2-7B / Mistral-v0.1 , Mixtral , Mixtral 8x7B / Mistral 8x7B Instruct / Mixtral 46.7B / Mixtral 8x7B Instruct / Mixtral-instruct-8x7b , MPT , O1 / OpenAI-o1-preview , O3 , O4-mini , PaLM 2 , Phi-3 , Pythia , Qwen1.5 , Qwen 2 , Qwen 2.5 72B Instruct , Stable Diffusion , Sybil , Vicuna Scale-dependent behaviour also in Aquila-7B , BEiT , BERT , BLOOM , Claude 2.1 , Claude 3 , Claude 3.5 , CLIP / CLIP-ViT (LC) , Code Llama , CodeGen , Cohere Command R , DeepSeek LLM , DeepSeekMoE , DeiT-III , DINO , DINOv2 , EquiformerV2 , ESCN , Falcon , FLAN-T5 , Gemini 1.0 Pro , Gemini 1.5 / Gemini Pro 1.5 , Gemma , Gemma 2 , GPT-2 , GPT-3 / GPT base , GPT-3.5 / ChatGPT-3.5 , GPT-4 / ChatGPT4 / GPT-4 Code Interpreter / GPT-4 Technical Report , GPT-J , GPT-Neo , I3D , Idefics , InternLM-2.5-7B , InternLM-XComposer2-VL , InternLM2 , InternVL-1.5 , InternVL2 , LLaMA , Llama 2 / Llama 2 base , Llama 3 , Llama 3.1 , Llama 3.2 , Llama-3.2-3B , LLaVA-1.5 / LLaVA-v1.5 , LLaVA-NeXT / LLaVA 1.6 , LongVA-7B , MAE , Mistral 7B / Mistral / Mistral 3 7B / Mistral-0.2-7B / Mistral-v0.1 , Mixtral , Mixtral 8x7B / Mistral 8x7B Instruct / Mixtral 46.7B / Mixtral 8x7B Instruct / Mixtral-instruct-8x7b , Moirai , MPT , MViT V2 , O1 / OpenAI-o1-preview , OLMo / OLMo base , OpenCLIP , OpenFlamingo , OpenLLaMA , OPT , PaLM 2 , Phi-3 , Platypus2-Instruct-70B , Pythia , Qwen , Qwen1.5 , Qwen 2 , Qwen 2.5 72B Instruct , Qwen2-VL , Qwen-Audio , Qwen2.5 , RedPajama-INCITE , ResNet / ResNet-152 / ResNet-101 / ResNet-50-BN , SlowFast , Solar 10.7B , Stable Diffusion , StableLM , Swin Transformer , T5 , TerraMind , text-ada-001 , TigerBot-70B-Chat , TimesFormer , TSM , Tulu 2 , Uniformer , Vicuna , VideoMAE , ViLA-8B , Wizardlm , X3D , XGLM , Xwin-LM-70B-v0.1 , Yi Shortcut also in BakLLaVA , BLIP-2 , Claude 3 , Claude 3.5 , CLIP / CLIP-ViT (LC) , CLIP4Clip , CLIPBERT , DALL·E 2 , DALL·E 3 , DeepSeek-VL2 , Eurus-RM-7B , Falcon , FLUX / FLUX1 , Gemini , Gemini 1.5 / Gemini Pro 1.5 , Gemma , Gemma 2 , GPT-2 , GPT-3 / GPT base , GPT-3.5 / ChatGPT-3.5 , GPT-4 / ChatGPT4 / GPT-4 Code Interpreter / GPT-4 Technical Report , GPT-J , iFlytekSpark-13B , InstructBLIP , Internlm2-Reward , InternVideo , InternVL2 , LLaMA , Llama 2 / Llama 2 base , Llama 3 , Llama-3-2-Vision , LLaVA , LLaVA-1.5 / LLaVA-v1.5 , LLaVA-Med , LLaVA-NeXT / LLaVA 1.6 , Med-Flamingo , Merlot Reserve , MiniGPT-4 , Mistral 7B / Mistral / Mistral 3 7B / Mistral-0.2-7B / Mistral-v0.1 , Mixtral , Mixtral 8x7B / Mistral 8x7B Instruct / Mixtral 46.7B / Mixtral 8x7B Instruct / Mixtral-instruct-8x7b , Molmo , mPLUG-2 , mPLUG-Owl3 , Nova Canvas , O1 / OpenAI-o1-preview , OpenCLIP , OPT , Otter , PaLM 62B , Pythia , Qwen , Qwen 2 , Qwen-VL , Qwen2.5 , RadFM , ResNet / ResNet-152 / ResNet-101 / ResNet-50-BN , Stable Diffusion , Swin Transformer , Sybil , TerraMind , Tulu 2 , UniPerceiver , UniVL , Vicuna , Video-LLaMA , VideoCLIP , VindLU , VioLET , X-CLIP