Light Dark text · generative · anchor · artifact
Suite of language models trained on one corpus in a fixed data order across sizes, built to make training dynamics comparable between scales. Cited in the corpus from 410M to 2.8B.
Note anchor found by search rather than in a citing paper, and checked against this entry's own description before it was recorded: "Pythia: A Suite for Analyzing Large Language Models Across Training and Scaling" Variants Pythia 410M , Pythia 1B , Pythia 1.4B , Pythia 2.8B , Pythia-6.9B , Pythia-12B , Pythia-160m , Pythia-70m , Pythia-70m-deduped Findings IC-015 Truncating MLP weights in Pythia-1b increases the probability of the correct answer in an Indirect Object Identification task IC-016 Truncating MLP weights in Pythia-1b increases the probability of the correct answer in a factual recall task IC-032 Off-policy DPO causes a squeezing effect in LLMs where probability mass shifts to the most confident token, explaining degenerate repetition IC-043 Five ~7B decoder-only LLMs develop a high-intrinsic-dimensionality phase in intermediate layers that marks the transition from surface-form to abstract linguistic processing, with earlier onset predicting better next-token prediction IC-060 SAE features in Pythia-160m and Mamba-130m exhibit high cross-architecture similarity with a depth-scaled correspondence IC-1015 GPT-J and 10 other LLMs exhibit overthinking: calibrated accuracy given incorrect few-shot demonstrations peaks at a critical layer then declines, and ablating 5 false induction heads in late layers reduces the accuracy gap by 38.9% on average IC-1127 The LM head in GPT-2, GPT-J, BLOOM, Pythia, and LLaMA-2 projects all input token hidden states into interpretable token distributions over the vocabulary, and these distributions converge approximately monotonically toward the final layer's distribution IC-1317 Llama-2 and Pythia models contain linear representations of space and time that improve with depth and model scale IC-1317 Llama-2 and Pythia models contain linear representations of space and time that improve with depth and model scale [context] IC-1369 Successor heads that increment ordinal-sequence tokens exist in Pythia, GPT-2, and Llama-2 models from 31M to 12B parameters IC-1370 MLP0 representations of ordinal-sequence tokens in Pythia-1.4b contain linearly decodable mod-10 features that are causally important for incrementation IC-1371 The Pythia-1.4b successor head l12h0 exhibits interpretable polysemanticity, performing successorship, acronym prediction, copying, and greater-than behaviors on natural language data IC-1372 Successor heads in Pythia-1.4b exhibit a greater-than bias: the OV circuit assigns systematically higher logits to tokens with greater ordinal values than the input, impairing decrementation IC-1380 Pythia and OPT small models exhibit non-trivial performance on BigBench tasks that is invisible under beam search but revealed by extensive random sampling IC-1386 Fact recall in OPT and LLaMA models degrades by more than 5% relative accuracy when more than 30% of weights are pruned, and similarly when moving from the 30B to the 13B dense model IC-1387 In-context learning capabilities in OPT and LLaMA models remain within 5% of dense-model accuracy even at 60-70% sparsity, and show less than 2% difference between the 30B and 1.3B dense OPT models IC-146 Pythia models show increasing robustness to off-policy RLHF data as policy size scales from 410M to 2.8B IC-1549 All 28 evaluated LMs exhibit gender bias on non-stereotypical sentence pairs, with fairness scores between 9% and 41% IC-1550 All evaluated LMs systematically prefer male pronoun completions in the non-stereotypical portions of Winobias and Winogender, with margins exceeding 40% IC-1551 No consistent relationship between model size and gender fairness scores is observed across six LM families IC-1552 Deduplication of pretraining data does not consistently improve gender fairness in Pythia models IC-160 Pythia-70m and Gemma-2-2b implement subject-verb agreement across a relative clause via a circuit of number detectors, PP/RC boundary detectors, and verb form promoters, with Gemma-2-2b additionally using NP number trackers IC-161 Linear probes on Pythia-70m and Gemma-2-2b trained on the ambiguous Bias in Bios set rely on gender as a spurious feature, with gender accuracy far exceeding profession accuracy IC-162 The majority of subject-verb agreement performance in Pythia-70m is explained by approximately 100 SAE feature nodes and in Gemma-2-2b by approximately 500 nodes, compared to approximately 1500 and 50000 neurons respectively IC-1630 Llama and Pythia models represent entity-attribute bindings via additive binding id vectors that form a continuous subspace with metric structure IC-1631 Binding id mechanism fidelity increases with model size in both Llama and Pythia families IC-167 Adding a PCA-derived control vector to the middle-layer residual stream improves logit-based reasoning accuracy on Pythia-1.4b, Pythia-2.8b, and Mistral-7B-Instruct IC-185 In Pythia-1B and Amber-7B, the probability of memorizing a training sequence scales log-linearly with both the number of repetitions in the corpus and the z-complexity of the sequence IC-186 The memorization status of sequences in Pythia-1B and Amber-7B is stationary throughout training: KL-LD fluctuations are mean-reverting with fixed variance, rejecting a random-walk model with p < 10⁻⁸ IC-187 Latent memorized sequences in Pythia-1B and Amber-7B can be recovered by adding random Gaussian noise of magnitude 2×10⁻³ to model weights, while un-memorized and unseen sequences cannot IC-218 LLaMA3-8B and other LLMs solve arithmetic via a bag of independent heuristic neurons in middle and late MLP layers rather than a robust algorithm IC-220 In Pythia-6.9B, the bag-of-heuristics mechanism emerges gradually during training and is the primary arithmetic mechanism from the earliest checkpoint showing good performance (23k steps) IC-248 Instruction fine-tuning causes context reliance under knowledge conflicts to initially increase then decrease (context-parametric inversion) in Llama2-7B, Pythia-6.9B, and Mistral-7B IC-249 The GitHub data-refined LLC identifies the induction circuit heads in Pythia-70m by distinguishing previous-token and induction heads from other head types across layers 2 and 3 IC-296 The degree to which SAE features are active at multiple residual-stream layers increases with model size in Pythia, Gemma 2, Llama 3.2, and GPT-2 IC-297 Applying tuned-lens transformations to the residual stream decreases the apparent multi-layer SAE feature activity from 54–88% to 37–41% of total variance IC-370 Knowledge entropy (sparsity of FFN memory coefficients) decreases consistently during pretraining for OLMo 1B, 7B, and Pythia 1.4B, and this decrease strongly correlates with reduced knowledge acquisition and increased forgetting in continual learning IC-372 Language models universally decompose retrieval tasks into request processing in middle layers and entity retrieval in late layers at the last token position IC-373 In Pythia-2.8B, the specific attention heads and MLPs implementing retrieval depend on superficial input features, and request-patching preserves the natural retrieval mechanism IC-374 Pythia models are vulnerable to prompt injection via distractor text, and request-patching from a single trusted input restores most of their accuracy IC-424 Structural in-context learning is transient in MultiBERTs and Pythia-1.4B, disappearing after early training IC-427 Newer base models (post-November 2023) outperform older ones by 7.3 points on MMLU and 19.1 points on GSM8K controlling for pretraining compute, but this gap vanishes after fine-tuning all models on the same task-relevant data IC-428 Qwen 1.5 appears to Pareto-dominate Pythia and LLaMA 2 on MMLU and GSM8K, but after adjusting for test task training all three model families exhibit equivalent scaling IC-429 The point of emergence for MMLU shifts from approximately 1.3×10²² flops to 5.6×10²⁰ flops as models train on 64,000 task-relevant examples, and the log-linear fit R² improves from 0.632 to 0.950 IC-432 56 LLMs from 19 families exhibit u-shaped scaling on hard questions and inverted-U scaling on easy questions, with the opposing trends explaining emergent ability stagnation IC-451 Transformer block coupling of Jacobian singular vectors positively correlates with benchmark performance across 30+ LLMs, more strongly than parameter count, depth, or embedding dimension IC-452 Transformer block coupling is absent at initialization and increases persistently throughout training in Pythia 12B and 6.9B, with layer-wise locality emerging IC-453 Hidden representation trajectories in trained LLMs exhibit considerable linearity (mean LSS 4.25) compared to 6.54 at initialization, and linearity increases with training IC-454 Most hidden trajectories in trained LLMs exhibit exponential growth in norm as a function of depth, a property that emerges with training IC-471 Pythia models exceeding 100M parameters show a consistent leftward shift of the multifractal spectrum (increasing regularity) during training that is absent in the 14M and 31M variants IC-471 Pythia models exceeding 100M parameters show a consistent leftward shift of the multifractal spectrum (increasing regularity) during training that is absent in the 14M and 31M variants [context] IC-472 The degree of emergence metric derived from Pythia's internal structure positively correlates with benchmark performance across training epochs IC-473 ResNet-18 lacks a clear multifractal structure while ResNet-152 shows one with irregular shifts, and a 160M diffusion model exhibits lower degree of emergence than Pythia 160M IC-808 Sparse autoencoder features in Pythia-70m's residual stream are more interpretable than PCA, ICA, random, and default-basis directions, with the advantage declining from early to late layers IC-809 Sparse dictionary features in Pythia-410m enable more precise causal localisation of indirect object identification behaviour than PCA, requiring fewer patches and smaller edit magnitudes for the same KL divergence IC-810 Individual sparse autoencoder features in Pythia-70m-deduped are monosemantic and have predictable causal effects on output logits, as demonstrated by an apostrophe feature whose ablation primarily suppresses the 's' token IC-815 RLHF on general-purpose preference data increases stereotypical bias and decreases truthfulness in Pythia and Llama-7B models IC-816 RLHF on general-purpose preference data increases privacy leakage in Pythia and Llama-7B models IC-817 RLHF on general-purpose preference data improves machine ethics performance in Pythia and Llama-7B models IC-818 RLHF on general-purpose preference data has negligible net effect on toxicity in Pythia and Llama-7B models IC-913 In OPT-2.7B, Pythia-70M/1.4B/6.9B, and BERT-base, the stable rank of MLP lower layers shows a drop-and-bounce pattern during training that is more salient in top layers while bottom layers show suppressed dropping curves IC-914 In Pythia models (70M through 2.8B), BERT-base, OPT-6.7B, LLaMA-2-7B, and ViT-Huge, the MLP out-projection vectors are almost orthogonal throughout training IC-915 In Pythia-70M and Pythia-160M, individual MLP hidden neurons are activated by multiple irrelevant token combinations (pattern superposition) IC-916 Pythia models show scale-dependent last-layer averaging barriers: 70M exhibits a barrier of ~13 while 410M shows ~1 IC-916 Pythia models show scale-dependent last-layer averaging barriers: 70M exhibits a barrier of ~13 while 410M shows ~1 [context] IC-945 LLaMA-2, MPT, Falcon, Pythia, and BERT-base-uncased allocate disproportionate attention to initial tokens regardless of their semantic content IC-946 LLaMA-2-7B, MPT-7B, Falcon-7B, and Pythia-12B do not consistently improve in perplexity as the StreamingLLM cache size increases Shared mechanisms Depth-dependent structure also in Baichuan 2 , BERT , BLIP-2 , BLOOM , Chameleon , CLIP / CLIP-ViT (LC) , DeepFloyd IF , DeiT-III , DINO , DINOv2 , Falcon , Gemma , Gemma 2 , GPT-2 , GPT-J , GPT-NeoX-20B , Griffin , I3D , Idefics , InstructBLIP , LLaMA , Llama 2 / Llama 2 base , Llama 3 , Llama 3.1 , Llama 3.2 , Llama-3.2-3B , LLaVA , LLaVA-1.5 / LLaVA-v1.5 , LLaVA-Phi , MAE , MAE-B/16 , Mamba , MiniGPT-4 , Mistral 7B / Mistral / Mistral 3 7B / Mistral-0.2-7B / Mistral-v0.1 , Mixtral 8x7B / Mistral 8x7B Instruct / Mixtral 46.7B / Mixtral 8x7B Instruct / Mixtral-instruct-8x7b , mPLUG-Owl , MPT , MultiBERTs , MViT V2 , OLMo / OLMo base , OpenCLIP , OPT , Phi-2 , Qwen 2 , Qwen2-VL , Qwen2.5 , RoBERTa / RoBERTa-L , RWKV , SALMONN , SAM , SlowFast , Stable Diffusion , Swin Transformer , TerraMind , TimesFormer , TSM , Uniformer , Vicuna , VideoMAE , ViT , X3D , Yi Distance preservation also in CLIP / CLIP-ViT (LC) , CoPlace , DINO , DINOv2 , Gemma , Gemma 2 , ImageBind , LanguageBind , LLaMA , Llama 2 / Llama 2 base , Llama 3 , Llama 3.1 , Llama-3.2-3B , LLaVA-1.5 / LLaVA-v1.5 , LLaVA-Med , MAE , MAE-B/16 , OpenCLIP , ResNet / ResNet-152 / ResNet-101 / ResNet-50-BN , SigLIP , SLIP , TerraMind , ViT Explanation faithfulness also in BakLLaVA , CF2 , Claude 3 , Claude 3.5 , CLIP / CLIP-ViT (LC) , DRUM , Fuyu , GEM , Gemini 1.5 / Gemini Pro 1.5 , Gemma 2 , GPT-3.5 / ChatGPT-3.5 , GPT-4 / ChatGPT4 / GPT-4 Code Interpreter / GPT-4 Technical Report , GPT-4o , GPT-J , Idefics , Llama 3.1 , LLaVA-NeXT / LLaVA 1.6 , MAE-B/16 , MobileNetV2 , mPLUG-Owl3 , OpenFlamingo , PGExplainer , RCExplainer , ResNet / ResNet-152 / ResNet-101 / ResNet-50-BN , SigLIP , SigLIP-2 , Stable Diffusion , TAGExplainer , ViT Failure mode also in AASIST , ADM , Aegis-Guard-Defensive , Alpaca , AnyLoc , AutoTikZ / DataTikZ , Baichuan , Baichuan 2 , Baichuan2-13B , BakLLaVA , Bard , BEiT , BERT , BingChat , BLIP , BLIP-2 , BLOOM , BSRNet , CF2 , Chat-UniVi-7B , ChatGLM-6B / ChatGLM-6b-2 , ChatGLM2 , ChatGPT , CLAP , Claude 1.3 , Claude 2.0 , Claude 2.1 , Claude 3 , Claude 3.5 , CLEAR , CLIP / CLIP-ViT (LC) , CLIP4Clip , CLIPBERT , CLIPCap , CLMBR-T-BASE , CloFNet , Code Llama , CodeGeex2 , CodeGen , CodeLlama-13B , CodeLlama-34B , CogVLM2 , Cohere Command R , CoMEt , Command R+ , CONCH , CycleGAN , DALL-E , DALL·E 2 , DALL·E 3 , DASR , DECAF , DeepSeek-2-Chat , DeepSeek-2-Coder , DeepSeek-V2-0628 , DeepSeek-VL , DeepSeek-VL2 , DeiT , DeiT-III , Depth Anything , DETR-R50 , DimeNet++ , DINO , DINO-FocalNet-Large , DINOv2 , EGNN , Emu2 , EquiformerV2 , ESCN , ESM-2 , ESM3 , ESRGAN , EVA-CLIP , EVE , Falcon , Faster R-CNN / Faster R-CNN R50 / Faster R-CNN X101 , FLAN-T5 , Florence-2 , FLUX / FLUX1 , FoveaBox , Fuyu , Galactica-6.7B , GAT , 3D Gaussian Splatting , GCN , Gemini , Gemini 1.5 / Gemini Pro 1.5 , Gemma , Gemma 2 , GIN , GLIDE , GLM-4 , GLM-4V , GloVe , GP-UNIT , GPT-2 , GPT-3 / GPT base , GPT-3.5 / ChatGPT-3.5 , GPT-4 / ChatGPT4 / GPT-4 Code Interpreter / GPT-4 Technical Report , GPT-4.1 , GPT-4o , GPT-J , GPT-NeoX-20B , GraphSAGE , Grounding DINO , Guanaco , GVP , Hawkeye , HiFaceGAN , HPS , HuggingGPT , IDDPM , Idefics , Idefics2 , ImageBind , ImageBind-LLM-7B , Imagen Video , ImageReward , 12-in-1 , InstructBLIP , InstructGPT , InternLM-2.5-7B , InternLM-XComposer2-VL , InternVideo , InternVL-1.5 , InternVL2 , Koala , LegalBERT , LLaMA , Llama 2 / Llama 2 base , Llama 3 , Llama 3.1 , Llama 3.2 , Llama-3.2-3B , Llama-3-2-Vision , LLaMA-Adapter v2 , Llama Guard , Llama Guard 2 , Llama-Guard 3 , Llama-VID , LLaVA , LLaVA-1.5 / LLaVA-v1.5 , LLaVA-Med , LLaVA-NeXT / LLaVA 1.6 , LLaVA-OneVision , LongVA-7B , LOVT , LWM-1M-JAX , MACE , MAE , Med-Flamingo , Merlot Reserve , MGCA , MiDaS , MiniCPM-V , MiniGPT-4 , Mip-Splatting , Mistral 7B / Mistral / Mistral 3 7B / Mistral-0.2-7B / Mistral-v0.1 , Mistral Large 2 , Mistral Large V2 , Mixtral , Mixtral 8x7B / Mistral 8x7B Instruct / Mixtral 46.7B / Mixtral 8x7B Instruct / Mixtral-instruct-8x7b , MobileNetV2 , Molmo , MolmoE-7B , Momentor , Moondream2 , Moonshot-v1-8k , mPLUG-2 , mPLUG-Owl , mPLUG-Owl3 , mPLUG-Owl2 , MPT , MSA Transformer , MultiBERTs , Nova Canvas , Nova Lite , Nova Pro , O1 / OpenAI-o1-preview , O3 , O4-mini , OLMo / OLMo base , OneLLM , OpenAI Moderation , OpenChat-3.5-0106 , OpenCLIP , OpenFlamingo , OPT , Otter , Otter-7B , PaLM 2 , PaLM 62B , PandaGPT-7B , PerSAM , Phi-3 , Phi-3.5 Mini Instruct , PickScore , PLIP , Prismatic , ProGen-2 , Qwen1.5 , Qwen 2 , Qwen 2.5 72B Instruct , Qwen2-VL , Qwen-Audio , Qwen-VL , Qwen2.5 , Qwen2-Audio , R2D2 , RadFM , RCExplainer , RD-SR , RealESRNet , Reprover , ResNet / ResNet-152 / ResNet-101 / ResNet-50-BN , RetinaNet-R50 , RivaGAN , RS-LDS , RT-1 , RT-2 , SALMONN , SAM , SAM 2 , SAULLM 54B , Scaffold-GS , SchNet , Seed-LLaMA-8B , SGC , SIREN , Sketch Transformer , SLD-max , SLD-medium , SLD-strong , SLDS , SLIMG , SpeechGPT , SphereNet , SRResNet , Stable Diffusion , StackLLaMA , Starcoder , StegaStamp , StyleGAN2-ADA , Swin Transformer , T5 , TD-MPC , TerraMind , TimeChat , TranceptionEVE , TreeRing , Tulu 2 , UnifiedQA , UniPerceiver , UNITER , UniVL , Van Gogh Diffusion , VERA , VGG / VGG13 , Vicuna , Video-Chat-7B , Video-ChatGPT , Video-LLaMA , Video-LLaMA-2-13B , Video-LLaVA , VideoCLIP , ViLA-8B , ViLBERT , VindLU , VioLET , ViRTex , ViT , ViV1T , VTG-LLM , WildGuard , Wizardlm , X-CLIP , X-InstructBLIP-7B , XGen-MM , Xlm-R , Zephyr-7B-beta Linear representation also in BLOOM , Cambrian-1 , Chameleon , CLIP / CLIP-ViT (LC) , DINOv2 , EVA-CLIP , Falcon , Gemma , Gemma 2 , GPT-2 , GPT-J , HPSv2 , ImageBind , InstructBLIP , LanguageBind , LLaMA , Llama 2 / Llama 2 base , Llama 3 , Llama 3.1 , Llama-3.2-3B , LLaVA-1.5 / LLaVA-v1.5 , LLaVA-NeXT / LLaVA 1.6 , MAE , Mamba , Mistral 7B / Mistral / Mistral 3 7B / Mistral-0.2-7B / Mistral-v0.1 , OLMo / OLMo base , OpenCLIP , Phi-3 , PickScore , Qwen2-VL , Qwen2.5 , ResNet / ResNet-152 / ResNet-101 / ResNet-50-BN , SALMONN , SAM , SigLIP , TerraMind , Tulu 2 , Vicuna , ViT Method artefact also in Baichuan , CLIP / CLIP-ViT (LC) , ConvNeXt , EfficientNet , Falcon , Gemma , Gemma 2 , GPT-2 , GPT-3.5 / ChatGPT-3.5 , GPT-4o , GPT-J , InternLM , Llama 2 / Llama 2 base , Llama 3 , Llama 3.1 , MAP-NEO , Mistral 7B / Mistral / Mistral 3 7B / Mistral-0.2-7B / Mistral-v0.1 , OLMo / OLMo base , OpenCLIP , OpenLLaMA , OPT , Qwen1.5 , Qwen 2 , Qwen2.5 , RedPajama , ResNet / ResNet-152 / ResNet-101 / ResNet-50-BN , Skywork , Stable Diffusion , StableLM , TerraMind , ViT , Yi , ZiYA2 Positional bias also in BERT , ChatGPT , Claude 3 , Claude 3.5 , Falcon , Fuyu , Gemini , Gemini 1.5 / Gemini Pro 1.5 , Gemma , Gemma 2 , GLIDE , GPT-2 , GPT-3.5 / ChatGPT-3.5 , GPT-4 / ChatGPT4 / GPT-4 Code Interpreter / GPT-4 Technical Report , GPT-4.1 , GPT-4o , GPT-J , InstructGPT , LLaMA , Llama 2 / Llama 2 base , Llama 3 , Mistral 7B / Mistral / Mistral 3 7B / Mistral-0.2-7B / Mistral-v0.1 , Mixtral , Mixtral 8x7B / Mistral 8x7B Instruct / Mixtral 46.7B / Mixtral 8x7B Instruct / Mixtral-instruct-8x7b , MPT , O1 / OpenAI-o1-preview , O3 , O4-mini , PaLM 2 , Phi-3 , Qwen1.5 , Qwen 2 , Qwen 2.5 72B Instruct , Stable Diffusion , Sybil , Vicuna Scale-dependent behaviour also in Aquila-7B , BEiT , BERT , BLOOM , Claude 2.1 , Claude 3 , Claude 3.5 , CLIP / CLIP-ViT (LC) , Code Llama , CodeGen , Cohere Command R , DeepSeek LLM , DeepSeekMoE , DeiT-III , DINO , DINOv2 , EquiformerV2 , ESCN , Falcon , FLAN-T5 , Gemini 1.0 Pro , Gemini 1.5 / Gemini Pro 1.5 , Gemma , Gemma 2 , GPT-2 , GPT-3 / GPT base , GPT-3.5 / ChatGPT-3.5 , GPT-4 / ChatGPT4 / GPT-4 Code Interpreter / GPT-4 Technical Report , GPT-4o , GPT-J , GPT-Neo , I3D , Idefics , InternLM-2.5-7B , InternLM-XComposer2-VL , InternLM2 , InternVL-1.5 , InternVL2 , LLaMA , Llama 2 / Llama 2 base , Llama 3 , Llama 3.1 , Llama 3.2 , Llama-3.2-3B , LLaVA-1.5 / LLaVA-v1.5 , LLaVA-NeXT / LLaVA 1.6 , LongVA-7B , MAE , Mistral 7B / Mistral / Mistral 3 7B / Mistral-0.2-7B / Mistral-v0.1 , Mixtral , Mixtral 8x7B / Mistral 8x7B Instruct / Mixtral 46.7B / Mixtral 8x7B Instruct / Mixtral-instruct-8x7b , Moirai , MPT , MViT V2 , O1 / OpenAI-o1-preview , OLMo / OLMo base , OpenCLIP , OpenFlamingo , OpenLLaMA , OPT , PaLM 2 , Phi-3 , Platypus2-Instruct-70B , Qwen , Qwen1.5 , Qwen 2 , Qwen 2.5 72B Instruct , Qwen2-VL , Qwen-Audio , Qwen2.5 , RedPajama-INCITE , ResNet / ResNet-152 / ResNet-101 / ResNet-50-BN , SlowFast , Solar 10.7B , Stable Diffusion , StableLM , Swin Transformer , T5 , TerraMind , text-ada-001 , TigerBot-70B-Chat , TimesFormer , TSM , Tulu 2 , Uniformer , Vicuna , VideoMAE , ViLA-8B , Wizardlm , X3D , XGLM , Xwin-LM-70B-v0.1 , Yi Shortcut also in BakLLaVA , BLIP-2 , Claude 3 , Claude 3.5 , CLIP / CLIP-ViT (LC) , CLIP4Clip , CLIPBERT , DALL·E 2 , DALL·E 3 , DeepSeek-VL2 , Eurus-RM-7B , Falcon , FLUX / FLUX1 , Gemini , Gemini 1.5 / Gemini Pro 1.5 , Gemma , Gemma 2 , GPT-2 , GPT-3 / GPT base , GPT-3.5 / ChatGPT-3.5 , GPT-4 / ChatGPT4 / GPT-4 Code Interpreter / GPT-4 Technical Report , GPT-4o , GPT-J , iFlytekSpark-13B , InstructBLIP , Internlm2-Reward , InternVideo , InternVL2 , LLaMA , Llama 2 / Llama 2 base , Llama 3 , Llama-3-2-Vision , LLaVA , LLaVA-1.5 / LLaVA-v1.5 , LLaVA-Med , LLaVA-NeXT / LLaVA 1.6 , Med-Flamingo , Merlot Reserve , MiniGPT-4 , Mistral 7B / Mistral / Mistral 3 7B / Mistral-0.2-7B / Mistral-v0.1 , Mixtral , Mixtral 8x7B / Mistral 8x7B Instruct / Mixtral 46.7B / Mixtral 8x7B Instruct / Mixtral-instruct-8x7b , Molmo , mPLUG-2 , mPLUG-Owl3 , Nova Canvas , O1 / OpenAI-o1-preview , OpenCLIP , OPT , Otter , PaLM 62B , Qwen , Qwen 2 , Qwen-VL , Qwen2.5 , RadFM , ResNet / ResNet-152 / ResNet-101 / ResNet-50-BN , Stable Diffusion , Swin Transformer , Sybil , TerraMind , Tulu 2 , UniPerceiver , UniVL , Vicuna , Video-LLaMA , VideoCLIP , VindLU , VioLET , X-CLIP