IC-575Four released LLMs (LLaMA-3.1-8B, Mistral-7B, Qwen2-7B, Yi-1.5-9B) can perform in-context learning on continuous vector representations projected into their embedding space, matching or outperforming few-shot ICL across text, time-series, graph, and fMRI tasks
The authors project continuous vectors from diverse encoders into the embedding space of four frozen LLMs using lightweight trained projectors, then use the resulting box tokens as in-context demonstrations. Across text reconstruction, function regression, text classification, summarization, molecule captioning, time-series classification, graph classification, and fMRI decoding, the LLMs learn from the projected vector context. With pretrained projectors, vector-ICL matches or outperforms few-shot ICL on 4 of 6 directly comparable tasks; with finetuned projectors it consistently surpasses both few-shot ICL and domain-specific baselines.
Experiments did not cover all possible combinations of encoders, LLMs, modalities, and tasks; only single-token encoders were tested; the effect of instruction tuning on vector-context understanding was not explored.