IC-577An encoder's text reconstruction performance positively correlates with its effectiveness in downstream vector-ICL classification tasks across 15 encoder-LLM-dataset configurations
The authors measure each of four text encoders on a text reconstruction task (as a proxy for information preserved in embeddings) and then rank them by downstream text classification accuracy when used with vector-ICL. Across 5 classification datasets and 3 LLMs (15 configurations total), Spearman rank correlations between the two rankings are consistently positive, with 4 of 15 configurations showing correlations approaching 1. This suggests reconstruction quality is a reliable predictor of an encoder's utility as a vector-ICL input.
Evidence
correlational
Key metric
Spearman rank correlation positive across 15 configurations; 4 of 15 configurations show values approaching 1
Caveat
The analysis is limited to text classification as the downstream task and to four specific encoders; the correlation may not hold for other task types or encoder families.