Chinchilla

anchor

Findings

• IC-1338Chinchilla 70B and Llama 2 7B, trained primarily on text, compress ImageNet patches and Librispeech audio better than domain-specific compressors PNG and FLAC
• IC-1339Chinchilla 1B's compression rate improves with increasing sequence length across text, image, and audio, demonstrating in-context learning without gradient updates
• IC-1340Chinchilla 70B produces coherent autoregressive continuations of text, image, and audio data when used as a compressor, outperforming gzip in sample quality