Empirical Neural Tangent Kernel

anchor

Kernel formed from the gradients of a trained network's outputs with respect to its parameters, used to characterise how it generalises.

Findings

• IC-032Off-policy DPO causes a squeezing effect in LLMs where probability mass shifts to the most confident token, explaining degenerate repetition [supporting]