IC-361Mistral 7B's circular representation of days of the week is continuous, mapping intermediate time-of-day values to positions between adjacent weekdays
Joshua Engels, Eric J Michaud, Isaac Liao, Wes Gurnee, Max Tegmark
The authors construct a synthetic dataset of prompts like '[very early/very late] on [monday/tuesday/.../sunday]' and project the layer-30 activations into the PCA plane of the weekday representations. The 'very early' version of each weekday maps towards the previous weekday and the 'very late' version maps towards the next weekday along the circle. A similar result is shown for '[morning/evening] on [weekday]' prompts. This demonstrates that the circular feature encodes a continuous notion of time rather than just discrete weekday labels.
Evidence
observational
Caveat
The continuity is demonstrated on a small synthetic set of prompts and is not quantified with a formal metric; the authors note continuity is not a requirement of their irreducibility definition.