Interpretability After the Transformer: We Changed the Architecture and Kept the Microscope
Mechanistic interpretability was built for softmax transformers. SSMs, diffusion LMs, and sparse MoE each dissolved one of its assumptions. Here's what that actually cost.