Mechanistic interpretability was built for softmax transformers. SSMs, diffusion LMs, and sparse MoE each dissolved one of its assumptions. Here's what that actually cost.
HiPPO's polynomial-projection idea is the through-line from S4 to Mamba-3 to HiSS. And it argues that long-context LLMs and long-context prediction are not the same problem.
A practitioner's guide to Mamba and State Space Models — how selective state spaces achieve linear scaling, when to use SSMs vs Transformers vs hybrids, and production-ready models.