Stream of Consciousness

The Prompt & The Ponder

Tech insights, life reflections, and the occasional existential tangent.

Latest

Interpretability After the Transformer: We Changed the Architecture and Kept the Microscope

August 11, 2026

·mixture-of-experts·machine-learning·transformers

Interpretability After the Transformer: We Changed the Architecture and Kept the Microscope

Mechanistic interpretability was built for softmax transformers. SSMs, diffusion LMs, and sparse MoE each dissolved one of its assumptions. Here's what that actually cost.

Read →

Written by Serendeep Rudraraju — navigating the intersection of technology and humanity.