News
Transformers Resist Their Own Architecture " Less Wrong
2+ hour, 49+ min ago (1395+ words) > This is the first entry in a sequence of posts which compare a mathematical theory of attention against trained transformers. "...
NLAs read thoughts beyond the J-space " Less Wrong
2+ hour, 59+ min ago (1442+ words) TLDR: " * On Llama-3. 3-70 B, I found thoughts it cannot see that are actively steering its behavior; and Anthropic's released NLA (Natural Language A...
Free will as a model parameter " Less Wrong
6+ hour, 26+ min ago (701+ words) I think machine learning has a better answer, and it is not a metaphor. The problem is that epsilon is still global, and still set by the user. We need something per-dimension and learned. Throwing in an equation for good…...
Subliminal Learning Happens at Every Rank, Given the Right Learning Rate and Enough Data " Less Wrong
10+ hour ago (1050+ words) Subliminal learning is the phenomenon where a language model picks up a behavioral trait'such as fondness for cats'by training on data from a trait-c...
Don't train away eval awareness until you know why it's there " Less Wrong
13+ hour, 16+ min ago (310+ words) In this post, I argue that metagaming can emerge in models in different ways, different types of it have different origins, and will probably require different mitigation strategies, so if model developers want models to stop metagaming, they need to…...
How much slower does takeoff go with 10" less compute? " Less Wrong
17+ hour, 42+ min ago (408+ words) About 6x slower in the median case, with an 80% confidence interval of 3. 5x to 8x. "...
Balancing Rigor and Utility: A Review of "A Pragmatic Vision for Interpretability" " Less Wrong
1+ day, 1+ hour ago (473+ words) By Sohybe Ibrahim Abdelwahab Amer|June 2026 This shows the importance and effectiveness of proxy tasks as sanity checks for the interp tools, the superiority of a minimal vector steering as an interp tool, which shows how simple tools, like vector…...
the polysemanticity of polysemanticity in language models " Less Wrong
1+ day, 1+ hour ago (403+ words) Polysemanticity is one of the most important concepts when it comes to mechanistic interpretability, basically, studying the internal representations...
No Space Like J-Space " Less Wrong
1+ day, 5+ hour ago (1411+ words) There is a new very cool Anthropic paper: Verbalizable Representations Form a Global Workspace in Language Models. You can read the blog post verison...
The Geometry of Yes: Mapping Sycophancy Inside an LLM's Emotion Space " Less Wrong
1+ day, 22+ hour ago (213+ words) Summary LLMs have internal emotion representations that causally shape their behaviour. This was recently observed in Claude: positive emotions like...