Learning Layer Paper Reading Club - Week 38 - Neural Networks Are Already Clean

This week's reading: Neural Networks Are Already Clean (Nick Cammarata, Goodfire)

Link: https://already-clean.goodfire.com/

About the session:

A common view in interpretability is that neural networks are tangled knots: individual neurons do several unrelated jobs (polysemanticity), because networks cram more concepts than they have room for into shared neurons (superposition). If that is right, you cannot study neurons directly and need tools like sparse autoencoders to unfold the network first.

This essay argues the opposite. It revisits classic examples such as the InceptionV1 "cat and car" neuron and re-reads them as ordinary features like V-shape and pose detectors. It then looks at vision models of very different sizes and at language-model neurons in Qwen3-32B. The author's claim is that most apparent polysemanticity comes from fixable causes (weak visualization tools, missing architecture features, thin datasets) and from neurons growing in several directions over training, not from superposition.

Thomas will walk through the post, then step back and cover some wider AI interpretability concepts at a high level, so you do not need a background in the field.

Relevant to anyone curious about how models work inside, from newcomers to interpretability to people who have used SAEs. No preparation required. Come read and discuss with us.

Thomas will lead the discussion this week.

What is the Learning Layer Labs Paper Reading Club?

An initiative from https://www.learninglayer.ai, a lab with the goal of reducing AI anxiety in the world.

What is the format?

Discussion based. Expect a low pressure environment to share insights and opinions with the group.

What are the group goals?

Stay on top of AI research and improve understanding of AI fundamentals + math.

Who is welcome?

Everyone! Try to put in at least some time on the paper and come prepared with questions or things you'd like to discuss, but it's ok to just show up!

Learning Layer Labs team:

• Thomas Redfern ( The backbone who runs every paper reading )

• Mat Allen (The new fella bring order and organization)

• Devinder Sodhi (The guy you talk to for sponsoring)