Layer 3 · Code

The Model Landscape

Write code to inspect a model's architecture and compute the active-vs-total parameter split for an MoE layer.

12 min read110 XP

config.json — architecture metadata

Router / gate weights

Expert feed-forward blocks (N total)

Shared attention + embedding layers

What you'll actually inspect when reading a model's files.

You rarely need to guess a model's architecture — Hugging Face's config.json and the model card spell it out. Here's how to pull the numbers that actually distinguish models, rather than trusting marketing copy.

Go deeper: L4 Foundations