How Gemma 4's Per-Layer Embeddings Actually Work — And Why E2B Punches Above 2B (opens in new tab)

How Gemma 4's Per-Layer Embeddings Actually Work — And Why E2B Punches Above 2B This is a submission for the Gemma 4 Challenge: Write About Gemma 4 Every benchmark article about Gemma 4 E2B says roughly the same thing: "Despite being a 2B model, it outperforms older 7B models on reasoning tasks." Then they move on. Nobody explains why. Nobody opens the hood. The answer is a mechanism called Per-Layer Embeddings (PLE) — and once you understand it, you'll never read an LLM parameter count the s...