snimu’s blog snimu.github.io

Trying out other peoples’ ideas

snimu.github.io·29w

modded-nanogpt medium world record: Re-using intermediate activations in the output latents

snimu.github.io·30w

modded-nanogpt world record: Decoupling embedding size from model dimension

snimu.github.io·32w

modded-nanogpt medium world record: adding value embeddings

snimu.github.io·32w

modded-nanogpt: Analyzing value-embedding-, UNet-, and x0-lambdas

snimu.github.io·40w

Separating Simulacra: tags for smarts and safety

snimu.github.io·48w

Model stacking doesn’t work

snimu.github.io·50w

Infinite Tool Use

snimu.github.io·51w·Hacker News

Schizo embeddings: Initializing special tokens the complicated way

snimu.github.io·56w

My dream VLM

snimu.github.io·58w

Multi-layer language heads: the output latent is for text (and nothing else)

snimu.github.io·59w

Tokens vs. Bytes

snimu.github.io·63w

The Tick-Tock-Boom Cycle: A Strategic Pattern for LLM Development

snimu.github.io·65w

Sorting shuffled data as a verifiable task

snimu.github.io·65w

Model stacking

snimu.github.io·65w

Forward-Backward prediction

snimu.github.io·65w

Log in to enable infinite scrolling