2026-01-09

I saw Alan Ma’s tweet about building a tiny TPU in less than a month and kept thinking, “Is it really easier than it sounds?” So I tried to walk the same path and see if I could ship something in a week.

This is my personal experience of what worked, what broke, and what I learned.

Starting point: a 4x4 systolic array

I started from a simple 4x4 systolic array I had built before with @cupx0j0e. I fixed a few pipeline issues, cleaned up the design, and then decided to make it a real peripheral instead of a standalone block.

Making it a real peripheral

I added AXI support, dropped it into a Vivado block design, connected it to MicroBlaze, and mapped weights/inputs/outputs as BRAM tiles.

This was my first time actually setting up Vivado, block design, and MicroBlaze end-to-end, so I spent a lot of time learning how the system actually boots and resets.

Bring-up pain (and a lot of debugging)

I got stuck for a while because the reset logic never deasserted MicroBlaze. The proc_sys_reset blocks were not behaving like I expected, and I had to learn how Vivado’s debugging tools actually work instead of just guessing.

Once I fixed that, I wrote a simple systolic array smoke test. It took more time than I expected because the AXI I wrote had timing issues with the BRAM setup.

Feeding it from PyTorch

After the smoke test, I made helpers so the systolic array could run PyTorch GEMMs over UART. At first the firmware sent one tile at a time (painfully slow but correct). Later I switched to multi-tile batching, which finally made it feel fast and usable.

Real models and pushing limits

To get a real model running, I took their own horizontal and vertical differentiator and ran it on my FPGA. That actually worked beautifully.

Then I tried to push it:

• Multi-tile batching in firmware.
• Using DSPs for the PEs instead of pure LUT math.

After those changes, performance felt instantly better.

Finally, I tried an MNIST classifier: a small 2-layer MLP with 8 hidden units. It worked a few times, but it was slow (around 30 seconds to print results) and fragile. The tricky part was the inference path from pygame into an MNIST-like input format. That pipeline was the first time I had built something end-to-end like this, and it showed.

MNIST demo shots

A few screenshots and a short capture from the pygame drawing loop + inference run.

What I learned

• Turning a neat RTL block into a real peripheral is half the work.
• Vivado bring-up is its own craft: resets, clocks, and timing can kill you.
• AXI plus BRAM timing needs discipline, not just “it simmed once.”
• Small firmware changes (like batching) can be bigger wins than RTL tweaks.

Where I stopped

After the MNIST attempt, I gave up on pushing further. It still felt like a win: I got an actual model to run on hardware I built, and I learned how these systems fit together instead of just reading about them.

If I revisit this, I want to make the host-to-FPGA data path cleaner and build a more reliable input pipeline. But for a week-long experiment, it was more fun than I expected.