Hi!

Just wanted to share my upgraded monster-server! I have bought the largest chassi I could reasonably find (Phanteks Enthoo pro 2 server) and filled it to the brim with GPU:s to run local LLM:s alongside my homelab. I am very happy how it has evloved / turned out!

I call it the "Monster server" :)

Based on my trusted old X570 Taichi motherboard (extremely good!) and the Ryzen 3950x that I bought in 2019, that is still PLENTY fast today. I did not feel like spending a lot of money on a EPYC CPU/motherboard and new RAM, so instead I maxed out what I had.

The 24 PCI-e lanes are divided among the following:

3 GPU:s - 2 x RTX 3090 - both dual slot versions (inno3d RTX 3090 x3 and ASUS turbo RTX 3090) - 1 x RTX 4090 (an extremely chonky boi, 4 slots! ASUS TUF Gaming OC, that I got for reasonably cheap, around 1300USD equivalent). I run it on the "quiet" mode using the hardware switch hehe.

The 4090 runs off an M2 -> oculink -> PCIe adapter and a second PSU. The PSU is plugged in to the adapter board with its 24-pin connector and it powers on automatically when the rest of the system starts, very handy! https://www.amazon.se/dp/B0DMTMJ95J

Network: I have 10GB fiber internet for around 50 USD per month hehe... - 1 x 10GBe NIC - also connected using an M2 -> PCIe adapter. I had to mount this card creatively...

Storage: - 1 x Intel P4510 8TB U.2 enterprise NVMe. Solid storage for all my VM:s! - 4 x 18TB Seagate Exos HDD:s. For my virtualised TrueNAS.

RAM: 128GB Corsair Vengeance DDR4. Running at 2100MHz because I cannot get it stable when I try to run it faster, but whatever... LLMs are in VRAM anyway.

So what do I run on it? - GPT-OSS-120B, fully in VRAM, >100t/s tg. I did not yet find a better model, despite trying many... I use it for research, coding, and generally instead of google sometimes... I tried GLM4.5 air but it does not seem much smarter to me? Also slower. I would like to find a reasonably good model that I could run alongside FLUX1-dev-fp8 though, so I can generate images on the fly without having to switch. I am evaluating Qwen3-VL-32B for this

- Media server, Immich, Gitea, n8n

- My personal cloud using Seafile

- TrueNAS in a VM

- PBS for backups that is synced to a offsite PBS server at my brothers apartment

- a VM for coding, trying out devcontainers.

-> I also have a second server with a virtualised OPNsense VM as router. It runs other more "essential" services like PiHole, Traefik, Authelia, Headscale/tailscale, vaultwarden, a matrix server, anytype-sync and some other stuff...

--- FINALLY: Why did I build this expensive machine? To make money by vibe-coding the next super-website? To cheat the stock market? To become the best AI engineer at Google? NO! Because I think it is fun to tinker around with computers, it is a hobby...

Thanks Reddit for teaching me all I needed to know to set this up!