Merry Christmas and happy winter holidays in general to you all!
My Christmas events occurred on Christmas Eve and are coming a few days later, so Christmas day itself I am working on the lab, new components for which are proof that Santa knows what's up.
This is a definite work in progress, but I am writing this from the new rack mount system so this is a new milestone. Pictured:
- The Thermaltake Core W200 case I thought I was going to use; the StarTech 15U rack and Rosewill RSV-AI01 I think I'm going to use. The Core W200 is currently housing the BTRFS RAID1 SATA drives because the SFF-8654 (SlimSAS) to SFF-8643 (HD Mini SAS) cables to use the built-in backplane have not arrived yet. The drives should soon migrate into the 8 hot swap bays of the RSV-AI01.
Visible: optical HDMI cable. The server rack is going to sit much farther back in the room than the prior build, so I'm using a Ruipro optical HDMI cable to bridge the gap.
Also visible: Cat6 cable for Ethernet. One wonder's why everything can't just flow over a single fiberoptic cable....
- The dominating design decision of the RSV-AI01 case is that the PSU rests on a shelf almost immediately above the CPU. This basically forces water cooling. Unfortunately, as I discovered at 3am, my AIO cooler is for the SP5 socket rather than the correct sTR5 Threadripper 7000 and 9000 series socket. Undaunted, I bent the PSU support shelf away and forged ahead with my existing Arctic Freezer 4U-M air cooling radiator, placing the PSU for now near the drive bays.
You can also see the substitute of three Silverstone 12cm fans (nabbed from the unused AIO cooler) in place of the hot swap fans.
- The absolute screamers of Rosewill 12038 fans which jet-engined the neighborhood awake in the early hours. I subbed in some Silverstones which are working much better. It seems that the 12038 form factor is aimed at absolute noise; if I can't find something quieter, I'll probably remove the hot swap assembly and attach fans directly.
In background: cat; Thermaltake T12-500 NAS which I mean to port over to a rack build; Netgear 10Gb ethernet switch; TP-Link / Archer A7 router running OpenWRT.
Components for the forthcoming NAS rebuild, and some hats. Things are so cluttered in this place right now it's driving me insane. The new build is also ASUS WRX90 but with the 16-core 9000 series Threadripper Pro. The use case is high-speed storage for machine learning training. Not pictured are the 2x Nvidia Connect-X 7 400Gb optical network interfaces meant to connect the NAS and the GPU workstation.
The other three RTX 6000 Adas I hope to fit into the RSV-AI-01. The plan involves using two PCI-E "risers" extension cables.
NVMEs for the NAS. Also a pair of 400Gb QSFP-DD optical transceivers to go with the Nvidia NICs. The plan is to directly connect the two servers optically---no switch for now.
Next steps:
- Bring the SATA drives in to the hot swap bays once the proper cables arrive
- Investigate using a custom cooling setup to best handle the space constraints. AIOs are quite restrictive in terms of placement, I'm not sure I could make one work with all the GPUs installed.
- Once we have a water cooling block on the CPU, we can bend the shelf back and install the PSU properly
- Either find some quiet 12038 fans, or switch permanently to Noctuas.
Thoughts:
The StarTech frame seems like a winner. The construction is very solid.
The Rosewill RSV-AI01 case is okay... but not great. The included mounting rails do not roll smoothly at all. The PSU-over-CPU design is very frustrating, and the fans are completely unworkable for a home lab situation. It might end up doing the job, but what really will I be getting from it versus a steel box? The 11 PCI-E slots; the 8 drive hot swap bay and backplane. That's basically it.
The Thermaltake Core W200 also seems great; if it would handle 2x SSI-EEB motherboards as I expected then I'd be sticking with it. (It can only do E-ATX on one side, and that isn't SSI-EEB.)
The Ruipro optical HDMI also seems like a winner.
