Running your own network video recorder (NVR) from home usually involves plenty of processing power. Throw in multiple IP camera streams, motion detection, object recognition, and other AI enhancements, and you’ve got quite the mix requiring some serious compute resources. Without dedicated hardware, you’ll rely on the CPU to handle pretty much everything, which is where everything can grind to a halt, especially on weaker systems. I had this problem before switching to Frigate and throwing in a mid-range GPU for good measure.

This open-source NVR is magic, enabling me to create not only a smarter home, but one that’s better secured against those who may wish to do us harm. Hardware acceleration is game-changing for video decoding with software such as Jellyfin or Plex, and the same goes for Frigate. This software takes it a step further with object detection, which can allow the system to interpret what’s present within any given frame, be it an animal, human, or moving vehicle. We record continuously and have everything configured to fire out real-time notifications.

The best part? I have Frigate running on the same Proxmox node as countless other smart home and home lab software, and it runs flawlessly with the right setup.

How Frigate solved everything

Powerful, open, free

Frigate is one of those puzzle pieces that just drop into place when used with a smart home running Home Assistant. The platform fully supports it and integrates well. I didn’t want to be locked into an ecosystem with proprietary hardware running commercial software from Reolink, Hikvision, and others, but I also wanted more from the system that Frigate would run on. ZoneMinder and Blue Iris are great, but they’re also system hogs when it comes to using the CPU. Frigate is the best of both worlds.

I get all the privacy-focused features offered by DIY solutions and the ease, performance, and connectivity of commercial options. Because Frigate is designed to be efficient with a wide variety of hardware, it’s easy to get an install up and running without sending your CPU usage to 100%. This is primarily down to video decoding being offloaded to hardware accelerators, which come in the form of Intel Quick Sync, Nvidia NVDEC, and AMD VCN. There’s also Raspberry Pi OMX if you’re running Frigate on a single-board computer (SBC).

These optimizations enable the CPU itself to take a backseat and offload much of the grunt work to a capable component, typically a discrete or onboard GPU or TPU device. These components can power through video decoding and object detection like it’s easy work, so long as you have enough compute to handle the workload — more IP camera feeds at higher resolutions will negatively impact performance. Running five IP cameras at 1080p with full motion and object detection on an Nvidia GeForce RTX 3060 Ti allows the assigned vCPU cores to run at around 10%.

I wrote up quite a detailed insight into my Frigate setup, and it’s largely the same, aside from using an RTX 3060 Ti instead of a T1000 and running on the same Proxmox system as other software. Dropping the enterprise hardware was a necessity thanks to the increased power draw, the ability to consolidate to a single system, and the difficulty in sourcing replacement parts at reasonable prices.

Detection on a frame-by-frame basis

Protecting the network and the home

Getting everything set up with an Nvidia passthrough on Proxmox was relatively easy, requiring just a few steps in the console. I had to run a few additional commands due to how my Proxmox box was configured with an already present GPU for AI inference. It’s one of a few ways I could make a multi-GPU system make sense outside of gaming in the 2000s. What’s also awesome is that because everything was consolidated onto a single server, power usage has more than halved compared to running multiple rack systems.

All Proxmox needs for you to do to add an Nvidia GPU (or two) for passthrough is to install the PVE helper script to prep the installation. After that, simply install Nvidia’s drivers and nvtop, and you’re good to go! Passing the actual PCI device through to Frigate is as easy as loading up all the Nvidia device locations and adding them to the Frigate virtual machine (VM) instance. My esteemed colleague, Adam Conway, also wrote a guide for doing the same with AMD cards!

I’ve saved money and computer resources, thanks to offloading as much as I can to dedicated hardware. Running large language models (LLMs) through OpenWeb UI would require an incredible amount of CPU usage for very little performance, which was solved by adding the RTX 4060 Ti. The RTX 3060 Ti is perfectly suited to handling Friogate and our array of IP cameras. FFmpeg hardware acceleration makes quick work of all the incoming feeds. Each feed is made up of frames, much like any video output, which are then monitored for detecting motion.

This is where the neural network takes over, again powered by the RTX 3060 Ti. You could get away with a Google Coral TPU or something similar, but a capable GPU with the right parts will work just as well. Doing so for both video and detection ensures the CPU is only tasked with handling transfers and requests, yet none of the heavy lifting processing. The difference is night and day when toggling hardware acceleration for each camera feed. Having just one feed running without the GPU helps bog down the CPU entirely.

The best part is that my setup is largely overkill with a dedicated GPU. All you really need is a TPU and integrated graphics, something which even SBCs like a Raspberry Pi can provide. What this does provide for my own setup is longevity and space to grow the network of cameras.

Home Assistant is the icing on the cake

Integration is key

One of the major selling points of Frigate for me was its integration with Home Assistant, something we rely on for controlling a bunch of smart devices and services at home. Keeping my CPU usage low through Frigate allows the rest of the chip to be dedicated to other tasks, including running Home Assistant OS. I’ve got countless connections, integrations, and hundreds of entities loaded into Home Assistant, and it has become a vital piece of infrastructure for creating the ultimate smart home.

If you’re looking to build a DIY NVR system with minimal CPU usage, I cannot recommend Frigate and Proxmox enough. So long as you have the available graphical processing and optional TPU, your CPU will barely be touched like mine, allowing you to run other things on the same Proxmox node, or keep it segregated on low-power hardware.