Why cudaMalloc fails on NVIDIA Jetson Orin Nano Super — and the one flag that fixes it (opens in new tab)

If you've tried running a GGUF model via llama.cpp on a Jetson Orin Nano Super and hit this error: NvMapMemAllocInternalTagged: error 12 cudaMalloc failed: out of memory ...while tegrastats shows the GPU idle and several GB of RAM free — this post is for you. The hardware context The Jetson Orin Nano Super (8 GB) uses unified memory. There is no separate VRAM pool — the CPU and GPU share one physical 8 GB block. This is how all Jetson Orin-series SoCs work, and it's what makes them cost-effec...