It took me weeks to figure this out, so want to share!

A good base model choice is MOE with low activated experts, quantized to NVFP4, such as Qwen3-Next-80B-A3B-Instruct-NVFP4 from huggingface. Thor has a lot of memory but it's not very fast, so you don't want to hit all of it for each token, MOE+NVFP4 is the sweet spot. This used to be broken in NVIDIA containers and other vllm builds, but I just got it to work today.

- Unpack and bind my pre-built python venv from https://huggingface.co/datasets/catplusplus/working-thor-vllm/tree/main - It's basically building vllm and flashinfer from the latest GIT, but there is enough elbow grease that I wanted to share the prebuild. Hope later NVIDIA containers fix MOE support - Spin up nvcr.io/nvidia/vllm:25.11-py3 docker container, bind my venv and model into it and give command like: /path/to/bound/venv/bin/python -m vllm.entrypoints.openai.api_server --model /path/to/model –served-model-name MyModelName –enable-auto-tool-choice --tool-call-parser hermes. - Point Onyx AI to the model (https://github.com/onyx-dot-app/onyx, you need the tool options for that to work), enable web search. You now have capable AI that has access to latest online information.

If you want image gen / editing, QWEN Image / Image Edit with nunchaku lightning checkpoints is a good place to start for similar reasons. Also these understand composition rather than hallucinating extra limbs like better know diffusion models.

All of this should also apply to DGX Spark and it's variations.

Have fun!