I've updated my glorified Llama fork (LLM Inference Server) for P40's to utilise MTP + TurboQuant + DFlash (opens in new tab)

Updates, ideas, and inspiration from GitHub to help developers build and design software.

Subreddit to discuss about Llama, the large language model created by Meta AI.