My AI Learning Journey – Part 4 (opens in new tab)

In part 3 of this series, I’ve set-up my headless local LLM execution environment with Ollama so I’m ready for further experiments. One of the questions I wanted to answer with this setup is how much faster LLMs run on a GPU vs. the CPU. Part of the LLMs one can download is a configuration file that defines how the LLM is set-up for execution. One parameter in this file defines how many of the neuron layers of the LLM are to be executed on the GPU and how many of them should be run on the CPU...