Chat With Your Documents Locally Using AnythingLLM and Ollama (opens in new tab)

A private RAG system where you drop in PDFs, Word docs, and code files and ask questions. Runs on any machine, no cloud dependency. What You Need Any computer (GPU optional - CPU works fine) Ollama installed About 10 minutes Architecture Component Role AnythingLLM Desktop/server app with RAG, agents, built-in vector DB Ollama Serves local LLM for chat + embeddings Qwen3 14B Default model for answering questions Setup 1. Install Ollama # Install from ollama.com, or run with Docker: docker run ...