AI Assistant Architecture: LLM, Memory, Tools, Routing, Observability (opens in new tab)

A production AI assistant is not "an LLM with a prompt". It is a system that accepts intent, keeps state, decides when to retrieve or act, and exposes enough runtime detail to debug failures. That systems-level view is what the AI Systems cluster explores when assistants move beyond a single model invocation. OpenAI describes agents as applications that plan, call tools, collaborate, and keep enough state for multi-step work, while Anthropic frames the same problem as a managed harness that c...