Stop letting the prompt be your state machine (opens in new tab)

Stop letting the prompt be your state machine You shipped an LLM feature six months ago. Now the same user input produces wildly different outputs depending on... nothing you can point to. Something in the sampling? The time the context filled up and a chunk got dropped? Nobody knows. This is what happens when the prompt becomes your runtime. The trap: the prompt as an accidental runtime Here is what the trap looks like in TypeScript: async function handleUserRequest(input: string): Promise<s...