I built a distributed job queue in Go to understand how they actually work (opens in new tab)
I have used job queues my whole developer life without knowing what was inside them. So I built one. Not a wrapper around an existing queue. A full implementation from scratch with Redis, PostgreSQL, goroutines, and real failure handling. Here is everything I learned. Why Dual Storage Most job queues use one store. Redis is fast. PostgreSQL is durable. I wanted both. Redis handles dispatch via a sorted set priority queue. Fast enqueue, fast dequeue. PostgreSQL is the source of truth. Every jo...
Read the original article