— or why I wrote procjail
The problem I kept running into
I’ve worked with containers long enough to be comfortable with them. Docker, Kubernetes, YAML, dashboards — all of that is familiar.
And yet, every serious production incident followed the same pattern:
- OOM kills that didn’t make sense
- Processes that ignored SIGTERM
- Containers that refused to shut down
- Zombie processes quietly piling up
When that happened, the abstractions stopped helping.
At some point, I had to answer questions like:
- What process is actually running?
- Who is PID 1?
- What does the kernel think is happening?
I realized something uncomfortable:
I could use containers very well, but I could not always explain their failures without hand-waving.
The idea: remove eve…
— or why I wrote procjail
The problem I kept running into
I’ve worked with containers long enough to be comfortable with them. Docker, Kubernetes, YAML, dashboards — all of that is familiar.
And yet, every serious production incident followed the same pattern:
- OOM kills that didn’t make sense
- Processes that ignored SIGTERM
- Containers that refused to shut down
- Zombie processes quietly piling up
When that happened, the abstractions stopped helping.
At some point, I had to answer questions like:
- What process is actually running?
- Who is PID 1?
- What does the kernel think is happening?
I realized something uncomfortable:
I could use containers very well, but I could not always explain their failures without hand-waving.
The idea: remove everything
Instead of adding more tooling, I decided to remove it.
I wanted the smallest possible program that would:
- create Linux namespaces explicitly
- apply cgroup resource limits directly
- run a process as PID 1
- forward signals manually
- do nothing “helpful” behind the scenes
That program became procjail.
You can find it here: github.com/Emmanuel326/procjail
The uncomfortable truth about containers
Building this forced me to internalize something very simple:
A container is just a Linux process.
More precisely:
- a process with a modified view of the system (namespaces)
- a process constrained by kernel-enforced limits (cgroups)
- often running as PID 1, whether it expects to or not
Everything else is tooling layered on top.
When things break, it is the kernel you are debugging — not Docker.
PID 1 is not a normal process
If procjail taught me one thing clearly, it’s this:
PID 1 has responsibilities that most programs are not written to handle.
As PID 1:
- signal handling semantics change
- ignored signals stay ignored
- zombie reaping becomes your job
- exiting tears down the entire environment
Many real-world container bugs are just this fact surfacing late.
What surprised me most
What surprised me wasn’t how complex this was.
It was how little code it took to reproduce real production failure modes.
A few syscalls. A few mounts. One badly behaved process.
And suddenly:
- SIGTERM doesn’t shut things down
- children outlive their parents
- memory limits kill processes abruptly
Containers are thin abstractions. That’s not a criticism — it’s a warning.
Why procjail stops here
I deliberately stopped adding features.
No networking. No image pulling. No overlay filesystems.
Those solve Day-1 problems.
Procjail exists for Day-2 — when you are staring at a broken system and the kernel is the only thing left that tells the truth.
The moment the kernel pushed back
This stopped being an academic exercise the time I tried to shut my laptop down.
The system didn’t power off. It hung.
Then the kernel panicked and told me to reboot.
I ended up forcing the machine off by holding the power button. When it came back up, everything looked fine — but I wasn’t.
Earlier that day, I had been working on procjail. Creating mount namespaces. Mounting /proc. Running processes as PID 1. Experimenting with cgroups.
Nothing crashed. Nothing failed loudly. Everything appeared to work.
And yet, during shutdown — when the kernel tried to tear the system down cleanly — it hit something it couldn’t reconcile.
That was the uncomfortable moment:
Did I actually do this?
I wasn’t debugging a userspace crash. I wasn’t chasing a panic caused by load or memory pressure.
I had violated an assumption the kernel makes about cleanup.
And it only surfaced at the very end.
Containers don’t usually fail at startup. They fail at teardown.
A leaked mount. A namespace that wasn’t as isolated as I thought. A lifecycle I misunderstood.
The kernel wasn’t being fragile. It was being strict.
That incident permanently changed how I think about containers.
Closing thoughts
I’m not suggesting anyone stop using Docker or Kubernetes. They are incredibly useful tools.
But I do think we should stop treating them as magic.
The kernel is doing the real work. When things fail, that’s what you’re debugging — whether you realize it or not.
Procjail exists to make that impossible to forget.
— Emmanuel