Put Your Runbook in Your README

During software development and maintenance we use an array of commands to build, test, deploy and release our software. When nobody writes these commands down, they’re tacit knowledge: the floor-level of computing know-how that’s necessary to contribute to the project. This is widely acknowledged as a barrier to entry and an accessibility pain point for Free software.

Tacit knowledge of development workflows can also lead to confusion and unnecessary friction between maintainers of a multi-user repository who develop slightly different versions. This is a key aspect of the famous “It Works On My Machine” development methodology.

When we write down our workflows, this constitutes **a run…

When we write down our workflows, this constitutes a runbook.

What I want out of a runbook

Writing a runbook takes time. It’s rare to write anything down for a script I intend to run one time and discard. In order to get value out of my runbook, I want other people to find it, including my future self. Therefore one of the universal goals of every runbook is discoverability.

Just writing anything down at all can be handy, but I can further increase the value by adding specificity. Exact command invocations, dependency versions, preparatory & cleanup steps, dependencies, and environment setups increase the likelihood that I can reproduce my workflows in the future, or enable another person to succeed in doing so.

Software practices tend to drift over time, as requirements change and trends shift. If practice changes but the runbook doesn’t, it becomes a form of tech debt, misleading contributors who aren’t aware of the shift and newcomers. My ideal runbook would always be current and reflective of the practice actually used to produce the most recent builds & test results.

How we maintain runbooks now

The usual practice is to scatter pieces of the runbook in files throughout the software repository, such as:

targets in a Makefile (or Rakefile, Justfile &c)
tasks in a language-specific config file like package.json or pyproject.toml
workflows in a config directory for some CI provider, like .github/workflows, Jenkinsfile, .travis.yml, or many others
shell scripts1
READMEs, HOWTOs, blog posts, and web documentation

Let’s analyze these in quadrants:

Low discoverability, likely to get outdated

shell scripts
targets in Makefile or similar
language-specific tasks in config files

These tend to bitrot quick if they aren’t used in CI. Those instances which are on the CI “hot path” get moved into the next category.

Old blog posts also tend to drop in discoverability while simultaneously bitrotting.

Low discoverability, current

workflows in CI config
those scripts, tasks and Makefile targets which are exercised in CI

High discoverability, likely to get outdated

READMEs & HOWTOs
recent blog posts
web documentation

These are easy for folks to find and are often the first on-ramp for new users and contributors, but they also tend to bitrot fast since they’re not on the “hot path” and software projects typically go through many update/extend/refactor cycles for each documentation refresh/blogging cycle.

High discoverability, current

Here we expect to find our most exceptional option, but instead it’s crickets. In practice, projects do not maintain current runbooks in a place that’s optimized for discovery and accessibility, and that’s a shame.

Put your runbook in your README

If the README is among the highest-impact, most discoverable places to document workflows, why don’t we put our scripts there? The typical marquee reason is that READMEs are non-executable: they’re prose, not code. This makes them suffer in the critical “specificity” criterion for runbook technology. Enter xc, the task runner for your README that works seamlessly with your CI, shell, and editor.

xc2 is a task runner similar to Make or npm run, that aims to be more discoverable and approachable

To use it, take all the commands you run to build, debug, test & deploy and document them in your README as code blocks, each under a descriptive header, like “Build”. Then run xc build and it’ll find that header in your docs and run the code block there.

Crucially, put your CI steps in your README and run them on each merge or release. Quality of life here is good: it supports dependencies between blocks, script arguments and plenty more.

You can also include usage instructions, caveats, deprecation notices, links, and other context right there next to the task’s script. And it’ll actually look nice, since you have all the info presentation tools normally available in READMEs at your disposal, instead of just spartan code comments.

Is it any good?

Yes! I’ve found xc especially nice to use:

It’s a single small binary that adds practically zero runtime overhead.
It supports both Markdown and org-mode READMEs.
I don’t have to export tasks into local scripts and then run them. There’s just one step: execute.
The niceties you’d expect are all there: shell completion, CI and IDE integration, and an interactive TUI mode.

Discoverable, executable runbooks that stay current

xc solves a need in the current dev tool space by putting workflows in the most prominent location and keeping them on the “hot path” so that they don’t get out of date. It’s Free software (MIT license) and migrating has been straightforward. This is a tool I’m glad to have in my working set.

Project scripts have their own whole D&D alignment chart.

Scripts go in: