Recently, I’ve had to rediscover some of my knowledge on makefiles for a work project. As these things go, when I had knowledge on makefiles it had all seemed obvious; so, no need to write it down, right?

Anyway, this is my attempt at properly archiving this knowledge for when I need it again five to ten years from now.

In case you don’t know, make is a build tool originally created by Stuart Feldman in 1976 at Bell Labs. While make is most commonly associated with C and C++ projects, it can be used in any context where you want to rebuild only the relevant parts of something after a dependency has changed.

Nowadays, many programming languages ship with their own build system, like Rust/Cargo, Zig, or Go; but make is not limited to just pure programming languages or build processes that yield a library or executable.

Let’s imagine, for example, that we wanted to write posts for our blog in Markdown. In order for people on the internet to read those posts, the Markdown files will need to be converted to HTML. Usually, this part is done by a static site generator, such as Hakyll, or Pelican. This blog also used to be built using Hakyll, as you can read here.

These tools work great and they make it easy to get started! But let’s assume for a moment, that we had a lazy afternoon, out wife is out of town, we’re sitting on the couch with our cat, and we have all this newly rediscovered knowledge on makefiles. Could we build our blog with just that? How hard could it be, right?

pandoc - Turning Markdown into HTML Since 2006

The tool that many static site generators use internally is called pandoc, a universal document converter.

pandoc really deserves its own blog post, so for the purposes of our little project here, it’s enough to know that it essentially converts any markup format - such as Markdown - into any other markup format - such as HTML. It can also work with templates which describe how a standalone document should look like.

If you’ve looked at the templating languages used by various static site generators, you may have noticed that they are all very similar (not to say identical). The reason is that they use pandoc internally, and the templating language is the one pandoc uses. Case in point, we will not have to adjust our existing templates for this project.

The command for turning one of our Markdown posts (foo.md) into the appropriate HTML (foo.html) looks like this:

pandoc --standalone --template=templates/post.html -o foo.html foo.md

The Structure of a Blog

The source code of our blog has a pretty simple structure; you have some posts written in Markdown, some templates that describe how to turn this Markdown into HTML, and some CSS to make it look a little more fancy:

posts/
2022-05-10-foo.md
...
templates/
post.html
...
css/
style.css

The index.html will have a list of recent blog posts and some general information about the blog, like an Atom feed, a git forge, and a contact email address.

From a programmer’s point of view, we have a compilation step which turns each Markdown file into an HTML file, followed by the generation of the index.html and atom.xml, and finally bundling these outputs with the CSS into a single directory with the following structure:

out/
atom.xml
index.html
css/
style.css
posts/
2022-05-10-foo.html
...

Up to this point, we could theoretically perform these steps manually each time we write a blog post, especially if we don’t publish a new one very frequently. However, that’s not really satisfying, is it?

Let’s imagine we had many blog posts and compilation of each individual blog post took a long time. We would only ever want to recompile what’s necessary leaving everything else untouched, right? That’s where make comes in.

make - Building Software Since 1976

make works based on three things: targets, their prerequisites, and the recipes of how to build the targets. In a makefile, this looks like this:

[target]: [prerequisites]
[recipe]

For example, we can use our pandoc call as a recipe to build our foo.html whenever the prerequisite foo.md or the post.html template changes. Let’s put that in a file called Makefile.

foo.html: foo.md templates/post.html
pandoc --standalone --template=templates/post.html -o foo.html foo.md

We can then build this by running this on the command line:

make foo.html

To determine whether there is anything that needs to be done, make checks whether the last-modified time of the target is older than the last-modified time of any of its prerequisites. If so, it rebuilds the target, otherwise it does not do anything.

Running make without specifying a target will build the first target in the list, so we can specify a top level target that builds our whole blog by specifying all relevant targets as a prerequisite:

blog: out/foo.html out/bar.html

out/foo.html: posts/foo.md templates/post.html
pandoc --standalone --template=templates/post.html -o out/foo.html posts/foo.md

out/bar.html: posts/bar.md templates/post.html
pandoc --standalone --template=templates/post.html -o out/bar.html posts/bar.md

So far, we’re only compiling the Markdown files into HTML files and putting them in the out/posts directory. We’re still missing the atom.xml feed and the index.html, so let’s define two additional targets and add them to the prerequisites for our main target.

blog: out/foo.html out/bar.html out/index.html out/atom.xml

out/index.html: out/foo.html out/bar.html build_index.sh templates/index_top.html templates/index_bottom.html
sh build_index.sh

out/atom.xml: out/foo.html out/bar.html
sh build_feed.sh

For the sake of brevity, we will skip over how we actually generate these two files. We encapsulate the logic in two small bash scripts. You can find them in the repo for this blog if you are interested.

Now, this is already a workable setup, but we still have to change our makefile every time we create a new blog post and add the new dependency. This would become rather tedious. Ideally, we want to just type make after adding a post and see that all the files are created/updated correctly without ever touching our makefile. To achieve this, we will use some built-in functions of make.

OUT_DIR := out

sources := $(wildcard posts/*.md)
posts := $(subst .md,.html,$(sources))
posts := $(subst posts/,$(OUT_DIR)/posts/,$(posts))

First we define a variable OUT_DIR to hold the name of the directory where we want to create all the files, so that it’s easy to change if we ever feel like it. Then we define another variable sources which holds the names of all Markdown files in posts/. Then we replace the .md with .html, and prefix the names with the value in OUT_DIR.

For our example above, the posts variable looks like this: out/posts/foo.html out/posts/bar.html.

This allows us to make the prerequisites for our various targets static:

blog: $(posts) $(OUT_DIR)/index.html $(OUT_DIR)/atom.xml

$(OUT_DIR)/index.html: $(posts) build_index.sh templates/index_top.html templates/index_bottom.html
sh build_index.sh

$(OUT_DIR)/atom.xml: $(posts)
sh build_feed.sh

But we still need to create a new target for each blog post. To get around this we can use pattern rules:

$(posts): $(OUT_DIR)/posts/%.html: posts/%.md templates/post.html
pandoc --standalone --template=templates/post.html -o $@ $<

Don’t worry, this looks much worse than it is. This essentially goes through the individual targets in posts and tries to match the pattern $(OUT_DIR)/posts/%.html where % is a wildcard. If the pattern matches, it defines a corresponding target with the prerequisites posts/%.md templates/post.html where % contains the value that was matched in the wildcard before.

Finally, $@ contains the current target, and $< contains the first prerequisite. These are built-in automatic variables in make. There are a few more of these, but we won’t need them.

As for the CSS, it doesn’t have any prerequisites (other than itself) or need for code generation, so we simply copy it into the appropriate directory and add it as a prerequisite for the blog target.

blog: $(posts) $(OUT_DIR)/index.html $(OUT_DIR)/atom.xml $(OUT_DIR)/css/style.css

$(OUT_DIR)/css/style.css: css/style.css
cp $< $@

And that’s it, now we can just type make on the command line, and all changes will be updated on our blog.

The Extra Mile and Phoniness

So, what’s left? Well, it’s common practice in makefiles to define a clean target which removes some or all build results, for example, intermediate files that are no longer needed after compilation has finished. It would also be nice to have a shortcut to actually publishing the updated version of the blog, e.g., a publish target.

Both of these targets, as well as our top-level blog target have one thing in common: they are not actual files on the file system. While we do not plan to have any files named like this in our blog, it is a good opportunity to introduce one final concept of make: phoniness.

A target marked as .PHONY will always be built when called. Even if there is a file that matches the target name and none of its prerequisites are newer than it. So, let’s define our final two targets and add some phoniness to the relevant targets.

.PHONY: blog clean publish

blog: $(posts) $(OUT_DIR)/index.html $(OUT_DIR)/css/style.css $(OUT_DIR)/atom.xml

publish: blog
tar -C $(OUT_DIR) -cvz . > $(OUT_DIR).tar.gz
hut pages publish -d blog.erikwastaken.dev $(OUT_DIR).tar.gz

clean:
rm -rf $(OUT_DIR)

Further Reading

If this has made you interested in make and makefiles, there’s more information at Learn Makefiles. I also recommend Clay Dowling’s blog series on makefiles. They’re what got me started on makefiles years ago and are quite enjoyable reads!

• OMG WTF Makefiles
• Easy Makefiles