26 Sep 2024, 9:20 a.m.

Changelogs and Release Notes

• Advice
• Open Source and Free Culture
• maintainers

My friend Ned Batchelder posted, “A list of commits is not a changelog!” and spurred this post.

Summary: We’d all benefit from restoring the distinction between a detailed changelog and brief release notes, but that’s hard to do if a project relies solely on GitHub as its communication platform.

Context: Ned maintains and uses open source software. He was talking to fellow open source maintainers, as am I.

Wording: I’m going to assume you know what open source software is, and understand what I mean when I use terms such as “commit” (the noun), “mailing list,” “issue”, “deprecation”, and “beta” in this context.

Where I’m coming from: I have a lot of advice for maintainers on how to communicate with their users. I think changelogs suit some of those communication needs and not others. In my guide “Marketing, Publicity, Roadmaps, and Comms” I discuss

• your audiences and what you need to tell them
• the roles of roadmaps, mailing lists, blogs, mailing lists, in-application notifications, group chats, videos, and more (with links to examples)
• a sample schedule for what to do 1 month before a release, 1 week before, at release, and one month afterwards So I’m thinking of changelogs in that context. A maintainer can use several announcement and documentation tools to systematically communicate to their users, upstreams, and other interested people about new releases, new features, deprecations, and so on. And one of them is the changelog.

A word definition: Karl Fogel, in Producing Open Source Software, writes that:

There should also be a CHANGES file (sometimes called NEWS), explaining what’s new in this release. The CHANGES file accumulates changelists for all releases, in reverse chronological order, so that the list for this release appears at the top of the file....

The sample CHANGES file is a list of succinct lines such as “Added regular expression queries (issue #53)” and fixed reindexing bug (issue #945)“. A reader who wants more details can follow up by reading the referenced issues.

The list can be as long as necessary, but don’t bother to describe every little bugfix and feature enhancement in detail. The point is to give users an overview of what they would gain by upgrading to the new release, and to tell them about any incompatible changes.

Karl wrote that nearly 20 years ago. More recently, Olivier Lacan has urged developers to “Keep a Changelog”, saying that a changelog “contains a curated, chronologically ordered list of notable changes for each version of a project”.

Change size, categories, and completeness: We agree that, usually, a list of commits isn’t suitable as a changelog. As my spouse Leonard Richardson puts it, each changelog entry should reflect an issue resolved, because that is the unit of someone caring about a change. One issue resolution often happens over multiple commits, so it’s better to summarize it in one log entry. Also, the first line of a commit message often isn’t suitable as a changelog entry – authors (very reasonably) communicate in jargon, abbreviations, etc. Aurélien Gâteau’s shared some examples. So someone needs to translate and edit and write a changelog targeting end users, and this can happen either iteratively alongside writing and merging contributions (MediaWiki example), or in a batch before each release. The “how” of that is a big enough topic that I won’t address it here.

I’m fine with a project either keeping one giant changelog (as pip does) or moving the previous releases’ changelog into a separate file (as MediaWiki does with HISTORY).

When the list of changes carried in a specific release gets long, it’s sensible to bucket them into categories, so different kinds of users can easily find changes that will affect them. For example, MediaWiki’s changelog for 1.41 has sections for end users, system administrators, API users, translators, and open source contributors (also relevant to downstreams who incorporate the software as a dependency). Or, for tools only used by other developers, you could break the list up as Zack Weinberg did in the autoconf 2.72 NEWS list: “Backward incompatibilities” (also known as “breaking changes”), “New features”, “Obsolete features and new warnings”, “Notable bug fixes”, and “Known bugs”.

We agree that a changelog is meant to be comprehensive, listing all the changes in the software since the previous release. David Brownman disagrees, opining that, for example, documentation updates and repository configuration changes don’t affect end users of a library, and thus don’t belong. But I disagree, and I’ll get into why in the next section.

(I do make an exception for truly trivial changes, such as whitespace modifications and documentation typo fixes, and I agree with the approach pip maintainers take in omitting stuff like that from the NEWS file.)

How detailed?: But we disagree on how detailed each entry should be. Karl writes, “don’t bother to describe every little bugfix and feature enhancement in detail.” Ned writes, “Tell me about the things I need to know. Give me the implications, and links to docs. ‘Updated dependency xyz to v4.3’ is useless.” And some projects do this. Dreamwidth calls it a “code tour”. Every time Dreamwidth deploys a new update to the site, someone writes explanations for each change, links to the GitHub pull request, and credits the authors. Example entry from ilyena_sylph’s June 2023 code tour:

Issue 3035: Update color picker element to something more modern (pull request) Category: Site Modernization; Prettifying Patch by: momijizukamori Description: You may or may not know that if you go to customize your style (https://www.dreamwidth.org/customize/options?group=colors), if you click on the little color swatches, you get a colorpicker? It opens in a new window, and is 1) ancient, 2) terrible, 3) inaccessible, and 4) apparently difficult for a lot of people to find. It has now been nuked to oblivion and replaced with a nice little library that pops up in the page and which has way better support for screenreaders and keyboard navigation. It also comes in dark mode to match all you Gradation users out there.

That was #16 of 45 changes. If you read it and think, “That’s way too much detail! How was a user supposed to skim this list for what mattered to them?” then I have good news for you: along with the code tour, Dreamwidth published an announcement that summarized salient changes in 9 bullet points. Which is to say: they provided release notes.

Release notes: We used to have a clear division, in open source software documentation, between changelogs and release notes. Release notes are a prose summary of what’s changed, which exist in addition to a changelog (the release notes might link to the changelog or include a copy at the bottom), and which focus on changes the user might perceive.

Zulip’s 9.0 announcement is a good example; scroll to “Release highlights”. Note that the medium (a blog post) gives author Tim Abbott room to give context, share screenshots, connect this release to Zulip’s product roadmap, ask for feedback, and so on. This approach makes sense for consumer-facing projects such as GUI applications. Another example: GNOME, which published separate user-facing and developer-facing release notes for GNOME 47 (“Denver”) as web pages.

Release notes for libraries and developer-targeted command-line tools often don’t need to include screenshots. Leonard Richardson’s announcement email for Beautiful Soup 4.13.0 beta 2 is a plain text release notes example. He started,

For the past few months I’ve been working on adding type hints to the Beautiful Soup code base. This process exposed a number of very small inconsistencies which couldn’t be fixed without changing behavior.

and then explained how to test the beta, asked for feedback from people who use a few specific features, pointed out a particular deprecation warning, and set schedule expectations for the next release. Similarly, my pip 20.2 announcement email linked to more detailed release notes which I’d published as a python.org blog post. This release included a beta version of a disruptive change, so I linked to a part of our documentation where we’d explained how to test and migrate, setups to test with special attention, our reasons for the change, the deprecation timeline, and so on. And I thanked our funders.

You can use release notes to announce things that don’t quite fit in a changelog, but that your users ought to know. “This is the penultimate release in the 3.x series.” “I just took a full-time job, so the time horizon for the next release is probably months, not weeks.” “Without new funding or volunteer assistance, this will be the last release that integrates with foo.” “We moved our discussion forum to a more usable platform.”

Release notes don’t replace changelogs, but supplement them. They balance each other. If a reader reads release notes and wants more specifics, they can go read the comprehensive changelog. If a changelog is too overwhelming and an end user is left thinking, “but what matters to me?” they can go read the release notes.

So that’s why I disagree with David Brownman. Changelogs should be comprehensive. Sharing documentation updates, repository configuration changes, schema migrations, architecture overhauls, new automated tests, etc. in a changelog helps readers appreciate the invisible maintenance work you do, and informs downstreams who are maintaining forks. And excluding these changes from a changelog sends a message about your values and about how you think about open source. It says that you think of your users primarily as consumers rather than as partners and potential contributors. In contrast, when you include “under-the-hood” changes in a changelog, that can tell supporters where their money’s gone, showcase the work of more volunteers, and provide inviting examples for people who had been thinking of getting involved.

And you can do all this safely knowing that users who only want a summary of user-perceptible changes can find that – in the release notes.

Skill: I’ve witnessed some maintainers’ apprehension at writing release notes. Anytime someone’s trying to write in a medium or genre they’re not used to, it can take a bit of practice and editing to get the tone right. (And this is something coaching and training can help with - Heidi Waterhouse is good at that, as am I.)

Changes in communication mediums: Most of the examples I’ve used here are projects that predate GitHub. Open source projects used to use a constellation of digital services for different kinds of activities: a code repository, an archive of packaged releases, an issue tracker, a mailing list, and a documentation site might all be separate, not to mention blogs, wikis, and chat.

And we used to assume that open source projects maintained some textual communication platform to officially announce releases – a Usenet group, a mailing list, a blog, or something like that – other than GitHub.

GitHub discourages release notes (compared to changelogs): But now, a ton of projects pretty much live on GitHub.* And GitHub’s interaction design privileges changelogs over release notes.

GitHub hosts and displays the CHANGELOG.md format just fine. A changelog file in the top level directory of a repository shows up reasonably prominently in a repository’s main page, where a lot of people expect to find it.

You can create a “Release” on GitHub and add release notes (example: Hometown v1.1.1). But, as Lacan notes, “The current version of GitHub releases is also arguably not very discoverable by end-users, unlike the typical uppercase files (README, CONTRIBUTING, etc.).”

Relatedly, adoption is pretty uneven. I’ve only witnessed projects doing this when they use the GitHub workflow to manage and publish their releases through GitHub, and many don’t. Example: vim. If you look in the sidebar, in the “Releases” section, you can click on the “17,469 tags” link, which leads you to a list of Git tags – and then if you select the “Releases” tab there, you’re told, “There aren’t any releases here.” That’s because vim publishes release notes (example: 9.1), and links to downloads, on the vim website, and GitHub doesn’t offer a way to transclude those from another data source into what GitHub displays, or redirect that “Releases” link to a webpage you control.

GitHub does make it easy to automatically generate a draft text for your release notes, which I’m guessing Ruby 3.3.5 used for theirs. It gives you a list-style changelog, and while I’m sure some maintainers edit it into release notes (summarizing, emphasizing the key changes, and adding context), clearly some do not.

Also, to create a GitHub Release for a particular version of your project, you have to associate that release with a Git tag (which isn’t how some projects manage their Git workflow).

And I get that this would be difficult to solve. I’m sure that product managers at GitHub have scars that ache at the mere memory of dozens of past battles waged over the precious real estate that is a repository’s main page. I fully admit that I’m not offering a solution.

* (Maybe they also communicate on microblogging platforms – X, Mastodon, Bluesky – and on unlinkable Discord servers and Slacks – but generally, none of those provide a stable, public-web-searchable venue for a textual announcement that might be hundreds of words long.)

Conclusion: The difference between a changelog and release notes is the difference between a “reference” and an “explanation” in Divio’s “four kinds of documentation” framework. A changelog, like an inventory of available API methods, is comprehensive, reliably consistent, and uses section headings and other formatting to be skimmable. A set of release notes, like a “why we made this controversial architectural choice” blog post, is distilled, often narrative, and uses brevity and focus on the reader’s priorities to be quickly readable. In Heidi Waterhouse’s words, it can be a derivation of the decision log.

They are both useful and both worth making, even if GitHub doesn’t make it easy to publicize the latter. And that’s one of several good reasons to maintain a place you control – outside of GitHub – where people can reliably find and subscribe to information updates about your project without having to depend on a totalizing vendor. An email list/newsletter or a blog pays excellent dividends, and this is just one of them.

Hope this is helpful! I’m Sumana Harihareswara, and I can help you with this sort of thing – including learning how to quickly draft release notes – through my consultancy, Changeset Consulting. I do project management, coaching, training workshops, and more. Anyone reading this is eligible for a short free 30-minute consultation call; email me.

And I’m working on a book on managing existing open source projects, so you can learn how to get them unstuck. You can read three sample chapters by signing up for my newsletter.