“Bus factor” is a somewhat morbid term used in software engineering (and possibly elsewhere), meaning the number of people who would have to get “hit by a bus” for a project to fall apart. In programming terms, it’s often used to mean the number of people who understand a codebase. It’s also called the “lottery factor” sometimes to try to have a less morbid spin; the number of people who would win the lottery and quit working. That doesn’t quite feel appropriate for open source projects done by volunteers, because potentially those people would not stop working on a project just because of petty concerns like winning the lottery (lol).

In 2021, Mosè Giordano and I wrote a blog post for the Julia blog that includes some stats on the estimated bus factor for Julia packages, placing the median number of contributors at 2. This includes any contribution, so the real “understanding the full codebase” bus factor is probably 1. These stats are in-line with other broader studies of open source projectsE.g. A Novel Approach for Estimating Truck Factors (2016) by Avelino, Passos, Hora, and Valente found 65% of repositories they studied had bus factor less than or equal to 2.. This is typically and rightfully understood as a fragility problem at the ecosystem level, although open package ecosystems tend to develop multiple competing solutions, packages with overlapping scope, and forks which can help mitigate the impactas someone who helps maintain Julia’s General registry, I have some thoughts on that written in this github comment that I might try to expand into a blog post at some point. But I probably should do a bit more research first..

LLMs let you write bus factor 0 software: code for which no human has ever fully read and understood the codebase. The code could be good or bad, and that distinction does matter for LLM-written code, but the capability to even have “real” code that no one has read is a pretty new and interesting thing. Now, code generation as a whole is not new, and you could make a pedantic argument about no one reading generated assembly (although compiler devs do) or FFTW code generation or similar projects, but I maintain generating high-level code that before say 2024 was 99+% written by humans is a new twist in that journey. LLMs also potentially let you “maintain” bus factor 0 code (“Claude, my user reported this error, please fix”), and again quality will vary widely.

Bus factor 1

I think to understand bus factor 0, it makes sense to start with bus factor 1. Clearly, bus factor 1 code is usable to some extent. The fact that many important systems end up relying on various single-point-of-failure bus factor 1 systems (obligatory XKCD#2347) points to the fact that these pieces of software work well enough to get incorporated into bigger projects. It’s also widely recognized as a major point of fragilityFor example, the Open Source Security Foundation (OpenSSF), which itself is a Linux Foundation project, has a best practices badge system, and says projects should have a bus factor of at least 2 to qualify.. If an open source maintainer stops maintaining a piece of software for whatever reason, someone else may be able to onboard onto the codebase and take the reigns, or it may languish and fall apartthere’s actually some numbers on this from On the abandonment and survival of open source projects by Amaral Avelino, Constantinou, Valente, and Serebrenik (2019). They found in their survey 16% of open source packages at some point lost all of their developers, and of those, 41% of the projects survived by gaining new developers..

But bus factor 1 is also the case for many hobby projects, academic projects, and so forth. It’s a marker of fragility, but it’s OK for some projects to be fragile, or in other words, sometimes it’s better for the thing to exist in a state of fragility than not exist at all. Sometimes the code itself is not the point; instead of a existing to be a maintainable ongoing system, it exists to investigate something or explore something or prove a point. Which is why academic code is known for being terrible quality; typically the code is not the end-goal. The problems of course come in with scope creep: the one-off code that becomes load-bearing.

My rather obvious point here is the same dynamics basically exist for bus factor 0 code, just extremized one notch further. When no human understands the codebase, the code is quite fragile. It may have various defects, some of which might be quite subtle. It may have design issues that cut deep. And it may be hard to fix when problems arise. None of that is certain of course; the code might be basically fine, just “unread”. But the fact that it has never been understood by human opens these risks, similar to human code written by copy-pasting stackoverflow snippets together until it “works”, without understanding how or why.

However, fragility alone does not necessarily contravene productive use. It is a risk like any other. So when is it appropriate? Like with bus factor 1 code, it’s most appropriate when it is being used in a way in which this risk does not lead to severe consequences. For example, low risk one-off tasks, or for exploration or discovery, when those outcomes are not predicated on the code being fully correct. Something very interesting in this vein are problems in which the solution can be verified efficiently (e.g. NP-hard decision problemsI say “decision problems”, but I’m actually thinking more about formal verification, capabilities & compiler invariants. I just feel less confident saying anything there.).

Aside: if LLMs get really really good (or are really good already), should their “understanding” count as bus factor 1? I think no, but that isn’t to say high quality LLM analysis is not useful. I just think it is capturing different. Potentially someday this distinction will be less relevant.