In 2021 I wrote an article encouraging people not to build general purpose APIs for their own front-ends. (You should probably read it before reading this one.) It got featured on Hacker News twice, albeit with a worse reception (and more heated discussion) the second time around. My guess is, more front-enders showed up. 😛

Having observed this approach for 6 years, I’ve only grown more confident in its success. Part of that confidence comes from seeing it play out. Our team vastly simplified maintenance, reduced bugs, and boosted performance by making the jump. (The jump was made possible by Long Term Refactors btw.) Another reason for my confidence is the many comments I’ve received over the years confirming that it works for other teams too. Finally, I’ve received a number of challenges and condemnations, although they were either entirely theoretical, or based on a misunderstanding of the article. Since I consider it somewhat my fault for not being clear enough, I want to address these misunderstandings. So here they are, organized into topics, each one addressed in turn.

1. You reinvented HTML!

Multiple readers have informed me that I can apparently serve HTML directly from the server, instead of doing all that JSON payload nonsense. Having started building websites in 2003, I get it. However, the reality is that today we work with front-end teams, they work with React, and React comes with certain practices. 15 years of them in fact, in addition to the baggage of pre-built components. That’s worth some respect. If a JSON payload is preferred, and I have no trouble delivering it, why would I mind? Personally, I much prefer the good old Ruby on Rails-esque stack with server-side HTML rendering, but we work in teams, and should probably play to each other’s strengths.

That said, the bigger issue with the “you rediscovered HTML” crowd is that they’ve misunderstood the advice. HTML involves content, structure, and style (esp. in the case of Tailwind) — which is basically everything you can serve, aside from assets. In our JSON world we only serve content, a faint hint of structure, and no style at all. That’s way higher level than HTML. It’s important to understand that I’m not asking the back-end to serve JSON.dump(html) to the front-end, only the gaps the page requires to be filled. The rest of the page’s static content should just be hardcoded on the front-end.

I do, however, need to correct one mistake. My old advice went: “content and structure come from the back-end”, but I didn’t realize that folks would interpret it as literally as “serialize the entire HTML into JSON”. That’s not at all what I meant. You only need the bare minimum JSON structure to help the front-end engineer understand which values should go into which parts of the page. For example, if your HTML is something like this:

<div>
<article>
<h1>Title</h1>
<p>Body</p>
</article>
</div>

You don’t need this: { "div": { "article": { "h1": "Title", "p": "Body" } } }. You simply supply the keyword arguments to this otherwise hardcoded ArticlePage’s constructor:

{
"title": "Title",
"body": "Body"
}

If there are multiple articles, you make an array of these. As simple as that. Cater to the needs of the page.

2. Pages will load slower without async!

There exists a concern that with my approach you can’t load parts of pages asynchronously, and that this would result in bad performance. If you are a front-end engineer who has only ever worked with generic API endpoints, I get why you have this impression. You needed 10 endpoints to render a single page, you parallelized the requests, you saw that they can be flaky and slow at random, you prioritized some over others. Right? I’m sorry, but that pain was self-inflicted. If you asked a back-end engineer to give you all of that data in one bundle, the server would most likely produce it in under 30 milliseconds in a single streamlined 200ms roundtrip rather than juggling 10x200ms roundtrips competing with one another.

The whole idea of asynchronously loading pages is a truism of the front-end world that only feels correct in theory. In practice, it’s like hiring 10 trucks to deliver 10 USB drives in parallel, realizing how slow it is to manage 10 trucks, and concluding “maybe I need some more trucks”. Async loading only makes sense when you are actually dealing with slow, heavy, or streaming data sources. If it’s your own back-end, attached to your own database, spitting out kilobytes of content, you are making things hundreds if not thousands of times slower by running parallel requests. On top of that, you’re making it harder for the back-end engineering team to bundle, optimize, and cache data, because it must be sent piecemeal.

Then there’s the reverse concern as exemplified by this whole thread, and this comment. It argues that you should be able to submit individual form fields to the server, rather than entire forms. Is that something you should do? The real answer here is: sure, if you want to. Nothing about my approach prevents you from submitting forms in any way you like. But just like in the situation above, performance isn’t really a good reason to split data (in most cases).

3. This makes no sense in a Single-Page Application!

Some folks have struggled to see how serving pages can work in the context of a single-page application. The answer is: just rename “page” to “screen” in your mind and you’re good to go. When transitioning between them, ship the next screen’s worth of data to the front-end in one bundle. If you need to fetch a specific screen’s sections individually, feel free to provide special endpoints for those. Although, in most real-world cases, reloading the whole screen’s worth of data only to swap out a single section would still be insanely fast and simple. Only watch out that you don’t provide raw data from the database. For example, if the front-end is showing a table of items, give it a table_items endpoint for this specific screen, where all the data is already pre-arranged for display in this specific table. Don’t make the front-end do the extra legwork of accessing multiple endpoints to wire this table together.

4. Why not use GraphQL? Or an aggregation layer?

A couple of readers have wondered: why not use an aggregation layer over a generic API, or switch to GraphQL.

Not gonna lie, an aggregation layer sounds absurd to me. It might be a symptom of front-end-centric thinking. We first build a generic API, thereby creating the problem. Then we build another layer that hides the problem without solving it. This leaves us with twice as much back-end code, double/triple the back-end complexity, and all the same performance issues, while requiring more time and effort. Why?

As far as GraphQL goes, the reason you might want to avoid it is that it comes with an enormous complexity and development style trade-off. GraphQL is a web of infinite possibilities that back-end developers must be able to manage and secure. Supporting such an infinite maze starts to make a lot of sense when you consider how many types of clients Facebook has. Their clients span from computers and phones to TVs, fridges, and washing machines. For them, producing an insanely flexible layer that allows thousands of unique clients to fetch unique sets of data is probably worth it. How many unique types of clients do you have? Let me guess: one website and one mobile app (for most of you). Maybe not even. Should you be supporting an infinitely flexible query language for this?

Remember the problem you’re trying to solve: providing the data your front-end needs to display your software. Just go ahead and provide what’s needed. Problem solved.

5. You took away flexibility from the front-end!

Perhaps I didn’t communicate this clearly enough in the original article, but people are still telling me that having a generic API enables front-end flexibility. They talk about losing the ability of the front-end to build new features and do redesigns without communicating with the back-end.

There is no world where the front-end can build new features or redesigns without involving the back-end. Yes, they can use old endpoints in new, unexpected ways. But then, the back-end will have to catch up to the mess of unpredictable patterns of server bombardment that the front-end introduced, lose all track of what endpoints got used for what purpose, and try to post-hoc optimize inefficiencies, creating a half-baked version of the solution I proposed in the original article.

The cost of this is that the back-end will indefinitely have to support all unexpected usage patterns, and be afraid to change or remove anything, because it’s very hard to trace exactly how and where the front-end relies on specific endpoints. You will need to deal with API versioning, and a forever growing (i.e. never-shrinking) codebase. To make matters worse, there isn’t even that much flexibility gained, because for redesigns and new features, the front-end is still at the mercy of what the back-end provides, and will still probably require new endpoints built for it. Meanwhile, old endpoints will never be removed “just in case”.

On the other hand, you could vastly simplify your redesigns when the back-end serves the exact data needed for each page. You can redesign each page separately, and never wonder if something is being used in unexpected ways.

Bottom line is, there is no real flexibility to be gained from a naive CRUD API based on database records. The cost of making the back-end actually flexible for diverse use cases is much heavier than many teams realize. It only sounds nice in theory.

6. What do I put into the payload?

One of the most important questions I keep getting is exactly what data should be provided to the front-end, and how it should be structured and named. As mentioned before, some people erroneously think that I was suggesting to send essentially the entire HTML auto-translated into JSON. Others thought I was suggesting to create some sort of JSON-based UI-construction protocol. And yet others were just not sure what to do with state and static content that exists entirely on the front-end. Should it come from the back-end too?

The answer to all three is: you’re overthinking it. The front-end developer should hardcode a page as much as possible, and only leave gaps for what makes sense to come from the back-end. The data to fill those gaps should be arranged in a neat little JSON payload. End of story. Hardcode all static content on the front-end, keep all front-end state on the front-end, ship things that the back-end controls from the back-end.

Furthermore, do not try to make your JSON formalized and consistent across different pages. Each page should have its own custom code that takes JSON values and puts them in the right places in the right way for this one page. You may have certain very similar components on multiple pages that you might want to normalize arguments for, but don’t try to normalize overall page structure. It will only make things difficult. Just do what one specific page requires. Your JSON is nothing more than arguments into the constructor for this one page. A different page should have a different JSON structure. Any similarities between overall page structures should be accidental.

7. CRUD makes back-end easier to maintain!

Some comments insist that CRUD is easier to build, document and test than custom pages. This is a misunderstanding of the term. CRUD isn’t supposed to let the front-end CREATE/READ/UPDATE/DELETE your database records, it’s meant for doing that with resources. Figuring out what those resources are is the key to architecting your web app correctly. They may occasionally map directly to database records, but more often they won’t.

A page is a resource with a READ endpoint. The CREATE/UPDATE endpoints are rarely useful for pages, but they are useful for more granular resources, like various items and relationships. Sometimes they map to DB records, but more often you need a resource at a higher level of abstraction. “Form objects” can play that role. They can represent an end-user entity that “changes together”, served as a single form on the front-end. Your controller would then take the data from this object, and transactionally commit it to however many records in the DB are backing it.

All that is to say, the idea that “CRUD is easier to build, document, and test” is really saying “exposing my database records directly to the front-end is easier to build, document, and test”. By doing this, you skip the whole app, and expose low-level storage directly to your front-end. No wonder it’s easier. This shifts your entire app to the front-end, where composing data comes at a huge cost of network failure modes. In a way, I guess you did make development “easier”, because the front-end often gets a pass for omitting tests and docs.

8. What is “General Purpose API”?

Someone asked me for a definition.

When I say “don’t build a general purpose API”, I mean an API for a wide variety of public use cases, available for use by your customers. I’m instead advocating for a BFF (back-end for front-end) API, where you only cater to your front-end team’s requirements, and don’t make this API generally available. If you actually do need a general API, build it separately from your BFF API to avoid clashing requirements, and unnecessary release management + documentation overhead.

9. How is this applicable in the AI era?

This is a question I’m asking myself about all of my programming-related writing. Perhaps you can feed my articles into an AI, so it can follow my advice for you? Who am I kidding, they’ve already been ingested by LLMs and diluted in the ocean of other blog content. I don’t know. Be positive and have fun, I guess, everything is going to be okay.