Information systems have a funny way of becoming heterogeneous. No matter how hard you plan, no matter how comprehensive your pet framework is, there will always be that one indispensable service, that one business expediency, that will pollute an otherwise pristine stack. Heterogeneity in software is inevitable—especially so-called enterprise, and by induction Web-based systems—so we might as well be smart about it.

The Motivation

This idea has been quietly brewing for some time, but it finally came to a head with my election to the board of the Information Architecture Institute. A census of back-end systems revealed no fewer than thirteen different instances of blog/CMS platforms, wikis, hosted services and custom functionality, and none of them talk to each other. It’s exactly the kind of cruft consistent with a decade of slapdash volunteer effort: people had real work to do, but they still wanted to produce results for the organization. The net result, however, is a schizophrenic hodgepodge, incapable of delivering a consistent experience to the organization’s members and the public at large. While I have marked most of these systems for retirement, my board tenure will eventually come to a close. This effort is to ensure that after I’m gone, the wise things to do are still the easiest.

Protocols over Platforms

In order to achieve intelligent heterogeneity in an information system, we need to set aside the idea that there’s a product, platform, or framework out there that will solve all our problems. Instead, we look at the languages these systems speak: the network protocols and data exchange formats. We evaluate those languages in terms of their expressive range—whether or not they are capable of saying what we want them to say. Protocols are currency, and if they’re open standards, we can take our pick of specific vendors and implementations. And that change takes us miles away from politico-technological dependency, and towards doing things on our own terms.

Lesson number one for intelligent heterogeneity: Stop focusing on the things, and start looking at the things between the things. What follows are two specific ways I’m tackling the problem in all my work, using the IAI as a case study.

Identity Crisis

Each back-end of the IAI has its own incompatible user database, each with potentially different usernames, passwords, and access control policies. So in order to get one job done, you might have to log into three different systems, and fumble about for your other password because one of them won’t let you use the same password you use for everything else. Of course, your username is your email address in one place, your first name in another, and your first.last name elsewhere. Finally, when you manage to get all that nonsense sorted out, you can’t actually do what you set out to do because you aren’t authorized on that particular system.

This situation is a user experience nightmare. If I’m an IAI member, I had damn well better have access to the full range of functionality accorded to me by membership. If I’m a board member, I shouldn’t have to go monkeying around in the guts of the website just to get my volunteer work done. Desynchronized user databases create massive confusion for users, and overhead on the parts of staff and volunteers that could be put to better use. And that is to say nothing of the impression it leaves on lapsed members who still get dispatches from the mailing lists, or the monthly newsletter for that matter.

My solution here is to use the industrial-grade single sign-on protocol known as Kerberos to handle logging in, coupled with the equally industrial-grade directory protocol, LDAP, to encode user profiles and access rights. On top of those, I put an organization-wide authentication proxy which handles user sessions on the Web. With this stack in place, every Web property belonging to the IAI has access to the same user information, eliminating the identity crisis problem in one fell swoop. On the developer side, you can skip the housekeeping of access control and focus directly on the task at hand: most of the popular frameworks already support the integration, and it’s a quick one-time hack for everything else.

Relying on protocols rather than implementations means the latter become commodities. Don’t like MIT? Install Heimdal. Angry at OpenLDAP? Dump it for 389. Or if you’re like any organization with a physical office, hook your Web properties straight into your ubiquitous corporate Active Directory. Even the authentication proxy is disposable: I’m currently road-testing Stanford WebAuth, but I could ditch it for CoSign at a moment’s notice without registering a so much as a blip in the service.

Paint Job

The other perennial problem with a ménagerie of recalcitrant back-end systems is getting them all to look and act the same. There are at least as many template languages as there are middleware products, and all of them are, once again, in the very least slightly incompatible and often pretty ridiculous. Making organization-wide changes to information architecture, metadata and visual design thus explodes with a foetid plop into the Augean stable of Web development.

To stop everything from looking like shit, I’m using a tried-and-true method that hasn’t budged since I picked it up 13 years ago: XSLT. Why?

• It’s a standard: Which means you have your pick of dedicated processor implementations, some of which are very, very fast.
• It forces you to write well-formed markup: No more weird layout bugs caused by aberrant typographical errors.
• It only has access to information you give it: No security whoopsies because the template processor got tricked into dumping out the contents of the database.
• You can debug it offline: Because it’s been baked into your browser for over a decade.
• You can repurpose resources: Like turning Atom feeds into page content, or turning an HTML table into an SVG pie chart.
• You can nerf it for the n00bs: Because it’s modular, one person can write the hard transformations and designate specific, clutter-free spots for the HTML jockeys to drop in hunks of layout code. I had a visual design team working exactly this way for three years and they didn’t have any problems picking it up.

Perhaps most valuable though, is XSLT’s implementation-agnostic transclusion mechanism, which makes page compositing as easy as embedding an image. That means that your navigation could belong to one back-end while an inset was driven by another, all piled together onto a humble, static HTML page. With a little help from a reverse proxy, you can even have content from different servers show up seamlessly in the same source code.

How does it all come together? The template processor gets embedded into the server as a filter and operates on any markup that it has been told to transform. In each of the back-ends, you simply strip out all non-essential markup, leaving only that which constitutes the main content.

Making page components accessible directly as HTTP resources is consistent with the principles of REST, which, as an architectural style, is even more abstract than a protocol or data format. That means that once another standard comes along that does everything XSLT does but without all the awkwardness, the task of switching over will be a manageable one. But in the interim, I don’t want to hear any complaining! XSLT may seem like a blast from the past, but it does its one job extremely well.

Shearing Layers, Revisited

Versions of specific software products move extremely fast. It’s scarcely possible to go a day without being pestered for an update. New features get added, exploits get discovered, and bugs get fixed, putting us on a constant treadmill of software slavery. Developers are likewise continually releasing entirely new classes of middleware—and accompanying techniques—into the ecosystem. Even stolid frameworks, and the programming languages they’re based on, phase in and out of vogue every couple of years.

Protocol standards, by comparison, are glacial. If they have any currency, which is what makes them worth using in the first place, they outlive even the staunchest implementations. And when they finally are replaced, they often supply at least some inspiration to the new dialect.

Embracing a principle of intelligent heterogeneity means gearing down the churn, and making our systems less susceptible to the caprice of products, technologies and fashion, by taking the focus off individual implementations. I’m glad I’m getting the opportunity to demonstrate this principle with the IAI, so that I—and the design community at large—can create better systems going forward.