Photo by Sangga Rima Roman Selia on Unsplash

There is something very strange about automobiles. They are much faster and more dangerous than our brains are used to thinking about. This was more obvious to people when automoviles were new. In the early days, there was great consternation over the question of what would happen if two cars tried to occupy the same intersection at the same time. The speed and mass of the automobile were unprecedented, and the consequences of such a collision unimaginable. You couldn’t just say “oh, pardon me” and step around somebody. One solution mooted in the rural US was for a driver approaching an intersection to stop, exit his vehicle, and fire his shotgun in the air as a warning. Then, he could proceed. It was an extreme solution to an extreme problem.

In my last newsletter I asserted with no particular evidence that driving is a social process. More than that, I asserted that the social nature of driving was the key to both the failures and the potential future success of autonomous vehicles. Those are some big assertions, and it’s fair to ask me to back them up.

This question of the social nature of driving is a big one, and one that I won’t cover comprehensively today. It’s also controversial. When I give talks, I’ll often ask the audience to call out the most important skills for driving. People will suggest things like: knowing the rules of the road, understanding how to safely maneuver the car. They never say “the ability to interact socially with other humans”. Of course they don’t. It sounds kind of silly. We’re talking about driving, not small talk. So why do I say that driving is fundamentally social?

Perceptive Automata’s wonderful director of marketing, Anthony Cote (hire him!) once described rules as "social conventions written down". If everybody would agree to alternate taking turns at lights you’d never need stoplights. Speed limits are a way to establish the speed at which a considerate person should be driving anyways. Rules—social conventions—are agreements between you and another person. But the rules of the road comprise only those conventions that have been written down. They are conventions: the areas of social agreement that are unintuitive or unusual enough that you have to codify them. What percentage of interactions on the road, in a car, are centrally dependent on those rules? How fully do those rules describe how you should behave when driving?

You must yield to a road user with the right of way who wishes to cross your path. That is a fairly universal rule of the road. That seems like a pretty clear and robust rule. But if you decompose it at the level necessary to program a computer to follow it, it is surprisingly ill-defined. "With the right of way" means that with the balance of road conditions that obtains, vehicle A should get to go first. But determining who has the right of way, and who believes themselves to have the right of way gets tricky fast. People perceive themselves reaching intersections at different times and have different tolerances for gaps between vehicles. An unprotected left (that is, a left turn that is not controlled by a traffic signal) across multiple lanes of traffic turns out to be fantastically difficult in an autonomous car, because there is never a time where there is not an oncoming car that technically has the right of way. That situation is one of many where formally correct driving behavior according to the rules of the road leaves you no path from A to B.

The rules around unprotected lefts are social rules. If you are approaching a car waiting to turn left, how much will they expect you to defer? If you are making a left, how strongly can you assert your intention to turn? The actual maneuver happens in the context of a complex, real-time negotiation between drivers, most of which happens without those drivers even being aware of it.

How much of driving is like this? How much of driving behavior is governed by social interactions that are not covered by formal rules? A lot of it. Early on in the history of Perceptive Automata I dragged a camera out to intersections around Cambridge, Massachusetts and recorded the interactions I saw. In one thirty-second clip at a busy but not unusual suburban intersection I counted upwards of 40 interactions where one road user had to look at another and make a decision about how to interact with what they wanted.

This isn’t something that we notice when we’re driving. That’s because we aren’t doing it consciously. Social negotiation like this is something our brains are effortlessly great at. We evolved over millions of years to have a sophisticated social understanding of other humans. We might be better at this kind of thinking—social cognition—than any other animal. It comes so naturally to us that we don’t even notice we’re doing it.

You can notice it, though. The next time you drive, think about the vehicles and pedestrians you’re interacting with. Think about what you know about what they want. Think about how your behavior is influenced by that understanding. I used to talk through the subject of this essay with venture capitalists and potential customers, and more than once heard back later that they found themselves no longer able to drive the same way, because they were now uncomfortably aware of constantly making social judgments, and had no idea how they were doing it.

The rules of the road, the social agreements that we had to codify, are the things that are hard for humans. We had to make them explicit, signaled by colored lights and high visibility signs and enforced with real consequences, because otherwise we wouldn’t pay attention to them. We had to use technology to make those social conventions easier for us to follow.

There’s a concept in machine learning and robotics known as Moravec’s paradox. It was defined by the pioneering roboticist Hans Moravec. Moravec’s paradox states that the things that are easiest for humans are the hardest for machines. Autonomous vehicles have a pretty easy time following the rules of the road. An easier time than humans, actually, as they’re never frustrated or inattentive. The things that are easy for humans, on the other hand, which have not needed to be codified into rules: treating an oncoming driver or a pedestrian like an agent with desires and beliefs that wants specific things out of the interaction with you are largely impossible for today’s autonomous vehicles. We had it figured out at Perceptive Automata—and when we finish selling our IP, the buyer will have a big leg up on figuring it out for themselves—but why we couldn’t sell it is a story for another day.

I’ll return to this point, that driving is a fundamentally social process, a lot. I think it’s the biggest blind spot for autonomous vehicle companies. It has enormous explanatory power for the ways their vehicles do and do not behave the way you would expect them to. But step one is to notice the way that humans drive. There is a great hidden undercurrent of abilities that supports our ability to learn to drive around other people. Most prominent in that hidden world is the ability to understand other humans.

Thanks for reading Apperceptive! Subscribe for free to receive new posts and support my work.