macOS Terminal - still missing the mark Apple!

(Note: I’m resurrecting this dormant blog after a long period of inactivity. But this thing bothered me so much that I felt I needed to write a blog about it.)

TLDR: The macOS Terminal.app is still buggy and not recommended. Instead consider switching to a 3rd party emulator. The rest of this post describes the faults in far more detail.

One of the more lauded features in macOS 26 is the updated Terminal.app.

As some of you may know, I’m the author of a very popular library for interacting with terminals and terminal emulators, at least within the Go ecosystem.

I was excited to hear about the new Terminal.app. Finally, we would get support for 24-bit color, years after everyone else has had it. (Even Microsoft beat Apple to the punch here, leading by something like half a decade.)

So yes, the good news is that macOS Terminal.app in does support 24-bit color.

But in nearly every other interesting case it is far inferior.

The rest of this is probably only interesting to terminal nerds, but for those of you who care about these things, the new Terminal.app is a major disappointment.

Identification Woes

The first and most egregious failing is that there is no good way to remotely identify the macOS terminal, or even what features it might support.

Normally we would like to be able to query and get useful feedback via one of the following mechanisms:

Primary Device Attributes (obtained via CSI c): Terminal.app reports that it is a vanilla VT100 with no extra features, it does not claim to be a VT220 or similar. 1.

2. Extended Device Attributes (obained via CSI > q): Terminal.app reports nothing here. Most modern terminal emulators store their application name and version here. While I dislike hardcoding capabilities for a specific terminal emulator (so-called "quirks"), this isn’t even an option because we can’t know that this is Terminal.app. 1.

3. DECRQM - DEC mode queries. This is supported officially starting with VT320 and up, but pretty much every other software emulator supports it. This can be used to query whether a given DEC mode (including things like mouse support and features, automatic margin wrap, etc.) are present and modifiable. It has been used for new things like synchronized output (to fix tearing during a resize) and to identify support for grapheme clusters (very important in a Unicode world). Terminal.app does not support these queries, but it even gets it wrong in a worse way. This is a CSI sequence, so the reasonable behavior would be to parse to the end of the sequence, and discard it if you don’t support it. Terminal.app instead aborts the parsing early, emitting the final character of the sequence on the screen. So during early startup we see these mysterious "p" characters show up as Tcell tries to query for support.

Ok, this is just about identification. What about TERM? Well Terminal.app identifies as xterm-256color - although it clearly is not compatible with genuine xterm. What about $TERM_PROGRAM? Well, yes, we could use that. But note that this environment variable is not passed through ssh by default, which makes it useful only for locally running applications.

Unicode Woes

Modern applications and users expect to be able to use things like skin-tone modifiers for emoji, national flags, and similar "characters" (well graphemes technically, which is just another way of saying a user-perceived character). In Unicode, these graphemes are composed by combining multiple Unicode "characters" together.

This approach is also used with certain languages, such as Arabic, or even in Western languages where a character might be modified to include the diaeresis using two code points joined by a special character called the "zero-width-joiner". (For example, in Bengali, র্য (No, I do not read or speak Bengali.)

How does Terminal.app fare here? Well its mixed results. For some, such as 👨‍🚒 and the Bengali example, it works reasonably well. But for others, like the flag emoji 🇨🇭 it miscalculates the width. (In this case it calculates the width of the flag as just one cell, instead of two, even though it renders it using two cells!).

This behavior could be detected by an application emitting the composed sequence, then querying the resulting position using CSI 6 n.

It would be easier, and better if Apple would implement the DECRQM and advertise mode 2027. It could be hard coded to 3 (forced on) in this case, unless Apple were prepared to add the complexity to support both legacy and modern grapheme support.

Hashimoto also has something to say about Terminal.app and grapheme clustering:

🤡 Living in its own cursed little world

(Apparently it miscalculated the width a joined sequence as 6, probably because it considered the ZWJ as occupying two cells.)

Broken State Machine

So modern terminals, which includes everything starting from VT100 or claiming to support ANSI X3.64 or ECMA 48, are expected to fully parse (but may not implement) certain classes of escape sequences. For example, consider the sequence emitted by this command:

printf "\x1b]something\x1b\\"

This is a OSC (operating system command). The "\x1b" values are escape codes, and the command is "something". Now no terminal (probably) has a command "something". (Usually OSC sequences begin with a number followed by a semicolon, followed by arguments, but this isn’t required.)

Every reasonable implementation should parse the entire string, using a simple state machine that recognizes "\x1b]" is the start of OSC sequence, which is terminated by either "\x1b\“ (formally the ST sequence in ECMA-48), or "\x07" (due to historical accident.) Note that ST can also be encoded in a single eight bit character as "\x9c", but 8-bit encodings are uncommon due to potential confusion with other national encodings (but not UTF-8, or ISO-8859, or any EUC or ISO-2022 encoding).

So what does Terminal.app do when it sees this sequence? It prints "omething" on screen. (It consumed the leading "s".) One possible interpretation is that they implement all OSC commands by printing the entirety of the command minus the first character, to the screen. Is this a legal interpretation? Technically, yes. Is it a reasonable interpretation?

I believe this is an egregious violation of Postel’s Law. (By the way, if you work in software, you should be well familiar with Postel’s Law, which was first invoked by the late Jon Postel, whose name is listed as an author of most of the foundational RFCs that define the Internet.)

More importantly, this interpretation means that you cannot safely use OSC sequences at all on Terminal.app, although you can reasonably do this for real xterm. But Terminal.app is not a real xterm, so that should be fine – except that Terminal.app does nothing to make itself discoverable. It doesn’t define its own value of $TERM, it doesn’t answer any of the standard queries to test for functionality, nor does it answer the extended attributes to help a user program identify that it should avoid any of the normal things it might expect to be able to do with a real implementation of xterm.

Btw, this is not limited to just the OSC sequences. A similar, but slightly different behavior occurs with DCS sequences (device control sequences). It just emits the entire string (including the first character).

What about APC, SOS and PM (which are generally unused by modern terminals)? They behave as DCS. The content goes straight to screen.

Then there is that strange case of CSI sequences that it does not understand. Remember DECRQM above? That is a valid CSI sequence:

CSI ?Pm$ p

(The final character here is "p" and the Pm is a numeric mode number such as 7 for automatic margin wrapping.) As we have seen, the parser in Terminal.app does not swallow these, but acts as if "$" were the legal character. (Technically "$" is considered an "intermediate byte" in the ECMA specification.)

Modern Key Events

Modern terminal applications are looking for ways to make better use of the keyboard, for things like control-keys for short cuts. Historically, it was impossible to discriminate between certain keys. For example, Ctrl-I is the same as TAB, and Control-M is the same as Enter. (This goes all the way back to decisions made in the 1970s.)

In answer to this, there are a few standards available. The most widely popular one is a protocol designed by the author of kitty, and lets terminal applications fully discriminate different key sequences and can even report key repeat and release events. (Xterm has its own way to do this, which is a subset of the functionality, and Windows Terminal invented its own scheme called win32-input-mode, which offers the full richness but has some drawbacks in efficiency and safety – you cannot recover from this mode if you turn it on by mistake in a shell.)

Of course, Terminal.app has none of these, and is stuck in the 1970s for key reporting.

So What To Do?

With all these problems with Terminal.app, one might wonder what recourse there is. Fortunately, I have good news.

There are a plethora of excellent options available which are all far superior to Terminal.app, and which are free. (There are non-free options as well.)

Personally, I use the excellent Ghostty, but in the past I had great results with the old stand by iTerm2 as well as kitty. The up and coming Rio is also pretty excellent. The *Alacritty *program is popular but I found it inferior to these other options (no grapheme clustering, and no advanced keyboard support). I found WezTerm to be brittle (it would occasionally hang), so I don’t recommend it personally, but it has many adherents.

What About Windows Users?

For Windows users, Rio might be the best option, although for advanced keyboard handling the modern Windows 11 Terminal is quite excellent. It still lacks some things I’d like to have, like support for CSI > q identification, and better support for grapheme clusters, but for most applications it behaves quite well. One I specifically do not recommend is ConEmu, as it has many different ways in which it is broken. It also appears to be abandonware, in spite of its once great popularity.

Popular posts from this blog

The Hand May Be Forced

Well, as you may have read , Oracle has decided that at some point very soon, we’re going to lose normal regular access to the source code for OS/Net. (I.e. the Solaris kernel and supporting programs.) While I would have vastly preferred for Illumos to have a cooperative and collaborative relationship with Oracle , it appears that Oracle doesn’t value this. In fact, the exact words were from the management at Oracle were as follows: Solaris is not something we outsource to others, it is not the assembly of someone else’s technology, and it is not a sustaining-only product. While I understand the need to own the technology, there are few things that could be stated that show a stronger NIH attitude than this. Its unlikely that there will ever be a way for Oracle and the greater community to have a collaborative relationship. This is a dark day for OpenSolaris – its effectively dead now. (Its parent, Solaris, lives on however.) How unfortunate. For Oracle that is. Because...

GNU grep - A Cautionary Tale About GPLv3

My company, DEY Storage Systems , is in the process of creating a new product around the illumos operating system. As you might imagine, this product includes a variety of open and proprietary source code. The product itself is not delivered as a separate executable, but as a complete product. We don’t permit our customers to crack it open, both from the sense of protecting our IP, but also to protect our support and release engineering organizations – our software releases consist only of a single file and we don’t supply tools or source for other parties to modify that file. One of the pieces that we wanted to integrate into the tree is an excellent little piece of software called Zookeeper , produced by the Apache organization. Like illumos, Zookeeper has a nice non-viral copyleft license, which makes it nice for integration into our product. However, I discovered that as part of our integration, one of my engineers had decided to integrate GNU grep. ...

MacOS X 10.10.3 Update is *TOXIC*

As a PSA (public service announcement), I’m reporting here that updating your Yosemite system to 10.10.3 is incredibly toxic if you use WiFi. I’ve seen other reports of this, and I’ve experienced it myself. What happened is that the update for 10.10.3 seems to have done something tragically bad to the WiFi drivers, such that it completely hammers the network to the point of making it unusable for everyone else on the network. I have late 2013 iMac 27", and after I updated, I found that other systems started badly badly misbehaving. I blamed my ISP, and the router, because I was seeing ping times of tens of seconds ! (No, not milliseconds, seconds!!! In one case I saw responses over 64 seconds.) This was on other systems that were not upgraded. Needless to say, that basically left the network unusable. (The behavior was cyclical – I’d get a few tens of seconds where pings to 8.8.8.8 would be in the 20 msec range, and then it would start to jump up v...