Unix-based systems have been around for more than 50 years now. Although the best design ideas still prevail to this day, the evolution of the computing industry has forced operating system designers to rethink the way they work, multiple times over time.

From a device driver point of view, the most important change was the change from fixed, compile-time hardware configuration, enumerated upon boot and never changing afterwards, to a more dynamic model, where devices can come and go: storage devices first, with the first hotplug-capable SCSI controllers in the first half of the 1990s, and complete devices shortly later, first with the introduction of the PCMCIA bus on laptops, then with USB and Firewire, which were not limited to laptops.

While PCMCIA support in open source operating systems had lingered for a few years before being integrated (both in Linux with "pcmcia-cs" and in FreeBSD with the "laptop package"), by the time USB support was being worked on, the required changes to accept/allow that devices may show up or disappear at any time had been completed and tested, and the kernel had no excuse not to cope with removable devices.

Today’s story is the story of a device driver which caused some kernel assumptions to no longer stand, and the work done to remediate this situation, letting the kernel cope with the new world order.

---

This story starts on the 12th of march 2009. Theo de Raadt is travelling to Japan to visit fellow OpenBSD developer Ryan McBride, best known for implementing the CARP network high-availability protocol in OpenBSD, who has been living in Japan for a few years already.

Every time he goes to Japan, Theo never misses the opportunity to visit Akihabara in Tokyo, in order to find new computing devices to play with.

Among the devices he picks this time, is a small display, the size of a smartphone, with an usb cable, shown here running an X server under OpenBSD for the first time, a few months later. (picture courtesy of Marcus Glocker)

The company making these USB displays is called DisplayLink.

DisplayLink provides binary-only drivers for Microsoft Windows and Apple Mac OSX, and nothing more. This caused some frustration in the free software world, and Florian Echtler, with the help of Chris Hodges, worked on reverse engineering the device, with the intent to get it working under Linux. All of their work is nowadays described at https://floe.butterbrot.org/matrix/hacking/dlnk/.

Of course, at this point, there is no support for these devices in OpenBSD either, and they attach as generic USB devices when connected:

<deraadt> ugen0 at uhub0 port 3 "DisplayLink LCD-8000U" rev 2.00/0.02 addr 2
<deraadt> Picked up that too

Moments later, Matthieu Herrb points to Florian Echtler’s information:

<matthieu> <http://floe.butterbrot.org/displaylink/doku.php>

(which nowadays redirects to https://github.com/floe/tubecable/tree/master/doc)

The DisplayLink device gets passed from Theo de Raadt to developer Claudio Jeker, who lives in Switzerland close to another developer who has been recently working on USB devices, and will be the hero of this story: Marcus Glocker.

Once told he will be given the device, Marcus does what everyone in his place would have done: he asks the manufacturer politely for documentation on april 6th.

There was no contact information at DisplayLink, only a "contact us" webform in which you could enter your name and email address, and your message.

Name: Marcus Glocker
Message:
Dear Ladies and Gentleman,

We are thinking about developing an OpenSource driver for your
DL-120/DL-160 chipsets. This would allow several OpenSource projects
(e.g. NetBSD, FreeBSD, OpenBSD, Linux, X.org) to add support for
DisplayLink based USB LCD devices, and therefore extend your potential
user base.

Therefore we would like to talk with you about getting basic
documentation for your chipsets.

It would be nice if the right person for this topic could contact me for
further communication on the e-mail address stated in this form.

Regards,
Marcus Glocker, OpenBSD Developer

The answer was as unhelpful as possible:

Date: Mon, 6 Apr 2009 12:49:42 +0100
From: Jason Slaughter (DisplayLink)
To: Marcus Glocker
Cc: Bernie Thompson (DisplayLink)
Subject: Opensource Driver for OpenBSD

Hello Marcus, thank you for your email.

DisplayLink USB graphics are a software graphics card: graphics
functions are performed on the CPU and then compressed and sent across
the USB cable to be decompressed in a DisplayLink DL-120/DL-160 chip on
the other side. This means that - unlike most PC hardware - getting
basic documentation about the chip would not provide enough information
to implement a driver for BSD.

However, DisplayLink is current in the process of putting together a
library that will provide the functions necessary to drive a DisplayLink
USB graphics chips, and it is our intention to release this library
under an open source license in the very near future. This library would
provide you with all of the information necessary to develop an x.org
compatible driver for DisplayLink USB graphics devices on *BSD.

The best point of contact for you might be Bernie Thompson (cc'd), our
VP of Software Platforms based in Seattle, Washington. We could start
with a conversation on email, but Bernie and I would also be happy to
have a quick phone call with you to help you understand how DisplayLink
graphics work and what our plans are for supporting open source
platforms. I would also be curious to get your thoughts on how
DisplayLink device support might fit into BSD.

Thank you,

Jason

At the same time, Marcus was sharing the results of this discussion with the other OpenBSD developers on our private chatroom.

<mglock> wow.
<kettenis> wow?
<mglock> DisplayLink TM seems to be very communactive.
<mglock> asked the for specs for their DL-120/DL-160 chips, and got a detailed answer withing 4 hours.
<mglock> they want to call me :-)
<oga> they're actually responding?
<oga> wow...
<oga> I'd heard they were quite reticent. Someone on the xorg lists is rev engineering them for that reason. Not so much progress so far
<mglock> interessting.
<oga> Well, i asked him if he was reverse engineering, he said yes. I'm sure I asked if he'd asked for docs, but the reply from him doesn't include that bit.
<oga> If you can get specifications freely, I'd be interested in a copy
<oga> those things are pretty cool, in an oddball kinda way
<mglock> sure.  keep you informed.  lets see how far we come.
[...]
<mglocker> just wrote back to the DisplayLink guys.  lets see what happens.

The day after:

<mglocker> re
<mglocker> got the DisplayLink device from claudio.
<mglocker> and an answer from DisplayLink themselfs.
<mglocker> we will get no docs.
<kettenis> bastards!
<mglocker> but the seem to release the opensource library soon.  based on libusb.
<mglocker> library will be LGPL licensed ...

However, Marcus does not remain inactive waiting for a hypothetical code release from DisplayLink. After weeks of trials, he reaches a point where he has a terminal (text-mode emulation) running on it, albeit slowly, and talks about it on may 1st, leading to a productive brainstorming session with a few other OpenBSD developers:

<mglocker> hmm, looks bad for the USB display driver for wscons.  all that many little USB bulk transfers which are necessary to draw the fonts make the display rendering slow.
<mglocker> i think there is no way to improve it further.
<oga> mglocker: make a shadow area and blit the whole lot in one go
<oga> so make a line of text in one go the blit all of it
<oga> and use blits to move lines of text around for scrolling
<mglocker> nah, you need to be able to draw one character.
<oga> hmmm
<oga> did they release that library by the way?
<deraadt> there must be memory on the thing, to pre-load the fonts there
<mglocker> even if you preload the fonts, it doesn't help.  to bit blit one character you also need to setup a bulk xfer.
<oga> oh, offscreen memory if it has, then it's just blitting chars.
<oga> oh jesus.
<mglocker> it's the same amount of bulk xfers in the end.
<mglocker> yes, i tried exactly that.
<oga> how slow is slow?
<mglocker> it is exactly as slow as before.
<oga> I mean how slow is the output all told?
<mglocker> i think it's not the content of the bulk xfer making it slow, but the little peaces of bulk xfers.
<mglocker> ah.
<mglocker> on my amd64 it's like a 19200 modem connection or such :-)
<deraadt> how do they make it fast, then?
<oga> what info do you have on it?
<drahn> can multiple 'blits' be combined?
<oga> all I heard about was some libarary that they will release, and the reveng stuff
<sthen> if you have a shadow buffer, can't you update multiple chars in one go?
<mglocker> you could.  but then i would need to "collect" lines?
<sthen> yep.
<mglocker> shudder.
<mglocker> sounds ugly.
<oga> welcome to graphics, mate.
<mglocker> heh
<mglocker> i think i quit that area ;-)
<drahn> a timeout to gather up several or polling waiting for the previous one to complete?
<oga> what information do you have on the hardware itself?
<mglocker> http://floe.butterbrot.org/displaylink/doku.php
<mglocker> that's what we have.
<oga> oh, that's the reveng stuff?
<oga> yeah, seen it.
<mglocker> there is compression, which doesn't seem to work yet.  but i'm not sure if the compression stuff would help us for wscons.
<mglocker> since again, you need to setup a bulk xfer for each character.
<oga> good for bulk data moves. less good for char-by-char
<mglocker> and the thing which make it slow seems to be the setup overhead, not the content itself.
<mglocker> yes.
<mglocker> i don't know how the driver on windows works, but i don't guess they do something like char-by-char, as we do for wscons.
<oga> is this a problem for scrolling, or for typing?
<oga> scrolling you can coalesce as long as the timeout is short.
<mglocker> the scrolling seems fine.
<oga> and it'd be a LOT better than char-by-char.
<oga> so it's just typing?
<mglocker> the problem is for rendering the chars.  e.g. "ls -l".
<oga> if you put a timeout say (out of my arse) 10ms. all chars that come in then go out at the same time
<oga> no one types that fast, so you see char by char, but ls gets done in one go
<mglocker> that would help for sure.
<oga> try that. one transfer per char is not going to help you
<mglocker> the question is how to implement this in a nice manner ... sigh.
<oga> several ways present themselves
<oga> font in offscreen memory, when you get a char make blit command for that char to where you want it. multi commands in one bulkt transfer (which is apparently ok)
<mglocker> and then send it after the timeout is done (if there is something)?
<oga> when the timeout fires, launch off all you've got
<oga> exactly
<oga> just queue up commands and put htem all in one bulk
<oga> the timeout will need tuning to be a good average between latency and bandwidth though
<drahn> scrolls are a bit extra info in the queue
<mglocker> putting them in one bulk would improve it for sure.
<oga> give it a try, and mail me the diff so I can see what you're doing
<oga> i'm curious, even though I don't understand the usb stack at all
<mglocker> the usb part is not difficult for this device, really.
<oga> oh good. means I might get it.
<mglocker> "ls -l /etc" takes about 5 seconds now to render.  that's not nice ...
<mglocker> ok, thanks for the hints.  maybe i can get something together which helps ...
<oga> for wsfb you'll want to provide a mmap area and sync that as well. that will suck though
<oga> horribly
<oga> it'll need a proper driver to be usabel for X
<mglocker> yes.  there are different opinions for the X driver.
<mglocker> some say, use wsfb, some say you need to write a own accel driver for X :-)
<oga> wsfb iirc mmaps the framebuffer, so you'd need to shadow it and update every X times a second
<oga> you'd need the compression at least for it to not suck
<mglocker> exactly.
<mglocker> jup.  and the compression part is missing from the rev. eng. as it looks.
<oga> it'd help to know more about the hardware it really would
<oga> without compression wsfb will be FAR too slow
<mglocker> i believe so, too, now.
<oga> no mmap over usb :(
<mglocker> no way.
<miod> mglocker: can't you copy the font image to offscreen video memory and do screen-to-screen blt to display chars?
<miod> that's the canonical trick.
<oga> that was my suggestion, with batching to keep from too many bulks.
<miod> ah
<mglocker> miod: i thought about that, too, but the bulk xfers keep the same.
<miod> (not caught up with logs yet)
<mglocker> one per bitblit.
<mglocker> i've even tested it.
<mglocker> it doesn't help at all.  it's exactly as slow as before.
<miod> can you queue the xfers and schedule a batch of them 25 times per second?
<deraadt> you can only request one blt per bulk?
<oga> the rev eng says you can do more, hence the batching
<deraadt> sounds like a retarded protocol if you can only do one blt per bulk
<mglocker> miod: that's what oga suggested.
<mglocker> yes you can queue more than one bitblit.
<mglocker> but that would be i would need to "collect" chars.  what oga said.
<mglocker> and fire them on a timeout.

After going back to the drawing board, the driver performance improves, and it becomes worth commiting to the OpenBSD source code repository, so that people can try it and developers can work on improving it.

The first stab of a driver hits the tree on may 9th.

Three days later, possibly related to that commit, but we’ll never know...

Date: Tue, 12 May 2009 20:08:44 +0100
From: Jason Slaughter (DisplayLink)
To: Marcus Glocker
Cc: Bernie Thompson (DisplayLink)

Hello Marcus,

Just an update on this: our source code library will be going live this
Friday and you'll be able to find it by going to http://displaylink.org
around Friday afternoon.

We would be happy to send you a few DL-160 based USB to DVI graphics
adapters if you think you would get good use out of them. If so, let me
know how many you'd like and send me your mailing address. With any luck
they'll get to you before the code goes live on displaylink.org.

Thank you,

Jason

which leads to this conversation at OpenBSD headquarters:

<mglocker> oho.  DisplayLink is answering again.
<deraadt> heh
<mglocker> the will release the opensource driver this friday.
<mglocker> and they want to send me DL-160 based USB to DVI graphic adapters.
<oga> the library you mean?
<mglocker> jup.
<oga> i'm sure i found a dl-160 going for 50 quid on amazon the other day
<oga> out of stock though
<mglocker> so, who wants to have such a toy?
<oga> I'd love one.
<mglocker> ok, one for you, one for me, anyone else?
<mglocker> Florian Echtler will be glad to hear this.
<mglocker> because the compression stuff is hard to crack.
<oga> I wonder how complete the library will be
<oga> what license, by the way?
<mglocker> pretty complete i hope ;-)
<oga> hope is definitely the word.
<mglocker> they wrote me about the license.  some GPL shiz if i remember right ...
<oga> well it's a libusb library anyway, so it'll want a rewrite
<oga> GPL? if they're targetting X then they should know X uses MIT.
<mglocker> *shrug*
<mglocker> after the last conversation i don't know how much they really know about X ...
<mglocker> probably they put the library out and hope somebody will write the X driver for them ...
<oga> time permitting i'll probably write one
<oga> it'll be either drm or wscons based depending on what I learn about the hardware
<oga> s/wscons/wsfb-style
<mglocker> oga: if you going to start on the X driver, we should sync with Florian maybe.  it would be probably good if not three people start to write on the X driver when the library has been released I guess ...
<oga> yeah.
<oga> if we're doing stuff with wscons for the driver though, then it won't work on linux.
<oga> I don't know how fbdev works.
<mglocker> and he started some sourceforge page today, to find out about the compression stuff more :-)  i'll forward him the mail from displaylink now.

Three more days pass, and indeed, on may 15th...

<mglock> re
<mglock> http://www.freedesktop.org/wiki/Software/libdlo
<mglock> the library is online.
<mglock> read first feedback from the displaylink rev. eng. mailling-list.
<mglock> it seems to be useless.
<mglock> since the whole compression stuff is missing.
<mglock> need to look at it myself later.

(the libdlo url is now https://libdlo.freedesktop.org/wiki/

)

5 hours later:

<oga> mglock: displaylink rev. eng have a mailing list?
<oga> where?
<mglock> erm, yes.
<mglock> https://lists.sourceforge.net/lists/listinfo/displaylink-devel
<oga> mglock: thanks
<oga> hmmm, glancing at the library it is not that useful
<oga> disappointing

2 hours later:

<oga> mein gott! libdlo is ugly
<oga> ERR_GOTO(function(args)) ...

and 2 more hours later:

<mglocker> i've setup another mail to displaylink asking them why they didn't include the compression stuff in their library :-)

On the same day, on the displaylink-devel list:

Date: 2009-05-15 13:59:30
From: Florian Echtler
To: displaylink-devel mailing list
Subject: Re: [Displaylink-devel] News from Displaylink

> Sigh, pretty disappointing.  LGPL licensed first and then it doesn't
> contain the compression stuff.  What do the think how people can
> write an usable X.Org driver with this?  I gonna ask them.
All right, this is getting a bit ridiculous. Looking in dlo_usb.c, they
labeled the "set encryption" request as "select channel", and the
"null-key" for disabling encryption is called STD_CHANNEL. Okay, maybe
just terminology.

However, in dlo_data.h, there's suddenly always an DLO_MODE_ENABLE_*
sequence (never seen that before), and the DLO_MODE_DATA stuff looks
totally random. Hey, look, in mode_select(), it's always sending the
ENABLE sequence to the "select channel" command, then the binary blob
and then DLO_MODE_POSTAMBLE. Hmm, this POSTAMBLE looks just like the
STD_CHANNEL default key! Oh, what a surprise, the register sets for the
modes are _still_ encrypted - never mind that we can decrypt this since
Christmas last year.

Displaylink, I really don't get this. I'll have a closer look at the way
the stride registers are implemented; this is one of the smaller parts
which I still would like to figure out, but in general, this library is
absolutely useless to the wider opensource community. It's obviously
designed for some embedded shops which want to use it for non-realtime
stuff like LCD signs etc., but it's just ridiculous to still try and
obfuscate parts of an opensource library.

Florian
--
0666 - Filemode of the Beast

(emphasis mine)

Two more weeks pass, during which Florian Echtler continues his reverse engineering efforts and figures out the compression scheme used by the DisplayLink chip.

<mglocker> for those who are interessted in the displaylink stuff ... the compression has been cracked.  florian echtler did a library and showed it to the displaylink guys ;-)
<mglocker> http://lists.freedesktop.org/archives/libdlo/2009-May/000092.html
<mglocker> http://floe.butterbrot.org/external/tubecable-0.1.tar.bz2
<mglocker> ~mglocker/dldemo2_mglocker.tar.gz
<mglocker> if you want to have a kind of C-style demo for the compression.

Minor fixes to the udl driver occur on june 1st, then there is not much activity until Marcus resumes working on it, in order to add compression support.

On august 14th, he mentions the need for a compression table which could be stored on the filesystem, rather than in the kernel image:

<mglocker> anyone mind if we would keep a 300kB huffman table in /etc/firmware?
<miod> that's not really firmware... can't you generate the table at runtime? is this for udl evilness?
<mglocker> jup, for the huffman pixel difference compression.  i could store the table into an *.h file, but i'm not sure if this is what we want.
<mglocker> i don't think we can "generate" the table.
<miod> but yes, although it's not technically firmware, it's loadable data, so why not
<mglocker> yeah.  it's probably worth an exception in that case.
<mglocker> i can draw some compressed stuff already, but some issues left to fix.

And on august 25th,compression support is added to the driver.

As more and more developers are trying the driver on as many platforms as possible, we start hitting situations where heavy screen adjustments would cause the output to stall, or some display updates to get lost.

Investigating these showed that the device was overflowed with requests, and needed time to process all the pending operations until further requests could be made.

This was not very different from a serial line link, where the serial chip has a small (and sometimes nonexistent) FIFO queue, and no more characters can be transmitted when it gets full, until the characters get transmitted over the wire.

For serial links, the kernel handles this nicely, by forcing writers to sleep when the FIFO gets full, and waking them when space becomes available in the FIFO.

But for display terminal emulations, there was no such thing, because it was assumed that every display operation (output a character, draw the cursor shape, scroll the display in any direction) could be done without having to wait.

And in fact, until the DisplayLink driver came into use, this was the case: display drivers either had full access to the display memory and could do the required changes directly in memory or, as e.g. for the Vax gpx driver [which I will write the story of soon...], by instructing the graphics controller to perform these changes.

But here, over the USB bus, bandwidth has to be shared with all the other USB devices on the controller, and the USB controller itself might not be able to queue enough requests for the udl needs - in other words, this is similar to the serial FIFO.

The graphics display being, from the kernel point of view, a special case of serial line, we could benefit from the FIFO behaviour, even though this had never been the case.

So a cheap and easy way to solve the "display gets overflowed with requests" problem would be to simply let the display operation routines (draw the cursor, paint a character...), which until now where `void` operations, return a value letting their caller know whether they had been able to perform the requested operation or not. The DisplayLink specific implementation of these requests could then return failure when the device FIFO is full, and let the upper layers know that they need to wait and force the process currently writing to the display, to sleep.

This would be a large, but mostly mechanical, change to all the display drivers in the OpenBSD source tree.

On august 30th, I suggested going in this direction:

Date: Sun, 30 Aug 2009 20:42:29 +0000
From: Miod Vallat
To: private mailing list
Subject: wsdisplay asynchronous processing

After discussing udl(4) behaviour with mglocker@, we came to the
following thoughts which I believe are worth sharing.

wscons - and its wsdisplay output part - has been written 10 years ago
with tga(4) and vga(4) style bitmapped devices in mind, with synchronous
operations.

This assumption was true, until we started to support video devices on
which the video memory is not directly accessible. Examples of this are
cfxga(4), gpx(4) on vax, and udl(4).

The first two such drivers have been coerced to fit the synchronous
model, because sending commands to the hardware was fast and did only
involve the device registers.

udl(4), however, is completely alien to this. Commands are sent with USB
pipes, there might be other devices on the USB bus, and processing is
asynchronous in nature.



Now, if you look at wsdisplay, there are two conditions upon which
wsdisplay will want to update the display contents:
- userland writes to the wsdisplay terminal. wsdisplay is invoked from
the tty layer, and has a process context, so it can sleep.
- kernel writes to the console device. wsdisplay can not assume
anything.

So if we are able to flag devices such as udl(4) as not being able to be
a console output device, then it will know that it can sleep at the
wsdisplay level.



Why am I talking about sleeping? Because udl(4) is currently the only
wsdisplay(4) driver unable to behave synchronously. In order to complete
its operation, it needs to get fifo resources and whatnot, and if none
of them are available, there won't be any new resources available until
the usb stack has a chance to run: udl(4) needs to sleep.

Now if udl(4) is allowed to sleep, and the driver<->wsdisplay interfaces
are modified so that a driver can return EAGAIN if it is not a console
device, then wsdisplay(4) can sleep. Problem solved.

and since the best way to figure out if such a proposal is worth doing is to write a proof-of-concept, the next day I shared a diff with Marcus.

But that diff was larger and trickier that what my previous message implied.

The reason for that, is that the graphics terminal is not simply a "line printer on a CRT". (and, writing this in 2025, I suppose some of my younger readers will not be familiar with Cathode Ray Tube displays)

An important improvement of CRT displays above line printers is that most CRT terminals have an addressable cursor position; this opens the road towards fancy displays (nowadays with colours!), first to implement IBM 3270 terminal emulators, later for more generic needs, thanks to the termcap and/or terminfo abstractions, and eventually the curses library.

The OpenBSD workstation console terminal emulation tries to mimic two well-known console devices, the Digital VT220 terminal (minus several seldom used features difficult to implement cleanly, such as double-width characters) on most platforms, and the Sun console (with extra features such as colour) on sparc and sparc64 ports. (I tried to have the luna88k port, which defaults to black on white display like the Sun systems, to use the sun terminal emulation, but got outpowered).

Both these terminal emulations have in common that they recognize specific escape sequences, e.g. to change the cursor position, change output colours, or other specific operations. Also, simply outputting a regular character expands to at least three display operations: hide the cursor, draw the character, and draw the cursor in its new position; and if the character had been the last on the last line, the display needs to scroll, which involves one more operation.

If display drivers were able to fail any operations, as allowed in my proposal, we could end up with a particular output operation processed only partly. But the tty layer in the kernel has no way to know that a character has been, sort of, "partially output". In this layer, a character has either been output, or needs to be output, and nothing in-between. Any finer-grained state needs to be maintained in a lower layer, such as the vt220 emulation code.

So, in addition to allowing display drivers to fail operations, I wrapped the terminal emulation processing into what I called an "abort state", remembering at which point we encountered a failure, so that attempting to output a character would skip the operations which had succeeded already.

With my changes, a character output would only be "validated" if all the display operations it would cause had been successful. If not, the process writing to the display would be forced to sleep, until the display driver reports it is able to process requests again. Processing of the display operations would note which operations had been successful, so as to skip them and only perform the operations which had failed or had no chance to start earlier.

Date: Mon, 31 Aug 2009 22:00:27 +0000
From: Miod Vallat
To: Marcus Glocker
Subject: early wscons `ok to stall' diff

This should be enough for an i386 or amd64 kernel to compile. Many frame
buffer drivers still need to be modified due to interface and prototypes
changes.

How does it work? Well, this adds an error path from the display driver
to the tty layer.

So we have this path:

1. void wsdisplay_emulops routines now return values. The driver
implementing them can return 0 if it did the work, or nonzero
(preferrably EAGAIN) if it couldn't. Note that generic rasops
routines, accessing frame buffer memory, never fail and always return
zero.

2. the return values of the emulops are now tested in the wsdisplay
emulation code. This is the horrible part with a lot of changes,
which will need careful review.

3. I designed this so that, when the emulops return an error, the emulation
state machine is moved back to a sane state, allowing the operation
to be tried again later. This involves undoing logical cursor moves,
and other internal state changes.

Note the code currently assumes an operation involving several emulops
can fail in the middle and be restarted from the beginning - this is
wrong, e.g. when scrolling the tty, since we copy rows and then clear
the bottom row. If the copy works but the clearing fails, we'll
restart with the copy.

I am aware of this and working on an `abort state' part of the
emulation state machine (which is currently the FALSE_ABORT bit in
the per-emulation flags, but needs to move into its own thing).

(This is why this is an alpha diff...)

4. The error condition detected by the emulation causes the tty write to
abort early. The MI wsdisplay code will now know how many chars have
been processed, and if it detected an early abort, it will not try to
write more characters, even if there are any left in the tty queue.

5. The same function already has logic to schedule a timeout if more
tty data is pending. This timeout will try to feed the driver more
work, but until it fires the driver can hopefully get some interrupts
and gather resources to do so.

What is left to do:

1. Update all MD frame buffer drivers (mechanical).

2. Finish the abort state design and correctly recover from a partial
operation.

3. Update udl to return EAGAIN in the emulops.

Known problems which won't be fixed soon:

1. Some operations do not come from the tty layer, but by keyboard
events (emulation reset sequence), ioctl or timeouts (screen burner).
We do not necessarily have a process context there, so sleeping is
not an option.

I need to extend some interfaces for the affected routines to know
whether the caller can recover automatically (tty context), or not,
and if not, whether it's ok to sleep or not.

2. There is no way, yet, for the driver which has returned EAGAIN to
cause the tty timeout to be triggered earlier (i.e. as soon as it
regains resources). I'll think about it eventually.

Miod (and now time for some zzz)

The next day, I received the best testimonial ever for a diff:

<mglock> hi from udl over wscons with initial EAGAIN support :-)
<mglock> miod is evil.

After more testing, on september 1st, a new version of that diff was shared:

Date: Tue, 1 Sep 2009 20:19:02 +0000
From: Miod Vallat
To: private mailing list
Subject: Re: wsdisplay asynchronous processing

In case people are interested, I have a monster diff (which will be
split in 4 different pieces), which implements error path from the
display drivers up to the tty layer, so that the driver can cause tty
output to stop if it is overflowed.

The tty layer (well, the wsdisplay tty code) will then nicely recover
from this, and everything is fine.

This diff has the disavantage of adding about 1KB of code to the kernel
(for kernels with vt100 emulation), so this might be a problem for the
floppies. And there is no way to disable this if SMALL_KERNEL. Unless
you want to dive in your own sea of macro hell filled with sharks.

Due the large size of the diff (about 200KB, affecting 51 files), I will
not append it to this mail. People interested in it can find it in

cvs:~miod/wscons-stall2.vari

Note that, for it to be really useful on udl(4) devices - since they are
the reason for this work - you need another diff from mglocker@, adding
the necessary EAGAIN code in udl(4).

$ wc wscons-stall2.vari
7017   26187  194603 wscons-stall2.vari

Miod

The monster diff started with the following description:

This diff is large because many frame buffer drivers need to be modified due to
interface and prototype changes.

How does it work? Well, this adds an error path from the display driver
to the tty layer.

So we have this path:

1. void wsdisplay_emulops routines now return values. The driver
implementing them can return 0 if it did the work, or nonzero
(preferrably EAGAIN or EINTR) if it couldn't. Note that generic rasops
routines, accessing frame buffer memory, never fail and always return
zero.

2. the return values of the emulops are now tested in the wsdisplay
emulation code. This is the horrible part with a lot of changes,
which will need careful review.

3. I designed this so that, when the emulops return an error, the emulation
state machine is moved back to a sane state, allowing the operation
to be tried again later. This involves undoing logical cursor moves,
and other internal state changes.

Note there might be bugs in the undoing so far, I need to review this
carefully. And test too (-:

There are comments in wsemulvar.h trying to explain how I keep track
of failures occuring in the middle of a `character' (from the tty
layer point of view) output.

4. The error condition detected by the emulation causes the tty write to
abort early. The MI wsdisplay code will now know how many chars have
been processed, and if it detected an early abort, it will not try to
write more characters, even if there are any left in the tty queue.

5. The same function already has logic to schedule a timeout if more
tty data is pending. This timeout will try to feed the driver more
work, but until it fires the driver can hopefully get some interrupts
and gather resources to do so.

Note that it is not necessary to implement a faster output resume path (e.g.
if the driver gets an interrupt and frees resources), as the timeout will run
only 8ms later (1/128 second). Keep in mind the human persistance of vision
is about 1/25 second, so in the blink of an eye the wsdisplay code can
resume stalled output several times.

Problems left with this code:

1. You may notice resetop() does not check for emulops failure. This because
this is an out-of-tty-layer processing (but ioctl issued to the tty device).
I know how to make it able to recover, but this will need a few more emulops
prototypes changes, and I would like to keep the focus of this diff minimal
(har, har), i.e. trying to only address one problem. A later diff will
address that area.

2. The same comments apply to the screen burner code. Again, I have plans for
this.

How I intend to split this work in individual commits:

1. internal wsemul changes to change the state machine functions from
returning u_int to returning void (and updating emul state structs
directly), so that they can later be changed again to return errors.
(no functional change, but little code growth)

2. emulops prototype changes (and all the rasops / driver part of this diff).
(again, no functional change, but little code growth)

3. wsemul_ops change of output() to return the number of chars consumed, and
the corresponding logic in wsdisplaystart(), with the emulation code
returning the number of chars it has been given
(again, no functional change, but little code growth)

4. the error path handling in the emulation code, i.e. the evil part of this
diff.

Of course, testing exposed a few bugs in that diff, which led to a new version:

Date: Wed, 2 Sep 2009 16:17:06 +0000
From: Miod Vallat
To: private mailing list
Subject: Re: wsdisplay asynchronous processing

New diff fixing a few bugs in the previous diff (description of changes
at the head of the new diff).

cvs:~miod/wscons-stall3.vari

$ wc wscons-stall3.vari
7074   26494  196861 wscons-stall23vari

Miod

And while writing this, I am dissatisfied with past me. It was not a wise idea to put the details in a file which is now long gone, and I should have put them both in the file and in the email. But I did not think that future me would want to tell this story, years later.

• Fortunately, it turns out that I had sent that diff directly to Marcus minutes later:*

Date: Wed, 2 Sep 2009 16:13:51 +0000
From: Miod Vallat
To: Marcus Glocker
Subject: latest wscons diff

This is diff #3.
Changes since last diff:
- rearrange changes to wsdisplaystart() so as not to introduce a new goto.
- minor simplification in wsemul_sun when backing out state changes because
of failed scrollup operation.
- rearrange some double-wide array updates in vt100 to be able to correctly
recover from operations aborted in the middle. This double width feature is
something noone uses anyway.
- wrap the COPYCOLS and ERASECOLS operations in vt100 within WSEMULOP so that
they will not be reissued by mistake if failure occurs after they're done.
This ought to have been in the previous diff but I forgot to do this.

And some time later:

Date: Thu, 3 Sep 2009 22:00:06 +0000
From: Miod Vallat
To: Marcus Glocker
Subject: Re: udl

Even better, with a diff that still compiles after the untested last
minute change.

This is diff #4.
Changes since last diff:
- removed udl.c changes, get them from mglocker@
- fixed an unitialized variable in wsemul_vt100_output() causing cursor
image display to sometimes be skipped.
- extended the abort state with four different states (ok, failed to display
the cursor image, failed to jump scroll, failed a regular operation) instead
of two (ok/fail) and having the `fail cursor' a particular state of failure.
This allows me to make things a bit more clean (arguably) and two fix two
important bugs:
+ after a regular failure, jump scroll would be attempted before the
other operation; if the failure had happened after N operations we would
then skip N operations during the jump scroll (usually causing the
scroll not to happen, so lines would be overwritten instead of
scrolling).
+ after a jump scroll failure, we need to retry it with the same number
of lines as the failed operation, so that the copy/erase parts of the
scrolling operation are consistent with each other.

Eventually Marcus confirms it works:

From: Miod Vallat
To: Marcus Glocker
Subject: Re: udl

> This diff works pretty fine for me!  I can't spot any bugs anymore yet,
> even when running in a screen session with crazzy apps like irssi and
> mutt.

Excellent.

Having been able to wrap these operations in specfic macros to hide the note-and-restart logic, this allowed installation media kernels (which would not embed the DisplayLink driver) to use the previous "nothing can fail" logic, in order not to grow these kernels and still allow them to fit on 3"½ floppy installation media. This work eventually got commited on september 5th, in multiple steps and then some...

...and udl made use of it immediately.

On september 11th, this allowed to nicely fix a diresome situation.

And since I am only human, a small bugfix was needed on the 14th.

After these changes, there have been no problem reports with DisplayLink devices.

Later this month, an X server, based upon the "damage" extension which describes areas which need to be redrawn, was also added to the OpenBSD source tree, and support for the DisplayLink devices was now complete.

This allowed "serial console-only" platforms such as the armish and landisk ports to use a graphics console and run an X server.

And even if there had been no X server support, the DisplayLink driver forced the console code to face new challenges and solve them in a way which will benefit future drivers.

---

Today, DisplayLink still exists as part of Synaptics, and their most recent chips are supported neither by Linux nor by OpenBSD with open source drivers. Synpatics provides a binary driver for Ubuntu, which to this day hasn’t been reverse engineered, yet.

The usefulness of these devices has apparently gone away, people do not seem to be interested by these devices anymore those days.