Since the start of Wayland, the input situation has been a bit iffy for certain specialized cases of global keybindings. For most, this is a minor inconvenience, where an additional shortcut can be defined via a settings application or configuration file. For the visually impaired, who require a screen reader for their computer to function, this is completely debilitating. Just to find out what application you’re currently on requires active, global keybindings. Uniquely, screen readers in particular often need conditional keybindings based on the current location of a user. For example, the keybindings h, i, or t, with no modifiers are valid, consumable keybinding under certain circumstances, but not others. The point here is that a screen reader often needs to determine in real time whether a keybinding is valid or not.

For accessibility to become better on Wayland, a protocol for intercepting, and conditionally consuming any events should be created for Wayland. Even GNOME folks are on board with a Wayland protocol for this behaviour, although their suggestions are a bit more complex; I am not even sure if I completely understand them:

a grabbed-key event could be synchronously acked by the AT along a list of grab adds/dels, so that grabbing updates can be atomic with the grabbed-key event

As of now, I have created a very simple draft for this protocol, and I’m looking for input on how to improve it. I will be extremely active in discussions on this, since I am being payed to create and implement the protocol, hopefully by mid September. I don’t know a ton about Wayland, so there may be some mistakes in the XML itself. Hopefully others can chime in and help get me antiquated with my mistakes.

I’d like to ping @emersion specifically, since I’m personally invested in Sway’s accessibility, and I know that y’all prefer Wayland protocols. Inclusion in Sway bodes well for other, smaller Wayland compositors.

<?xml version="1.0" encoding="UTF-8"?>
<protocol name="a11y_input_v1">

<copyright>
Copyright © 2023 Tait Hoyem

Permission to use, copy, modify, distribute, and sell this
software and its documentation for any purpose is hereby granted
without fee, provided that the above copyright notice appear in
all copies and that both that copyright notice and this permission
notice appear in supporting documentation, and that the name of
the copyright holders not be used in advertising or publicity
pertaining to distribution of the software without specific,
written prior permission. The copyright holders make no
representations about the suitability of this software for any
purpose. It is provided "as is" without express or implied
warranty.

THE COPYRIGHT HOLDERS DISCLAIM ALL WARRANTIES WITH REGARD TO THIS
SOFTWARE, INCLUDING ALL IMPLIED WARRANTIES OF MERCHANTABILITY AND
FITNESS, IN NO EVENT SHALL THE COPYRIGHT HOLDERS BE LIABLE FOR ANY
SPECIAL, INDIRECT OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES
WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN
AN ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION,
ARISING OUT OF OR IN CONNECTION WITH THE USE OR PERFORMANCE OF
THIS SOFTWARE.
</copyright>

<interface name="ext_a11y_input_v1" version="1">
<description summary="accessibility input controls">
This interface is designed to allow any assistive technology
to control various input mechanisms, and conditionally consume
the input events.

The assistive technology application can request access to directly control
which events are incomming, and based on internal logic, determine whether to pass
this event on through to the compositor, or consume the event as a command to
the assistive technology.

It is left to the compositor to determine if an assistive technology is permitted to
access this interface, and what should occur if a response is taking too long.
It may cause a timeout to trigger in the compositor, passing through keys after a certain length of time;
it may revoke access outright to the requester;
or, it may simply wait until a response is made, no matter the length of time.
The compositor may also depend on policy or user configuration to determine
which behaviour will be used.

This object is a singleton global.
</description>

<request name="destroy" type="destructor">
<description summary="unbind from the accessibility input interface">
Informs the server that the client will not be using this protocol
object anymore. You must destroy any wp_a11y_input created from the
wp_a11y_input_control before.
</description>
</request>

<request name="get_binding">
<description summary="creates a binding object">
Creates a binding object. It will be used by the application to bind
one or more actions and get the corresponding events.
</description>

<arg name="id" type="new_id" interface="zwp_a11y_input_v1"
summary="the new binding interface id" />
</request>
</interface>

<interface name="ext_a11y_input_v1" version="1">
<description summary="binding to an an accessibility input interface">
This interface represents a binding to an assistive technology control process.
</description>

<enum name="consume_behaviour">
<entry name="passthrough" value="0"
summary="continue processing the event as if the assistive technology does not exist" />
<entry name="consume" value="1"
summary="immediately drop the event without processing it further" />
</enum>

<enum name="error">
<entry name="same_priority" value="0"
summary="an assistive technology is already using this interface at the same priority." />
<entry name="invalid_event_id" value="1"
summary="the event id being used is not valid" />
</enum>

<request name="consume_event">
<description summary="notify server whether to continue processing the event">
Tell the Wayland server whether the event id should be passed through, or if it should be
consumed by the assistive technology process.
</description>

<arg name="id" type="uint" summary="an event id" />
<arg name="consume" type="consume_behaviour" summary="should the event be consumed" />
</request>

<request name="destroy" type="destructor">
<description summary="unbind the events">
The client no longer wants to receive events.
</description>
</request>

<request name="bind">
<description summary="bind an assistive technology to receive events">
Bind an assistive technology to the protocol so it may recieve events.
The priority is so that two separate assistive technologies can bind to the protocol without issue.
Much like `nice`, the lower the priority number, the earlier it will be served.
</description>

<arg name="priority" type="uint" summary="the priority level of the assistive technolgoy" />
</request>

<event name="started">
<description summary="the action started">
This is sent when an input event is started.
</description>
<arg name="id" type="uint" summary="the event id" />
<arg name="event" type="string" summary="the event in question" />
</event>

<event name="stopped">
<description summary="the action stopped">
This is sent when an input event is stopped.
</description>
<arg name="id" type="uint" summary="the event id" />
<arg name="event" type="string" summary="the event in question" />
</event>
</interface>
</protocol>