Taking Notes

cause i'll forget

Accessibility Engineering and Low End Disruption

apple doesn’t succeed at making their platforms accessible because they care more, they don’t succeed because they are somehow better at programming

a capable, thoughtful manager, who is correctly assessing risk vs reward, both qualitatively and quantitatively, who is doing their job well, using all of the experience and skill that brought them to where they are, can, and likely will, decide to ship an app that could be accessible, but is not.

this is not a function of callousness (although they may also be quite callous).

this is not a function of engineering aptitude (plenty of our leading lights have let access fall to the wayside1 by making solid engineering choices that are completely defensible).

the apple accessibility teams are made up of disciplined, brilliant engineers. they all work incredibly hard on a problem with unimaginable scale. but their brilliance and discipline and hard work, while necessary, are not sufficient.

what matters here, what sets apple apart, is that they figured out how to go last and contain the risk enough to avoid a revolt from other teams or executive leadership. it costs them a lot to do so, has significant, measurable downsides, and requires a significant rethink of what it means to build good software.

bundles

some of the code that powers accessibility on apple platforms is just disgusting to look at and to work on.

most of the code that makes apple software accessible lives in what’s called an accessibility bundle. without diving into the minutia of the thing, bundles are a way to load something akin to a plugin into a cocoa app at runtime if an assistive technology is activated. it involves manipulating the app or framework class hierarchy and using objective-c dynamism to read app state and build up a usable accessibility hierarchy. insert a super class here, read an instance variable there, swizzle in a method and store the state for it in associated objects.2

bundles are not a good way to build software by any common measure. they don’t make you efficient, or your code correct. the performance can get bad. they’re hard to maintain and involve constant patching to keep them alive. huge chunks of bundle code are thrown away and rewritten multiple times during every major os release.

i found the initial adjustment to working on this code sometimes viscerally upsetting. “i can’t do that!!”, “clearly i should be protecting against…”, “if i do it that way it’ll break when they…”, etc.

the thing is, critically, if a user never activates an assistive technology like voiceover they wouldn’t know bundles even exist. many of the managers at apple don’t know much or care much about accessibility. some of them are borderline hostile to it. the bundles make it possible for the accessibility team to allocate their own resources and set their own priorities with only very limited coordination and fallout contained to people who would benefit most from the upside. they allow the accessibility team to fix the bugs they find, when they find them, with pretty limited opportunities for a manager who’s next promotion is on the line to get cold feet and pull resources or block fixes.

in the model of low end disruption, we can see bundles as a low end entrant, with a product that seems worse, but that solves an important problem in a way that the mature products, by their nature, can’t realistically address.

if the job to be done is “produce easy to understand, correct, maintainable code that engineers who care about those things will enjoy working on” then bundles aren’t even in the competition.

if the job is “manage your team resources responsibly, keep your bug count low, ship your features on time” then bundles aren’t the tool you reach for. why would you load a squirrelly plugin into your own app on purpose?

the chaotic morass of runtime hacks and macros for everything and nearly exclusive dependence on runtime validation3 make them not just not good, but almost aggressively bad. but if the job they are hired to do is make apps that aren’t accessible, they’re incredible. unparalleled. entirely without peer.

almost all the things a good engineer would do to make bundles less gross would make them worse in unacceptable ways. almost anything a high end product might do to accommodate the kind of autonomy bundles demand would make them unacceptable to their current clients.4

apple has a team whose only job is to make things accessible and they have infrastructure that lets that team mostly avoid people who have a different job saying no to them. put those together and you have the most successful accessibility software team (imho) on planet earth.

the gross stuff isn’t desirable, but is maybe necessary

anyone who is steeped in universal design that has made it this far has maybe broken out in hives? please just stick with me, it’ll all come out in the wash.

  • well designed apps that take accessibility into consideration from the beginning are better for everyone.
  • accessible ramps for getting into buildings are better than those terrible lifts they bolt into the sides of stairs in public buildings.
  • we all benefit from features in society that originate as accessibility affordances, like curb cuts for wheelchairs, that everyone with a stroller can now take for granted.

without question these statements are true.

i don’t worry too much about saying things that might make people think i’m a bad engineer or have bad ideas about building software. i do care a lot about not giving people the idea that we should all knock out our accessibility code real quick and sloppy at the end as an afterthought.

my thesis here is that where the effect of failing is very much acute, most of us do not adjust our approach accordingly. likely because we regard assistive technology users as one of many available audiences we may or may not wish to court.

if we ship an app that’s inaccessible, hey we’ll get around to it, we’ve got a roadmap, hasn’t bubbled up yet, software takes time. there are so many bugs in our support tracker and “doesn’t work for blind people” is just a tiny blip.

of course we should be methodical and plan ahead and build our software in a way that is both accessible and maintainable. we should seek to avoid compromising, but should the need to compromise emerge, the balance should unquestionably be tilted towards gross and unmaintainable code where that is the cost of accessibility. if you could fix that missing label but eww i’d have to wire up a very clumsy chain of delegates or fire a notification that clearly violates layering, without question, you should do it.5

we should be willing to find a blend of cultural and political and technical workarounds in our organizations that acknowledge that code can’t be allowed to go out the door that isn’t accessible and that a lot of our ideas about technical excellence and what a good engineering manager does wildly miscalculate the cost of our decisions to “do it right”.


  1. what’s good loren brichter?

  2. you can find some of the machinery for bundles on your mac if you want to go spelunking, but it involves liberal usage of macros so you’re only getting a piece of the picture.

  3. it’s hard to validate code at compile time that’s compiled at a different time from a separate source repository.

  4. it’s hard to overstate how impressive it is that objective-c managed to be a language that could credibly accommodate people who cared about correctness and performance and people who cared about doing wild stuff you probably shouldn’t do.

  5. you should also try to contain the wild gross nonsense you do so that the “good” engineers don’t chase you out of the building. the thing where bundles only load when voiceover is on might be their most important feature.

React Native Accessibility Is Pretty Bad

Allow me to preface this by saying that React Native sets out to solve problems I don’t think are very interesting to solve. In particular I don’t think that sharing a single UI between disparate platforms is desirable from any perspective that isn’t cost savings and that even from that perspective it’s dicey.

So I’ve been recently saddled with some React Native problems.

Hey, here’s some very good React. It’s idiomatic and uses the popular and recommended libraries. Isn’t it great?

~Them

So I do what I always do, I fire up VoiceOver. I had read the docs for React Native when it was new. I knew they were aware of accessibility, so maybe it wouldn’t be too bad.

(You’ll be very surprised to learn it was quite a bit too bad.)

Some of the issues were specific to the project. They didn’t understand VoiceOver or how accessibility works in React, so I spent a bit of time fixing those problems before I got into the weeds.

Where I realized there were some tricky problems, and maybe some intractable problems, was when I dug into React Navigation. This is the library recommended (but not maintained) by Facebook for navigation in new projects. (It’s very very cross platform and we all know that’s an inherent good.)

A detour to set expectations

If you navigate around an iOS app that uses built-in UIKit controls like UINavigationController, you get lots of great stuff for free and maybe the most important free thing is compatibility withassistive tech like VoiceOver. When you navigate to a new screen you hear a tone and VoiceOver jumps to a logical location on the new screen, usually the title in the navigation bar. If you want out of that screen you can do a two finger scrub (in roughly the shape of a Z) and boom you pop back one screen, again hearing the tone that tells you that you’re somewhere new and again VoiceOver moving it’s focus somewhere sensible.

Under the hood, this is all done with a straightforward accessibility API, primarily via notifications and actions, largely automatically. When the navigation controller pushes a new controller onto the stack, it can inspect the navigation bar for a sensible element for VoiceOver to focus on, then tell the system that there’s a new screen and here’s an element that should be considered for initial focus. The container view that holds the views of the controllers in a navigation controller can implement an action called escape (the scrub mentioned earlier) and when VoiceOver receives that gesture it can search up the view hierarchy until it finds someone implementing escape and it more than likely finds that navigation container view and navigates to the previous screen.

iOS has had a screen reader (VoiceOver) since iOS 3, it’s had an accessibility API since iOS 5. They’ve been hard at work for a long time now and have given us a toolchain where you have to make a lot of sketchy decisions to produce software that isn’t useful to people who use assistive technology.

React Navigation from an accessibility perspective, is one big sketchy decision, and React Native isn’t helping them do better

The accessibility API in React Native doesn’t even offer the API for accessibility notifications or for accessibility escape. For some reason they support an action called magic tap, an accessibility feature no app does (or should) implement, butskipped over escape, which you need if you want to create an accessible navigation library on iOS.

Given the lack of React Native support for important accessibility API, anyone building a new app and using the recommended library for navigation has to

  • Know these technical and usability problems even exist
  • Implement a native bridge for posting accessibility notifications
  • Implement a middleware to respond to React Navigation actions and post screen changed notifications using that bridge
  • Implement an action for the escape gesture, place a natively bridged component somewhere in your applications component hierarchy such that it can receive the event from the system and dispatch the action which will eventually find it’s way to dispatching a React Navigation action to pop it’s stack by 11
  • Figure out how to, in your middleware that watches for React Navigation actions, find an appropriate initial element for VoiceOver to focus on and get that element over the bridge to the accessibility notification API (I’m not sure how to do this in a way that’s at all generalized and not very project specific)

Most projects won’t even clear the first hurdle.

Buttons aren’t buttons

The various Touchable components in React Native don’t advertise themselves as buttons to the accessibility API. You can readily add the annotation. They even made enhancements to the API for doing just that in a recent version of React Native. But, as with navigation, you’d have to know this is something that’s needed.

Phantom Elements

Flex box as a layout idiom loves to have a hidden element to make stuff center properly. On top of being obnoxious to me personally, this makes it super easy to leave phantom elements floating around your UI that only VoiceOver users will encounter.

Missing API

The React Native accessibility is missing a LOT of the native iOS accessibility API. Much of which has been around almost as long (and in some cases longer) than React Native has existed.

Custom rotors are cool. Knowing what’s up with Zoom is cool. Annotating your views so Switch navigates in a more useful way is cool. React Native can do exactly none of those things without cumbersome bridging.

Accessibility is Table Stakes if you’re going to claim to be building “native” apps

If your fancy new library is 🎇 Whatever 🎇 Native 🎇, and the front page of your website is all about how everything is just like if you had written it with the platform vendors recommended toolchain, then you have a moral responsibility to make that statement true in the ways that it matters most. One of those ways is accessibility.


  1. Or you could submit a PR to the React Native project and hope one day someone actually looks at it.

Everything I Know (About Screen Readers (Part 0))

If you’ve been paying close attention, it’s been more than a week since my last post. I’ve been making steady progress on publishing the open source components that underly Speak Up:

I’ve been working on what I hope will be a multi-part blog post I’ve been calling Everything I Know About ScreenReaders. It’s been fun to work on, but it is still very much in the post it notes and white board scribbles phase. I’m not going to put a firm timeline on it, but I’d like to have the first posts up soon and rap it up by the end of this summer.

Stay tuned.

Speak Up

I’m working on a new thing

I left my job in December. For a lot of reasons. The main one being that I spend most of my time these days doing political organizing in Detroit, where I was born and had been away from for too long. But… it’s hard to do a lot of that work in Michigan, in the winter, so i fired up Xcode and started exorcising some ideas.

Two months later I’ve got something worth talking about.

Building a screen reader for macOS is an arcane science

Before VoiceOver shipped, the public accessibility APIs in AppKit and in HIServices were conceived as agnostic interfaces for anyone who wanted to build a screen reader. some of the quirks of these APIs are there because there was no real client to poke them and make sure they made sense.

There is still, theoretically, support for third party assistive technology (AT), the umbrella a screen reader would fall under, but it’s not a trivial task and it’s simply not possible without dipping in to SPI1.

But it doesn’t have to be

The current state of accessibility on all of the Apple platforms is, like almost everything in the Apple universe, organized in a top down fashion. An app becomes accessible because Apple does the work to make it accessible, because the app developer does the work to make it accessible. There’s very little that can be done, even by well motivated community members, to make an app neglected by Apple or by a third party developer better.

I believe it is possible to produce infrastructure that democratizes assistive technology for macOS. I want the barrier to entry to be as low as active interest and time. The future of fundamental technologies serving disabled people ought to be in disabled peoples hands.2

This coming week I’ll be prepping open source components I’ve built for release. Hopefully you’ll find them well organized and easy to jump into. Once I’ve got all the pieces out in the world I’ll be seeking community involvement to steer the future of the project and to figure out what problems are most useful to solve.

In addition, I’m working on the first app to sit on top of this infrastructure. A brand new screen reader for macOS, which i’ve given the project code name SpeakUp.

Accessibility as autonomy

SpeakUp, as well as the open source components it sits on top of, will support a robust plugin architecture to allow individuals or groups to tune, tweak, or totally overhaul the accessibility of an app.

Many of us that work in professional software development take for granted that we can choose from a large number of IDEs, editors, compilers, and any number of other tools because as professionals even seemingly small optimizations add up and make us happier worker bees. But when it comes to accessibility there’s not nearly as many options to make things just so, to get a workflow that’s just for you, just for your job, or hobby. It matters that when an app’s accessibility is lacking, or just not well suited to your needs, that there are options available.

In this spirit I’ve been hard at work developing several sample plugins. The one I’m happiest with right now is a plugin that provides an accessibility API for Sublime Text via it’s Python API. Sublime Text is a very popular programming text editor and a black hole for VoiceOver right now.

Here’s a recording of navigation using regular arrow keys through the Python that powers the plugin, line by line, along with the relevant Python file for comparison. You’ll hear sequential pops indicating leading indentation level and marked up output that includes all the control characters a programmer would care about.

connect.py

The specifics are subject to change, but there’s so much potential to make developer tools accessible and widen the community of people who can participate in building software on and for Apple platforms.

Stay tuned

Until I get a proper website up and running for the project, this will be the spot to watch for updates, which you should expect to see regularly going forward.


  1. SPI is apple jargon (it can mean other things in other contexts) for private API, for internal use, by first party apple software. it doesn’t have the same compatibility guarantees of published API.

  2. The number of blind Cocoa developers is relatively small considering the huge popularity of iOS among blind smartphone users. this is largely a reflection of the very poor state of developer tool accessibility.

If I Was a Liberal Speech Writer

The Trump win still has me reeling.

And on top of that I’ve spent this week watching Democrats seek to appease Donald Trump’s voters and Donald Trump himself.

I don’t think i can overstate what an enormous mistake this is. These people who wish to be seen as leaders are abandoning any and all concern for the trust and wellbeing of marginalized peoples, who they depend on to be elected and who are forced to depend on them to defend their rights. The people who show up to vote for them in the 90+% range, like Black women. This is deplorable.

I’ve had enough conversations about whether Democrats are being strategic or are simply capitulating that I would like to propose a framing for those Democrats who can’t bring themselves to call Trump voters the bigots they are, to at least make the case for not cooperating, and seek some base level of accountability.


Generic Democrat Approaches The Lectern

I’d like to talk about cooperation and accountability today.

I don’t think the majority of Trump’s voters set out to do harm.

Some definitely did, and we need to be conscious of them, & mindful of the people they seek to harm, but I don’t believe that is the majority.

What I do think the majority did was respond to a changing world by trying to put things back how they were. A man said he could make it how it was and they said, “that sounds good”.

I think there are two critical takeaways from that:

First, that man is lying to you.

He cannot and will not make things how they were. He seeks only to empower himself and will betray your trust.

And second, putting things back how they were is a dangerous idea for many, many people. For these people, these fellow Americans, equality is only recently a viable idea. In many ways, the equality they have found isn’t durable. Their place in our society is conditional on good behavior, not the inherent right it’s meant to be.

Enabling a man like Donald Trump, cooperating with him in any way, presents a very real, even life-threatening danger to these people. These people that are our friends, our families, our communities.

We need to be mindful of everyone’s struggles, but when making policies and choosing leaders we need to resist asking the marginalized peoples we share this country with to shoulder more than their fair share.

App Camp for Girls’ Seattle Camp

The first time I heard about App Camp I sent Jean an email asking if I could help with anything, however small. Even if it was just sweeping up after the camp. They never did take me up on the offer to sweep up, but this week they gave Sound Off a chance to do something not so small.

Sound Off is sponsoring a team at App Camp for Girls’ Seattle Camp. Unlike our usual rounds, this is something we’re sponsoring out of our operating budget, because it was time sensitive and an important cause to us.

App Camp for Girls is a truly special organization. Social pressure for young women to give up on STEM fields starts to take hold around middle school and App Camp makes a deliberate effort to get out ahead of that. The campers spend a week with adult women in tech who show them some real possibilities for themselves and their friends as developers, designers & testers. They get to build a real app they can put on their real phones. It’s an invaluable experience and one I’m proud to support.

App Camp Seattle still needs one team sponsor by this Friday. I’d love to see someone from #teamSoundOff step up to fill that gap!

If you’re interested get in touch with Kristina from App Camp.

Sound Off

This month I started a new thing with my friend Jeff.

It’s called Sound Off.

We sat down as a couple of white cis hetero dudes to explore what it would mean to help make working in tech a better place. How to contribute without being gatekeepers or trying to be in charge. We took a lot of inspiration from existing projects and asked for help.

Our organizing principle is that we’re not qualified to be the ones deciding what should be done, but we are qualified to draw attention to all the good work already going on. We can use our voices and ask our community to join us in finding funding for them.

I think we can help good people do good work and I’m excited about it.

Check out our first campaign to help fund live captioning and ASL interpreters at AlterConf.

At the Risk of Jon Ronsoning

I’m not much for telling other people when or how to be mad. This person said this, that person said that, none of it to me. Not my bag.

But…

There’s something worth saying about the hubbub on the tubes this week.

I don’t think the details of the disagreement are that important, but more the dynamics of what happened after.

There’s speculation that Bielefeld is being disingenuous about her identity. She might be, but I have no of way of knowing this, and it doesn’t matter.

What I do know, is that the pile on is very much about someone who is perceived to be a woman with an audience and a contrarian position, and a whole mess of people who won’t stand for that. The tone and scale of the blowback would have been fundamentally different for someone that people believed had “earned” their audience. And a lot of men have a very hard time believing any woman can ever earn an audience.

Some people have taken the time to point out there are real problems here. Some have not, and I find myself thinking less of them for it.

It is possible in the coming days and weeks that we’ll learn more and our attention will turn away from sexist backlash, but I hope we all remember this part next time we’re wondering why women writing about tech need to be very cautious about having an opinion or being heard by too many people.

Update: Arment was dealing with a family issue. Criticism of his silence retracted and I hope for the best with what must be a difficult time for them.

Free Advice

So let’s say you’re a company that’s seen a little criticism lately.

Let’s say you’re asked to comment on that criticism.

Let’s say you don’t think you did anything wrong and you’re being portrayed in an unfair light.

Something I think you should consider doing, is not defending yourself. Don’t present your side of the story. Don’t try to explain why that’s not representative of your culture. Don’t tell people how hard you’re working on whatever.

Just tell people that you’re going to investigate, that you’re going to do what you can to correct the situation, and that you’re going to make sure this isn’t a systemic issue.

Here’s a free template :)

Hi [Reporter],

Thanks for bringing this to our attention. Please give [subject of story] my contact info [appropriate phone number and email address for an actual adult person at your company empowered to handle this situation]. We’ll be sure to get to the bottom of this, correct what we find as best we’re able, and find out if this is part of a broader issue that we need work on. We’ll do our best to be transparent about how this turns out and our progress as we make it.

Thanks, [Actual adult at your company]

And then, ya know, do that stuff.

Radio Dysentery

Everybody has a podcast these days, but way way back, in the before times, me and some of my friends got together and recorded a show that was ostensibly about sharing odd ball music we liked.

It was called Radio Dysentery.

Mostly it was just weirdos being weird.

My favorite episode is our fake This American Life about food poisoning:

13 - This American Dysentery

Here are the first few episodes (which are more representative of what the show was usually like):

1 - Man Grappling

2 - The Revenge

3 - Return to the Fold

4 - Jesus Overslept

Also, here’s some Perfect Strangers 9/11 fan fic