“Image May Contain…” – Alt Text and Other Breakthroughs in Assistive Technology

Sometimes the screen has to go dark in the middle of an important presentation to make a point. That’s exactly what Matt King did during his, to show others in the room what he sees. King is an engineer at Facebook, and is also blind.Facebook’s automatic alt text uses AIKing was presenting the results of the project he was part of at Facebook, an AI-powered tool, officially unveiled by the company on April 4.In its press release, “Using Artificial Intelligence to Help Blind People ‘See’ Facebook,” written by Shaomei Wu, Software Engineer and Hermes Pique, Software Engineer on iOS, and Jeffrey Wieland, Head of Accessibility, Facebook introduced automatic alternative text, or automatic alt text, a new development for its applications that generates a description of an image using object recognition technology.This is how the feature works, according to the press release:

People using screen readers on iOS devices will hear a list of items a photo may contain as they swipe past photos on Facebook. Before today, people using screen readers would only hear the name of the person who shared the photo, followed by the term “photo” when they came upon an image in News Feed. Now we can offer a richer description of what’s in a photo thanks to automatic alt text. For instance, someone could now hear, “Image may contain three people, smiling, outdoors.”

Facebook notes that the alt text feature will be available on iOS screen readers set to English at first, but plans to make it available, without the dates set yet, in more languages and on more platforms.Microsoft unveiled Seeing AI at developer conference in MarchMicrosoft was not to be left behind with its unveiling of Seeing AI, a research project “that uses computer vision and natural language processing to describe a person’s surroundings, read text, answer questions and even identify emotions on people’s faces” at the company’s Build developer conference at the end of March.According to the company blog announcement, Seeing AI can be either used via Pivothead smart glasses or as a cell phone app. (No release date was announced yet.)Here’s a video on “Microsoft Cognitive Services: Seeing AI app”:

Netflix’ added audio descriptions to DaredevilBig companies aren’t strangers to the world of accessibility tools, and seem to be aware how important it is to tap into it. With estimated more than 39 million blind people, and more than 246 million visually impaired, it makes sense, and will keep happening.Last April, for instance, in response to numerous complaints and customer pressure, Netflix added audio descriptions to its original series Daredevil, which features a superhero who is blind. The company also announced expanding the feature to its other series, including Orange Is the New BlackHouse of CardsUnbreakable Kimmy Schmidt, and Marco Polo.Twitter wants you to “feel” imagesIn an effort to make accessing and understanding images more inclusive for the visually impaired, Twitter made alt text technology available on its platform.In a company blog post dated March 29, Staff engineer Todd Kloots explained how this assistive tech feature would work:

Starting today, people using our iOS and Android apps can add descriptions — also known as alternative text (alt text) — to images in Tweets. With this update, we’re empowering everyone to ensure content shared on Twitter is accessible to the widest possible audience.Enable this feature by using the compose image descriptions option in the Twitter app’s accessibility settings. The next time you add an image to a Tweet, each thumbnail in the composer will have an add description button. Tap it to add a description to the image. People who are visually impaired will have access to the description via their assistive technology (e.g., screen readers and braille displays). Descriptions can be up to 420 characters.”

twitter accessibility feature

As Aman Jain, who covers technology news for financial industry company ValueWalk, noted, until now, even with screen readers and braille technology, images have been out of reach for the visually impaired people. Now the descriptions of the images, once added, can be accessed like any other text post.To enable third parties to add alt text to images, Kloots noted that Twitter extended its platform products to both the REST API and Twitter Cards.“We know this is especially important for specialized Twitter clients for the visually impaired such asEasyChirp, Chicken Nugget, and The Qube,” he wrote.Should social media experience accessibility become the norm?Needless to say, the global reaction to such promising breakthroughs had been positive. Social media users living with sight loss can feel pretty isolated, as images are an integral part of our everyday social media experience.Terry Hawkins, Head of B2B solutions at RNIB, sight loss charity, explained in his article forHuffPost Tech UK what the recent changes in the social media accessibility mean to him:

[T]he most pleasing thing for me about this is that the announcement felt like a huge step in in the right direction towards making life easier for those living with sight loss — particularly in Generation Y. Working with partially sighted and blind people on a regular basis, I can vouch for many when I say Twitter’s change will be well received. Social media can be a poor performer when it comes to accessibility, which poses a particular problem for visually impaired millennials who rely hugely on these platforms.

Assistive technology can’t always keep up with the constant updates the social media platforms are undergoing, in order to meet user demands, Hawkins noted, and while tech companies do often have teams dedicated specifically to accessibility projects, “it’s sometimes debatable whether the needs of those living with sight loss are properly considered at the development stage. He wrote:

Due to the huge diversity of content shared across social media platforms, these organisations now have a responsibility to make it accessible to all — which can pose a challenge. And while it won’t be an easy ride, it’s a necessary one. As our reliance on technology and the web continues to increase, and we access more of our services online, this change has never been so crucial.

Michelle Hackman, in her aptly-titled article, “Why you should use Twitter’s new accessibility feature for blind users — even if you’re not blind,” concurs. Image captioning should become the norm for all of us. For companies and news outlets there’s also a “side benefit,” she noted: “Adding alt text to images makes them accessible to search engines, potentially giving them a boost in search engine rankings.”Developers, giant companies, news organizations, and your average social media user tend to agree: The recent developments in assistive tech can change lives. As summarized by Seth Fiengerman his Mashable article:

The same technology that some scoff at or even fear today — artificial intelligence, self-driving cars, voice-powered personal assistants and robotics — could fundamentally transform the lives of the visually impaired in the coming years.

Image source: Twitter (fair use).

Add comment

Security code

How Can You Help?

Get your profile made! We interview for 20 minutes and edit to 2. Audio or video. Buy a six-pack for your team! Email or phone SML at 540-324-7023 to schedule today!

A Wish Goes A Long Way

Amazon logo 8We are a vocational program in the media arts for the deaf, blind and uniquely able. Please support our programs, and check out our Wish List at Amazon.com.

Staunton Media Lab - Copyright 2021