Google Translate for the Visually Impaired
We’re very pleased to present a project we’ve been working on for some time at the Staunton Media Lab. We call it The Google Translate Trick. These are instructions for using Google Translate to read any printed document. For the blind and visually impaired, this technique has the potential to make life a little easier by reading things to you such as street signs, menus, legal documents, written instructions, books, magazines, newspapers, and almost any other printed document—in up to 90 different languages! Wow! That’s life-changing.
The Google Translate Trick is a crude scanner-to-voice system that is very close to being a great piece of assistive technology. With a couple tweaks, it could open up entire libraries to the blind without the need for embossing into Braille or producing talking books. Right now, it can be an incredible facilitator for the visually impaired who have enough sight to position the camera. They can use it to have legal documents or other important information—such as prescription labels or doctor’s instructions—read to them by their phones.
We have been trying to work out the bugs in this presentation, but there are just too many issues to deal with. We feel it is important to present the information we have and then continuously improve it. Below is a 2-minute video of the technique, followed by step-by-step instructions, and then an edited, audio-only version of the instructions. We sincerely welcome your feedback in the comments section about your experiences attempting The Google Translate Trick and your suggestions for improving our instructions. Thank You!
Step 1: Download and Install Google Translate
Google Translate is available as an app for Android phones, iPhones, and Windows phones, and as a Chromebook or Firefox extension. It works a little differently in each of these environments, but these instructions should work well in all environments. However, it is much easier to position the camera on a mobile phone than it is to take a picture with a laptop or notebook computer.
Step 2: Open Google Translate, select the language of the document you want to read and the language you want it translated into.
Here’s the funny thing: Google Translate will not let you select the same language for both input and output. Select the language of the document you want to read and if you want it read to you in the same language (English/English, for example), ignore the output language.
Step 3: Choose the camera as the source of the input document.
When you open Google Translate, it gives you four ways to input language: Typing, Writing, Speaking, and Image Scan. Image scan is represented by a camera icon. Screenreaders used by the blind will tell you where the camera icon is and will allow you to select it.
Step 4: Position the phone over the document to be read.
Once the camera is selected as the input source, the screen goes into camera mode, and the message “align text” appears. The camera must be held above the document to be read so that the text to be read appears upon the screen. Aligning the camera is nearly impossible for a blind person to accomplish, though it is very possible for those who have some small amount of vision but cannot read standard-size text. The phone should be held on a flat plane parallel to the surface of the document, at a distance of about 12-18 inches from the document. Practice makes perfect. If you can’t read it the first time, try again.
Once you have been successful reading a page from a book, every page of that book will have the same approximate camera alignment. Once you have successfully used Google Translate on your phone to read your computer screen to you, holding the camera at about the same distance from the screen next time will likely work. This trick of reading the computer screen with Google Translate will often work when your normal screen readers, such as JAWS or NVDA, get stuck. If you can’t progress on a computer because none of your screen readers work, try The Google Translate Trick to read what’s on the screen—it can be a real time saver.
You can also have friends position the phone for you. Friends can help you capture a document that you want to refer to later or repeatedly, such as a restaurant menu. In the future, we hope Google Translate will include an alignment feature using sounds or vibrations to indicate when the document is in alignment. A beeping noise or a pulsing vibration that gets faster as alignment approaches would be extremely helpful in guiding blind users to perfect alignment without assistance.
Step 5: When the document is aligned, double-tap the Scan icon to take a picture.
Again, this is very difficult for the blind to accomplish. A single touch is all that’s required for sighted users, but blind users using a screen reader have to tap twice. The tapping itself is usually enough to put the camera out of alignment. The frustration of trying to accomplish both alignment and tapping without sight will deter many blind users. However, it is worth the effort because mastering these techniques in theory makes every single printed document in the world readable by a blind person without conversion to Braille. It opens up the whole world of books and magazines and legal documents and written instructions to the blind and visually impaired. You will have trouble reading things like newspapers that have multiple columns, or artistic lettering in many advertisements, but the technology is expected to improve very quickly.
Step 6: Double-tap the “Select All” button to have the entire scanable document translated.
Google Translate lets you choose to read some of the words or the entire document. Tapping the “Select All” button twice instructs it to read whatever it has scanned. If you have a screen reader on, Google Translate will immediately begin reading the scan to you. If it failed to make a decent scan, use the “Cancel” button (back arrow icon) to go back Step 4 and try again.
Step 7: Select the right arrow button to access the translation.
If you are not using a screen reader, selecting the right arrow will take you to the translation, with the input language on top, and the output language down below. You can ask it to read the text to you in either language. You will most likely want the input language.
Step 8: Double-tap the microphone icon to have Google Translate read the document aloud.
When you select to have the text spoken to you, VoiceOver begins reading. It’s a pretty good screen reader, and you can control, to some extent, the voice it uses to read to you. For example, you can get English read back to you in a variety of accents, by a male or female voice, and at a fast or slow pace.
Step 9: The translation of the document can be saved for later use.
Google Translate will allow you to store the translated document for later access. However, it will only let you save the output document, not the input document. So if the output document is in the wrong language (Latin, for example), you can toggle the language to English (or your preferred language), and then save it by pressing the Star icon. [PROBLEM: Note that if you are using a screen reader, it is impossible to get to the screen that allows you to save the translated text. The only way to exit is to use the “Cancel” or back arrow key to back out.]
So that’s our first take on The Google Translate Trick. At the Staunton Media Lab, we are developing more detailed, step-by-step graphics to accompany these instructions. We are also improving our instructions with every comment we receive, so thank you for your help.
Coming soon: The Google Translate Trick for the Hearing Impaired. With Google Translate, deaf or hard-of-hearing people who can read can now carry on a conversation with anyone, anytime, anywhere, in any language, whether or not the other person knows sign language. Amazing! Stay tuned!
Images and multimedia courtesy of the Staunton Media Lab.