In September 2021, when Josh Miele, an Amazon principal accessibility researcher, got a text from someone at the MacArthur Foundation requesting a phone call, his heart leapt. For anyone in the arts and sciences, a MacArthur Fellowship, known as the “genius” grant, is akin to winning the lottery. You can’t apply for the $625,000 fellowship; it just arrives, mysteriously, out of the ether with a phone call from the foundation.
For Miele, who is blind and has spent his career developing tools to make the world more accessible for people who are blind and visually impaired, a MacArthur grant had long been a fantastical dream.
Joshua Miele, Adaptive Technology Designer | 2021 MacArthur Fellow
“Everybody has things that they imagine might happen to them,” Miele said. “And some things are more realistic than others. You think, ‘Wouldn’t it be nice to get married, have kids, get a great job at Amazon, and, yes, wouldn’t it be nice to get a MacArthur grant?’ Some dreams you can work on and make happen yourself, and some you have to wait and hope for. I won’t deny that one of my long-time fantasies was that I would get a MacArthur Fellowship.”
Assuming the call was to ask his opinion about a possible recipient, he was ecstatic to learn he was among the 25 2021 fellows selected by the foundation. The five-year grant provides money that recipients can use however they want. For someone like Miele, who spent years in a non-profit accessibility thinktank in an endless quest for grant money, the MacArthur news left him with sweaty palms, ringing ears, and pure joy.
“It was extraordinary,” he declared.
A devotion to accessibility
Yet given his life’s work, the grant was a surprise to no one who knows Miele. After graduating from the University of California, Berkeley with a PhD in psychoacoustics in 2003, Miele worked for 16 years at the non-profit Smith-Kettlewell Eye Research Institute in San Francisco as a principal scientist and researcher. He devoted his life to fostering accessibility for the blind and visually impaired.
Stephen Lam/Amazon
He began working with Amazon Lab126, which designs and engineers Amazon devices and services, in 2019. There Miele joined the group of designers and developers who built the Echo family of devices, Kindle, Fire tablets, Fire TV, Amazon Basics Microwaves, and a growing list of innovative products.
His work is with the accessibility team that seeks to make Amazon products intuitively useful for individuals with disabilities.
His aim is to ensure that the designers, product managers, and team members have as clear an understanding as possible of the user experience for customers with disabilities.
“What I’m doing is making sure that the folks who are doing the design understand the customers they are designing for,” Miele explained. “That’s a special and important part of the puzzle because some people doing the work don’t have disabilities themselves and don’t innately have the deep understanding of what the customer requirements are.”
Improving Show and Tell
For example, when Miele joined Lab126, the group was working on Show and Tell, an Alexa feature for Echo Show devices that uses the camera and voice interface to help people who are blind identify products. Employing advanced computer vision and machine learning models for object recognition, Show and Tell can be a vital tool in the kitchen of a customer who is blind or has low vision. A person holds up an object and asks, “Alexa, what am I holding?” and gets an immediate answer.
New Amazon Echo Feature Increases Accessibility (With Audio Description) | Amazon News
The project was stymied, however, by the developers’ struggle to match each product perfectly. If they didn’t get a 100-percent match, the team felt they had failed.
Miele helped the team understand that they needed only to provide useful context, even just a word or two, for a person who is blind or visually impaired to identify the product. The team focused on kitchen and pantry items — things that come in cans, boxes, bottles, and tubes. The goal: Recognize items in Amazon’s vast product catalogue, or if that wasn’t possible, recognize brands and logos that could give the customer enough information to know what they held in their hand.
“If I touch a can of something, I know it’s a can,” Miele explained, “but I don’t know if it’s a can of black beans or pineapple. So, if I’m making chili, and I open a can of pineapple, I’m going to be pretty irritated.”
“I helped design an interaction model that would look for an exact match,” he added, “but if Alexa didn’t get that, it would look for brand recognition. Alexa would look for logos or text that would offer at least some clue as to what was in the can.”
The team created what he called “a gentle letdown curve.” If Alexa can’t find the exact information, Alexa politely apologizes — but doesn’t give up. “I don’t know what it is, but I saw the words `whole black beans’ on the label,” Alexa says. It isn’t an exact match, Miele noted, at least you know what is in the can, which is still incredibly helpful.
Additionally, Miele worked with the team to create audio feedback where Alexa guides the customer to hold the object in the camera’s field of view. Without that, Alexa might be unable to identify the product and frustration could set in.
“Doing things the right way”
Miele also has provided important input into the team’s braille display support and Braille Screen Input technology.
“The most important thing I do is work with my colleagues to test these experiences with real blind and visually impaired customers,” Miele said. “Usability testing is a fairly well known art, but when you’re testing with specialized populations, like people who are blind or deaf, or people using wheelchairs, there are certain adjustments that you want to make to your research protocol so that you are doing things the right way.”
For example, Miele helped the design team understand that it was important that things like consent forms be made accessible.
“Designing research protocols so that people with disabilities are comfortable and properly accommodated is a really important part of research,” Miele said. “The very best way to ensure that the thing you’re designing works for the people you are designing it for is to have people on the team who are going to be customers for that experience.”
A remarkable journey
Miele’s story is a remarkable one, but he discourages others from focusing too much on what happened to him as a child, and instead to consider who he is and what he’s accomplished. He was blinded at age 4 when an assailant in his Brooklyn, New York neighborhood poured sulfuric acid over his head, which blinded and disfigured him.
Barbara Butkus
With the support of incredible parents, teachers and colleagues, he has never thought of himself as being less capable.
He has a full and accomplished life. Miele is married and has two teenage children. He plays the bass guitar and has developed a new braille code for writing jazz chord charts. He is a serious cook and woodworker and loves to hike. He is a proud member of the disability community.
He doesn’t focus on what happened to him because for him, it was simply the challenge life handed him.
“It wasn’t a choice I made,” he said about his positive outlook. “It never even occurred to me to feel sorry for myself. I just wanted to go through life and do the things I was interested in.”
An important epiphany
Miele’s love of science emerged early. He wanted to build rockets, become a space scientist, and explore outer space. Shortly after beginning an undergraduate physics degree at UC Berkeley, he interned with NASA, where he worked in planetary science. It wasn’t until later when he took a job at a software startup named Berkeley Systems, where he worked on some of the first graphical user interface (GUI) screen readers for the blind and visually impaired. That produced an epiphany.
“I realized that the work I was doing in accessibility was both rewarding to me and something that not many people could do at the level I was able to do it,” he recalled. “I thought, ‘There are plenty of people who could be great planetary scientists but there were not a lot of people who could design cool stuff for blind people and meet the needs of the people who were going to use it.’”
Jean Miele
His colleague at Berkeley Systems, Peter Korn, was recruited to join the Amazon Lab126 accessibility team in 2013. One of his first moves was to create an external advisory council of disability experts and he asked Miele to join the council. Korn offered council members a peek behind the curtain at some of the lab’s projects. After one of those sessions, Miele took Korn aside and said, “You know, I’d really love to play a bigger role in helping bring some of those technologies you’re talking about to life.”
Korn responded, “Well, I would love to have you.”
Korn, who has been a colleague and friend for 30 years, was among those who were not surprised at Miele’s MacArthur grant.
“I’ve been incredibly impressed by his creativity, his design sense, his energy and passion, and his inventiveness,” Korn said. “He has a really good sense of what somebody who doesn’t understand technology faces.”
“My superpower”
“I am blind and that is my superpower in this,” Miele said. “I’ve been working in accessibility for a really long time and not just for people who are blind and visually impaired, but for people with all kinds of disabilities. I not only have a fairly good understanding of what some of the basic requirements are for a wide range of disabilities, but I also know how to connect with those communities and bring their voices into the conversation with the designers, developers, and product managers.”
I love accessibility. There’s a social justice aspect to it. You’re working on inclusion and accessibility of information.
After many years in the non-profit sector, Miele is happy with his move to the technology industry.
“I love what I do,” he declared. “I love accessibility. There’s a social justice aspect to it. You’re working on inclusion and accessibility of information. You’re empowering people to do the things they want to do, which is extremely exciting for me. I’m strongly motivated to build cool things for blind people. I want blind people’s lives to be better. I also really like challenges, finding new, fun exciting things to work on and at Amazon, there is absolutely no shortage of cool, interesting, thought-provoking design challenges for accessibility.”
To learn more about Amazon Accessibility, please visit amazon.com/accessibility.
