Mind the gap!!
Peter Xu
UX designer
Zoom Accessibility Redesign
Better video meeting for deaf / hard of hearing users
Background
This is a 3 month long project during which I worked with 3 fellow MSHCI students at Georgia Tech to improve the video conferencing experience for deaf and hard of hearing Zoom users in small-mid size business settings.
My Role
My main contribution for the research aspect was reaching out to the deaf / hard of hearing users and conducting interview / prototype testing. I also redesigned the main meeting flow of Zoom to incorperate new accessibility features.
Why this project?
While use of video conferencing tool has blown up since the pandemic, many minority groups have issues with the new tech, such as deaf or hard of hearing (DHH) users, as such a new medium brought new challenges. How do they communicate back and forth with others? How do they know who's speaking? As remote working has become more popular, the need for a more accessible video conferencing experience for DHH is also rising, and we wanted to fill some of this gap through this class project.
So how do DHH do video calls???
Survey
Through early research, we learned most DHH involved work meetings have sign language interpreters, thus we reached out to both groups through Reddit, Discord and institutions to fill out a survey. The questions were mainly qualitative, and produced preliminary insights on their general attitue towards video calls.
Results showed that Zoom was the most used platform, and many described their Zoom experience as "frustrating", with frictions of use mostly involving software limitation, communication difficulty, as well as lack of awareness from hearing people.
Observation
I also conducted observation on how DHH users did video calls to understand the experience, communication dynamics, and see what challenges hearing people might face when communicating with DHH people.
I had the chance to hop in a Zoom call with DHH gamers playing World of Warcraft, and observed a 3 hour gaming session. I also chatted with many of them through text over breaks. Though the context was gaming, many insights are transferrable to broader situations, including professional settings.
DHH users use additional avenue forinformation intake. For instance, they used add-ons that display fight strategy as text in game.
Getting attention is hard for DHH users, especially when there are many people in the call. Their last resort is to spam waving their hand at the camera, as the change of color grabs attention.
There was usually a 15-20 second communication delay when I was chatting with the DHH gamers through text.
Interview with ASL interpreters
As many DHH users rely on interpreters, learning about their pain points was also important in improving the overall experience. We reached out to 3 ASL interpreters through connections at Georgia Tech and synthesized their pain points to 4 main categories.
Text interview with DHH
We reached out and conducted semi structured interviews with 10 DHH users through Discord about their video call experiences. The questions mainly focused on communication dynamics, online meeting procedure, and etiquitte. We discovered that a good deal of DHH's painpoints coincide with those of the interpeters.
Where could we improve on?
Using the data, we created personas and journey maps for interpreters and DHH users to consolidate the two groups' goals, frustrations, and emotions during online meeting. By doing so, we were able to identify where we could introduce design interventions along the users' journey.
Where does it all line up??
As we have already discovered, some of DHH and interpreters' pain points overlap. We tried to address as many of these bundled problems as possible when brainstorming design ideas.
So.. how might we empower DHH during video meetings & make engagement more organic?
chit chat
video check
DIY screen w/ more interactions
Sketch Feedbacks
We discussed the sketches with DHH and interpreters to learn about:
If they could understand those ideas
Their general reactions
How useful they think these are
Potential edge cases
Wireframes
Main functionalities of the new design:
Meeting host can upload glossary, etc. to meetings
Video check before entering the meeting
Waiting room repurposed to allow chit chat
Interactive conversation elements and actions geared towards DHH + interpreters
Resizable video feed for better focus
New animation for seeking attention
High Fi Design
Meeting Assist
Through Meeting Assist, hosts can help participants, especially the interpreters, prepare better for the meeting. Host can upload presentation files and glossaries etc, to be sent out with the meeting invite link. The feature now has its unique entry point instead of being part of "Edit meeting", as testing showed it had low discoverability and was often overlooked.
Chit chat
Leveraging Zoom’s waiting room, we aimed to make the wait time more meaningful, during which DHH users and interpreters can discuss meeting context, communication expectations, etc. Self identified interpreters and DHH users have icons next to their names, making them easier to identify each other. DHH user can also choose not to disclose identity for privacy. User initiates chit chat by clicking on others’ profiles to send out invite, and they will enter a separate private room.
Video Check
Users can run a video check before entering the meeting, similar to how you can test your audio device. When camera is turned on, the system automatically rates the user’s video quality based on three criteria. User will try adjust accordingly based on prompts, and when all checks are passed they can enter the meeting. We took feedback and gave the option to skip the check to reduce friction if it somehow couldn't be completed, or if users didn't want to do it.
The Main Meeting
More interactive elements are added to make it more streamlined, including auto generated transcript. User can quote directly from it or the chat feed to ask questions. Content in previously upoaded glossary is highlighted in the transcript if spoken out, interpreters can hover over to check its definition and sign.
Users can move and adjust others' video feeds on screen, while all participants can still be seen and interacted with through gallery view.
In research, we learned DHH users have difficulty seeking attention, and current emoji like "wave hand" wasn't a reliable solution as it was static and often overlooked. New reactions such as “Shine” and “Shake” were added, allowing more obvious visual cues to be sent out. The recipient sees the visuals on the sender's video feed in his own screen.
Not finished yet...
We tested out the prototype again with our DHH and interpreters again. The overall reaction was extremely positive as they believed some of the new features in the redesign were able to address issues they encoutered often. They also offered many detailed suggestions on how the redesign can be further improved.
The meaning of “Available identities” was confusing to new users. So we plan on changing the wording, and adding explanation tooltips.
Users appreciated the video check and wanted it during meeting as well to fix potential issues, so we intend to embed the function into the main meeting.
Users found it hard to notice the interactive elements in the transcript. To address this, we plan on using more visual cues such as higher contrast to make them more apparent.
What did I learn?
Reach out to the user base ASAP! A big mistake we made was not finding DHH / interpreters earlier. We did so after we nailed down the topic and it was quite late, while the medical nature further narrowed down the user base.
Try different methods of contact. Not only starting late, we initially struggled in finding users willing to share their experience with us. DHH users in traditional online forums such as Reddit were generally more reluctant to be contacted to take surveys, so I had to resort to Discord, and it was hugely successful!
Pilot the research tasks with users. A good example is when we found DHH users to fill out our survey, many of them advised us to change the wording of the questions to be more simple because DHH generally have worse understanding of English grammar due to difference in education, and that English could be considered a second language for many of them since most of them would be more fluent in sign languages. This was a point we didn't realize and we hope similar situations could be avoided in the future if we pilot the research methods more.
Lastly, big shoutout to my teammates Jensen, Xiao, and Elisa for their hard work throughout the project! This redesign wouldn't have been realized if it weren't for all of their help.