Peter Xu
UX designer

Zoom Accessibility Redesign

Better video meeting for deaf / hard of hearing users

Image alt tag
Zoom Accessibility Redesign

Background

This is a 3 month long project during which I worked with 3 fellow MSHCI students at Georgia Tech to improve the video conferencing experience for deaf and hard of hearing Zoom users in small-mid size business settings.

My Role

My main contribution for the research aspect was reaching out to the deaf / hard of hearing users and conducting interview / prototype testing. I also redesigned the main meeting flow of Zoom to incorperate new accessibility features.

Zoom Accessibility Redesign

Mind the gap!!

Why this project?

While use of video conferencing tool has blown up since the pandemic, many minority groups have issues with the new tech, such as deaf or hard of hearing (DHH) users, as such a new medium brought new challenges. How do they communicate back and forth with others? How do they know who's speaking? As remote working has become more popular, the need for a more accessible video conferencing experience for DHH is also rising, and we wanted to fill some of this gap through this class project.

So how do DHH do video calls???
Zoom Accessibility Redesign

Survey

Through early research, we learned most DHH involved work meetings have sign language interpreters, thus we reached out to both groups through Reddit, Discord and institutions to fill out a survey. The questions were mainly qualitative, and produced preliminary insights on their general attitue towards video calls.

Results showed that Zoom was the most used platform, and many described their Zoom experience as "frustrating", with frictions of use mostly involving software limitation, communication difficulty, as well as lack of awareness from hearing people.

Observation

I also conducted observation on how DHH users did video calls to understand the experience, communication dynamics, and see what challenges hearing people might face when communicating with DHH people.

I had the chance to hop in a Zoom call with DHH gamers playing World of Warcraft, and observed a 3 hour gaming session. I also chatted with many of them through text over breaks. Though the context was gaming, many insights are transferrable to broader situations, including professional settings.

Image alt tag

DHH users use additional avenue forinformation intake. For instance, they used add-ons that display fight strategy as text in game.

Image alt tag

Getting attention is hard for DHH users, especially when there are many people in the call. Their last resort is to spam waving their hand at the camera, as the change of color grabs attention.

Image alt tag

There was usually a 15-20 second communication delay when I was chatting with the DHH gamers through text.

Zoom Accessibility Redesign

Interview with ASL interpreters

As many DHH users rely on interpreters, learning about their pain points was also important in improving the overall experience. We reached out to 3 ASL interpreters through connections at Georgia Tech and synthesized their pain points to 4 main categories.

Image alt tag

Image alt tag

Image alt tag

Image alt tag

Zoom Accessibility Redesign

Text interview with DHH

We reached out and conducted semi structured interviews with 10 DHH users through Discord about their video call experiences. The questions mainly focused on communication dynamics, online meeting procedure, and etiquitte. We discovered that a good deal of DHH's painpoints coincide with those of the interpeters.

Image alt tag

Image alt tag

Image alt tag

Image alt tag

Where could we improve on?

Using the data, we created personas and journey maps for interpreters and DHH users to consolidate the two groups' goals, frustrations, and emotions during online meeting. By doing so, we were able to identify where we could introduce design interventions along the users' journey.

Zoom Accessibility Redesign
Image alt tag

Image alt tag

Where does it all line up??

As we have already discovered, some of DHH and interpreters' pain points overlap. We tried to address as many of these bundled problems as possible when brainstorming design ideas.

Image alt tag

Image alt tag

Image alt tag

Image alt tag

So.. how might we empower DHH during video meetings & make engagement more organic?

Zoom Accessibility Redesign
Image alt tag

chit chat

Image alt tag

video check

Image alt tag

DIY screen w/ more interactions

Zoom Accessibility Redesign

Sketch Feedbacks

We discussed the sketches with DHH and interpreters to learn about:

  • If they could understand those ideas

  • Their general reactions

  • How useful they think these are

  • Potential edge cases

Zoom Accessibility Redesign

Wireframes

Main functionalities of the new design:

  • Meeting host can upload glossary, etc. to meetings

  • Video check before entering the meeting

  • Waiting room repurposed to allow chit chat

  • Interactive conversation elements and actions geared towards DHH + interpreters

  • Resizable video feed for better focus

  • New animation for seeking attention

Zoom Accessibility Redesign

High Fi Design

Meeting Assist

Through Meeting Assist, hosts can help participants, especially the interpreters, prepare better for the meeting. Host can upload presentation files and glossaries etc, to be sent out with the meeting invite link. The feature now has its unique entry point instead of being part of "Edit meeting", as testing showed it had low discoverability and was often overlooked.

Zoom Accessibility Redesign

Chit chat

Leveraging Zoom’s waiting room, we aimed to make the wait time more meaningful, during which DHH users and interpreters can discuss meeting context, communication expectations, etc. Self identified interpreters and DHH users have icons next to their names, making them easier to identify each other. DHH user can also choose not to disclose identity for privacy. User initiates chit chat by clicking on others’ profiles to send out invite, and they will enter a separate private room.

Zoom Accessibility Redesign

Video Check

Users can run a video check before entering the meeting, similar to how you can test your audio device. When camera is turned on, the system automatically rates the user’s video quality based on three criteria. User will try adjust accordingly based on prompts, and when all checks are passed they can enter the meeting. We took feedback and gave the option to skip the check to reduce friction if it somehow couldn't be completed, or if users didn't want to do it.

Zoom Accessibility Redesign

The Main Meeting

More interactive elements are added to make it more streamlined, including auto generated transcript. User can quote directly from it or the chat feed to ask questions. Content in previously upoaded glossary is highlighted in the transcript if spoken out, interpreters can hover over to check its definition and sign.

Users can move and adjust others' video feeds on screen, while all participants can still be seen and interacted with through gallery view.

In research, we learned DHH users have difficulty seeking attention, and current emoji like "wave hand" wasn't a reliable solution as it was static and often overlooked. New reactions such as “Shine” and “Shake” were added, allowing more obvious visual cues to be sent out. The recipient sees the visuals on the sender's video feed in his own screen.

Not finished yet...

We tested out the prototype again with our DHH and interpreters again. The overall reaction was extremely positive as they believed some of the new features in the redesign were able to address issues they encoutered often. They also offered many detailed suggestions on how the redesign can be further improved.

Image alt tag

Image alt tag

Image alt tag

Image alt tag

The meaning of “Available identities” was confusing to new users. So we plan on changing the wording, and adding explanation tooltips.

Image alt tag

Users appreciated the video check and wanted it during meeting as well to fix potential issues, so we intend to embed the function into the main meeting.

Image alt tag

Users found it hard to notice the interactive elements in the transcript. To address this, we plan on using more visual cues such as higher contrast to make them more apparent.

Zoom Accessibility Redesign

What did I learn?

  • Reach out to the user base ASAP! A big mistake we made was not finding DHH / interpreters earlier. We did so after we nailed down the topic and it was quite late, while the medical nature further narrowed down the user base.

  • Try different methods of contact. Not only starting late, we initially struggled in finding users willing to share their experience with us. DHH users in traditional online forums such as Reddit were generally more reluctant to be contacted to take surveys, so I had to resort to Discord, and it was hugely successful!

  • Pilot the research tasks with users. A good example is when we found DHH users to fill out our survey, many of them advised us to change the wording of the questions to be more simple because DHH generally have worse understanding of English grammar due to difference in education, and that English could be considered a second language for many of them since most of them would be more fluent in sign languages. This was a point we didn't realize and we hope similar situations could be avoided in the future if we pilot the research methods more.

Lastly, big shoutout to my teammates Jensen, Xiao, and Elisa for their hard work throughout the project! This redesign wouldn't have been realized if it weren't for all of their help.