Cultural Bias in Voice AI Design
Jan 11, 2021
8 MIN READ

How to Overcome Cultural Bias in Voice AI Design

The COVID-19 pandemic has accelerated an already-growing adoption rate of voice technology. According to the 2020 Smart Audio Report published by NPR and Edison Research in April,  77% of US. adults changed their routine because of the COVID-19 pandemic, including an increase in the rate of multiple daily use of voice assistants. Since that time, voice assistant usage has continued to climb along with new use cases for voice technology.

77% of US. adults changed their routine because of the COVID-19 pandemic

As crucial influencers and leaders of the modern age, voice technology companies should become more sensitive to the potentially harmful biases that have been identified around AI and machine learning. Biases inherent in the way the technology is deployed will begin to have greater and greater influence on many aspects of our lives as advances in the technology extend its use to critical use cases, such as determining the results of an English proficiency test, dictating meetings, and influencing hiring decisions.

Companies hoping to remain competitive in the burgeoning voice technology market, should already be addressing the biases identified in current systems and guard against new instances. 

For instance, Claudia Lloreda Lopez’s article in Scientific American, highlights the problem of accent bias. She notes that she tried to get Siri to recognize the correct pronunciation of her name, After several attempts, she finally gave-up and opted for a more “Americanized” version. 

“Having to adapt our way of speaking to interact with speech recognition technologies is a familiar experience for people whose first language is not English or who do not have conventionally American-sounding names. I have even stopped using Siri because of it,” Lopez wrote.

Accent bias is only one aspect of the challenges for voice assistants in a multi-lingual, multi-gendered, and increasingly diverse world. The question voice AI providers need to ask themselves today is: “Does your technology safeguard against racial bias, gender bias, accent bias, age bias, and regional bias?” If not, what are you doing to ensure your consumers won’t be abandoning your product because they aren’t having the personalized, convenient experience you promised?

“Having to adapt our way of speaking to interact with speech recognition technologies is a familiar experience for people whose first language is not English or who do not have conventionally American-sounding names.”

Claudia Lloreda Lopez

Racial bias in voice AI today

In the annals of technology history, there are plenty of examples of racial bias. Remember the HP webcam in 2009 that would only follow a white person, but not a black person using facial recognition software? Or the stories of  Amazon using AI to parse through resumes for potential employees to hire with algorithms skewed towards hiring white males. Although these cases are a few years old, current voice AI technologies are not exempt from the same issues.

Researchers at Stanford University published “Racial Disparities in Automated Speech Recognition” in 2020, in which they reported findings from researching five ASR systems. The researchers found that the average word error rate (WER) for white subjects was 19% compared to 35% for black subjects speaking in African-American Vernacular English. 

These word error rates clearly illustrate the lack of data for a dialect of English commonly used in a large urban area of California and other cities, and may point to inherent bias in technology development based on who is creating speech models and testing them. 

Researchers found that the average word error rate (WER) for white subjects was 19% compared to 35% for black subjects speaking in African-American Vernacular English.

Conducting word error rate tests will help you determine which data is missing from your voice AI platform and identify areas of bias. Once identified, additional audio clips can be added to the data set. The Stanford researchers used the Corpus of Regional African-American Language. Then check the error rates for race. This came method can be used to test other biases by interchanging the testing corpi.

Gendered technology and gender bias

Gendered technology is technology that is associated with a certain sex. In our current society, many things are gendered, including career associations. For example, most people associate a doctor with a male and a nurse with a female. Even our household items are gendered and advertisers perpetuate the bias by depicting men using a grill  or power tools and women using a blender or oven in the kitchen. Voice technology is another area where helpful devices are gendered to appeal to target audiences.

Voice technology is the newest version of a gendered technology that has infiltrated our psychies as we accept the gendered norms of female names and voices as our helpful assistants. According to Katharine Schwab of Fast CompanyGoogle chose a female voice for its voice assistant because female voices sounded better (in their opinion) than the male voices that the algorithm was trained on. 

Voice technology is the newest version of a gendered technology that has infiltrated our psychies as we accept the gendered norms of female names and voices as our helpful assistants.

According to a research study by Voicebot.ai, U.S. consumers prefer synthetic female voices over male synthetic voices. When comparing human voices to synthetic voices Voicebot found that human male voices are actually preferential to any human or synthetic voice of any gender.

These findings bring into question the quality of the male synthetic voices in the testing environment. Is it possible that the preference for female synthetic voices over male synthetic voices is the result of less training data for the male synthetic voices—resulting in a lower quality voice synthesis model? It’s also possible that the data is skewed simply as a result of the labeling of voices as male or female, activating inherent bias among the respondents.

In terms of perpetuating gender bias, the automatic default to female voices for voice assistants is particularly problematic. The function of the voice assistant is to answer questions and perform routine tasks, similar to a personal assistant or secretary, a career long associated with women serving men in higher positions within a company. 

According to the Brookings article, “How AI bots and voice assistants reinforce gender bias,” Apple’s voice assistant, Siri, used to respond with “I’d blush if I could,” when told, “You are a bitch,” and Amazon’s Alexa replied, “Well, thanks for the feedback,” when told ‘”You’re a slut.” Although these responses have since changed, it’s important to acknowledge that these assistants were developed with the expectation that they would be verbally abused and their responses only changed when they started to receive negative attention. 

While developers and brands may want their voice assistants to be helpful and polite, programming them to not respond or respond with a neutral statement, creating a submissive personality with a female voice remains problematic.

It’s important to acknowledge that these assistants were developed with the expectation that they would be verbally abused and their responses only changed when they started to receive negative attention. 

When the female-oriented voice assistant refuses to respond, it may send a signal to the user that it’s ok to say rude and abusive things to a female-sounding voice. The dynamic between a voice assistant and the user was established early by mostly male engineering teams. Perhaps part of the solution to solving these inherent challenges is to start with more diverse teams at the development stage. 

Gender bias and the absence of women in engineering

As of 2018, only 12% of voice AI leading researchers identified as women. Although women are entering STEM careers at higher rates in recent years, they are still not advancing to managerial and senior positions at the same rate as their male counterparts. According to the Harvard Business Review, while there are many eager and capable women and minorities willing to take on higher level roles, they are rarely advanced to leadership positions. 

“The visibility of one’s technical skills influenced how valuable specific employees were perceived to be. This presented a conundrum for women. Since women were less likely to be represented on high-visibility technical projects, they were also less likely to be seen as having the kind of skill set most valued by leaders.”

Co-authors Shelley J. Correll and Lori Mackenzie.

The article concludes that women in STEM are viewed with less authority and possibly treated as engineering team members and not considered leadership material. The current pandemic has further impacted the role of women in the workplace. In a virtual workplace environment, three in five female employees feel like a promotion is more difficult to gain, and many are finding it harder to fully participate because they feel ignored or overlooked.

Women in STEM are viewed with less authority and possibly treated as engineering team members and not considered leadership material.

Furthermore the 2020 Women in the Workplace Report by McKinsey found that women—particularly women of color—are underrepresented at all levels and because of COVID-19, they are more likely to have been laid off, stalled in their careers and any potential advancement. In this environment, women are less likely to be part of voice AI teams and less likely to have influence over the gender biases now inherent in the technology.

The McKinsey report also noted that company profits and share performance can be close to 50% higher when women are well represented at the top. Additionally, women bring a more inclusive environment to work by championing for more gender and racial diversity and serving as mentors or sponsors to other women. 

“More than 50% of senior-level women say they consistently take a public stand for gender and racial equity at work, compared with about 40% of senior-level men.”

McKinsey Report

It’s important to note here that women are responsible for between 70% to 80% of consumer purchase decisions. It stands to reason that if you want women to buy your product, the product should be developed from their perspective and not solely the perspective of male engineers.

The influence of voice on our world

Frantz Fanon says in his book, Black Skin, White Masks“To speak is to exist absolutely for the other.” Speaking is an integral part of everyone’s individual identity, it carries the past and present, represents where one has lived, and who they’ve interacted with. Every time someone speaks, they are sharing a little part of themselves.

In the context of voice assistants, this statement takes on a little different meaning. As voice AI technology continues to improve and the role of the voice assistant becomes ever more personal, the biases currently inherent in these technologies continue to define who we are and how we are heard and acknowledged.

For voice AI developers, choosing between labeled and unlabeled data or using a combination may have a direct impact on the biases carried by your voice assistant. While labeled data from an existing dataset may be easier to use, these labels are sometimes biased or simply don’t hold all the information. For example, you could label Beyonce as a woman, or she could be labeled as a black woman, a singing black woman, or a Texan singer and actress. These are all acceptable labels.

While not all those labels are relevant to developing an unbiased voice assistant, developers should be focused on three main areas of bias, namely race, gender, and accent. Still, narrowing the fields may not solve all the challenges, since the difference of white men vs. white women and white women vs. mixed raced women have all been shown to affect accuracy rates, according to Joan Palmiter Bajorek.

While not all those labels are relevant to developing an unbiased voice assistant, developers should be focused on three main areas of bias, namely race, gender, and accent.

If you are using unlabeled data and labeling it on your own, the challenge is to not introduce your own (even subconscious) biases. One solution may be to ensure you are working with a diverse team and comparing labels. If you’re using an algorithm to label data, be aware of biases that may already exist in those pre-labeled algorithms.

The number one takeaway for developers of voice assistants, is to raise your own awareness and the awareness of your team of the biases that already exists and to work diligently to eliminate bias. In other words, don’t assume all your labels are correct and unbiased.

Moving beyond bias to diversity and inclusion in voice design

Eliminating bias should be approached from both the standpoint of the technology and hiring practices. Hire minorities and diversify your teams with non-programmers who have humanities backgrounds. Provide specific training and education to your engineering teams. There are tests such as Harvard’s Project Implicit and programs such as Equal AI that provide workshops to uncover unconscious biases. 

On the technological side, be meticulous with the data your algorithms are using and include more diverse data. In the testing phase, use a diverse consumer focus group or internal teams. Additionally, look at how your voice assistant performs in different environments through focus groups. 

However you approach eliminating biases in voice technology, begin with acknowledging the problem and then take steps toward a world where diversity and inclusion are communicated through a technology that will soon be an integral part of every aspect of our lives.

Jade Roberts

Jade Roberts started studying voice AI as an integral part of her Cognitive Science with Linguistics program at The University of Illinois. She is a member of Women in Voice and a passionate learner of technology’s effect on the human psyche and individual identity.

You Might Also be Interested In

Interested in Learning More?

Subscribe today to stay informed and get regular updates from SoundHound Inc.

Subscription Form Horizontal