When developing a voice user interface for your product, make testing and user research a key element in your workflow. Throughout the process, be sure to clearly communicate with all the stakeholders and employ a variety of types of research, including:

• Focus groups
• In-lab testing
• Remote testing
• Customer surveys
• AB testing

Here are a dozen tips and tricks to help you avoid the common pitfalls of usability testing and a review of when to use each type of research:

1. Focus groups

Focus groups are a good way to test out an idea to determine if your customers and prospects have a need for what you’ll be selling. Leading a focus group requires a certain amount of detachment and an ability to read people’s expressions and body language. It’s more than simply recording what they say, it’s noting their sentiment and the things they are communicating non-verbally.

Targeted focus groups are ideal for showing a product or function to a group of people and watching their reactions. If they show excitement and you determine the market is ready for the product or service, you can move forward with product development. 

2. In-lab usability testing

In-lab usability testing involves bringing someone from within your audience group to the lab, giving them a prototype or an early build of your product, and asking them to use it. The key to getting good results from in-lab usability testing is to resist guiding your participant. Instead, simply observe.

The researcher’s most important role during usability testing is to watch people using the product and record their experiences accurately. If the observer is inserted into the process, the results won’t accurately reflect the user’s experience with the product.

3. Remote testing

Remote user testing allows you to expand research to include a vast panel of users in any location. You can use this testing model to discover regional sentiment, like finding out how people on the East Coast of the US feel about their experiences on the Baltimore Ravens’ website. 

Using remote testing, you can gather data without physically traveling to the location. In addition, the testing cycle is faster and more cost effective than assembling people in a physical location. Using remote testing, results can be collected and insights garnered more quickly than with any other type of qualitative research. 

4. Customer surveys

Surveys are another good way to gauge interest and measure engagement with new features or functionalities in a product. Surveys can be used in many different phases of design and development. Pre-launch surveys have the benefit of providing benchmark information and post-launch surveys can supply immediate information about user satisfaction with your product. 

For instance, If you’ve just released a new feature or a major update, and you want to know the satisfaction level of your customers, begin with a user survey. If your customer satisfaction scores come back less than five out of five stars, you’ll know where to put your future efforts.

In addition to multiple choice questions, give your respondents an opportunity to write in and tell you what’s working and what’s not. Customer surveys can serve as your meat and potatoes measuring instruments for pre- and post-launch customer sentiment ratings.

5. AB testing

AB testing is pretty straightforward. Generally, we give users two choices and ask them which they like better. “Do you like the black or the dark blue design?” It’s really just a matter of gathering preferences for one thing over another. 

However, the real value in this type of testing for user research is taking it a step further to ask, “why?“. Your ability to take a deeper dive will depend on whether you’re conducting the research in person and if you have the right tools. Done right, AB testing can get at not just the “what” but also the “why.“

6. Involving all stakeholders

Every company has a different testing process, but the one thing that never changes is the need for regular communication with key stakeholders. Without proper communication, you may find yourself involved in a testing or research project that is completely useful to one part of the organization, but that doesn’t meet the needs of another key group.  When this happens, research teams often find themselves doubling their efforts in order to meet everyone’s needs.

Generally, the reason for a research mismatch between organizational groups is the result of a lack of communication. To avoid that, gather all the stakeholders together to set the goals of the testing and get on the same page. Involve everyone in the testing protocols and get approval for the testing script.

Even when it feels like you might be annoying people, it’s better to over communicate and get buy-in before the testing than to skimp on communication and have to double-up on testing efforts later. Obviously, your testing goals should be aligned with the business goals of the company.

For the best research outcomes, risk over-communicating to make sure you’re all on the same page around the goals. Communicate, communicate, communicate. It’s just like in relationships—you can’t get around it. Success is all about communication.

7. Testing your user population

In addition to lack of communication, people tend to not think about the sample enough—the people who will be taking part in the testing or survey. It’s important to always aim for testing the population that represents your actual user population.

For instance, if you’re testing a site that’s developed for engineers with very specific and tech heavy content, you won’t want to test it with the general population. On the other hand, if you send that same site to the right group of engineers, you’ll get appropriate feedback about what makes sense and what doesn’t. When there’s not enough time dedicated to determining the actual sample, the results reflect it.

8. Discovering the “why”

The most common pitfall of user research is failing to find out why your respondents like one design over the other or why they felt frustrated. There’s always a little bit more going on than what’s on the surface and to find out, researchers need to know how to probe for more information. 

It’s most helpful for designers to understand not just what people prefer, but why they prefer it. It’s possible that the feedback they receive will lead to a complete design change, different from the ones presented to the users. The reasoning behind their experiences and opinions is always infinitely more important than simply understanding preferences and watching what people are doing. You’ve got to dig a little deeper.

9. Allowing for discovery

Researchers are often trapped into explaining the tech too much without allowing the testers to try it out on their own. Because the researcher wants the testers to understand what they’re working with, they tell them a little bit too much without waiting for the users to understand the operation themselves.

If you haven’t designed a feature that testers can figure out how to use on their own, then your teams will need to figure out how to make it better—that’s the goal of the research. 

10. Knowing when to help

When you believe in the product, it’s often difficult not to intervene because you want the user to love it and to be able to interact with it smoothly. In some ways, that same instinct can muddy the research results when you have a prototype that you don’t feel is easy to use or doesn’t, in your opinion, get the message across. For researchers, the key to good research is to never project your own opinions onto the test participants. Leave the users to discover the usability of the product on their own. 

If your users get stuck during the testing session, you can prompt them to get started by instructing them to push a button or whatever the entry point is. However, it’s important that you include that intervention in your notes. It doesn’t make sense wasting the entire session with a test subject who can’t get past the first step. That’s not good research. That’s sloppy and it’s a waste of time.

Always give users the opportunity to try it first and give them time to figure it out on their own. If they can’t and it’s obvious that they’re going to give up, step in to get them started and then get the rest of the feedback.

11. Don’t skimp on research

The worst trap of user testing is deciding not to do any at all. As a researcher, you might find yourself advocating for testing when design teams would rather not have to deal with the fallout from possible negative feedback. There’s a tendency with designs that are complex to opt out of testing in the near term and to try to push it to later in the process, especially if testing requires complicated prototyping.

While this approach may be convenient for the moment, it will likely end up costing everyone a lot of time and wasted effort. Getting feedback early, even if it’s inconvenient feedback, saves developers a lot of time fixing features that users don’t want, don’t need, or can’t access. 

12. Accurate reporting

Give feedback to your design team as straight as possible. Don’t editorialize or fluff any egos or sugarcoat anything. If something is bad, your design team needs to know it. They may wish you would sugarcoat things a little bit more, but you can’t do that. Your job is to report what you really found. Back your statements up with clips of people being frustrated—or being delighted—so that stakeholders can see it firsthand.

Become a research evangelist by constantly reminding stakeholders and engineers to get their ideas in front of regular users. Even testing a product with one user is better than no user research at all, and any data is better than no data. There’s an old statistic that says that if you do five usability tests, you can uncover 80% of your issues, and that’s so much better than uncovering none of your issues. It begins with convincing the people in your organization that you need to test and that you need to be doing research. 

At SoundHound Inc., we have the expertise, the tools, and testing protocols you need to develop a voice assistant for your product, service, or mobile app. Explore Houndify’s independent voice AI platform or register for a free account. Want to learn more? Talk to us about how we can help you bring your voice strategy to life.

Val Schweisberger is a researcher, a nomad, an ok home cook, and a nerd. You can find her either wandering around a mountain or watching great TV.