The only way to get a product right is through user testing.
While assumptions can be useful, ultimately a product needs to be put in the hands of users to see if it solves their core needs.
In fact, pairing user testing alongside market and product fit is the trifecta in building your product strategy. When teams do these things, they are more efficient in every other area of development, accelerating product roadmaps and finding ROI faster.
This knowledge only comes from solving your user’s underlying problems, which comes from testing.
In this guide to user testing, we’ll take you through the types of user testing, when each should be used, and how to use them in a realistic product life cycle.
Types of User Tests
There are two categories of user testing, qualitative vs quantitative.
Qualitative testing is learning about a user’s interpretations and feelings, while quantitative is number-based and measurable. An example might be interviewing a single user about their needs and motivations vs A/B testing a single interface element or user flow.
When conducting your quantitative data, be sure to account for statistical significance, otherwise, your results could be by chance and not a real consensus of users.
For the best results, you need both qualitative and quantitative testing. Here are the top user tests and how they help.
Usability testing
One of the most important and popular tests, usability testing asks a user to complete a particular task with feedback while a moderator watches and documents. These kinds of tests are essential because they put a real human behind a device.
With this, you can learn about their problems, how they navigate, how they search for information, and what they like/dislike about a certain product.
Direct observation
With direct observation, you observe without interaction. Seeing a user easily find information or struggle to complete a sign-up can shed light on how easy your application is to use.
Prototyping
With prototyping, your user is reviewing a prototype, not a full-fledged app. Sometimes these prototypes are low fidelity wireframes, sometimes they’re high fidelity and clickable. Since development is expensive and time-consuming, you don’t want to begin developing without testing some kind of prototype.
A/B testing
Comparing two options to find the superior one can be a fast way to iterate. Is a user more likely to respond to a “Contact Us” CTA or “Let’s Talk”? Does a blue button work better than a red button? With A/B testing, you can get useful data and make rapid-fire decisions.
Treejack testing
Treejack testing is ideal for information architecture testing, which is essential for any complex website or product. You can ask questions like, “if you wanted to find the location nearest to you, what page would you click on?” and then record all interactions.
Eye-tracking and heat mapping
Eye-tracking and heat map testing can be expensive, but it’s worthwhile for some products. With these tests, you can measure where your users are looking and where they’re clicking on your app– to understand if you are providing the right information in the right place.
In-person vs remote
You can conduct all of these tests in-person, remote, or automatically. Remember to take these different environments into account when analyzing data, because different atmospheres can affect users’ opinions and how they interact with technology.
When & how often should a team test?
Like most answers to design-related questions, “It depends.”
Minimum viable products (MVPs) need different testing compared to a legacy app. A single-page web app needs different testing versus a large e-commerce store. So, unfortunately, there is no one size fits all solution.
A robust discovery session is a good practice. This process helps strategists understand the problem space and craft a program to tackle the issue. It also helps designers determine how to test and when.
However, we can provide some general guidelines which are:
Test often
A strong design process starts with discovery and definition, before diving into ideation. Then, repetition is essential.
Ideate – Prototype – Test as needed
Ideate – Prototype – Test as needed
Every time you iterate, you need to ask yourself, “should we test this?” While a single colour swap won’t require standalone testing, a new feature will.
You also have a budget to consider. Testing every small thing will cost you time and money, so have discussions, ask questions, and keep testing on your team’s mind.
Lightweight iterations = quantitative tests
There are always exceptions to this rule, but it’s generally true.
A “lightweight iteration” is difficult to define, but it’s likely something that can be A/B tested and decided upon with analytics.
Heavyweight iterations = qualitative tests
Also exceptions here, but when you do more intensive iterations, like developing new features or launching to a new audience, you need to sit down and talk to users. Understanding not just what they like, but why they like it can be incredibly insightful.
Internal vs external testing
If you’re in the very beginning stages of product development (think wireframes), we recommend testing with internal stakeholders.
Don’t go outside of your organisation until you have high-fidelity prototypes. After all, every product needs stakeholders to sign off before digging into the design.
Real product testing: eMoney
Rooted in comprehensive financial planning, eMoney’s products strengthen client relationships, streamline business operations, and drive overall growth.
eMoney’s goal was to extend its product to a new audience: millennials.
Together with eMoney, we came up with five concepts that could potentially reach their target of millennial users. We then kicked off a series of design sprints, where we built out prototypes for eMoney concepts and tested them with real users. It became clear that users gravitated towards one of the concepts more than others, which was a mobile app that could provide financial wellness advice and complement the existing eMoney software-as-a-service platform.
This was the beginning of what became eMoney Incentive.
We learned through testing and development that Incentive appealed to a wider audience beyond millennials.
Because of this rigorous testing, eMoney’s pilot program created a strong pipeline of interested users, including current users, retirement plan advisors, and employers.
If your product needs a user testing strategy put in place, reach out to the designers and product innovators at DEPT®.
More Insights?
View all Insights
Questions?
Senior Product Strategist