Creating Effective User Testing Questions

[vc_row][vc_column][vc_column_text]If you have great questions to ask your users, you will get great answers to apply to your testing goal. If you work for a business that has limited resources, getting the best feedback is important the first round testing, thus saving money.

Understand the Goal

Every user test needs to have at least one goal to be effective. If you work for a company, ask the product owner, product manager, or leader what the goal for the test is. If you work by yourself, come up with a goal that will allow you to compare and measure the results.

Once you have the goal(s), understand what that goal means. For example, if your goal is to determine if there are any usability issues on an interface, you can use quantitative data to ask the user if a particular task was easy or difficult.

Think Like a User

The best advice for writing great questions is to put yourself in your user’s shoes. Pretend you are a user that takes things literally and have never seen your product nor have any idea how it works. Ask yourself these questions:

• How did I get to this product?
• What is this product for?
• What am I supposed to do?
• What will I get?

Knowing the answers to the questions will help you write an introduction for your user to get in the mindset of your test.

Example

You were browsing your social media feed and see a title that looks interesting, so you click on it and come to our website.

Use a Prototype or Mockup as Reference

Always have access to a prototype or mockup when writing the questions for the user test. Having a visual of the interface in front of you will allow for you to see the path you want the user to take, and this will easily allow you to write the questions and the actual tasks you want them to perform.

Readily having the mockup or prototype available will also prevent you from accidentally skipping over an important step or feature of the interface when writing the questions.

Write Questions with Clarity

Here are some great quick tips to help you write clear questions.

Use Layman’s Terms

Please don’t use industry jargon when writing the questions! Your users will not understand the question clearly and will not provide good feedback.

Example

No: What are your thoughts on the interstitial on the application?
Yes: What are your thoughts on the pop-up with the discount information?

One Question at a Time

Keep the question focused to one thing. Do not pack in several things into a single question as this gets too difficult for the user.

Example

No: What is your opinion on how the search works and the way the results are given to you?
Yes: What is your opinion on how the search works?

Unbiased Feedback

Write questions in a way so you won’t get biased feedback from your users. Write your question as neutral as possible.

Example

Wrong: Was it hard to find the search box on the site?
Right: How easy or difficult was it to find the search bar on the site?

Get the Responses You Need

When writing your questions, you have two types of feedback based on the types of questions you write:

1.) Quantitative (can be easily measured)
2.) Qualitative (influenced by feelings or emotions)

To achieve the user testing goal ultimately, it is ideal to have a healthy mix of both quantitative and qualitative feedback from your users.

Quantitative Questions

Use the following types of systems to write your questions for easy to measure results from your users.

• Rating Scale (levels of difficulty, clarity)
• Yes/No (completion of task)
• Multiple Choice (to prevent a wide range of answers)

Qualitative Questions

Qualitative questions are best if you ask for the user to talk out loud what they are thinking or to write their answer and talk out loud as they do. Your best feedback will come because the user gets to speak their mind and express areas on your prototype that may be confusing or cause friction.

Examples

• Find a class that is convenient for you. Talk out loud as you demonstrate this.
• Is there any other way you would find a class on this page?
• Was there anything that seemed confusing or difficult while you were searching for a class? Please explain out loud as you type your answer.
• What frustrated you most about this site?

Avoid User Test Frustration and Issues

If you have ever conducted a user test and when going over the results find that there was confusion on the instructions or questions, there is nothing more frustrating than seeing your user wander away from the page or area you want feedback on.

Here are some helpful tips to keep your users on track and get the results you need.

This is Just a Test

Sometimes you have to let your users know that your prototype or mockup isn’t functional yet. Doing so will keep them from asking questions about links that don’t work or getting frustrated when they can’t move further along on your interface.

Example Statement

What you will be testing is a prototype, an idea where not everything is 100% functional or clickable, so as you explore you may find things that don’t work yet.

Unexpected Results from Free Reign Testers

If you give your users free reign to click around on your prototype (if it is functional), there is the possibility that your user may move on to the next question while on the page you did not intend to test. It is important to communicate with your user to return to the specific page you want them to be when moving on to the next question by simply adding a statement (and link if you really want to be nice) to the beginning of your next question.

Example

Be sure you are on the home page before continuing to the next question.

Allow the User to Vent

Sometimes, you can really get awesome feedback from allowing your user to vent, or tell you what frustrated them the most on your interface. The best way to do this is to allow them to type in their answer rather than talk out loud. The user will be more intentional about what they write and provide insightful feedback.

Example

What frustrated you most about this site (remember it’s a prototype, so items may not be clickable)?

Summing Up

To get the best possible results from user testing, clear, concise, easy to understand questions are essential to get the feedback you want to achieve your goal. Being able to think like your user is a huge plus when writing effective questions.

Carefully crafted qualitative and quantitative user test questions are the framework for clear feedback responses that can help you see if you achieved your user testing goal or not.

Most importantly, being able to avoid user frustration while taking the test will result in honest, helpful, and effective feedback that you can bring back to your business unit, stakeholders, leaders, or product owners.

[/vc_column_text][/vc_column][/vc_row]