What is the key to good Moderation?
Moderation can make or break a usability testing session. Be too passive and useful information can be missed out on. Be too involved and your bias can be left all over the study. So what is the key to good moderation?
The Moderation Guide
Moderation starts with good preparation (as Benjamin Franklin said, “failing to prepare is preparing to fail!”). Using the knowledge gained from expert reviews, analytics and known client concerns, a moderation guide should be created. This should contain the tasks that you are going to ask the participant to carry out, as well highlighting any elements of discussion that you wish to focus on. The level of detail is up to the individual moderator; some might want to include some additional ‘crib’ notes.
The moderation guide should also be trialled in advance, so conduct a pilot study to identify where changes may be required. This also provides an opportunity to test all the equipment is working. At this point, also make sure you have a backup plan should any fail unexpectedly during the session (anything that can go wrong, will go wrong!).
Add Your Style
As human beings, we all have our own personality traits, so it is important that you approach moderation in a way that comes naturally to you. However, there are a few things that you need to keep in mind.
When it comes to testing day, the moderator has to make the participants feel as comfortable as possible. This starts at the introduction. At this point the tone for the whole session is set. Engage the participant in normal conversation, don’t go straight into business. Discuss the weather, talk about their weekend or simply ask about their day.
Once they have settled in, you can get them focused on the task in hand. Be clear when explaining the task to the user and give the participant the opportunity to ask for clarification. However, don’t give away everything about what you are testing at the start – keep your description high level, telling them just enough to point them in the right direction, and otherwise let the participant use the product as they would if they were at home (or in the office) on their own.
Thinking Out Loud
Many usability testers will ask participants to think out loud when going through a process. However, this isn’t always easy for a participant to do and can detrimentally impact the natural feel to the test you are trying to create. Imagine trying to explain how to kick a ball at the same time as doing so; your actions would be different – less flowing and natural – than when kicking the ball without the description.
In addition, many participants will go into more detail when explaining and start looking for issues that they would not have noticed under normal circumstances, exaggerating the impact of these issues and skewing the test results. Participants may also say one thing, but actually do something else.
Technology such as eye tracking glasses makes spotting this easier, as you can observe where the participant is looking, what they are seeing, what they are missing and what is causing them confusion. However, not everyone will have access to this kind of equipment. This is where a skilled moderator will earn their stripes.
Let the session flow as normal – you’ll find that when a participant is really struggling they will probably say something to you. A really good moderator will be able to choose the right points to ask pertinent questions without unnecessarily interrupting the participant’s process.
During the session, the greatest skill is to stay neutral. Don’t be leading in your questions (e.g. “that button was easy to find, wasn’t it?”). You should also avoid being too quick to help the user, or answer their questions, if they begin to struggle or go off piste.
Knowing when to be silent during a session is as important as anything else – give the participant a chance to find a solution themselves. If they can’t, try to use it as an opportunity to understand why. If they ask you a question, ask them a question in return to prompt them to complete the task without help (e.g. Participant – “Should I click on that button?” Moderator – “What would you normally do in this situation?”), then let them carry on down the path that they feel most comfortable with.
Don’t overload the participant with tasks
Keep in mind that you won’t be able to test all the features you want in a single set of interactions, but you can always go back and set specific tasks to focus on certain features once the participant has completed the main tasks. For example, the participant may not use the filters to find what they are looking for, but at the end of the session you can take them back to that page and ask them to find a specific item that forces them to use the filter. I would recommend that you plan several plenary activities as part of your session.
Finally, remember that participants will work at different speeds so don’t try and rush those that you think are working too slowly or slow down those that you think are going to fast.
Moderation can be tricky to perfect initially, but it isn’t a dark art. Practice makes perfect, work on your approach and develop your own style.
Amrit Bhachu, UX Designer