Seven Mistakes to Avoid During Usability Testing
by Design Science
Let’s cut some red tape. Pharmaceutical companies, in particular, sometimes seem to approach usability testing as if it’s a clinical trial.
As such, they often spin their wheels worrying about exposure to risk, engage in all kinds of hand wringing about report formats, and generally overthink the whole process, until it looms frighteningly in their imagination. But usability testing is a completely different beast—and it can be, well, practically fun. But first, here are some things you shouldn’t do:
Institutional Review Boards (IRBs) are NOT always necessary.
The FDA has no specific requirement about the use of IRBs. In terms of risk, unless there’s something special, you can think of usability testing as you would market research.
Prototypes do NOT have to remain unchanged during a formative test.
Usability can be enhanced by making changes on the fly. There’s no reason you can’t make changes to prototypes and to the testing protocol in the middle of a formative study. This might make you uncomfortable if you are used to clinical trials. But flexibility can yield far better outcomes than rigid adherence to a test protocol that doesn’t give you the information you need.
Minor research errors are NOT necessarily important.
Unless the error fundamentally alters the implications for design, they don’t need to be a source of major concern. In fact, trying to improve an error rate from, say, 1% to 0.1% can be costly and probably won’t yield tangible benefits.
Inferential statistics are NOT usually necessary or appropriate.
For usability testing, FDA doesn’t want to see statistical arguments that “usability goals” have been met. They have rejected quantitative acceptance criteria for summative tests. Further, you will not likely need complex research designs. These are seldom appropriate for usability testing. Keep in mind that the central question is whether or not the device is as safe and effective as it feasibly can be.
Formative reports do NOT have to be formal.
You don’t need document templates—in fact, PowerPoint is an acceptable format. What you do want to think about is how you can improve understanding of the “takeaways” by the design team. Video links can enhance understanding of how users respond to a design. Remember that the focus should be effective use of the results. Too much formality can be a barrier to use by your design team.
Formative reports do NOT have to be tedious.
Less is often more. The simpler your account, the better everyone at your organization will be able to respond. Overly long-winded performance accounts are often counter-productive. The conclusions are more important than the details for formative testing. Details can be put in appendices. They are your friends.
A usability test is NOT a clinical trial.
This is not intended to be a statistical cost/benefit analysis. It is an assessment that you have designed a product to the best of the organization’s ability to make it usable.
Share this entry