Usability testing is one of the primary rung on the ladder for quantifying user experience and weeding out any issues with the final product. The challenges with usability testing today on mobile devices are: its not very well defined and a lot also depends on the context in which it is used. For example, We can uncover richer insights and identify points of failure when we observe a customer in a coffee shop than in a lab setting. We tried and perfected a few techniques for both lab and field based usability testing. To quantify user experience, depending on the product and medium, we measure : time on task, success/error rate, SUS score, and hedonistic qualities (or ways to measure “love”).
Embedded as part of the product & design team, I carried out regular usability testing. It involved identifying major tasks of the particular experience and observe customers perform the tasks. We measured success rates, number of times they needed prompting, their description of the process etc. Over the course of time, we collected data from different segments of customers including customers with accessibility issues. The average study involved 8 participants spread across target technographic segments and put our experiences to the litmus test.