Usability testing for PayPal digital wallet

Usability testing is one of the primary rung on the ladder for quantifying user experience and weeding out any issues with the final product. The challenges with usability testing today on mobile devices are:  its not very well defined and a lot also depends on the context in which it is used. For example, We can uncover  richer insights and identify points of failure when we  observe a customer in a coffee shop than in a lab setting. We tried and perfected a few techniques for both lab and field based usability testing. To quantify user experience, depending on the product and medium, we measure : time on task, success/error rate, SUS score, and hedonistic qualities (or ways to measure “love”).

Details
Embedded as part of the product & design team, I carried out regular usability testing. It involved identifying major tasks of the particular experience  and observe customers perform the tasks. We measured success rates, number of times they needed prompting, their description of the process etc. Over the course of time, we collected data from different segments of customers including customers with accessibility issues.  The average study involved 8 participants spread across target technographic segments and put our experiences to the litmus test.

My Contributions
  • Designed the study, helped plan research and set expectations with the team
  • Created recruitment criteria and helped recruit participants
  • Interview questionnaire
  • Moderated and facilitated participant interviews
  • Edited and collected insights from interview footage and notes
  • Presented top level and detailed reports
  • Submitted and tracked UX bugs with dev/qa
Methods
  • Structured interviews
  • Unmoderated usability studies
  • Eye tracking studies
  • Cognitive walkthroughs
Team
  • John Amir Abbassi (Research)
  • Laura Ward (Design lead)
  • Danny Yee (Visual design lead)
  • Amit Paka (Product)
  • IOS/Android dev team