August 23, 2020
Project Lead: Dean Zuk
UX Researcher: Bibi Tarin
August 23, 2020
Project Lead: Dean Zuk
UX Researcher: Bibi Tarin
Goals
Methodology
Summary of Results
Recommendations & Action Items
Appendix
The goal for this A/B Usability Test was to identify whether users preferred the "Danger Bubble" Flow A over the "Gauge" Flow B. Additionally, participants were asked questions about UX/UI elements such as "progress bar" indicating to the user where they were at each step of the process, "skip to subscribe" , and the arrangement of checkboxes with service information on the final page, to assess their impact and value in ensuring users complete the process and have a pleasant experience doing so
Participants were introduced to Confidently and their services, and were encouraged to share their thought process out loud throughout the usability test.
Enter Sign-in page, and proceed to next step.
Select icons corresponding to users internet activity and proceed to next step.
Interpret gauge/danger bubble and proceed to next step.
Thoughts/comments/insights on final page (organization of information, security, etc)
After both prototypes have been tested, participants were asked the following questions to gain a deeper understanding of how they felt about each product (Flow A/B) overall.
What was your impression of the product?
What was your favorite aspect of the Flow A/B?
What was the most confusing part of Flow A/B?
Would you continue using this product?
Usability tests were conducted on a 1-1 basis via zoom with participants who went through Flow A, and Flow B. Users were asked general questions about their experience throughout the session to identify pain points, confusion, mental models, general impressions, and user preferences.
Factors used to define usability
Here are the key findings:
of participants preferred Flow B's Gauge feature.
of participants preferred [Flow B] all cost and subscription information to be displayed at top of page, before providing credit card information.
of participants did not find the "Skip to Subscribe" feature useful.
of participants preferred [Flow A] progress bar, listing steps until completion.
of participants did not understand the significance of "109" on the Gauge meter.
of participants preferred the Gauge feature over the "Danger Bubbles", they enjoyed the concept and the overall experience of using the product
Breakdown
Learnability
Happiness / Enjoyability
Confusion
Relevance
Participants displayed increased satisfaction engaging with Flow B's "Gauge" feature because it was interactive, and instantly displayed their exposure to data brokers online. This feature also captured user curiosity, and made them want to learn more about their exposure, and the companies who had access to their data.
"The gauge along with the use of color here really creates an urgency to act."
75% of users did not understand what the number "109" indicated on the gauge. Participants suggested a pop up with additional information to be featured as a supplement.
On average, users took more time to understand the meaning of the Danger Bubble feature.
Participants were also confused about the graphic, because it lacked context when separated from the icon selection page.
Breakdown
Accuracy
Happiness / Enjoyability
Confusion
Relevance
Danger Bubble graphic was bold, and the use of color, alarming, and overall effective in capturing user attention.
Information was not clearly conveyed, users exerted more time and effort to interpret data, reducing the inclination to invest in understanding the implications of their exposure.
"I like the idea, but it's not hitting hard for me."
Consider implementing the "Gauge" feature, because your target audience invested more time into understanding their risk, the nature of Confidently's services, and enjoyed how interactive the experience was due to the layout.
Consider implementing a pop-up upon hovering over the "109 Data Brokers," because participants did not understand what that meant. I would also recommend keeping the "109 Data Brokers" as a static feature on the gauge, visible before users select online activity categories, so it's clear this is the baseline amount of data brokers with access to their information by virtue of users being online.
Consider implementing the progress bar feature from Flow A, that lists steps 1-3, because your target audience could anticipate how long it would take for them to complete the process, and felt at ease knowing a minimal time investment was required of them.
Consider eliminating the "Skip to Subscribe" feature from the second, and third frames because your target audience did not find it necessary after signing in on the main page. Users became invested in understanding their exposure, risk, and Confidently's services. This also provided them with enough information that would help them understand the value of Confidently's services, increasing the likelihood of them sharing credit card information.
Consider implementing the final frame in Flow B because your target audience wanted to know how much the service would cost after the free trial period, and exactly what benefits they would receive before sharing confidential information.
Recommendations
Consider fitting sign-in page elements to screen, because target audience preferred to see all features without scrolling.
Consider implementing auto-generated user name and information on final frame above credit card form because your target audience expressed it would feel personalized, and safer to share confidential information.
Link to individual usability test notes:
https://docs.google.com/document/d/1RK6OmDK-dVNzPxx86eG6b1gKiJJQNL0-fkQnWqSw7FM/edit?usp=sharing
Link to video recordings:
https://drive.google.com/drive/folders/15RV5sCEEfeDmOTaPFDDYq7v8-9j63id8?usp=sharing
“I really like the amount of white space [sign-in frame], it’s very clean, professional, the hierarchy of typography is eye catching and really draws you in and conveys trust.” - S. H.
“I understand exactly what we’re doing when we get to this page [gauge feature], this feels clear to me” - D. P.
“I’m emotionally invested at this point, and would like to see how I can take back control” - A. L.
“Very user friendly, very intuitive [sign-in page]” - M. R.
“If I were to walk my 80 year old mother into this, it would not make me want to bang my head against the wall” - M. R.