On-Boarding Checklist Experiment

Overview

In this successful experiment, we targeted users attempting to activate over several sessions. The variant introduced features to improve user reorientation after their initial session, such as clearly labeled buttons and helper text displaying the current status for each step.

Role

Product designer

Research (collaborator)

Workshop facilitator

output

Disposable designs

Final experiment mocks

Problem

Addressing Navigation Challenges to Improve Activation Funnel Completion

Poor navigation is causing our users to not complete the activation funnel. In addition, data suggests our users are exponentially less likely to complete the activation funnel in every session after their first.

Goal

Enhancing User Journey with Intuitive Navigation and Clear Action Insights for Activation Success

Positively impact our Activation metric by allowing users to easily navigate the checklist and by providing better insight into the last action taken on any given step of the checklist.

Research

Unveiling the need for better support across multi-session engagements

Our data analyst revealed a critical insight: users were significantly less likely to complete the activation funnel if they didn't do so during their first session. This finding underscored the importance of optimizing the initial user experience to enhance engagement and conversion rates.

Research

Enhancing Checklist Navigation Based on Past Experiment Success

Based on previous successful experiments we were extremely confident in the ordering of our checklist steps. So for this experiment we focused on how users would navigate throughout the steps’ respective workflows.





Output

Disposable designs

Once our team completed our design thinking workshop, we aligned on a few different approaches we could take for our experiment. I took those rough ideas, brought them up in fidelity, then presented them back to the team for our final vote on the solution experiment. I called them “Disposable designs” because I wanted to emphasize that despite the higher fidelity, we will only be using them as a visual reference to put together the final version of our experiment solution.

Output

Checklist cell permutations

As the complexity of our checklist cells increased, my engineers required a thorough audit of the possible states each cell could occupy. This served as a vital reference for them, guiding the implementation of enhanced state communication and new navigation patterns within the cells.

Edge case

E-comm users experience

Here’s what the checklist will include and how it will behave when a user identifies themselves as having an e-commerce store in the Sign-up flow.

Edge case

Skipped cell permutations

These designs illustrate the appearance of copy and visuals when a user opts to skip steps in the checklist. They provide a clear view of how the interface adapts to accommodate the user's choice to bypass certain tasks.

Solution

Final experiment mock

This is the completed flow for our checklist experiment. It includes visual callouts to relevant points of navigation from other parts of the product. It also includes visual callouts for when a decision was made in the absence of the rest of the team. I branded these particular callouts as Decision Tracker ™️ in an attempt to bring some whimsey to something that might otherwise feel weirdly authoritative.

Results

1% lift in our activation rate due to improvements made to our on-boarding checklist

The experiment yielded a 1% absolute lift in our Activation Rate, a modest yet significant achievement that marked one of our first successes of the year! It also laid the groundwork for a subsequent experiment that resulted in an even greater improvement: the Email Optimization Experiment.

Collaborators

Growth Product Manager
Data Analyst

That's a wrap for this project!

See more of my work from

MailChimp