Apptimize — Mobile A/B Testing & Analytics

When I started working at Apptimize, the biggest tasks that required immediate tackling was designing a consistent user experience and design across the entire product. My tasks were everything from redesigning how navigation worked to how users setup AB experiments and analyze their results. Throughout the project I helped design and spec out tasks for developers, debug, create interactive mockups, and designed a style guide to help bring more consistency in the product for the entire company.

Screenshot of an information architecture

A/B Experiments

To create a step process for A/B experiments can prove quite challenging, however one of my inspirations for the interactions designed was a shopping cart. With online shopping the user often has a product in mind they are searching for, and from there they will filter down to the brand, color, and type of product. This mindset helped set the foundation on the order of our steps.

One of the major changes we created in this onboarding was moving the step to setup your experiment goals step from being the last to second. Our reasoning was to encourage our users to think about why they are running the experiment before setup. We found many drop-offs or not finding value in the product was a result of customers running experiments and setting goals as an afterthought. When we launched this setup process, we found our users were taking more time to launch experiments and adding more goals relevant to their experiments both primary and secondary goals. As a result the number of people setting up experiments went down, however, the engagement for the experiments being run had increased.

Display of the steps to setting up an AB experiment

Configure the experiment

Configure needed to be very easy and concise, at the top we have a very simple switch box which allows the user to easily switch between visual, code block, or dynamic variable experiments. Within each of these are their own configuring panel (with the exception of code block).

Different configuration options

Target and launch

Once the experiment configuration is ready, we have the launchpad. This page can be considered the "shopping bag" of an ecommerce website. Our intention here again was to give the idea of checking out and finalization of the process. With an easy to understand slider for audience (with a legend directly below), and refinement of audience targeting. The user can then preview variants live on their own devices, and when ready launch the experiment.

Launchpad and review of experiment

Results of an experiment

On the results page one of our challenges was finding a way to show lots of data in a clean and concise way. Previously charts were on their own section, separated by the data tables. In this new interface we put all the data in a layout that not only is easy to ready, but allows for quick education on terms that may not be familiar to new users.

Results of an experiment