Testing the Efficacy of Your Website’s New Feature (Part 2)

Technically known as A|B testing or hypothesis testing

Photo by Markus Winkler on Unsplash
import pandas as pd
import numpy as np
df = pd.read_csv('ad_actions.csv')df.tail()
control 4264
experiment 3924
Name: group, dtype: int64
view 6328
click 1860
Name: action, dtype: int64
click_ids = set(df[df.action=='click']['id'].unique())
view_ids = set(df[df.action=='view']['id'].unique())
print("Number of viewers: {} \t Number of clickers: {}".format(len(view_ids), len(click_ids)))
print("Number of viewers who didn't click: {}".format(len(view_ids-click_ids)))
Number of viewers: 6328 Number of clickers: 1860
Number of viewers who didn't click: 4468
df['count'] = 1
control = df[df.group == 'control'].pivot(index='id', columns='action', values='count')
control = control.fillna(value=0)
control group
experiment = df[df.group == 'experiment'].pivot(index='id', columns='action', values='count')
experiment = experiment.fillna(value=0)
experimental group
print('Sample sizes: Control:{}  Experiment:{}'.format(len(control), len(experiment)))
print('Total clicks: Control:{} Experiment:{}'.format(control.click.sum(), experiment.click.sum()))
print('Average click-through-rate: Control:{} Experiment:{}'.format(control.click.mean(), experiment.click.mean()))
Sample sizes: Control:3332 Experiment:2996
Total clicks: Control:932.0 Experiment:928.0
Average click-through-rate:
Control:0.2797118847539016 Experiment:0.3097463284379172

I’m a recent Data Science graduate with a B.S. in Environmental Science. Currently seeking job opportunities. Constantly learning!

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store