SCAIM Seminar: Roger Donaldson

A common experiment performed by web companies is the so-called A/B test, which compares two software designs assigning different interfaces or functionalities to two groups of randomly-selected users.  Although it is usual to design these experiments as hypothesis tests, standard tests (such as t-tests) overestimate confidence, since web traffic is heavy-tailed.  It is typical for a small number of individuals to radically affect the statistics of their test group.

In this talk, I discuss my work as a consultant for deviantART, an online art community, as we developed reliable models for their traffic data, as well as subsequent hypothesis tests.

This talk is both a review of properties and methods for handling heavy-tailed data, and an advertisement for the utility of probabilistic modeling in industry.