Recipient session attribution for a/b experimentation

A computing system comprising:

• one or more processors;
• storage media;
• and
• computer-executable instructions stored in the storage media and which, when executed by the computing system, cause the computing system to perform the following operations:
• storing record of a plurality of end-user sessions that occurred while a single-randomization A/B experiment was conducted with an online service;
• for each end-user session, identifying one or more end-user activity alerts that triggered the end-user session, and
• for each end-user session, attributing the end-user session to one or more alert-triggering actions that caused the one or more end-user activity alerts that triggered the end-user session to be communicated by the online service;
• and
• for each alert-triggering action of a plurality of alert-triggering actions taken by end-users of the online service while a single-randomization A/B experiment was conducted with the online service, computing a recipient session attribution metric for the alert-triggering action based on the one or more end-user sessions, of the plurality of end-user sessions, attributed to the alert-triggering action.

One or more non-transitory storage media storing computer-executable instructions which, when executed by a computing system comprising one or more processors, cause the computing system to perform the following operations:

• storing record of a plurality of end-user sessions that occurred while a single-randomization A/B experiment was conducted with an online service;
• for each end-user session, identifying one or more end-user activity alerts that triggered the end-user session;
• for each end-user session, attributing the end-user session to one or more alert-triggering actions that caused the one or more end-user activity alerts that triggered the end-user session to be communicated by the online service;
• wherein the single-randomization A/B experiment has a treatment group to which end-users are assigned and a control group to which end-users are assigned;
• for each alert-triggering action of a first plurality of alert-triggering actions taken by end-users assigned to the treatment group, computing a recipient session attribution metric for the alert-triggering action based on the one or more end-user sessions, of the plurality of end-user sessions, attributed to the alert-triggering action;
• and
• for each alert-triggering action of a second plurality of alert-triggering actions taken by end-users assigned to the control group, computing a recipient session attribution metric for the alert-triggering action based on the one or more end-user sessions, of the plurality of end-user sessions, attributed to the alert-triggering action.