Attributing Loss of Engagement with an Online System Using Temporal Partitioning of Training Data for a Churn Prediction Model

A computer program product comprising a non-transitory computer readable storage medium having instructions encoded thereon that, when executed by a processor, cause the processor to perform steps comprising:

• obtaining a training dataset comprising a plurality of training examples, wherein each training example is generated by:
• retrieving historical interactions by a user of a plurality of users with an online system, wherein the historical interactions for the user include a churn event indicating a loss of engagement of the user with the online system,
• segmenting the historical interactions into at least a first time interval and a second time interval that is after the first time interval,
• obtaining a set of user features from the historical interactions occurring in the first time interval,
• obtaining a set of online system features from the historical interactions occurring in the second time interval, wherein the historical interactions occurring in the second time interval include one or more causal events, and
• labeling an attribution of the churn event to one or more of the causal events;
• training a churn prediction model, wherein the churn prediction model is trained by:
• applying the churn prediction model to each training example of the training dataset to generate an attribute score for each of the causal events, the attribute score for a causal event comprising a predicted probability of the attribution of the churn event the causal event,
• scoring the churn prediction model using a loss function and the label of the training example, and
• updating one or more parameters of the churn prediction model by backpropagation based on the scoring until one or more criteria are satisfied;
• identifying a new churn event associated with a target user of the online system;
• applying the churn prediction model to a set user features and online system features from historical interactions by the target user with the online system, the churn prediction model outputting an attribution score for each of the one or more causal events;
• attributing a causal event to the new churn event based on the attribution scores;
• and
• implementing a remedial action corresponding to the attributed causal event, wherein implementing the remedial action mitigates a reduction in a predicted probability of the target user interacting with the online system.

A system comprising:

• a processor;
• and
• a non-transitory computer readable storage medium having instructions encoded thereon that, when executed by the processor, cause the processor to perform steps comprising:
• obtaining a training dataset comprising a plurality of training examples, wherein each training example is generated by:
• retrieving historical interactions by a user of a plurality of users with an online system, wherein the historical interactions for the user include a churn event indicating a loss of engagement of the user with the online system,
• segmenting the historical interactions into at least a first time interval and a second time interval that is after the first time interval,
• obtaining a set of user features from the historical interactions occurring in the first time interval,
• obtaining a set of online system features from the historical interactions occurring in the second time interval, wherein the historical interactions occurring in the second time interval include one or more causal events, and
• labeling an attribution of the churn event to one or more of the causal events;
• training a churn prediction model, wherein the churn prediction model is trained by:
• applying the churn prediction model to each training example of the training dataset to generate an attribute score for each of the causal events, the attribute score for a causal event comprising a predicted probability of the attribution of the churn event the causal event,
• scoring the churn prediction model using a loss function and the label of the training example, and
• updating one or more parameters of the churn prediction model by backpropagation based on the scoring until one or more criteria are satisfied;
• identifying a new churn event associated with a target user of the online system;
• applying the churn prediction model to a set user features and online system features from historical interactions by the target user with the online system, the churn prediction model outputting an attribution score for each of the one or more causal events;
• attributing a causal event to the new churn event based on the attribution scores;
• and
• implementing a remedial action corresponding to the attributed causal event, wherein implementing the remedial action mitigates a reduction in a predicted probability of the target user interacting with the online system.