US 10,313,262 B1Granted

System for management of content changes and detection of novelty effects

AssigneeAmazon Technologies, Inc.

InventorsAlexandra Juliet Brasch, Brent Russell Smith, Douglas R. Hains, Corry Lee-Boehm

Life of this patent

Estimating the term…

Loading patent…

Abstract

Systems are provided to analyze user behavior with respect to different versions of content, and to identify the behavioral effects that can be attributed to the differences between the versions of content. Systems can determine whether an observed difference in user interactions with different versions of content is caused by the specific changes that have been made to content (e.g., an actual improvement or other substantive change), or whether the observed difference in user interactions is caused by the mere fact that the versions of content are different (e.g., due to the novelty of seeing a different version of content for the first time, regardless of whether the difference is a substantive improvement). In some cases, the observed effect may be caused both by the novelty of the content being changed and by the substance of the changes.

Claims20

Dependent claims

A computer-implemented method comprising:

as performed by a computing system configured to execute specific instructions,
determining an initial effect that a difference between a first version of a network resource and a second version of the network resource has on a metric during an initial period of time, wherein the metric relates to actions performed after accessing the network resource, and wherein the difference comprises at least one of:
a portion of the first version is generated using a different algorithm than a corresponding portion of the second version, the portion of first version is presented in a different manner than the corresponding portion of the second version, or interaction with the portion of the first version causes a different action than interaction with the corresponding portion of the second version;
determining a number of exposures to the network resource occurring during a subsequent period of time following the initial period of time;
determining an expected effect that the difference between the first version and the second version is likely to have on the metric during the subsequent period of time, wherein the expected effect is determined based at least in part on the initial effect and the number of exposures;
determining an observed effect that the difference between the first version and the second version has on the metric during the subsequent period of time, wherein the observed effect is determined based at least in part on data regarding actions performed during the subsequent period of time, and wherein an action performed by an individual user of the network resource during the subsequent period of time is performed after a user device of the individual user receives either the first version or the second version;
determining to modify the network resource based at least in part on a degree the observed effect differs from the expected effect;
and
modifying the network resource.

6 more elements

A non-transitory computer storage medium storing executable code, wherein the executable code configures a computing system to perform a process comprising:

determining an initial effect that a difference between a first version of a network resource and a second version of the network resource has on a metric during an initial period of time, wherein the metric relates to actions performed after accessing the network resource, and wherein the difference comprises at least one of:
a portion of the first version is generated using a different algorithm than a corresponding portion of the second version, the portion of first version is presented in a different manner than the corresponding portion of the second version, or interaction with the portion of the first version causes a different action than interaction with the corresponding portion of the second version;
determining a number of exposures to the network resource occurring during a subsequent period of time following the initial period of time;
determining an expected effect that the difference between the first version and the second version is likely to have on the metric during the subsequent period of time, wherein the expected effect is determined based at least in part on the initial effect and the number of exposures;
determining an observed effect that the difference between the first version and the second version has on the metric during the subsequent period of time, wherein the observed effect is determined based at least in part on data regarding actions performed during the subsequent period of time, and wherein an action performed by an individual user of the network resource during the subsequent period of time is performed after a user device of the individual user receives either the first version or the second version;
determining to modify the network resource based at least in part on a degree the observed effect differs from the expected effect;
and
modifying the network resource.

5 more elements

A system comprising:

computer-readable memory storing executable instructions;
and
one or more processors in communication with the computer-readable memory and configured by the executable instructions to at least:
determine an initial effect that a difference between a first version of a network resource and a second version of the network resource has on a metric during an initial period of time, wherein the metric relates to actions performed after accessing the network resource, and wherein the difference comprises at least one of:
a portion of the first version is generated using a different algorithm than a corresponding portion of the second version, the portion of first version is presented in a different manner than the corresponding portion of the second version, or interaction with the portion of the first version causes a different action than interaction with the corresponding portion of the second version;
determine a number of exposures to the network resource occurring during a subsequent period of time following the initial period of time;
determine an expected effect that the difference between the first version and the second version is likely to have on the metric during the subsequent period of time, wherein the expected effect is determined based at least in part on the initial effect and the number of exposures;
determine an observed effect that the difference between the first version and the second version has on the metric during the subsequent period of time, wherein the observed effect is determined based at least in part on data regarding actions performed during the subsequent period of time, and wherein an action performed by an individual user of the network resource during the subsequent period of time is performed after a user device of the individual user receives either the first version or the second version;
determine to modify the network resource based at least in part on a degree the observed effect differs from the expected effect;
and
modify the network resource.

8 more elements

Dependencies are inferred from the claim text.

Facts

Application: 15/385,178Filed 20 Dec 2016
Granted: 4 Jun 2019
Term ends: Not estimatedThe source records don’t support an estimate.

Classification5 codes

H04L 67/535
Network services: tracking the activity of the user
H04L 47/70
Traffic control in data switching networks: admission control; Resource allocation
H04L 67/10
Protocols: in which an application is distributed across nodes in the network
H04L 67/565
Provisioning of proxy services: conversion or adaptation of application format or content

Family

No related US applications are recorded in the indexed filing history.

Timeline

Citations

Open elsewhere

Building an agent or pulling many patents? Use the API or MCP server instead of scraping pages. Get an API key