# US 2021/0241158 A1 · Machine-learning models applied to interaction data for facilitating experience-based modifications to interface elements in online environments

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2021/0241158 A1
- Document ID: US20210241158A1
- Status: Published application
- Kind: A1
- Application number: 17/236,506
- Filed: 2021-04-21
- Published: 2021-08-05
- Earliest claimed: 2018-04-06
- Assignees: Adobe Inc.
- Applicants: Adobe Inc.
- Inventors: Atanu R. Sinha; Deepali Jain; Nikhil Sheoran; Deepali Gupta; Sopan Khosla
- Web page: https://patentgrep.com/patent/US20210241158A1

## CPC

- G06N 3/08: Learning methods
- G06F 17/11: for solving equations {, e.g. nonlinear equations, general mathematical optimization problems (optimization specially adapted for a specific administrative, business or logistic context G06Q10/04)}
- G06F 18/2148: {characterised by the process organisation or structure, e.g. boosting cascade}
- G06F 18/23: Clustering techniques
- G06F 18/2414: {Smoothing the distance, e.g. radial basis function networks [RBFN]}
- G06N 3/044: Recurrent networks, e.g. Hopfield networks
- G06N 3/0442: characterised by memory or gating, e.g. long short-term memory [LSTM] or gated recurrent units [GRU]
- G06N 3/09: Supervised learning
- G06N 7/01: Probabilistic graphical models, e.g. probabilistic networks
- G06N 20/00: Machine learning
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)

## Abstract

In some embodiments, a computing system computes, with a state prediction model, probabilities of transitioning from a click state represented by interaction data to various predicted next states. The computing system computes an interface experience metric for the click with an experience valuation model. To do so, the computing system identifies base values for the click state and the predicted next states. The computing system computes value differentials for between the click state's base value and each predicted next state's base value. Value differentials indicate qualities of interface experience. The computing system determines the interface experience metric from a summation that includes the current click state's base value and the value differentials weighted with the predicted next states' probabilities. The computing system transmits the interface experience metric to an online platform, which can cause interface elements of the online platform to be modified based on the interface experience metric.

## Claims

### Claim 1

A method comprising:
operating an environment evaluation system for working with a host system configured for:
performing a modification of an arrangement of interface elements of an online platform based on an interface experience metric computed from interaction data from the online platform; and
performing, with one or more processing devices of the environment evaluation system, operations comprising:
computing probabilities of transitioning from a click state to different next states based on the interaction data from the online platform;
computing value differentials between a current base value for the click state and subsequent base values for the different next states;
computing the interface experience metric from a sum of the current base value and the value differentials weighted with the probabilities of transitioning; and
providing the host system with the interface experience metric.

### Claim 2 (depends on 1)

The method of claim 1, wherein the probabilities of transitioning include (i) a first probability of transitioning from the click state represented by the interaction data to a first predicted next state and (ii) a second probability of transitioning from the click state to a second predicted next state, wherein the value differentials include (i) a first value differential computed from a difference between the current base value and a first subsequent base value for the first predicted next state and (ii) a second value differential computed from a difference between the current base value and a second subsequent base value for second predicted next state, wherein the first value differential indicates a higher quality of interface experience than the second value differential, wherein the operations further comprise:
identifying (i) a first transition importance for transitioning from the click state to the first predicted next state and (ii) a second transition importance for transitioning from the click state to the second predicted next state, wherein identifying a transition importance comprises determining, from historical interaction data, a statistical likelihood of a transition leading to a target outcome;
weighting the first value differential with the first transition importance in addition to the first probability; and
weighting the second value differential with the second transition importance in addition to the second probability,
wherein the interface experience metric is computed with (i) the first value differential weighted with the first transition importance in addition to the first probability and (ii) the second value differential weighted with the second transition importance in addition to the second probability.

### Claim 3 (depends on 2)

The method of claim 2, wherein identifying each of the first transition importance and the second transition importance comprises:
computing a representation vector from the interaction data or entity data;
accessing a vector space having (i) a first cluster generated from the historical interaction data or the entity data and (ii) a second cluster generated from additional historical interaction data or the entity data; and
selecting, based on the representation vector defining a point that is closer to the first cluster than the second cluster, the first transition importance and the second transition importance from a transition importance dataset corresponding to the historical interaction data.

### Claim 4 (depends on 2)

The method of claim 2, wherein the operations further comprise:
computing a state duration weight based on a statistical deviation of a duration for the click state from historical durations for the click state; and
applying the state duration weight to both the first value differential and the second value differential,
wherein the interface experience metric is computed with (i) the first value differential weighted with the state duration weight in addition to the first probability and the first transition importance and (ii) the second value differential weighted with the state duration weight in addition to the second probability and the second transition importance.

### Claim 5 (depends on 1)

The method of claim 1, wherein the operations further comprise training an experience valuation model, which computes the interface experience metric, by performing training operations comprising:
accessing training interaction data having training click states; and
modifying one or more tuning parameters applied to a sum of (i) one or more current base values and (ii) value differentials weighted with probabilities of transitioning from a click state to next states, wherein the one or more tuning parameters are modified to minimize violations of a constraint function,
wherein the constraint function specifies that a first relationship between a training interface experience metric computed for a training click state and a base value for the training click state corresponds to a second relationship between the base value for the training click state and an additional base value for a subsequent training click state.

### Claim 6 (depends on 5)

The method of claim 5, wherein the one or more tuning parameters are modified to have (i) one or more first one or more tuning parameter values for a first segment and (ii) one or more second one or more tuning parameter values for a second segment, wherein the training operations further comprise:
segmenting the training interaction data into a first training dataset and a second training dataset, wherein segmenting the training interaction data comprises:
computing representation vectors for entities corresponding to respective subsets of the training interaction data corresponding to respective user devices;
clustering the representation vectors within a vector space;
identifying the first training dataset based on first representation vectors for the first training dataset being included in a first one of the clusters, and
identifying the second training dataset based on second representation vectors for the second training dataset being included in a second one of the clusters;
modifying the one or more tuning parameters based on the first training dataset to compute the first one or more tuning parameter values for the first segment; and
modifying the one or more tuning parameters based on the second training dataset to compute the second one or more tuning parameter values for the second segment.

### Claim 7 (depends on 6)

The method of claim 6, wherein the operations further comprise:
computing a representation vector for an entity associated with the interaction data;
determining, from the representation vector, that the interaction data indicates the entity in the first segment; and
applying the experience valuation model using the first one or more tuning parameter values.

### Claim 8

A system comprising:
one or more processing devices configured to:
operate an environment evaluation system for working with a host system configured for:
performing a modification of an arrangement of interface elements of an online platform based on an interface experience metric computed from interaction data generated by interactions with the interface elements of the online platform; and
perform, with the environment evaluation system, operations comprising:
computing probabilities of transitioning from a click state to different next states based on the interaction data from the online platform;
computing value differentials between a current base value for the click state and subsequent base values for the different next states; and
computing the interface experience metric from a sum of the current base value and the value differentials weighted with the probabilities of transitioning; and
a network interface device configured for transmitting the interface experience metric to the host system.

### Claim 9 (depends on 8)

The system of claim 8, wherein the probabilities of transitioning include (i) a first probability of transitioning from the click state represented by the interaction data to a first predicted next state and (ii) a second probability of transitioning from the click state to a second predicted next state, wherein the value differentials include (i) a first value differential computed from a difference between the current base value and a first subsequent base value for first predicted next state and (ii) a second value differential computed from a difference between the current base value and a second subsequent base value for second predicted next state, wherein the operations further comprise:
identifying (i) a first transition importance for transitioning from the click state to the first predicted next state and (ii) a second transition importance for transitioning from the click state to the second predicted next state, wherein identifying a transition importance comprises determining, from historical interaction data, a statistical likelihood of a transition leading to a target outcome;
weighting the first value differential with the first transition importance in addition to the first probability; and
weighting the second value differential with the second transition importance in addition to the second probability,
wherein the interface experience metric is computed with (i) the first value differential weighted with the first transition importance in addition to the first probability and (ii) the second value differential weighted with the second transition importance in addition to the second probability.

### Claim 10 (depends on 9)

The system of claim 9, wherein identifying each of the first transition importance and the second transition importance comprises:
computing a representation vector from the interaction data or entity data;
accessing a vector space having (i) a first cluster generated from the historical interaction data or the entity data and (ii) a second cluster generated from additional historical interaction data or the entity data; and
selecting, based on the representation vector defining a point that is closer to the first cluster than the second cluster, the first transition importance and the second transition importance from a transition importance dataset corresponding to the historical interaction data.

### Claim 11 (depends on 9)

The system of claim 9, wherein the operations further comprise:
computing a state duration weight based on a statistical deviation of a duration for the click state from historical durations for the click state; and
applying the state duration weight to both the first value differential and the second value differential,
wherein the interface experience metric is computed with (i) the first value differential weighted with the state duration weight in addition to the first probability and the first transition importance and (ii) the second value differential weighted with the state duration weight in addition to the second probability and the second transition importance.

### Claim 12 (depends on 8)

The system of claim 8, wherein the operations further comprise training an experience valuation model by performing training operations comprising:
accessing training interaction data having training click states; and
modifying one or more tuning parameters applied to a sum of (i) one or more current base values and (ii) value differentials weighted with probabilities of transitioning from a click state to next states, wherein the one or more tuning parameters are modified to minimize violations of a constraint function,
wherein the constraint function specifies that a first relationship between a training interface experience metric computed for a training click state and a base value for the training click state corresponds to a second relationship between the base value for the training click state and an additional base value for a subsequent training click state.

### Claim 13 (depends on 12)

The system of claim 12, wherein the one or more tuning parameters are modified to have (i) one or more first one or more tuning parameter values for a first segment and (ii) one or more second one or more tuning parameter values for a second segment, wherein the training operations further comprise:
segmenting the training interaction data into a first training dataset and a second training dataset, wherein segmenting the training interaction data comprises:
computing representation vectors for entities corresponding to respective subsets of the training interaction data corresponding to respective user devices;
clustering the representation vectors within a vector space;
identifying the first training dataset based on first representation vectors for the first training dataset being included in a first one of the clusters, and
identifying the second training dataset based on second representation vectors for the second training dataset being included in a second one of the clusters;
modifying the one or more tuning parameters based on the first training dataset to compute the first one or more tuning parameter values for the first segment; and
modifying the one or more tuning parameters based on the second training dataset to compute the second one or more tuning parameter values for the second segment.

### Claim 14 (depends on 13)

The system of claim 13, wherein the operations further comprise:
computing a representation vector for an entity associated with the interaction data;
determining, from the representation vector, that the interaction data indicates an entity in the first segment; and
applying the experience valuation model using the first one or more tuning parameter values.

### Claim 15

A non-transitory computer-readable medium having program code that is stored thereon, the program code executable by one or more processing devices to cause the one or more processing devices to:
operate an environment evaluation system for working with a host system configured for:
performing a modification of an arrangement of interface elements of an online platform based on an interface experience metric computed from interaction data; and perform, with the environment evaluation system, operations comprising:
computing probabilities of transitioning from a click state to different next states based on the interaction data from the online platform;
computing value differentials between a current base value for the click state and subsequent base values for the different next states;
computing the interface experience metric from a sum of the current base value and the value differentials weighted with the probabilities of transitioning; and
providing the host system with the interface experience metric.

### Claim 16 (depends on 15)

The non-transitory computer-readable medium of claim 15, wherein the probabilities of transitioning include (i) a first probability of transitioning from the click state represented by the interaction data to a first predicted next state and (ii) a second probability of transitioning from the click state to a second predicted next state, wherein the value differentials include (i) a first value differential computed from a difference between the current base value and a first subsequent base value for first predicted next state and (ii) a second value differential computed from a difference between the current base value and a second subsequent base value for second predicted next state, wherein the operations further comprise:
identifying (i) a first transition importance for transitioning from the click state to the first predicted next state and (ii) a second transition importance for transitioning from the click state to the second predicted next state, wherein identifying a transition importance comprises determining, from historical interaction data, a statistical likelihood of a transition leading to a target outcome;
weighting the first value differential with the first transition importance in addition to the first probability; and
weighting the second value differential with the second transition importance in addition to the second probability,
wherein the interface experience metric is computed with (i) the first value differential weighted with the first transition importance in addition to the first probability and (ii) the second value differential weighted with the second transition importance in addition to the second probability.

### Claim 17 (depends on 16)

The non-transitory computer-readable medium of claim 16, wherein identifying each of the first transition importance and the second transition importance comprises:
computing a representation vector from the interaction data or entity data;
accessing a vector space having (i) a first cluster generated from the historical interaction data or the entity data and (ii) a second cluster generated from additional historical interaction data or the entity data; and
selecting, based on the representation vector defining a point that is closer to the first cluster than the second cluster, the first transition importance and the second transition importance from a transition importance dataset corresponding to the historical interaction data.

### Claim 18 (depends on 16)

The non-transitory computer-readable medium of claim 16, wherein the operations further comprise:
computing a state duration weight based on a statistical deviation of a duration for the click state from historical durations for the click state; and
applying the state duration weight to both the first value differential and the second value differential,
wherein the interface experience metric is computed with (i) the first value differential weighted with the state duration weight in addition to the first probability and the first transition importance and (ii) the second value differential weighted with the state duration weight in addition to the second probability and the second transition importance.

### Claim 19 (depends on 15)

The non-transitory computer-readable medium of claim 15, wherein the operations further comprise training an experience valuation model, which computes the interface experience metric, by performing training operations comprising:
accessing training interaction data having training click states; and
modifying one or more tuning parameters applied to a sum of (i) one or more current base values and (ii) value differentials weighted with probabilities of transitioning from a click state to next states, wherein the one or more tuning parameters are modified to minimize violations of a constraint function,
wherein the constraint function specifies that a first relationship between a training interface experience metric computed for a training click state and a base value for the training click state corresponds to a second relationship between the base value for the training click state and an additional base value for a subsequent training click state.

### Claim 20 (depends on 19)

The non-transitory computer-readable medium of claim 19, wherein the one or more tuning parameters are modified to have (i) one or more first one or more tuning parameter values for a first segment and (ii) one or more second one or more tuning parameter values for a second segment, wherein the training operations further comprise:
segmenting the training interaction data into a first training dataset and a second training dataset, wherein segmenting the training interaction data comprises:
computing representation vectors for entities corresponding to respective subsets of the training interaction data corresponding to respective user devices;
clustering the representation vectors within a vector space;
identifying the first training dataset based on first representation vectors for the first training dataset being included in a first one of the clusters, and
identifying the second training dataset based on second representation vectors for the second training dataset being included in a second one of the clusters;
modifying the one or more tuning parameters based on the first training dataset to compute the first one or more tuning parameter values for the first segment;
modifying the one or more tuning parameters based on the second training dataset to compute the second one or more tuning parameter values for the second segment;
computing a representation vector for an entity associated with the interaction data;
determining, from the representation vector, that the interaction data indicates the entity in the first segment; and
applying the experience valuation model using the first one or more tuning parameter values.
