Back to results
US 2024/0420197 A1Published application
Reinforcement learning-based digital twin models
AssigneePlaystudios US, LLC
InventorsHeinrich Fidencio Terborg, Peter Kendall Landi, Shih-Chien Lee
Life of this patent
Estimating the term…
Abstract
Data describing how multiple users interact with an online system is gathered. For each user, a digital twin model is trained using reinforcement learning on their data to predict their interactions in various virtual environment variants. This model is then run in several candidate virtual environment variants to simulate how the user might interact with each. A score is assigned to each environment based on the likelihood of the user interacting in a specific way. The virtual environment variant with the best score is selected and displayed to the user.
Claims20
A method comprising:
- collecting user data describing how a plurality of users interact with an online system;
- for each of the plurality of users,
- applying reinforcement learning to the user data of a corresponding user to train a digital twin model that predicts user interactions in any given virtual environment variant;
- configuring a plurality of virtual environment variants, each differing in at least one of layout, product placement, navigational element, or promotional strategy;
- executing the digital twin model across the plurality of virtual environment variants to simulate user interactions with each of the plurality of virtual environment variants;
- for each of the plurality of virtual environment variants,
- logging the user interactions simulated by the digital twin model in the virtual environment variant;
- and
- analyzing the logged user interactions to determine a set of engagement metrics associated with the virtual environment variant;
- comparing the sets of engagement metrics associated with the plurality of virtual environment variants to identify a set of engagement metrics that indicates a higher user engagement or satisfaction;
- selecting a virtual environment variant associated with the identified set of engagement metrics that indicates the higher user engagement or satisfaction;
- and
- providing for display of the selected virtual environment variant to the user.
10 more elements
A non-transitory computer-readable storage medium comprising stored instructions thereon that, when executed by a processor system, cause the processor system to:
- collect user data describing how a plurality of users interact with an adversarial system;
- for each of the plurality of users,
- apply reinforcement learning to the user data of a corresponding user to train a digital twin model that predicts user interactions in any given virtual environment variant;
- configure a plurality of virtual environment variants, each differing in at least one of layout, product placement, navigational element, or promotional strategy;
- execute the digital twin model across the plurality of virtual environment variants to simulate user interactions with each of the plurality of virtual environment variants;
- for each of the plurality of virtual environment variants,
- log the user interactions simulated by the digital twin model in the virtual environment variant;
- and
- analyze the logged user interactions to determine a set of engagement metrics associated with the virtual environment variant;
- compare the sets of engagement metrics associated with the plurality of virtual environment variants to identify a set of engagement metrics that indicates a higher user engagement or satisfaction;
- select a virtual environment variant associated with the identified set of engagement metrics that indicates the higher user engagement or satisfaction;
- and
- provide for display of the selected virtual environment variant to the user.
10 more elements
A computing system, comprising
- a processor system comprised of one or more processors;
- and
- a non-transitory computer-readable storage medium comprising stored instructions thereon that, when executed by one or more processors, cause the processor system to:
- collect user data describing how a plurality of users interact with an adversarial system;
- for each of the plurality of users,
- apply reinforcement learning to the user data of a corresponding user to train a digital twin model that predicts user interactions in any given virtual environment variant;
- configure a plurality of virtual environment variants, each differing in at least one of layout, product placement, navigational element, or promotional strategy;
- execute the digital twin model across the plurality of virtual environment variants to simulate user interactions with each of the plurality of virtual environment variants;
- for each of the plurality of virtual environment variants,
- log the user interactions simulated by the digital twin model in the virtual environment variant;
- and
- analyze the logged user interactions to determine a set of engagement metrics associated with the virtual environment variant;
- compare the sets of engagement metrics associated with the plurality of virtual environment variants to identify a set of engagement metrics that indicates a higher user engagement or satisfaction;
- select a virtual environment variant associated with the identified set of engagement metrics that indicates the higher user engagement or satisfaction;
- and
- provide for display of the selected virtual environment variant to the user.
13 more elements
Dependencies are inferred from the claim text.