Gesture training for skill adaptation and accessibility
AssigneeSony Interactive Entertainment Inc.
InventorsSteven Osman, Olga Rudi, Mahdi Azmandian, Jorge Arroyo Palacios, Ravi Coimbatore Madhavan
Life of this patent
Estimating the term…
Abstract
A system that teaches players gestures, for instance during the introduction of the game, and asks the player to invoke the gesture. Rather than asking the player to repeat over and over until the player succeeds, the game looks for commonality in the player's attempts, and after a small number of attempts, the game can learn how that player interprets the gesture given the player's own ability. The game can then adapt itself to look for that pattern to trigger the action.
Claims20
A computer-implemented method, comprising:
- for a particular computer simulation command, obtaining sensor data associated with a user performing a gesture multiple times;
- for each performance of the gesture by the user, generating a training data set for training at least one machine learning (ML) model that correlates input sensor data to output computer simulation commands, the training data set comprising (i) a portion of the sensor data that corresponds to the performance of the gesture by the user, and (ii) a label identifying the particular computer simulation command as being correlated to the sensor data;
- training the at least one ML model using the training data sets;
- during execution of the computer simulation, obtaining additional sensor data associated with the user performing the gesture;
- providing the additional sensor data to the ML model;
- receiving from the ML model an indication that the particular computer simulation command is correlated to the additional sensor data;
- and
- in response to receiving the indication from the ML model, executing the computer simulation according to the particular computer simulation command.
A system comprising one or more computers and one or more storage devices storing instructions that are operable, when executed by the one or more computers, to cause the one or more computers to perform operations comprising:
- for a particular computer simulation command, obtaining sensor data associated with a user performing a gesture multiple times;
- for each performance of the gesture by the user, generating a training data set for training at least one machine learning (ML) model that correlates input sensor data to output computer simulation commands, the training data set comprising (i) a portion of the sensor data that corresponds to the performance of the gesture by the user, and (ii) a label identifying the particular computer simulation command as being correlated to the sensor data;
- training the at least one ML model using the training data sets;
- during execution of the computer simulation, obtaining additional sensor data associated with the user performing the gesture;
- providing the additional sensor data to the ML model;
- receiving from the ML model an indication that the particular computer simulation command is correlated to the additional sensor data;
- and
- in response to receiving the indication from the ML model, executing the computer simulation according to the particular computer simulation command.
One or more non-transitory computer readable storage media storing instructions that when executed by one or more computers cause the one or more computers to perform operations comprising:
- for a particular computer simulation command, obtaining sensor data associated with a user performing a gesture multiple times;
- for each performance of the gesture by the user, generating a training data set for training at least one machine learning (ML) model that correlates input sensor data to output computer simulation commands, the training data set comprising (i) a portion of the sensor data that corresponds to the performance of the gesture by the user, and (ii) a label identifying the particular computer simulation command as being correlated to the sensor data;
- training the at least one ML model using the training data sets;
- during execution of the computer simulation, obtaining additional sensor data associated with the user performing the gesture;
- providing the additional sensor data to the ML model;
- receiving from the ML model an indication that the particular computer simulation command is correlated to the additional sensor data;
- and
- in response to receiving the indication from the ML model, executing the computer simulation according to the particular computer simulation command.
Dependencies are inferred from the claim text.