# US 11,896,902 B1 · Emotion based music style change using deep learning > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 11,896,902 B1 - Document ID: US11896902B1 - Status: Granted - Kind: B1 - Application number: 18/179,725 - Filed: 2023-03-07 - Granted: 2024-02-13 - Earliest claimed: 2021-04-13 - Assignees: Electronic Arts Inc. - Applicants: Electronic Arts Inc. - Inventors: Jie Sheng; Hulya Duygu Yigit; Chong Zhao - Web page: https://patentgrep.com/patent/US11896902B1 ## CPC - A63F 13/54: involving acoustic signals, e.g. for simulating revolutions per minute [RPM] dependent engine sounds in a driving game or reverberation against a virtual wall - A63F 13/40: Processing input control signals of video game devices, e.g. signals generated by the player or derived from the environment - A63F 13/67: adaptively or by learning from player actions, e.g. skill level adjustment or by storing successful combat sequences for re-use - A63F 13/79: involving player-related data, e.g. identities, accounts, preferences or play histories - A63F: CARD, BOARD, OR ROULETTE GAMES; INDOOR GAMES USING SMALL MOVING PLAYING BODIES; VIDEO GAMES; GAMES NOT OTHERWISE PROVIDED FOR ## Abstract Various aspects of the subject technology relate to systems, methods, and machine-readable media for changing music of a video game based on a player's emotion. The method includes receiving indicators of emotion comprising in-game attributes of a player in a video game. The method also includes predicting an emotion of the player based on the indicators of emotion from the video game. The method also includes receiving original music from the video game. The method also includes determining an original tone of the original music. The method also includes determining a transformed tone based at least in part on the emotion of the player that was predicted. The method also includes transforming the original tone of the original music to the transformed tone. The method also includes generating transformed music from the original music based on the transformed tone. ## Claims ### Claim 1 A computer-implemented method for changing music of a video game based on a player's emotion, comprising: receiving indicators of emotion comprising in-game attributes of a player in a video game; receiving original music from the video game; determining an original tone of the original music using a model trained to recognize an original tone from the original music; and generating transformed music from the original music by a variational autoencoder, wherein the variational autoencoder: compresses the original music into a dimensional representation and reconstructs the dimensional representation based at least in part on the indicators of emotion. ### Claim 2 (depends on 1) The computer-implemented method of claim 1, further comprising predicting an emotion of the player based on the indicators of emotion from the video game. ### Claim 3 (depends on 2) The computer-implemented method of claim 2, wherein inputs to the model comprise at least one of a piano roll, a hold roll, and/or a velocity roll. ### Claim 4 (depends on 2) The computer-implemented method of claim 2, wherein the in-game attributes of the player comprise at least one of rate of game progress, damage taken, damage caused, number of kills, and/or a status of game action. ### Claim 5 (depends on 1) The computer-implemented method of claim 1, wherein the original tone may be based on a category of the original music. ### Claim 6 (depends on 1) The computer-implemented method of claim 1, wherein inputs to the model comprise at least one of a piano roll, a hold roll, and/or a velocity roll. ### Claim 7 (depends on 1) The computer-implemented method of claim 1, further comprising outputting the transformed music to the video game. ### Claim 8 (depends on 1) The computer-implemented method of claim 1, wherein the emotion comprises at least one of happy, sad, angry, afraid, or motivated. ### Claim 9 A system for changing music of a video game based on a player's emotion, comprising: a processor; and a memory comprising instructions stored thereon, which when executed by the processor, causes the processor to perform: receiving indicators of emotion comprising in-game attributes of a player in a video game; receiving original music from the video game; determining an original tone of the original music using a model trained to recognize the original tone from the original music; and generating transformed music from the original music by a variational autoencoder, wherein the variational autoencoder: compresses the original music into a dimensional representation and reconstructs the dimensional representation based at least in part on the indicators of emotion. ### Claim 10 (depends on 9) The system of claim 9, wherein the in-game attributes of the player comprise at least one of rate of game progress, damage taken, damage caused, number of kills, and/or a status of game action. ### Claim 11 (depends on 9) The system of claim 9, further comprising predicting an emotion of the player based on the indicators of emotion from the video game. ### Claim 12 (depends on 11) The system of claim 11 wherein the memory further comprises instructions stored thereon, which when executed by the processor, causes the processor to determine a transformed tone from the variational autoencoder based at least in part on the emotion of the player that was predicted. ### Claim 13 (depends on 9) The system of claim 9 wherein the memory further comprises instructions stored thereon, which when executed by the processor, causes the processor to transform the original tone of the original music to a transformed tone by the variational autoencoder. ### Claim 14 (depends on 9) The system of claim 9, further comprising stored sequences of instructions, which when executed by the processor, cause the processor to perform: training a model to recognize a tone of music. ### Claim 15 (depends on 9) The system of claim 9, further comprising stored sequences of instructions, which when executed by the processor, cause the processor to perform: outputting the transformed music to the video game. ### Claim 16 A non-transitory computer-readable storage medium comprising instructions stored thereon, which when executed by the one or more processors, cause the one or more processors to perform operations for changing music of a video game based on a player's emotion, the operations comprising: receiving indicators of emotion comprising in-game attributes of a player in a video game; receiving original music from the video game; determining an original tone of the original music using a model trained to recognize the original tone from the original music; transforming the original tone of the original music to a transformed tone by a variational autoencoder; and generating transformed music from the original music based on the transformed tone by the variational autoencoder, wherein the variational autoencoder: compresses the original music into a dimensional representation and reconstructs the dimensional representation based at least in part on the indicators of emotion. ### Claim 17 (depends on 16) The non-transitory computer-readable storage medium of claim 16, wherein the in-game attributes of the player comprise at least one of rate of game progress, damage taken, damage caused, number of kills, and/or a status of game action. ### Claim 18 (depends on 16) The non-transitory computer-readable storage medium of claim 16, wherein the original tone may be based on a category of the original music. ### Claim 19 (depends on 16) The non-transitory computer-readable storage medium of claim 16, further comprising instructions stored thereon, which when executed by one or more processors, cause the one or more processors to perform operations comprising predicting an emotion of the player based on the indicators of emotion from the video game. ### Claim 20 (depends on 19) The non-transitory computer-readable storage medium of claim 19, further comprising determining a transformed tone based at least in part on the emotion of the player that was predicted.