# US 2023/0264104 A1 · Systems and Methods for Dynamic Modification of Audio Playback with User Feedback > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 2023/0264104 A1 - Document ID: US20230264104A1 - Status: Published application - Kind: A1 - Application number: 18/309,431 - Filed: 2023-04-28 - Published: 2023-08-24 - Earliest claimed: 2017-09-20 - Assignees: Sony Interactive Entertainment Inc. - Applicants: Sony Interactive Entertainment Inc. - Inventors: Michael Taylor; Glenn Black - Web page: https://patentgrep.com/patent/US20230264104A1 ## CPC - A63F 13/54: involving acoustic signals, e.g. for simulating revolutions per minute [RPM] dependent engine sounds in a driving game or reverberation against a virtual wall - A63F 13/60: Generating or modifying game content before or while executing the game program, e.g. authoring tools specially adapted for game development or game-integrated level editor - A63F 13/67: adaptively or by learning from player actions, e.g. skill level adjustment or by storing successful combat sequences for re-use - A63F 2300/6009: for importing or creating game content, e.g. authoring tools during game development, adapting content to different platforms, use of a scripting language to create content - A63F 2300/6081: generating an output signal, e.g. under timing constraints, for spatialization - A63F: CARD, BOARD, OR ROULETTE GAMES; INDOOR GAMES USING SMALL MOVING PLAYING BODIES; VIDEO GAMES; GAMES NOT OTHERWISE PROVIDED FOR ## Abstract A method for dynamically modifying audio playback of a video game is provided. The method includes ascertaining game music data associated with the video game. The game music data may include a plurality of game soundtracks classified according to predetermined criteria. The method may include collecting feedback while the user is playing the video game, the feedback including data associated with the user’s performance; an express user action; an evaluation of game results; the user’s emotional state; and a user input. The method may continue with determining one or more replacement soundtracks based on criteria associated with the one or more replacement soundtracks. The method may further include dynamically modifying the game music data while the user is playing the video game by replacing at least one of the plurality of game soundtracks with the one or more replacement soundtracks to obtain modified game music data. ## Claims ### Claim 1 A system for dynamically modifying audio playback of a video game, the system comprising: a processor configured to: while a user is playing a video game: ascertain game music data associated with the video game; collect feedback over a period of time while the user is playing the video game, the feedback including at least one of the following: data associated with a performance of the user in the video game; an express action of the user; an evaluation of game results; an emotional state of the user; and an input of the user; and based on the feedback, dynamically modify the game music data. ### Claim 2 (depends on 1) The system of claim 1, wherein the input includes at least one of: a tactile input, an audio input, a video input, and a biometric input. ### Claim 3 (depends on 1) The system of claim 1, wherein the emotional state of the user includes one or more of the following: a biometrical response, a facial expression, a facial motion, and a stress level. ### Claim 4 (depends on 1) The system of claim 1, wherein the express action includes one or more of the following: pressing a “like” button, pressing a “thumbs up” button, pressing a “thumbs down” button, enabling a pop-up, the pop-up showing at least a title of a song provided to the user in the modified game music data, and disabling the pop-up. ### Claim 5 (depends on 1) The system of claim 1, the processor being further configured to: access a music source, the music source including one or more replacement soundtracks; determine audio parameters for each of the one or more replacement soundtracks; classify the one or more replacement soundtracks based on audio parameters; and index the one or more replacement soundtracks based on the classifying to provide replacement soundtrack indexes, wherein the replacement soundtrack indexes are used to match a classification of the one or more replacement soundtracks with a classification of a plurality of game soundtracks. ### Claim 6 (depends on 5) The system of claim 5, wherein the classification of each of the plurality of game soundtracks and the classification of the one or more replacement soundtracks is determined by a machine-learning system; and wherein the machine-learning system is trained based on historical user data stored in a historical data database. ### Claim 7 (depends on 1) The system of claim 1, wherein the video game comprises a computer game, a console game, an online game, a network game, a mobile device game, a multiplayer game, or a virtual reality game. ### Claim 8 A computer-implemented method for dynamically modifying audio playback of a video game, the computer-implemented method comprising: providing a processor, the processor configured to: while a user is playing a video game: ascertain game music data associated with the video game; collect feedback, the feedback including at least one of the following: data associated with a performance of the user in the video game; an express action of the user; an evaluation of game results; an emotional state of the user; and an input of the user; based on the feedback, determine one or more replacement soundtracks; and dynamically modify the game music data. ### Claim 9 (depends on 8) The computer-implemented method of claim 8, further comprising storing the feedback in a historical data database, the historical data database storing a plurality of feedbacks associated with a plurality of users collected while each of the plurality of users is playing the video game. ### Claim 10 (depends on 8) The computer-implemented method of claim 8, wherein the input includes at least one of: a tactile input, an audio input, a video input, and a biometric input. ### Claim 11 (depends on 8) The computer-implemented method of claim 8, wherein the emotional state of the user includes one or more of the following: a biometrical response, a facial expression, a facial motion, and a stress level. ### Claim 12 (depends on 8) The computer-implemented method of claim 8, wherein the express action includes one or more of the following: pressing a “like” button, pressing a “thumbs up” button, pressing a “thumbs down” button, enabling a pop-up, the pop-up showing at least a title of a song provided to the user in the modified game music data, and disabling the pop-up. ### Claim 13 (depends on 8) The computer-implemented method of claim 8, further comprising: searching in a historical data database for one or more users having parameters similar to parameters of the user; selecting one or more game soundtracks recommended or preferred by the one or more users; and wherein the dynamically modifying of the game music data further includes replacing at least one of a plurality of game soundtracks with the one or more game soundtracks recommended or preferred by the one or more users. ### Claim 14 (depends on 8) The computer-implemented method of claim 8, further comprising: accessing a music source, the music source including the one or more replacement soundtracks; determining audio parameters for each of the one or more replacement soundtracks; classifying the one or more replacement soundtracks based on the audio parameters; and indexing the one or more replacement soundtracks based on the classifying to provide replacement soundtrack indexes, wherein the replacement soundtrack indexes are used to match a classification of the one or more replacement soundtracks with a classification of a plurality of game soundtracks. ### Claim 15 (depends on 14) The computer-implemented method of claim 14, wherein the classification of each of the plurality of game soundtracks and the classification of the one or more replacement soundtracks is determined by a machine-learning system; and wherein the machine-learning system is trained based on historical user data stored in a historical data database. ### Claim 16 (depends on 14) The computer-implemented method of claim 14, wherein the audio parameters include at least one of: a beats-per-minute value, a tonality value, a pitch value, a timbre value, a harmonics value, a loudness value, a rhythm value, a sound envelope, a music genre, a release date, and an emotional classification. ### Claim 17 (depends on 8) The computer-implemented method of claim 8, further comprising identifying sound triggers based on a gameplay of the video game, actions of the user in the video game, or actions of further users in the video game, wherein replacement of at least one of a plurality of game soundtracks with the one or more replacement soundtracks is performed at times associated with the sound triggers. ### Claim 18 (depends on 15) The computer-implemented method of claim 15, wherein the classification of the one or more replacement soundtracks includes an emotional classification, the emotional classification being determined based on one or more of: the audio parameters, the game music data, gameplay, game content, language associated with gameplay, and the sound triggers. ### Claim 19 (depends on 8) The computer-implemented method of claim 8, further comprising temporarily displaying, within the video game, an identifier of the one or more replacement soundtracks used to replace at least one of a plurality of game soundtracks. ### Claim 20 A non-transitory processor-readable medium having instructions stored thereon, which when executed by one or more processors, cause the one or more processors to implement a method for dynamically modifying audio playback of a video game, the method comprising: while a user is playing a video game: ascertaining game music data associated with a video game; collecting feedback, the feedback including at least one of the following: data associated with a performance of the user in the video game; an express action of the user; an evaluation of game results; an emotional state of the user; and an input of the user; based on the feedback, determining one or more replacement soundtracks; and dynamically modifying the game music data.