# US 12,190,851 B2 · Audio generation methods and systems > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 12,190,851 B2 - Document ID: US12190851B2 - Status: Granted - Kind: B2 - Application number: 17/865,869 - Filed: 2022-07-15 - Published: 2023-01-19 - Granted: 2025-01-07 - Earliest claimed: 2021-07-16 - Assignees: Sony Interactive Entertainment Europe Limited - Applicants: Sony Interactive Entertainment Europe Limited - Inventors: Adrian Barahona Rios - Web page: https://patentgrep.com/patent/US12190851B2 ## CPC - G10H 1/0008: {Associated control or indicating means} - G10K 15/02: Synthesis of acoustic waves (synthesis of speech G10L13/00) - A63F 13/54: involving acoustic signals, e.g. for simulating revolutions per minute [RPM] dependent engine sounds in a driving game or reverberation against a virtual wall - G10L 21/04: Time compression or expansion - G10H 2220/135: Musical aspects of games or videogames; Musical instrument-shaped game input interfaces - G10H 2250/235: Fourier transform; Discrete Fourier Transform [DFT]; Fast Fourier Transform [FFT] - G10H 2250/311: Neural networks for electrophonic musical instruments or musical processing, e.g. for musical recognition or control, automatic composition or improvisation - G10H: ELECTROPHONIC MUSICAL INSTRUMENTS; INSTRUMENTS IN WHICH THE TONES ARE GENERATED BY ELECTROMECHANICAL MEANS OR ELECTRONIC GENERATORS, OR IN WHICH THE TONES ARE SYNTHESISED FROM A DATA STORE - G10K: SOUND-PRODUCING DEVICES; METHODS OR DEVICES FOR PROTECTING AGAINST, OR FOR DAMPING, NOISE OR OTHER ACOUSTIC WAVES IN GENERAL; ACOUSTICS NOT OTHERWISE PROVIDED FOR - A63F: CARD, BOARD, OR ROULETTE GAMES; INDOOR GAMES USING SMALL MOVING PLAYING BODIES; VIDEO GAMES; GAMES NOT OTHERWISE PROVIDED FOR - G10L: SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING ## Abstract A method of generating audio assets, comprising the steps of: receiving an input audio asset having a first duration, generating an input image representative of the input audio asset, training a generative model on the input image and implementing the trained generative model to generate an output image representative of an output audio asset having a second duration different to the first duration, and generating the output audio asset based on the output image. ## Claims ### Claim 1 A method of generating audio assets, comprising the steps of: receiving an input audio asset having a first duration, generating an input image representative of the input audio asset, training a generative model on the input image and implementing the trained generative model to generate an output image representative of an output audio asset having a second duration different to the first duration, and generating the output audio asset based on the output image, wherein the input image and output image each comprise an axis representative of time duration, and the step of generating an output image comprises retargeting the input image along the axis representative of time duration, and wherein the output image has a larger dimension along the axis representative of time duration than the input image. ### Claim 2 (depends on 1) A method according to claim 1, wherein the output image has a dimension along the axis representative of time duration which is multiplied by a scale factor. ### Claim 3 (depends on 1) A method according to claim 1, wherein the input image is an input spectrogram, and the output image is an output spectrogram. ### Claim 4 (depends on 3) The method according to claim 3, wherein the step of generating an input image comprises performing a Fourier transform on the input audio asset, and the step of generating the output audio asset comprises performing an inverse Fourier transform on the output image. ### Claim 5 (depends on 1) The method according to claim 1, wherein the generative model is a single-image generative model comprising a generative adversarial network, GAN, having a generator and a patch discriminator. ### Claim 6 (depends on 1) A method according to claim 1, wherein the step of receiving an input audio asset comprises receiving, from a video game environment, video game information, and the step of generating the output image comprises feeding the video game information into the generative model such that the output image is influenced by the video game information. ### Claim 7 (depends on 1) A method according to claim 1, further comprising the step of storing the trained generative model on a memory, configured to be accessed to generate further audio assets. ### Claim 8 (depends on 1) A method according to claim 1, wherein the step of receiving an input audio asset comprises receiving a second input audio asset, and the step of generating an input image comprises generating a multi-channel image, comprising a first channel having an image representative of the input audio asset, and a second channel having an image representative of the second input audio asset. ### Claim 9 A non-transitory computer-readable medium having stored thereon a computer program comprising computer-implemented instructions that, when run on a computer, cause the computer to implement a method of generating audio assets, comprising the steps of: receiving an input audio asset having a first duration, generating an input image representative of the input audio asset, training a generative model on the input image and implementing the trained generative model to generate an output image representative of an output audio asset having a second duration different to the first duration, and generating the output audio asset based on the output image, wherein the input image and output image each comprise an axis representative of time duration, and the step of generating an output image comprises retargeting the input image along the axis representative of time duration, and wherein the output image has a larger dimension along the axis representative of time duration than the input image. ### Claim 10 A system for generating audio assets, the system comprising: an asset input unit configured to receive an input audio asset having a first duration, and to convert the input audio asset into an input image, an image generation unit configured to implement a generative model to generate one or more output images based on the input image, the output image representing an output audio asset having a second duration different to the first duration, and an asset output unit configured to generate an output audio asset based on the output image, wherein the input image and output image each comprise an axis representative of time duration, and the step of generating an output image comprises retargeting the input image along the axis representative of time duration, and wherein the output image has a larger dimension along the axis representative of time duration than the input image. ### Claim 11 (depends on 10) A system according to claim 10, further comprising a transform unit configured to perform Fourier transform operations and inverse Fourier transform operations to convert between audio and graphical files, and wherein the asset input unit is configured to access the transform unit to convert the input audio asset into an input image, and the asset output unit is configured to access the transform unit to convert each output image into an output audio asset. ### Claim 12 (depends on 10) A system according to claim 10, further comprising a video game data processing unit, configured to process video game information derived from or relating to a video game environment and feed through to one or more of the asset input unit, the image generation unit and the asset output unit, and the image generation unit is configured to implement the generative model based at least in part on the video game information. ### Claim 13 (depends on 10) A system according to claim 10, configured to store the generative model on the memory.