# US 12,118,976 B1 · Computer-implemented method and computer system for configuring a pretrained text to music AI model and related methods > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 12,118,976 B1 - Document ID: US12118976B1 - Status: Granted - Kind: B1 - Application number: 18/622,365 - Filed: 2024-03-29 - Granted: 2024-10-15 - Earliest claimed: 2024-03-29 - Assignees: Futureverse IP Limited - Applicants: Futureverse IP Limited - Inventors: Boyu Chen; Peike Li; Yao Yao; Yijun Wang - Web page: https://patentgrep.com/patent/US12118976B1 ## CPC - G10L 13/027: Concept to speech synthesisers; Generation of natural phrases from machine-based concepts (generation of parameters for speech synthesis out of text G10L13/08) - G10L: SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING ## Abstract The method involves configuring a pretrained text to music AI model that includes a neural network implementing a diffusion model. The process includes receiving audio sample data corresponding to a specific audio concept, generating a concept identifier token based on the audio sample data, adapting a loss function of the diffusion model based on the concept identifier token, selecting pivotal parameters in weight matrices in a self-attention layer of the neural network of the AI model based on the audio sample data, and further training the pivotal parameters of the AI model, to optimize the AI model for the specific audio concept. ## Claims ### Claim 1 A computer-implemented method for configuring a pretrained text to music artificial intelligence (AI) model that includes a neural network implementing a diffusion model, the method comprising: receiving audio sample data corresponding to a specific audio concept; generating at least one concept identifier token based on the audio sample data, wherein the at least one concept identifier token represents unique characteristics of the audio sample data; adapting a loss function of the diffusion model based on the at least one concept identifier token; selecting pivotal parameters in weight matrices in a self-attention layer of the neural network of the AI model based on the audio sample data; and further training the pivotal parameters of the AI model, to thereby optimize the AI model for the specific audio concept, whereby the diffusion model is able to generate music corresponding to the specific audio concept. ### Claim 2 (depends on 1) The method of claim 1, wherein the specific audio concept is the style of a specified artist. ### Claim 3 (depends on 1) The method of claim 1, wherein the specific audio concept is the sound of a specified musical instrument. ### Claim 4 (depends on 1) The method of claim 1, wherein the step of selecting pivotal parameters comprises: initializing a trainable mask which has the same shape as the self-attention layer; elementwise multiplying the trainable mask with parameters of the self-attention layer during calculation for the neural network to derive a refined mask form the trainable mask; and selecting, as the pivotal parameters, subset of the parameters having a high variation between the trainable mask and the refined mask. ### Claim 5 (depends on 4) The method of claim 4, wherein the subset comprises a predetermined percentage of the parameters. ### Claim 6 (depends on 4) The method of claim 4, wherein the subset comprises a predetermined number of the parameters. ### Claim 7 (depends on 1) The method of claim 1, wherein the at least one concept identifier token comprises two or more concept identifier tokens. ### Claim 8 (depends on 1) The method of claim 1, wherein further training the pivotal parameters of the AI model, to thereby optimize the AI model for the specific audio concept comprises training only the pivotal parameters. ### Claim 9 (depends on 1) The method of claim 1, wherein the specific concept is at least one of a music genre, an artist's style, and a musical instrument. ### Claim 10 A computer system for configuring a pretrained text to music artificial intelligence (AI) model that includes a neural network implementing a diffusion model, the method comprising: at least one computer hardware processor; and at least one memory operatively couple to the at least one computer hardware and having computer executable instructions stored therein which, when executed by the at least one computer processor, cause the at least one computer processor to carry out a method comprising: receiving audio sample data corresponding to a specific audio concept; generating at least one concept identifier token based on the audio sample data, wherein the at least one concept identifier token represents unique characteristics of the audio sample data; adapting a loss function of the diffusion model based on the at least one concept identifier token; selecting pivotal parameters in weight matrices in a self-attention layer of the neural network of the AI model based on the audio sample data; and further training the pivotal parameters of the AI model, to thereby optimize the AI model for the specific audio concept, whereby the diffusion model is able to generate music corresponding to the specific audio concept. ### Claim 11 (depends on 10) The system of claim 10, wherein the specific audio concept is the style of a specified artist. ### Claim 12 (depends on 10) The system of claim 10, wherein the specific audio concept is the sound of a specified musical instrument. ### Claim 13 (depends on 10) The method of claim 10, wherein the step of selecting pivotal parameters comprises: initializing a trainable mask which has the same shape as the self-attention layer; elementwise multiplying the trainable mask with parameters of the self-attention layer during calculation for the neural network to derive a refined mask form the trainable mask; and selecting, as the pivotal parameters, subset of the parameters having a high variation between the trainable mask and the refined mask. ### Claim 14 (depends on 13) The system of claim 13, wherein the subset comprises a predetermined percentage of the parameters. ### Claim 15 (depends on 13) The system of claim 13, wherein the subset comprises a predetermined number of the parameters. ### Claim 16 (depends on 10) The system of claim 10, wherein the at least one concept identifier token comprises two or more concept identifier tokens. ### Claim 17 (depends on 10) The system of claim 10, wherein further training the pivotal parameters of the AI model, to thereby optimize the AI model for the specific audio concept comprises training only the pivotal parameters. ### Claim 18 (depends on 10) The system of claim 10, wherein the specific concept is at least one of a music genre, an artist's style, and a musical instrument.