# US 2026/0073302 A1 · Method, apparatus, device, and storage medium for training model

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2026/0073302 A1
- Document ID: US20260073302A1
- Status: Published application
- Kind: A1
- Application number: 19/321,042
- Filed: 2025-09-05
- Published: 2026-03-12
- Earliest claimed: 2024-09-06
- Assignees: Beijing Zitiao Network Technology Co., Ltd.; Lemon Inc.
- Applicants: Beijing Zitiao Network Technology Co., Ltd.; Lemon Inc.
- Inventors: Yiqing LU; Zhongyi Huang; Qingqing Huang; Jitong Chen; Shuo Zhang; Lamtharn Hantrakul
- Web page: https://patentgrep.com/patent/US20260073302A1

## CPC

- G06N 20/00: Machine learning
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS

## Abstract

A method, an apparatus, a device, and a storage medium for training a model are provided. First audio content associated with vocal content is extracted from a music sample. First annotation information is generated based on the first audio content, and the first annotation information includes text content corresponding to the first audio content and first melody information of the first audio content. A first training sequence is constructed based on the text content and the first melody information. The first training sequence is input to a music generation model to generate a first set of music encoded representations. The music generation model is trained based on the first set of music encoded representations and a second set of music encoded representations of the music sample.

## Claims

### Claim 1

A method of training a model, comprising:
extracting, from a music sample, first audio content associated with vocal content;
generating first annotation information based on the first audio content, the first annotation information comprising text content corresponding to the first audio content and first melody information of the first audio content;
constructing a first training sequence based on the text content and the first melody information;
inputting the first training sequence to a music generation model to generate a first set of music encoded representations; and
training the music generation model based on the first set of music encoded representations and a second set of music encoded representations of the music sample.

### Claim 2 (depends on 1)

The method of claim 1, wherein generating the first annotation information comprises:
processing the first audio content with a text recognition model to determine the text content; and/or
processing the first audio content with a melody recognition model to determine the first melody information.

### Claim 3 (depends on 2)

The method of claim 2, wherein constructing the first training sequence based on the text content and the first melody information comprises:
constructing a first sequence part corresponding to the text content, based on the text content and first time information corresponding to the text content;
constructing a second sequence part corresponding to the first melody information, based on the first melody information and second time information corresponding to the first melody information; and
constructing the first training sequence based on the first sequence part and the second sequence part.

### Claim 4 (depends on 3)

The method of claim 3, wherein:
the first sequence part indicates a plurality of lyrics elements and a time distribution corresponding to the plurality of lyrics elements, and/or
the second sequence part indicates a first set of melody elements and a time distribution corresponding to the first set of melody elements.

### Claim 5 (depends on 1)

The method of claim 1, further comprising:
processing the first audio content with a trained discrete encoder to determine the second set of music encoded representations.

### Claim 6 (depends on 1)

The method of claim 1, wherein training the music generation model based on the first set of music encoded representations and the second set of music encoded representations of the music sample comprises:
determining a training loss based on a comparison of the first set of music encoded representations and the second set of music encoded representations; and
updating a model parameter of the music generation model based on the training loss.

### Claim 7 (depends on 1)

The method of claim 1, wherein inputting the first training sequence to the music generation model comprises:
encoding the first training sequence with a text digital hybrid encoder to generate a hybrid encoded representation; and
inputting the hybrid encoded representation to the music generation model.

### Claim 8 (depends on 1)

The method of claim 1, further comprising:
obtaining sampled second audio content, the second audio content corresponding to a voice track;
obtaining second annotation information of the second audio content, the second annotation information comprising phoneme information corresponding to the second audio content and second melody information of the second audio content; and
fine-tuning the music generation model based on the second annotation information of the second audio content.

### Claim 9 (depends on 8)

The method of claim 8, wherein the second annotation information indicates:
a plurality of phonemes corresponding to the second audio content, and a first time distribution corresponding to the plurality of phonemes; and/or
a second set of melody elements corresponding to the second audio content, and a second time distribution corresponding to the second set of melody elements.

### Claim 10 (depends on 9)

The method of claim 9, further comprising:
processing the second audio content with a melody recognition model to generate a third set of melody elements; and
updating the third set of melody elements based on the first time distribution of the plurality of phonemes to determine the second set of melody elements.

### Claim 11 (depends on 10)

The method of claim 10, wherein updating the third set of melody elements based on the first time distribution of the plurality of phonemes comprises:
determining, for a target time segment of the first time distribution corresponding to a target phoneme, at least one melody element corresponding to the target time segment based on the third set of melody elements;
determining a target melody element corresponding to the target time segment based on the at least one melody element; and
updating the third set of melody elements with the target melody element.

### Claim 12 (depends on 11)

The method of claim 11, wherein determining the target melody element corresponding to the target time segment based on the at least one melody element comprises:
determining a fundamental frequency median corresponding to the target time segment based on the at least one melody element; and
determining the target melody element based on the fundamental frequency median.

### Claim 13

An electronic device, comprising:
at least one processor; and
at least one memory coupled to the at least one processor and storing instructions for execution by the at least one processor, the instructions, when executed by the at least one processor, causing the electronic device to perform acts comprising:
extracting, from a music sample, first audio content associated with vocal content;
generating first annotation information based on the first audio content, the first annotation information comprising text content corresponding to the first audio content and first melody information of the first audio content;
constructing a first training sequence based on the text content and the first melody information;
inputting the first training sequence to a music generation model to generate a first set of music encoded representations; and
training the music generation model based on the first set of music encoded representations and a second set of music encoded representations of the music sample.

### Claim 14 (depends on 13)

The electronic device of claim 13, wherein generating the first annotation information comprises:
processing the first audio content with a text recognition model to determine the text content; and/or
processing the first audio content with a melody recognition model to determine the first melody information.

### Claim 15 (depends on 14)

The electronic device of claim 14, wherein constructing the first training sequence based on the text content and the first melody information comprises:
constructing a first sequence part corresponding to the text content, based on the text content and first time information corresponding to the text content;
constructing a second sequence part corresponding to the first melody information, based on the first melody information and second time information corresponding to the first melody information; and
constructing the first training sequence based on the first sequence part and the second sequence part.

### Claim 16 (depends on 13)

The electronic device of claim 13, wherein the acts further comprise:
processing the first audio content with a trained discrete encoder to determine the second set of music encoded representations.

### Claim 17 (depends on 13)

The electronic device of claim 13, wherein training the music generation model based on the first set of music encoded representations and the second set of music encoded representations of the music sample comprises:
determining a training loss based on a comparison of the first set of music encoded representations and the second set of music encoded representations; and
updating a model parameter of the music generation model based on the training loss.

### Claim 18 (depends on 13)

The electronic device of claim 13, wherein inputting the first training sequence to the music generation model comprises:
encoding the first training sequence with a text digital hybrid encoder to generate a hybrid encoded representation; and
inputting the hybrid encoded representation to the music generation model.

### Claim 19 (depends on 13)

The electronic device of claim 13, wherein the acts further comprise:
obtaining sampled second audio content, the second audio content corresponding to a voice track;
obtaining second annotation information of the second audio content, the second annotation information comprising phoneme information corresponding to the second audio content and second melody information of the second audio content; and
fine-tuning the music generation model based on the second annotation information of the second audio content.

### Claim 20

A non-transitory computer-readable storage medium having a computer program stored thereon, wherein the computer program is executable by a processor to perform acts comprising:
extracting, from a music sample, first audio content associated with vocal content;
generating first annotation information based on the first audio content, the first annotation information comprising text content corresponding to the first audio content and first melody information of the first audio content;
constructing a first training sequence based on the text content and the first melody information;
inputting the first training sequence to a music generation model to generate a first set of music encoded representations; and
training the music generation model based on the first set of music encoded representations and a second set of music encoded representations of the music sample.
