# US 2022/0016527 A1 · Method and device for audio generation

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2022/0016527 A1
- Document ID: US20220016527A1
- Status: Published application
- Kind: A1
- Application number: 17/375,192
- Filed: 2021-07-14
- Published: 2022-01-20
- Earliest claimed: 2020-07-17
- Assignees: Beijing Dajia Internet Information Technology Co., Ltd.
- Applicants: Beijing Dajia Internet Information Technology Co., Ltd.
- Inventors: Jian SHI; Xu DAI; Jiayun LI
- Web page: https://patentgrep.com/patent/US20220016527A1

## CPC

- A63F 13/54: involving acoustic signals, e.g. for simulating revolutions per minute [RPM] dependent engine sounds in a driving game or reverberation against a virtual wall
- A63F 13/35: Details of game servers
- A63F 13/44: involving timing of operations, e.g. performing an action within a time slot
- G06F 3/165: {Management of the audio stream, e.g. setting of volume, audio stream path}
- G10H 1/0025: {Automatic or semi-automatic music composition, e.g. producing random music, applying rules from music theory or modifying a musical piece (automatically producing a series of tones G10H1/26)}
- G10H 1/40: Rhythm
- A63F 2300/6063: for sound processing
- G10H 2210/061: for extraction of musical phrases, isolation of musically relevant segments, e.g. musical thumbnail generation, or for temporal structure analysis of a musical piece, e.g. determination of the movement sequence of a musical work
- G10H 2210/131: Morphing, i.e. transformation of a musical piece into a new different one, e.g. remix
- G10H 2240/325: Synchronizing two or more audio tracks or files according to musical features or musical timings
- Y02B 20/40: Control techniques providing energy savings, e.g. smart controller or presence detection
- A63F: CARD, BOARD, OR ROULETTE GAMES; INDOOR GAMES USING SMALL MOVING PLAYING BODIES; VIDEO GAMES; GAMES NOT OTHERWISE PROVIDED FOR
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G10H: ELECTROPHONIC MUSICAL INSTRUMENTS; INSTRUMENTS IN WHICH THE TONES ARE GENERATED BY ELECTROMECHANICAL MEANS OR ELECTRONIC GENERATORS, OR IN WHICH THE TONES ARE SYNTHESISED FROM A DATA STORE
- Y02B: CLIMATE CHANGE MITIGATION TECHNOLOGIES RELATED TO BUILDINGS, e.g. HOUSING, HOUSE APPLIANCES OR RELATED END-USER APPLICATIONS

## Abstract

The present disclosure relates to a method and device for audio generation. The method includes: obtaining a target rhythm, a target verse melody and a target chorus melody; configuring the target rhythm as a first audio track, the target verse melody as a second audio track, and the target chorus melody as a third audio track; generating a target audio by aligning start playing time of the first audio track, the second audio track and the third audio track to beat occurrence time of a first beat, a second beat and a third beat in a first metronome data respectively.

## Claims

### Claim 1

A method for audio generation, comprising:
obtaining a target rhythm, a target verse melody and a target chorus melody, wherein the target rhythm is corresponding to a target game scene, and the target verse melody and the target chorus melody are respectively corresponding to at least one of the target game scene and data of behavior state of a player character;
configuring the target rhythm as a first audio track, the target verse melody as a second audio track, and the target chorus melody as a third audio track;
generating a target audio by aligning start playing time of the first audio track, the second audio track and the third audio track to beat occurrence time of a first beat, a second beat and a third beat in a first metronome data respectively, wherein the first metronome data comprises a first offset, and occurrence time of each beat in the first metronome data is determined based on a first reference time and the first offset.

### Claim 2 (depends on 1)

The method of claim 1, wherein obtaining the target rhythm, the target verse melody and target chorus melody comprises:
determining a second rhythm corresponding to the target game scene from a preset first rhythm based on a first correspondence and the target game scene, and obtaining the target rhythm in the second rhythm, wherein the first rhythm comprises at least one second rhythm, and the first correspondence is a correspondence between a game scene and rhythm data;
determining a second verse melody corresponding to the target game scene from a preset first verse melody based on a second correspondence and at least one of the target game scene and the data of behavior state of the player character, and obtaining the target verse melody in the second verse melody, wherein the first verse melody comprises at least one second verse melody, and the second correspondence is a correspondence between a verse melody and at least one of a game scene and data of behavior state of a player character;
determining a second chorus melody corresponding to the target game scene from a preset first chorus melody based on a third correspondence and at least one of the target game scene and the data of behavior state of the player character, and obtaining the target chorus melody in the second chorus melody, wherein the first chorus melody comprises at least one second chorus melody, and the third correspondence is a correspondence between a chorus melody and at least one of the game scene and the data of behavior state of the player character.

### Claim 3 (depends on 1)

The method of claim 1, wherein the first audio track comprises at least one rhythm segment, the second audio track comprises at least one verse melody segment, and the third audio track comprises at least one chorus melody segment; the first beat comprises at least one sub-beat, the second beat comprises at least one sub-beat, and the third beat comprises at least one sub-beat.

### Claim 4 (depends on 3)

The method of claim 3, wherein said generating the target audio by aligning the start playing time of the first audio track, the second audio track and the third audio track to the beat occurrence time of a first beat, a second beat and a third beat in first metronome data respectively, comprises:
aligning a start playing time of each rhythm segment in the first audio track to an occurrence time of each sub-beat in the first beat respectively;
aligning a start playing time of each verse melody segment in the second audio track to an occurrence time of each sub-beat in the second beat;
aligning a start playing time of each chorus melody segment in the third track to an occurrence time of each sub-beat in the third beat; and
generating the target audio based on the aligned first audio track, the second audio track, and the third audio track.

### Claim 5 (depends on 3)

The method of claim 3, wherein said generating the target audio by aligning the start playing time of the first audio track, the second audio track and the third audio track to the beat occurrence time of a first beat, a second beat and a third beat in a first metronome data respectively, comprises:
aligning a start rhythm segment in the first audio track to beat occurrence time of a start sub-beat in the first beat;
aligning a start verse melody segment in the second audio track to beat occurrence time of a start sub-beat in the second beat;
aligning a start chorus melody segment in the third audio track to beat occurrence time of a start sub-beat in the third beat; and
generating the target audio based on the aligned first audio track, the second audio track, and the third audio track.

### Claim 6 (depends on 1)

The method of claim 1, further comprising:
obtaining actual playing time of the first audio track, the second audio track, and the third audio track in the target audio respectively during playing the target audio;
adjusting the first reference time in the first metronome data to the actual playing time, in a case that a difference between the actual playing time of the first audio track, the second audio track, or the third audio track and an aligned beat occurrence time thereof is greater than a preset threshold, and adjusting an occurrence time of a target beat based on the adjusted first reference time and the first offset,
wherein, the target beat is a beat that occurs after the actual playing time in the first metronome data.

### Claim 7 (depends on 1)

The method of claim 1, further comprising:
obtaining first target data of behavior state of the player character;
adjusting the first offset based on the first target data and a fourth correspondence,
wherein the fourth correspondence is a correspondence between data of behavior state of a player character and an offset.

### Claim 8 (depends on 1)

The method of claim 1, wherein the target verse melody and the target chorus melody are corresponding to the target game scene and the data of behavior state of the player character;
the method further comprises:
obtaining a second target data of behavior state of the player character and a game scene during playing the target audio;
switching playing the target chorus melody to playing a verse melody in a case that the game scene is the target game scene and the second target data meets the preset state.

### Claim 9 (depends on 1)

The method of claim 1, further comprising:
receiving a piece of music uploaded;
identifying a second beat data corresponding to the music;
controlling the target object in the target game scene to execute a target action based on an occurrence time of each beat in the second beat data,
wherein the second beat data comprises a second offset, and an occurrence time of each beat in the second metronome data is determined based on a second reference time and the second offset.

### Claim 10

A device for audio generation, comprising:
a processor;
a memory for storing executable instructions of the processor;
wherein, the processor is configured to performing acts of:
obtaining a target rhythm, a target verse melody and a target chorus melody, wherein the target rhythm is corresponding to a target game scene, and the target verse melody and the target chorus melody are respectively corresponding to at least one of the target game scene and data of behavior state of a player character;
configuring the target rhythm as a first audio track, the target verse melody as a second audio track, and the target chorus melody as a third audio track;
generating a target audio by aligning start playing time of the first audio track, the second audio track and the third audio track to beat occurrence time of a first beat, a second beat and a third beat in a first metronome data respectively, wherein the first metronome data comprises a first offset, and occurrence time of each beat in the first metronome data is determined based on a first reference time and the first offset.

### Claim 11 (depends on 10)

The apparatus of claim 10, wherein the processor is configured to performing acts of:
determining a second rhythm corresponding to the target game scene from a preset first rhythm based on a first correspondence and the target game scene, and obtaining the target rhythm in the second rhythm, wherein the first rhythm comprises at least one second rhythm, and the first correspondence is a correspondence between a game scene and rhythm data;
determining a second verse melody corresponding to the target game scene from a preset first verse melody based on a second correspondence and at least one of the target game scene and the data of behavior state of the player character, and obtaining the target verse melody in the second verse melody, wherein the first verse melody comprises at least one second verse melody, and the second correspondence is a correspondence between a verse melody and at least one of a game scene and data of behavior state of a player character;
determining a second chorus melody corresponding to the target game scene from a preset first chorus melody based on a third correspondence and at least one of the target game scene and the data of behavior state of the player character, and obtaining the target chorus melody in the second chorus melody, wherein the first chorus melody comprises at least one second chorus melody, and the third correspondence is a correspondence between a chorus melody and at least one of the game scene and the data of behavior state of the player character.

### Claim 12 (depends on 10)

The apparatus of claim 10, wherein the first audio track comprises at least one rhythm segment, the second audio track comprises at least one verse melody segment, and the third audio track comprises at least one chorus melody segment; the first beat comprises at least one sub-beat, the second beat comprises at least one sub-beat, and the third beat comprises at least one sub-beat.

### Claim 13 (depends on 12)

The apparatus of claim 12, wherein the processor is configured to performing acts of:
aligning a start playing time of each rhythm segment in the first audio track to an occurrence time of each sub-beat in the first beat respectively;
aligning a start playing time of each verse melody segment in the second audio track to an occurrence time of each sub-beat in the second beat;
aligning a start playing time of each chorus melody segment in the third track to an occurrence time of each sub-beat in the third beat; and
generating the target audio based on the aligned first audio track, the second audio track, and the third audio track.

### Claim 14 (depends on 12)

The apparatus of claim 12, wherein the processor is configured to performing acts of:
aligning a start rhythm segment in the first audio track to beat occurrence time of a start sub-beat in the first beat;
aligning a start verse melody segment in the second audio track to beat occurrence time of a start sub-beat in the second beat;
aligning a start chorus melody segment in the third audio track to beat occurrence time of a start sub-beat in the third beat;
generating the target audio based on the aligned first audio track, the second audio track, and the third audio track.

### Claim 15 (depends on 10)

The apparatus of claim 10, wherein the processor is configured to performing acts of:
obtaining actual playing time of the first audio track, the second audio track, and the third audio track in the target audio respectively during playing the target audio;
adjusting the first reference time in the first metronome data to the actual playing time in a case that a difference between the actual playing time of the first audio track, the second audio track, or the third audio track and an aligned beat occurrence time thereof is greater than a preset threshold, and adjusting an occurrence time of a target beat based on the adjusted first reference time and the first offset,
wherein, the target beat is a beat that occurs after the actual playing time in the first metronome data.

### Claim 16 (depends on 10)

The apparatus of claim 10, wherein the processor is configured to performing acts of:
obtaining first target data of behavior state of the player character;
adjusting the first offset based on the first target data and a fourth correspondence,
wherein the fourth correspondence is a correspondence between data of behavior state of a player character and an offset.

### Claim 17 (depends on 10)

The apparatus of claim 10, wherein the target verse melody and the target chorus melody are corresponding to the target game scene and the data of behavior state of the player character;
the processor is configured to performing acts of:
obtaining a second target data of behavior state of the player character and a game scene during playing the target audio;
switching playing the target chorus melody to playing a verse melody in a case that the game scene is the target game scene and the second target data of behavior state meets the preset state.

### Claim 18 (depends on 10)

The apparatus of claim 10, wherein the processor is configured to performing acts of:
receiving a piece of music uploaded;
identifying a second beat data corresponding to the music;
controlling the target object in the target game scene to execute a target action based on an occurrence time of each beat in the second beat data,
wherein the second beat data comprises a second offset, and an occurrence time of each beat in the second metronome data is determined based on a second reference time and the second offset.

### Claim 19

A non-transitory computer-readable storage medium having instructions stored thereon, wherein when instructions are executed by a processor of a server, the server is caused to execute a method for audio generation, and the method comprises:
obtaining a target rhythm, a target verse melody and a target chorus melody, wherein the target rhythm is corresponding to a target game scene, and the target verse melody and the target chorus melody are respectively corresponding to at least one of the target game scene and data of behavior state of a player character;
configuring the target rhythm as a first audio track, the target verse melody as a second audio track, and the target chorus melody as a third audio track;
generating a target audio by aligning start playing time of the first audio track, the second audio track and the third audio track to beat occurrence time of a first beat, a second beat and a third beat in a first metronome data respectively, wherein the first metronome data comprises a first offset, and occurrence time of each beat in the first metronome data is determined based on a first reference time and the first offset.

### Claim 20 (depends on 19)

The storage medium of claim 19, wherein obtaining the target rhythm, the target verse melody and target chorus melody comprises:
determining a second rhythm corresponding to the target game scene from a preset first rhythm based on a first correspondence and the target game scene, and obtaining the target rhythm in the second rhythm, wherein the first rhythm comprises at least one second rhythm, and the first correspondence is a correspondence between a game scene and rhythm data;
determining a second verse melody corresponding to the target game scene from a preset first verse melody based on a second correspondence and at least one of the target game scene and the data of behavior state of the player character, and obtaining the target verse melody in the second verse melody, wherein the first verse melody comprises at least one second verse melody, and the second correspondence is a correspondence between a verse melody and at least one of a game scene and data of behavior state of a player character; and
determining a second chorus melody corresponding to the target game scene from a preset first chorus melody based on a third correspondence and at least one of the target game scene and the data of behavior state of the player character, and obtaining the target chorus melody in the second chorus melody, wherein the first chorus melody comprises at least one second chorus melody, and the third correspondence is a correspondence between a chorus melody and at least one of the game scene and the data of behavior state of the player character.
