# US 2025/0168464 A1 · Multimedia Data Generating Method, Apparatus, Electronic Device, Medium, and Program Product

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2025/0168464 A1
- Document ID: US20250168464A1
- Status: Published application
- Kind: A1
- Application number: 19/033,358
- Filed: 2025-01-21
- Published: 2025-05-22
- Earliest claimed: 2021-10-28
- Applicants: Beijing Zitiao Network Technology Co., Ltd.
- Inventors: Jiajin CAO
- Web page: https://patentgrep.com/patent/US20250168464A1

## CPC

- G06F 16/48: Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually
- H04N 21/8106: {involving special audio data, e.g. different tracks for different languages}
- G10L 13/02: Methods for producing synthetic speech; Speech synthesisers
- G11B 27/031: Electronic editing of digitised analogue information signals, e.g. audio or video signals
- G11B 27/34: Indicating arrangements {(indicating means incorporated in magazine or cassette G11B23/046 and G11B23/0875; indicating measured values in general G01D)}
- H04N 21/4394: {involving operations for analysing the audio stream, e.g. detecting features or characteristics in audio streams (arrangements characterised by components specially adapted for monitoring, identification or recognition of audio in broadcast systems H04H60/58)}
- H04N 21/4884: {for displaying subtitles}
- H04N 21/854: Content authoring
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION
- G10L: SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
- G11B: INFORMATION STORAGE BASED ON RELATIVE MOVEMENT BETWEEN RECORD CARRIER AND TRANSDUCER (recording measured values in a way that does not require playback through a transducer G01D9/00; recording or playback apparatus using mechanically marked tape, e.g. punched paper tape, or using unit records, e.g. punched or magnetically marked cards G06K; transferring data from one type of record carrier to another G06K1/18; circuits for coupling output of reproducer to radio receiver H04B1/20; gramophone pick-ups or like acoustic electromechanical transducers or circuits therefor H04R)

## Abstract

Disclosed are a multimedia data generating method, apparatus, electronic device, medium, and program product, applied to the field of multimedia data processing technologies. The method includes: receiving text information inputted by a user; displaying, in response to a recording trigger operation for the text information, the text information and acquiring a first reading speech of the text information; and generating a first multimedia data based on the text information and the first reading speech and displaying the first multimedia data; the first multimedia data includes the first reading speech and a video image matched with the text information; the first multimedia data includes a plurality of first multimedia segments; the plurality of first multimedia segments correspond to a plurality of text segments included in the text information, respectively. The disclose may improve quality of multimedia data generation.

## Claims

### Claim 1

A multimedia data generating method, comprising:
receiving text information;
displaying the text information and acquiring a first reading speech of the text information; and
generating a first multimedia data based on the text information and the first reading speech and displaying the first multimedia data;
wherein the first multimedia data comprise the first reading speech and a video image matched with the text information, the first multimedia data comprise a plurality of first multimedia segments, the plurality of first multimedia segments corresponding to a plurality of text segments included in the text information, respectively; wherein a first target multimedia segment comprises a first target video segment and a first target speech segment, the first target multimedia segment referring to a first multimedia segment in the plurality of first multimedia segments corresponding to a first target text segment in the plurality of text segments, the first target video segment including a video image matched with the first target text segment, the first target speech segment including a reading speech of the first target text segment, the first target video segment and the first target speech segment is displayed in editing tracks.

### Claim 2 (depends on 1)

The method of claim 1, further comprising:
converting the text information into speech data in response to a multimedia synthesis operation; and
generating second multimedia data based on the text information and the speech data, and displaying the second multimedia data;
wherein the second multimedia data comprise the speech data and the video image matched with the text information; the second multimedia data comprise a plurality of second multimedia segments, the plurality of second multimedia segments corresponding to a plurality of text segments included in the text information. respectively; a second target multimedia segment comprises a second target video segment and a second target speech segment, the second target multimedia segment referring to a second multimedia segment in the plurality of second multimedia segments corresponding to a second target text segment in the plurality of text segments, the second target video segment including a video image matched with the second target text segment, the second target speech segment including a reading speech of the second target text segment.

### Claim 3 (depends on 2)

The method of claim 2, further comprising:
displaying, in response to a recording trigger operation after the second multimedia data are generated, the text information, and acquiring a second reading speech of the text information; and
generating a third multimedia data based on the text information and the second reading speech, and displaying the third multimedia data to overwrite the second multimedia data;
wherein the third multimedia data comprise the second reading speech and the video image matched with the text information; the third multimedia data comprise a plurality of third multimedia segments, the plurality of third multimedia segments corresponding to a plurality of text segments included in the text information, respectively; a third target multimedia segment includes a third target video segment and a third target speech segment, the third target multimedia segment referring to a third multimedia segment in the plurality of third multimedia segments corresponding to a third target segment in the plurality of text segments, the third target video segment including a video image matched with the third target text segment, and the third target speech segment including a reading speech of the third target text segment.

### Claim 4 (depends on 1)

The method of claim 1, further comprising:
deleting, in response to a rerecording operation for the first target speech segment, the first target speech segment;
displaying the first target text segment corresponding to the first target speech segment, and acquiring a read segment of the first target text segment; and
displaying the read segment in an area corresponding to the first target speech segment.

### Claim 5 (depends on 1)

The method of claim 1, further comprising:
marking, in a case of detecting that a match rate between the first target speech segment and the first target text segment is lower than a match rate threshold while acquiring the first reading speech, the first target speech segment and the first target text segment.

### Claim 6 (depends on 1)

The method of claim 1, further comprising:
moving, in response to a speech segment sliding operation and sliding of a first cursor pointing to the first reading speech till the first target speech segment, a second cursor pointing to the text information till the first target text segment.

### Claim 7 (depends on 1)

The method of claim 1, further comprising:
prominently displaying a text segment currently read by the user while acquiring the first reading speech.

### Claim 8 (depends on 1)

The method of claim 1, further comprising:
editing, after the first multimedia data are generated and displayed, the text information in response to an edit operation for the text information to obtain a modified target text information;
displaying, in response to a recording trigger operation for the target text information, the target text information and acquiring a target reading speech of the target text information;
updating the first reading speech based on the target reading speech to obtain a third reading speech; and
generating a fourth multimedia data based on the target text information and the third reading speech, and displaying the fourth multimedia data;
wherein the fourth multimedia data comprise the third reading speech and the video image matched with the text information; the fourth multimedia data comprise a plurality of fourth multimedia segments, the plurality of fourth multimedia segments corresponding to a plurality of text segments included in the text information, respectively; a fourth target multimedia segment comprises a fourth target video segment and a fourth target speech segment, the fourth target multimedia segment referring to a fourth multimedia segment in the plurality of fourth multimedia segments corresponding to a fourth target text segment in the plurality of text segments, the fourth target video segment including the video image matched with the fourth target text segment, the fourth target speech segment including the reading speech of the fourth target text segment.

### Claim 9 (depends on 1)

The method of claim 1, further comprising:
subjecting, after the first reading speech is acquired, the first reading speech to voice change processing and/or speed change processing to obtain a fourth reading speech;
wherein the generating a first multimedia data based on the text information and the first reading speech and displaying the first multimedia data comprises:
generating the first multimedia data based on the text information and the fourth reading speech, and displaying the first multimedia data.

### Claim 10

An electronic device, comprising:
a processor configured to execute a program stored in a memory, wherein the computer program, when being executed by the processor, causes the processer to:
receive text information;
display the text information and acquire a first reading speech of the text information; and
generate a first multimedia data based on the text information and the first reading speech and display the first multimedia data;
wherein the first multimedia data comprise the first reading speech and a video image matched with the text information, the first multimedia data comprise a plurality of first multimedia segments, the plurality of first multimedia segments corresponding to a plurality of text segments included in the text information, respectively; wherein a first target multimedia segment comprises a first target video segment and a first target speech segment, the first target multimedia segment referring to a first multimedia segment in the plurality of first multimedia segments corresponding to a first target text segment in the plurality of text segments, the first target video segment including a video image matched with the first target text segment, the first target speech segment including a reading speech of the first target text segment, the first target video segment and the first target speech segment is displayed in editing tracks.

### Claim 11 (depends on 10)

The electronic device of claim 10, wherein the processor is further caused to:
convert the text information into speech data in response to a multimedia synthesis operation; and
generate second multimedia data based on the text information and the speech data, and display the second multimedia data;
wherein the second multimedia data comprise the speech data and the video image matched with the text information; the second multimedia data comprise a plurality of second multimedia segments, the plurality of second multimedia segments corresponding to a plurality of text segments included in the text information. respectively; a second target multimedia segment comprises a second target video segment and a second target speech segment, the second target multimedia segment referring to a second multimedia segment in the plurality of second multimedia segments corresponding to a second target text segment in the plurality of text segments, the second target video segment including a video image matched with the second target text segment, the second target speech segment including a reading speech of the second target text segment.

### Claim 12 (depends on 11)

The electronic device of claim 11, wherein the processor is further caused to:
display, in response to a recording trigger operation after the second multimedia data are generated, the text information, and acquire a second reading speech of the text information; and
generate a third multimedia data based on the text information and the second reading speech, and display the third multimedia data to overwrite the second multimedia data;
wherein the third multimedia data comprise the second reading speech and the video image matched with the text information; the third multimedia data comprise a plurality of third multimedia segments, the plurality of third multimedia segments corresponding to a plurality of text segments included in the text information, respectively; a third target multimedia segment includes a third target video segment and a third target speech segment, the third target multimedia segment referring to a third multimedia segment in the plurality of third multimedia segments corresponding to a third target segment in the plurality of text segments, the third target video segment including a video image matched with the third target text segment, and the third target speech segment including a reading speech of the third target text segment.

### Claim 13 (depends on 10)

The electronic device of claim 10, wherein the processor is further caused to:
delete, in response to a rerecording operation on the first target speech segment, the first target speech segment;
display the first target text segment corresponding to the first target speech segment, and acquire a read segment of the first target text segment; and
display the read segment in an area corresponding to the first target speech segment.

### Claim 14 (depends on 10)

The electronic device of claim 10, wherein the processor is further caused to:
mark, in a case of detecting that a match rate between the first target speech segment and the first target text segment is lower than a match rate threshold while acquiring the first reading speech, the first target speech segment and the first target text segment.

### Claim 15 (depends on 10)

The electronic device of claim 10, wherein the processor is further caused to:
move, in response to a speech segment sliding operation and sliding of a first cursor pointing to the first reading speech till the first target speech segment, a second cursor pointing to the text information till the first target text segment.

### Claim 16 (depends on 10)

The electronic device of claim 10, wherein the processor is further caused to:
prominently display a text segment currently read by the user while acquiring the first reading speech.

### Claim 17 (depends on 10)

The electronic device of claim 10, wherein the processor is further caused to:
edit, after the first multimedia data are generated and displayed, the text information in response to an edit operation for the text information to obtain a modified target text information;
display, in response to a recording trigger operation for the target text information, the target text information and acquire a target reading speech of the target text information;
update the first reading speech based on the target reading speech to obtain a third reading speech; and
generate a fourth multimedia data based on the target text information and the third reading speech, and display the fourth multimedia data;
wherein the fourth multimedia data comprise the third reading speech and the video image matched with the text information; the fourth multimedia data comprise a plurality of fourth multimedia segments, the plurality of fourth multimedia segments corresponding to a plurality of text segments included in the text information, respectively; a fourth target multimedia segment comprises a fourth target video segment and a fourth target speech segment, the fourth target multimedia segment referring to a fourth multimedia segment in the plurality of fourth multimedia segments corresponding to a fourth target text segment in the plurality of text segments, the fourth target video segment including the video image matched with the fourth target text segment, the fourth target speech segment including the reading speech of the fourth target text segment.

### Claim 18 (depends on 10)

The electronic device of claim 10,
wherein the processor is further caused to:
subject, after the first reading speech is acquired, the first reading speech to voice change processing and/or speed change processing to obtain a fourth reading speech;
and wherein the processor is further caused to:
generate the first multimedia data based on the text information and the fourth reading speech, and displaying the first multimedia data, so as to generate a first multimedia data based on the text information and the first reading speech and displaying the first multimedia data.

### Claim 19

A non-transitory computer-readable storage medium on which a computer program is stored, wherein the computer program, when being executed by a processor, causes the processer to:
receive text information;
display the text information and acquire a first reading speech of the text information; and
generate a first multimedia data based on the text information and the first reading speech and display the first multimedia data;
wherein the first multimedia data comprise the first reading speech and a video image matched with the text information, the first multimedia data comprise a plurality of first multimedia segments, the plurality of first multimedia segments corresponding to a plurality of text segments included in the text information, respectively; wherein a first target multimedia segment comprises a first target video segment and a first target speech segment, the first target multimedia segment referring to a first multimedia segment in the plurality of first multimedia segments corresponding to a first target text segment in the plurality of text segments, the first target video segment including a video image matched with the first target text segment, the first target speech segment including a reading speech of the first target text segment, the first target video segment and the first target speech segment is displayed in editing tracks.

### Claim 20 (depends on 19)

The non-transitory computer-readable storage medium of claim 19, wherein the processor is further caused to:
convert the text information into speech data in response to a multimedia synthesis operation; and
generate second multimedia data based on the text information and the speech data, and display the second multimedia data;
wherein the second multimedia data comprise the speech data and the video image matched with the text information; the second multimedia data comprise a plurality of second multimedia segments, the plurality of second multimedia segments corresponding to a plurality of text segments included in the text information. respectively; a second target multimedia segment comprises a second target video segment and a second target speech segment, the second target multimedia segment referring to a second multimedia segment in the plurality of second multimedia segments corresponding to a second target text segment in the plurality of text segments, the second target video segment including a video image matched with the second target text segment, the second target speech segment including a reading speech of the second target text segment.
