# US 11,308,952 B2 · Text and voice information processing method and terminal

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 11,308,952 B2
- Document ID: US11308952B2
- Status: Granted
- Kind: B2
- Application number: 16/483,688
- Filed: 2017-05-23
- Published: 2020-01-23
- Granted: 2022-04-19
- Earliest claimed: 2017-02-06
- Assignees: Huawei Technologies Co., Ltd.
- Applicants: Huawei Technologies Co., Ltd.
- Inventors: Xiaojuan Li; Wenmei Gao
- Web page: https://patentgrep.com/patent/US11308952B2

## CPC

- G06F 3/167: {Audio in a user interface, e.g. using voice commands for navigating, audio feedback}
- G06F 3/165: {Management of the audio stream, e.g. setting of volume, audio stream path}
- G06F 40/20: Natural language analysis (semantic analysis of natural language G06F40/30)
- G06F 40/279: Recognition of textual entities
- G06V 30/40: Document-oriented image-based pattern recognition
- G10L 15/08: Speech classification or search
- G10L 15/22: Procedures used during a speech recognition process, e.g. man-machine dialogue
- G06F 3/0481: based on specific properties of the displayed interaction object or a metaphor-based environment, e.g. interaction with desktop elements like windows or icons, or assisted by a cursor's changing behaviour or appearance
- G06V 30/10: Character recognition
- G10L 15/00: Speech recognition (G10L17/00 takes precedence)
- G10L 2015/088: {Word spotting}
- G10L 2015/223: {Execution procedure of a spoken command}
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G10L: SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING

## Abstract

A text and voice information processing method includes: photographing a document to obtain a first picture, wherein the document comprises a first text keyword and a second text keyword; recording audio to obtain a voice file corresponding to the document; obtaining a first voice segment matching the first text keyword and a second voice segment matching the second text keyword from the voice file; displaying a second picture with a first play button and a second play button; in response to a user input, playing back the first voice segment or playing back the second voice segment.

## Claims

### Claim 1

An electronic device comprising:
a memory configured to store instructions; and
a processor coupled to the memory and configured to execute the instructions, which causes the electronic device to:
photograph a document to obtain a first picture, wherein the document comprises a first text keyword and a second text keyword;
record audio to obtain a voice file corresponding to the document;
obtain, from the voice file, a first voice segment matching the first text keyword and a second voice segment matching the second text keyword;
display a second picture;
obtain a first coordinate position of the first text keyword in the second picture;
obtain a second coordinate position of the second text keyword in the second picture,
wherein a first play button is displayed corresponding to the first coordinate position of the first text keyword, and
wherein a second play button is displayed with corresponding to the second coordinate position of the second text keyword;
detect a first user input on the first play button;
play, in response to the first user input, the first voice segment;
detect a second user input on the second play button; and
play, in response to the second user input, the second voice segment.

### Claim 2 (depends on 1)

The electronic device of claim 1, wherein the instructions further cause the electronic device to:
recognize text information in the first picture; and
extract first key information of the text information, wherein the first key information comprises a text keyword set and word frequency corresponding to each text keyword, and wherein the word frequency is a quantity of occurrence times of each text keyword in the document.

### Claim 3 (depends on 2)

The electronic device of claim 2, wherein the instructions further cause the electronic device to:
map the first voice segment to the first text keyword when the word frequency of the first text keyword is greater than a threshold; and
map the second voice segment to the second text keyword when the word frequency of the second text keyword is greater than the threshold.

### Claim 4 (depends on 2)

The electronic device of claim 2, wherein the voice file comprises a voice keyword set and a voice segment corresponding to each voice keyword in the voice keyword set.

### Claim 5 (depends on 2)

The electronic device of claim 2, wherein the instructions further cause the electronic device to extract first key information based on preset feature information or a character recognition manner.

### Claim 6 (depends on 1)

The electronic device of claim 1, wherein the instructions further cause the electronic device to:
calibrate the first picture; and
recognize the first picture to obtain text information.

### Claim 7

A computer program product comprising computer-executable instructions stored on a non-transitory computer readable medium that, when executed by a processor, cause an apparatus to:
photograph a document to obtain a first picture, wherein the document comprises a first text keyword and a second text keyword;
record audio to obtain a voice file corresponding to the document;
obtain, from the voice file, a first voice segment matching the first text keyword and a second voice segment matching the second text keyword;
display a second picture;
obtain a first coordinate position of the first text keyword in the second picture;
obtain a second coordinate position of the second text keyword in the second picture,
wherein a first play button is displayed with corresponding to the first coordinate position of the first text keyword, and
wherein a second play button is displayed corresponding to the second coordinate position of the second text keyword;
detect a first user input on the first play button;
play, in response to the first user input, the first voice segment;
detect a second user input on the second play button; and
play, in response to the second user input, the second voice segment.

### Claim 8 (depends on 7)

The computer program product of claim 7, wherein the instructions further cause the apparatus to:
recognize text information in the first picture; and
extract first key information of the text information,
wherein the first key information comprises a text keyword set and word frequency corresponding to each text keyword, and
wherein the word frequency is a quantity of occurrence times of each text keyword in the document.

### Claim 9 (depends on 8)

The computer program product of claim 8, wherein the instructions further cause the apparatus to:
map the first voice segment to the first text keyword when the word frequency of the first text keyword is greater than a threshold; and
map the second voice segment to the second text keyword when the word frequency of the second text keyword is greater than the threshold.

### Claim 10 (depends on 8)

The computer program product of claim 8, wherein the voice file comprises a voice keyword set and a voice segment corresponding to each voice keyword in the voice keyword set.

### Claim 11 (depends on 8)

The computer program product of claim 8, wherein the instructions further cause the apparatus to extract first key information based on preset feature information or a character recognition manner.

### Claim 12 (depends on 7)

The computer program product of claim 7, wherein the instructions further cause the apparatus to:
calibrate the first picture; and
recognize the first picture to obtain text information.

### Claim 13

An image and voice processing method comprising:
photographing a document to obtain a first picture, wherein the document comprises a first text keyword and a second text keyword;
recording audio to obtain a voice file corresponding to the document;
obtaining, from the voice file, a first voice segment matching the first text keyword and a second voice segment matching the second text keyword;
displaying a second picture;
obtaining a first coordinate position of the first text keyword in the second picture;
obtaining a second coordinate position of the second text keyword in the second picture,
wherein a first play button is displayed corresponding to the first coordinate position of the first text keyword, and
wherein a second play button is displayed corresponding to the second coordinate position of the second text keyword;
detecting a first user input on the first play button;
playing, in response to the first user input, the first voice segment;
detecting a second user input on the second play button; and
playing, in response to the second user input, the second voice segment.

### Claim 14 (depends on 13)

The image and voice processing method of claim 13, further comprising:
recognizing text information in the first picture; and
extracting first key information of the text information, wherein the first key information comprises a text keyword set and word frequency corresponding to each text keyword, and wherein the word frequency is a quantity of occurrence times of each text keyword in the document.

### Claim 15 (depends on 14)

The image and voice processing method of claim 14, further comprising:
mapping the first voice segment to the first text keyword when the word frequency of the first text keyword is greater than a threshold; and
mapping the second voice segment to the second text keyword when the word frequency of the second text keyword is greater than the threshold.

### Claim 16 (depends on 14)

The image and voice processing method of claim 14, wherein the voice file comprises a voice keyword set and a voice segment corresponding to each voice keyword.

### Claim 17 (depends on 14)

The image and voice processing method of claim 14, wherein the text information comprises a text feature.

### Claim 18 (depends on 14)

The image and voice processing method of claim 14, further comprising extracting first key information based on preset feature information in a database.

### Claim 19 (depends on 14)

The image and voice processing method of claim 14, further comprising extracting first key information based on a character recognition manner.

### Claim 20 (depends on 13)

The image and voice processing method of claim 13, further comprising:
calibrating the first picture; and
recognizing the first picture to obtain text information.