# US 12,190,866 B2 · Voice based manual image review

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 12,190,866 B2
- Document ID: US12190866B2
- Status: Granted
- Kind: B2
- Application number: 17/707,029
- Filed: 2022-03-29
- Published: 2023-10-05
- Granted: 2025-01-07
- Earliest claimed: 2022-03-29
- Assignees: Conduent Business Services, LLC
- Applicants: Conduent Business Services, LLC
- Inventors: Lee D Roche
- Web page: https://patentgrep.com/patent/US12190866B2

## CPC

- G06Q 50/26: Government or public services (business processes related to the transportation industry G06Q50/40)
- G10L 15/08: Speech classification or search
- G06F 3/0481: based on specific properties of the displayed interaction object or a metaphor-based environment, e.g. interaction with desktop elements like windows or icons, or assisted by a cursor's changing behaviour or appearance
- G06F 3/167: {Audio in a user interface, e.g. using voice commands for navigating, audio feedback}
- G06Q 10/10: Office automation; Time management
- G06Q 50/18: Legal services
- G06Q 50/40: Business processes related to the transportation industry (shipping G06Q10/083)
- G10L 15/22: Procedures used during a speech recognition process, e.g. man-machine dialogue
- G10L 2015/088: {Word spotting}
- G10L 2015/223: {Execution procedure of a spoken command}
- G06Q: INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G10L: SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)

## Abstract

Methods and systems for manual-based image review can involve associating an image with a group of keyword utterances, the image displayable in a display screen of a computing device, the group of keyword utterances including different keyword utterances. A prompt for the user to utter a keyword utterance can be displayed in a first area of the image in the display screen and another prompt for the user to utter another keyword utterance can be displayed in another area of the image in the display screen. Audio of the keyword utterances displayed in the display screen can be captured and processed by natural language processing (NLP), when uttered by the user. The utterances can be displayed respectively as text in the first area and the other area of the image in response to processing by NLP of the audio. Thus, instead of users typing in the results and changing their focus between screen and keyboard, for example, the user can speak to the results, which increases the throughput of the results.

## Claims

### Claim 1

A method for manual-based image review, comprising:
associating an image with a plurality of keyword utterances, the image displayable in a display screen of a computing device, the plurality of keyword utterances including different keyword utterances, wherein each area of the display screen displays at least one keyword utterance among the plurality of keyword utterances;
displaying for a user, a prompt for the user to utter the at least one keyword utterance among the plurality of keyword utterances in a first area of the image displayed in the display screen and another prompt for the user to utter at least one other keyword utterance among the plurality of keyword utterances in another area of the image displayed in the display screen;
capturing and processing by natural language processing (NLP), audio of the at least one keyword utterance and the at least one other keyword utterance when uttered by the user through an audio device operable to operate with the NLP, for display of the at least one keyword utterance and the at least one other keyword utterance as text in the respective first area and the another area of the image, in response to processing by NLP of the audio, wherein the NLP is operable to understand the at least one keyword utterance and post NLP results resulting from the capturing and the processing by the NLP;
wherein the NLP understands the at least one keyword utterance and the at least one other keyword utterance uttered by the user posts NLP results to a back office after the processing of the audio; and
the NLP comprises at least one of: NLP based on a hybrid sequence-to-sequence approach, and NLP processing review and override based on confidence analysis.

### Claim 2 (depends on 1)

The method of claim 1 wherein the capturing and the processing by NLP of the at least one other keyword utterance further comprises: detecting the at least one other keyword utterance when uttered by the user.

### Claim 3 (depends on 1)

The method of claim 1 wherein:
the capturing and the processing by NLP of the at least one other keyword utterance further comprises: detecting the at least one other keyword utterance when uttered by the user; and
the NLP understands the at least one keyword utterance and the at least one other keyword utterance uttered by the user posts NLP results to a back office after the processing of the audio.

### Claim 4 (depends on 1)

The method of claim 1 wherein the image comprises an image of a vehicle.

### Claim 5 (depends on 4)

The method of claim 4 wherein the plurality of keyword utterances associated with the image includes at least one of: a license plate number of the vehicle, a state of the vehicle, and a type of the vehicle.

### Claim 6 (depends on 1)

The method of claim 1 wherein the image comprises an image of a vehicle and wherein the plurality of keyword utterances associated with the image includes at least one of: a license plate number of the vehicle, a state of the vehicle, and a type of the vehicle.

### Claim 7 (depends on 1)

The method of claim 1 wherein the image comprises an image of a vehicle and wherein the plurality of keyword utterances associated with the image includes a license plate number of the vehicle, a state of the vehicle, and a type of the vehicle.

### Claim 8

A system for manual-based image review, comprising:
a display screen for displaying an image associated with a plurality of keyword utterances, wherein the plurality of keyword utterances includes different keyword utterances;
a user interface for displaying for a user, a prompt for the user to utter at least one keyword utterance among the plurality of keyword utterances in a first area of the image displayed in the display screen and another prompt for the user to utter at least one other keyword utterance among the plurality of keyword utterances in another area of the image displayed in the display screen; and
an audio device for capturing and processing by natural language processing (NLP), audio of the at least one keyword utterance and the at least one other keyword utterance when uttered by the user through an audio device operable to operate with the NLP, for display of the at least one keyword utterance and the at least one other keyword utterance as text in the respective first area and the another area of the image, in response to processing by NLP of the audio, wherein the NLP is operable to understand the at least one keyword utterance and post NLP results resulting from the capturing and the processing by the NLP;
wherein the NLP understands the at least one keyword utterance and the at least one other keyword utterance uttered by the user posts NLP results to a back office after the processing of the audio; and
the NLP comprises at least one of: NLP based on a hybrid sequence-to-sequence approach, and NLP processing review and override based on confidence analysis.

### Claim 9 (depends on 8)

The system of claim 8 wherein the capturing and the processing by NLP of the at least one other keyword utterance involves detecting the at least one other keyword utterance when uttered by the user.

### Claim 10 (depends on 8)

The system of claim 8 wherein:
the capturing and the processing by NLP of the at least one other keyword utterance further comprises: detecting the at least one other keyword utterance when uttered by the user; and
the NLP understands the at least one keyword utterance and the at least one other keyword utterance uttered by the user posts NLP results to a back office after the processing of the audio.

### Claim 11 (depends on 8)

The system of claim 8 wherein the image comprises an image of a vehicle.

### Claim 12 (depends on 11)

The system of claim 11 wherein the plurality of keyword utterances associated with the image includes at least one of: a license plate number of the vehicle, a state of the vehicle, and a type of the vehicle.

### Claim 13 (depends on 11)

The system of claim 11 wherein the image comprises an image of a vehicle and wherein the plurality of keyword utterances associated with the image includes at least one of: a license plate number of the vehicle, a state of the vehicle, and a type of the vehicle.

### Claim 14 (depends on 11)

The system of claim 11 wherein the image comprises an image of a vehicle and wherein the plurality of keyword utterances associated with the image includes a license plate number of the vehicle, a state of the vehicle, and a type of the vehicle.

### Claim 15

A computer program product for facilitating manual-based image review of images, the computer program product comprising one or more non-transitory computer readable storage media and program instructions collectively stored on the one or more non-transitory computer readable storage media, the program instructions comprising program instructions to:
associate an image with a plurality of keyword utterances, the image displayable in a display screen of a computing device, the plurality of keyword utterances including different keyword utterances;
display for a user, a prompt for the user to utter at least one keyword utterance among the plurality of keyword utterances in a first area of the image displayed in the display screen and another prompt for the user to utter at least one other keyword utterance among the plurality of keyword utterances in another area of the image displayed in the display screen; and
capture and process by natural language processing (NLP), audio of the at least one keyword utterance and the at least one other keyword utterance when uttered by the user through an audio device operable to operate with the NLP, for display of the at least one keyword utterance and the at least one other keyword utterance as text in the respective first area and the another area of the image, in response to processing by NLP of the audio, wherein the NLP is operable to understand the at least one keyword utterance and post NLP results resulting from the capturing and the processing by the NLP;
wherein the NLP understands the at least one keyword utterance and the at least one other keyword utterance uttered by the user posts NLP results to a back office after the processing of the audio; and
the NLP comprises at least one of: NLP based on a hybrid sequence-to-sequence approach, and NLP processing review and override based on confidence analysis.

### Claim 16 (depends on 15)

The computer program product of claim 15 wherein the program instructions further comprise program instructions to detect the at least one other keyword utterance when uttered by the user.

### Claim 17 (depends on 15)

The computer program product of claim 15 wherein:
the image comprises an image of a vehicle; and
the plurality of keyword utterances associated with the image includes at least one of: a license plate number of the vehicle, a state of the vehicle, and a type of the vehicle.