# US 2023/0410802 A1 · Information processing system, information processing method, and storage medium

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2023/0410802 A1
- Document ID: US20230410802A1
- Status: Published application
- Kind: A1
- Application number: 18/227,741
- Filed: 2023-07-28
- Published: 2023-12-21
- Earliest claimed: 2018-07-18
- Assignees: NEC Corporation
- Applicants: NEC Corporation
- Inventors: Masamichi TANABE
- Web page: https://patentgrep.com/patent/US20230410802A1

## CPC

- G10L 15/22: Procedures used during a speech recognition process, e.g. man-machine dialogue
- G06Q 30/06: Buying, selling or leasing transactions
- G06Q 30/0613: {using intermediate agents}
- G06V 20/52: Surveillance or monitoring of activities, e.g. for recognising suspicious objects (recognising microscopic objects G06V20/69)
- G06V 40/10: Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
- G10L 15/063: {Training}
- G10L 15/08: Speech classification or search
- G10L 15/24: Speech recognition using non-acoustical features
- G10L 15/30: Distributed recognition, e.g. in client-server systems, for mobile phones or network applications
- G10L 15/1822: {Parsing for meaning understanding}
- G10L 2015/088: {Word spotting}
- G10L: SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
- G06Q: INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING

## Abstract

Provided is an information processing system including: a voice information acquisition unit that acquires voice information including an utterance made by a person; a status acquisition unit that acquires status information related to status of the person; and a support information generation unit that generates support information used for supporting operation of the person based on the voice information and the status information.

## Claims

### Claim 1

An information processing method executed by a computer, the information processing method comprising:
extracting a keyword from an utterance of a salesclerk or a customer;
detecting status of the salesclerk, from an image capturing the salesclerk, wherein the status includes position information representing whether the salesclerk is in a sales area or in an area other than the sales area;
generating support information based on the extracted keyword and the detected status; and
outputting the support information to an output device used by the salesclerk.

### Claim 2 (depends on 1)

The information processing method according to claim 1,
wherein the position information further includes information representing where the salesclerk is in the sales area.

### Claim 3 (depends on 1)

The information processing method according to claim 1,
wherein the status further includes information as to whether the customer is present near the salesclerk.

### Claim 4 (depends on 1)

The information processing method according to claim 1,
wherein the status further includes information as to whether other salesclerk is present near the salesclerk.

### Claim 5 (depends on 1)

The information processing method according to claim 1,
wherein the status further includes information as to whether a predetermined article is present near the salesclerk.

### Claim 6 (depends on 5)

The information processing method according to claim 5,
wherein the predetermined article is at least one of a sales counter, a register, a terminal apparatus, a product shelf, and a store computer in the backyard.

### Claim 7

An information processing apparatus comprising:
one or more memories storing instructions; and
one or more processors configured to execute the instructions to:
extract a keyword from an utterance of a salesclerk or a customer;
detect status of the salesclerk, from an image capturing the salesclerk, wherein the status includes position information representing whether the salesclerk is in a sales area or in an area other than the sales area;
generate support information based on the extracted keyword and the detected status; and
output the support information to an output device used by the salesclerk.
