# US 12,183,335 B2 · Information processing system, information processing method, and storage medium > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 12,183,335 B2 - Document ID: US12183335B2 - Status: Granted - Kind: B2 - Application number: 17/258,960 - Filed: 2019-07-05 - Published: 2021-04-29 - Granted: 2024-12-31 - Earliest claimed: 2018-07-18 - Assignees: NEC Corporation - Applicants: NEC Corporation - Inventors: Masamichi Tanabe - Web page: https://patentgrep.com/patent/US12183335B2 ## CPC - G10L 15/22: Procedures used during a speech recognition process, e.g. man-machine dialogue - G06Q 30/06: Buying, selling or leasing transactions - G06Q 30/0613: {using intermediate agents} - G06V 20/52: Surveillance or monitoring of activities, e.g. for recognising suspicious objects (recognising microscopic objects G06V20/69) - G06V 40/10: Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands - G10L 15/063: {Training} - G10L 15/08: Speech classification or search - G10L 15/24: Speech recognition using non-acoustical features - G10L 15/30: Distributed recognition, e.g. in client-server systems, for mobile phones or network applications - G10L 15/1822: {Parsing for meaning understanding} - G10L 2015/088: {Word spotting} - G10L: SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING - G06Q: INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR - G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING ## Abstract Provided is an information processing system including: a voice information acquisition unit that acquires voice information including an utterance made by a person; a status acquisition unit that acquires status information related to status of the person; and a support information generation unit that generates support information used for supporting operation of the person based on the voice information and the status information. ## Claims ### Claim 1 An information processing system comprising: a voice information acquisition unit that acquires voice information including an utterance made by a person that is a salesclerk of a shop; a status acquisition unit that acquires status information related to status of the person, the status information acquired based on an image including the person captured by a camera installed in the shop, the status information including information related to a position of the person; and a support information generation unit that generates support information used for supporting an operation of the person based on the voice information and the status information, wherein the support information generation unit withholds generation of the support information when a customer is near the salesclerk. ### Claim 2 (depends on 1) The information processing system according to claim 1, wherein the support information generation unit generates the support information by using a learning model generated by machine learning using training data including the voice information and the status information. ### Claim 3 (depends on 1) The information processing system according to claim 1, wherein the support information generation unit generates the support information by referencing a database in which the voice information, the status information, and the support information are associated with each other. ### Claim 4 (depends on 1) The information processing system according to claim 1, wherein the voice information acquisition unit acquires the voice information from an interaction between the salesclerk and a customer. ### Claim 5 (depends on 1) The information processing system according to claim 1, wherein the status information further includes information related to a work state of the person. ### Claim 6 (depends on 1) The information processing system according to claim 1 further comprising a notification unit that notifies the person of the support information by output of a voice or display on a display device. ### Claim 7 (depends on 1) The information processing system according to claim 1, wherein the support information contains an operation instruction to the person. ### Claim 8 (depends on 1) The information processing system according to claim 1, wherein the status acquisition unit further acquires the status information based on information acquired by a device that is different from a device that acquires an utterance made by a salesclerk or a customer. ### Claim 9 (depends on 1) The information processing system according to claim 1 further comprising a conversion unit that converts the voice information into text information. ### Claim 10 (depends on 9) The information processing system according to claim 9 further comprising an extraction unit that extracts a keyword to be input to the support information generation unit from the text information. ### Claim 11 (depends on 1) The information processing system according to claim 1, wherein the voice information is based on a voice acquired by a microphone worn by the person or a microphone installed in a shop. ### Claim 12 (depends on 1) The information processing system according to claim 1, wherein, based on at least one of the voice information and the status information, the support information generation unit further generates support information that is different from the support information. ### Claim 13 (depends on 1) The information processing system according to claim 1 further comprising a notification unit that notifies the person of the support information by output of a voice or display on a display device, wherein the notification unit provides a notification to the person based on at least one of the voice information and the status information via a device that is different from the display device. ### Claim 14 (depends on 1) The information processing system according to claim 1, wherein the support information generation unit controls a timing to provide the support information to the person based on at least one of the voice information and the status information. ### Claim 15 (depends on 1) The information processing system according to claim 1, wherein the status acquisition unit determines the status of the salesclerk based on the position of the salesclerk obtained from the image and information as to whether or not the customer, another salesclerk, or a predetermined article is present around the salesclerk. ### Claim 16 An information processing method comprising: acquiring voice information including an utterance made by a person that is a salesclerk of a shop; acquiring status information related to status of the person, the status information acquired based on an image including the person capture by a camera installed in the shop, the status information including information related to a position of the person; and generating support information used for supporting an operation of the person based on the voice information and the status information, wherein the generating support information comprises withholding generation of the support information when a customer is near the salesclerk. ### Claim 17 A non-transitory storage medium storing a program that causes a computer to perform an information processing method comprising: acquiring voice information including an utterance made by a person that is a salesclerk of a shop; acquiring status information related to status of the person, the status information acquired based on an image including the person captured by a camera installed in the shop, the status information including information related to a position of the person; and generating support information used for supporting an operation of the person based on the voice information and the status information, wherein the generating support information comprises withholding generation of the support information when a customer is near the salesclerk.