# US 11,445,120 B2 · Information processing method for adjustting an image capture region based on spoken utterance and apparatus therefor > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 11,445,120 B2 - Document ID: US11445120B2 - Status: Granted - Kind: B2 - Application number: 16/787,905 - Filed: 2020-02-11 - Published: 2021-06-03 - Granted: 2022-09-13 - Earliest claimed: 2019-11-29 - Assignees: LG Electronics Inc. - Applicants: LG Electronics Inc. - Inventors: In Jung Yang; Mi Seon Park; Ji-Won Kim - Web page: https://patentgrep.com/patent/US11445120B2 ## CPC - G06F 3/167: {Audio in a user interface, e.g. using voice commands for navigating, audio feedback} - H04N 23/695: Control of camera direction for changing a field of view, e.g. pan, tilt or based on tracking of objects - G06F 40/20: Natural language analysis (semantic analysis of natural language G06F40/30) - G10L 15/19: Grammatical context, e.g. disambiguation of the recognition hypotheses based on word sequence rules - G10L 15/22: Procedures used during a speech recognition process, e.g. man-machine dialogue - G10L 15/26: Speech to text systems (G10L15/08 takes precedence) - H04N 23/611: where the recognised objects include parts of the human body - H04N 23/69: Control of means for changing angle of the field of view, e.g. optical zoom objectives or electronic zooming - G10L 15/1822: {Parsing for meaning understanding} - G10L 2015/223: {Execution procedure of a spoken command} - H04N 23/90: Arrangement of cameras or camera modules, e.g. multiple cameras in TV studios or sports stadiums - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) - H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION - G10L: SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING ## Abstract Disclosed are an information processing method and information processing apparatus which execute an installed artificial intelligence (AI) algorithm and/or machine learning algorithm to process a spoken utterance of a user in a 5G communication environment. The information processing method according to an embodiment of the present disclosure may include receiving a spoken utterance of a user and extracting, from the spoken utterance, a demonstrative pronoun referring to a target indicated by the user, determining an image capture region to be scanned by a camera according to the type of the demonstrative pronoun, recognizing the target indicated by the user from a result of scanning the image capture region, and feeding back a result of processing the spoken utterance to the user on the basis of a result of recognizing the target indicated by the user. ## Claims ### Claim 1 An information processing method performed by an information processing apparatus, the information processing method comprising: receiving a spoken utterance of a user and extracting, from the spoken utterance, a demonstrative pronoun referring to a target indicated by the user; determining an image capture region to be scanned by a camera according to a type of the demonstrative pronoun; recognizing the target indicated by the user from a result of scanning the image capture region; and feeding back, to the user, a result of processing the spoken utterance based on a result of recognizing the target indicated by the user, wherein the determining the image capture region comprises selecting a wide-angle lens of a triple camera provided in the information processing apparatus and having the wide-angle lens, a normal lens, and a telescopic lens to determine a region between the user and the information processing apparatus as the image capture region to be scanned, in response to the extracted demonstrative pronoun being a first demonstrative pronoun referring to a target located near the user. ### Claim 2 (depends on 1) The information processing method of claim 1, wherein the determining the image capture region comprises determining the region between the user and the information processing apparatus by adjusting an intensity of output light outputted from a time-of-flight (TOF) camera provided in the information processing apparatus as the image capture region to be scanned by the camera, in response to the extracted demonstrative pronoun being the first demonstrative pronoun referring to the target located near the user. ### Claim 3 (depends on 1) The information processing method of claim 1, further comprising identifying a direction indicated by the user by obtaining image information comprising a gesture of the user upon receiving the spoken utterance, and wherein the determining the image capture region comprises determining which region among regions located in the direction indicated by the user is to be the image capture region to be scanned by the camera, according to the type of the demonstrative pronoun. ### Claim 4 (depends on 3) The information processing method of claim 3, wherein the identifying the direction indicated by the user comprises: obtaining skeleton information from the gesture of the user included in the image information; and identifying the direction indicated by the user using a position of shoulder joints and a position of knuckles of the user from the skeleton information. ### Claim 5 (depends on 3) The information processing method of claim 3, wherein the identifying the direction indicated by the user comprises identifying the direction indicated by the user included in the image information using a deep neural network model trained in advance to identify a direction indicated by an object by analyzing a gesture of the object included in image information. ### Claim 6 (depends on 3) The information processing method of claim 3, wherein the determining the image capture region comprises determining which region among the regions located in the direction indicated by the user is to be the image capture region to be scanned by the camera, in response to the extracted demonstrative pronoun being a second demonstrative pronoun referring to a target located far away from the user and the information processing apparatus. ### Claim 7 (depends on 3) The information processing method of claim 3, wherein the determining the image capture region comprises selecting the telescopic lens of the triple camera provided in the information processing apparatus to determine a peripheral region of the direction indicated by the user as the image capture region to be scanned, in response to the extracted demonstrative pronoun being a second demonstrative pronoun referring to a target located far away from the user and the information processing apparatus. ### Claim 8 (depends on 3) The information processing method of claim 3, wherein the determining the image capture region comprises increasing an intensity of output light outputted from a time-of-flight (TOF) camera provided in the information processing apparatus to a value higher than a reference value to determine a peripheral region of the direction indicated by the user as the image capture region to be scanned, in response to the extracted demonstrative pronoun being a second demonstrative pronoun referring to a target located far away from the user and the information processing apparatus. ### Claim 9 An information processing method performed by an information processing apparatus, the information processing method comprising: receiving a spoken utterance of a user and extracting, from the spoken utterance, a demonstrative pronoun referring to a target indicated by the user; determining an image capture region to be scanned by a camera according to a type of the demonstrative pronoun; recognizing the target indicated by the user from a result of scanning the image capture region; and feeding back, to the user, a result of processing the spoken utterance based on a result of recognizing the target indicated by the user, wherein the determining the image capture region comprises decreasing an intensity of output light outputted from a time-of-flight (TOF) camera provided in the information processing apparatus to a value lower than a reference value to determine a region between the user and the information processing apparatus as the image capture region to be scanned, in response to the extracted demonstrative pronoun being a first demonstrative pronoun referring to a target located near the user. ### Claim 10 An information processing apparatus comprising: an extraction processor configured to receive a spoken utterance of a user and extract, from the spoken utterance, a demonstrative pronoun referring to a target indicated by the user; a determination processor configured to determine an image capture region to be scanned by a camera according to a type of the demonstrative pronoun; a recognition processor configured to recognize the target indicated by the user from a result of scanning the image capture region; and a feedback processor configured to feed back, to the user, a result of processing the spoken utterance based on a result of recognizing the target indicated by the user, wherein the determination processor is configured to select a wide-angle lens of a triple camera provided in the information processing apparatus and having the wide-angle lens, a normal lens, and a telescopic lens to determine a region between the user and the information processing apparatus as the image capture region to be scanned, in response to the extracted demonstrative pronoun being a first demonstrative pronoun referring to a target located near the user. ### Claim 11 (depends on 10) The information processing apparatus of claim 10, wherein the determination processor is configured to determine the region between the user and the information processing apparatus by adjusting an intensity of output light outputted from a time-of-flight (TOF) camera provided in the information processing apparatus as the image capture region to be scanned by the camera, in response to the extracted demonstrative pronoun being the first demonstrative pronoun referring to the target located near the user. ### Claim 12 (depends on 10) The information processing apparatus of claim 10, wherein the determination processor is configured to decrease an intensity of output light outputted from a time-of-flight (TOF) camera provided in the information processing apparatus to a value lower than a reference value to determine a region between the user and the information processing apparatus as the image capture region to be scanned, in response to the extracted demonstrative pronoun being the first demonstrative pronoun referring to the target located near the user. ### Claim 13 (depends on 10) The information processing apparatus of claim 10, further comprising an analysis processor configured to identify a direction indicated by the user by obtaining image information comprising a gesture of the user, upon receiving the spoken utterance, and wherein the determination processor is configured to determine which region among regions located in the direction indicated by the user is to be the image capture region to be scanned by the camera, according to the type of the demonstrative pronoun. ### Claim 14 (depends on 13) The information processing apparatus of claim 13, wherein the analysis processor is configured to obtain skeleton information from the gesture of the user included in the image information, and identify the direction indicated by the user using a position of shoulder joints and a position of knuckles of the user from the skeleton information. ### Claim 15 (depends on 13) The information processing apparatus of claim 13, wherein the analysis processor is configured to identify the direction indicated by the user included in the image information using a deep neural network model trained in advance to identify a direction indicated by an object by analyzing a gesture of the object included in image information. ### Claim 16 (depends on 13) The information processing apparatus of claim 13, wherein the determination processor is configured to determine which region among the regions located in the direction indicated by the user is to be the image capture region to be scanned by the camera, in response to the extracted demonstrative pronoun being a second demonstrative pronoun referring to a target located far away from the user and the information processing apparatus. ### Claim 17 (depends on 13) The information processing apparatus of claim 13, wherein the determination processor is configured to select the telescopic lens of the triple camera provided in the information processing apparatus to determine a peripheral region of the direction indicated by the user as the image capture region to be scanned, in response to the extracted demonstrative pronoun being a second demonstrative pronoun referring to a target located far away from the user and the information processing apparatus. ### Claim 18 (depends on 13) The information processing apparatus of claim 13, wherein the determination processor is configured to increase an intensity of output light outputted from a time-of-flight (TOF) camera provided in the information processing apparatus to a value higher than a reference value to determine a peripheral region of the direction indicated by the user as the image capture region to be scanned, in response to the extracted demonstrative pronoun being a second demonstrative pronoun referring to a target located far away from the user and the information processing apparatus.