# US 11,908,055 B2 · Information processing device, information processing method, and recording medium > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 11,908,055 B2 - Document ID: US11908055B2 - Status: Granted - Kind: B2 - Application number: 17/434,970 - Filed: 2020-03-04 - Published: 2022-06-09 - Granted: 2024-02-20 - Earliest claimed: 2019-03-13 - Assignees: Sony Group Corporation - Applicants: Sony Group Corporation - Inventors: Mari Saito; Kenji Sugihara - Web page: https://patentgrep.com/patent/US11908055B2 ## CPC - G06F 3/011: {Arrangements for interaction with the human body, e.g. for user immersion in virtual reality (blind teaching G09B21/00)} - G06T 13/00: Animation - G06F 3/012: {Head tracking input arrangements} - G06F 3/013: {Eye tracking input arrangements (G06F3/015 takes precedence)} - G06F 3/015: {Input arrangements based on nervous system activity detection, e.g. brain waves [EEG] detection, electromyograms [EMG] detection, electrodermal response detection} - G06F 3/16: Sound input; Sound output (speech processing G10L) - G06F 3/167: {Audio in a user interface, e.g. using voice commands for navigating, audio feedback} - G06T 7/70: Determining position or orientation of objects or cameras (camera calibration G06T7/80) - G06T 19/006: {Mixed reality (object pose determination, tracking or camera calibration for mixed reality G06T7/00)} - G10L 15/22: Procedures used during a speech recognition process, e.g. man-machine dialogue - G10L 25/51: for comparison or discrimination - G06T 2207/30196: Human being; Person - G06T 2219/024: Multi-user, collaborative environment - G10L 2015/223: {Execution procedure of a spoken command} - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) - G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL - G10L: SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING ## Abstract An information processing device includes an acquisition unit that acquires position information, visual line detection information, and voice information of a first user, and position information of a second user, a determination unit that determines a positional relationship between the first user and the second user on a basis of the position information of the first user and the position information of the second user, a specification unit that specifies a voice dialogue area in which a voice dialogue is possible to be performed between the first user and the second user on a basis of the determined positional relationship, and an avatar control unit that sets a position of an avatar outside the specified voice dialogue area, in which in a case where the voice information of the first user is acquired when the first user is looking outside the voice dialogue area, the avatar control unit causes the avatar to execute a predetermined action, and in a case where the voice information of the first user is acquired when the first user is looking inside the voice dialogue area, the avatar control unit suppresses the avatar from executing the predetermined action. ## Claims ### Claim 1 An information processing device comprising: an acquisition unit that acquires position information, visual line detection information, and voice information of a first user, and position information of a second user; a determination unit that determines a positional relationship between the first user and the second user on a basis of the position information of the first user and the position information of the second user; a specification unit that specifies a voice dialogue area in which a voice dialogue is possible to be performed between the first user and the second user on a basis of the determined positional relationship; and an avatar control unit that sets a position of an avatar outside the specified voice dialogue area, wherein in a case where the voice information of the first user is acquired when the first user is looking outside the voice dialogue area, the avatar control unit causes the avatar to execute a predetermined action, and in a case where the voice information of the first user is acquired when the first user is looking inside the voice dialogue area, the avatar control unit suppresses the avatar from executing the predetermined action. ### Claim 2 (depends on 1) The information processing device according to claim 1, further comprising a detection unit that detects visual line direction information and a peripheral visual field of the first user, wherein the specification unit specifies the voice dialogue area on a basis of the visual line direction information of the first user when the first user is having a dialogue with the second user. ### Claim 3 (depends on 2) The information processing device according to claim 2, wherein the detection unit detects gaze points of the first user as the visual line direction information on a basis of the visual line detection information, and the specification unit specifies a gaze area that is an area in which the gaze points of the first user are distributed, and specifies the voice dialogue area on a basis of the gaze area and the positional relationship. ### Claim 4 (depends on 3) The information processing device according to claim 3, wherein in a case where the voice dialogue area is specified on a basis of the gaze points of the first user, the detection unit further detects a central visual field of the first user, and the avatar control unit sets the position of the avatar at a position outside the voice dialogue area, outside the central visual field of the first user, and inside the peripheral visual field of the first user. ### Claim 5 (depends on 2) The information processing device according to claim 2, wherein the detection unit detects a direction of a face of the first user as the visual line direction information on a basis of inertia information of the first user acquired as the visual line detection information, and the specification unit specifies the voice dialogue area on a basis of the direction of the face of the first user. ### Claim 6 (depends on 5) The information processing device according to claim 5, wherein in a case where the voice dialogue area is identified based on the direction of the face of the first user, the specification unit further specifies a visual line movement area that is an area in which the first user is able to move a visual line, and the avatar control unit sets the position of the avatar at a position outside the voice dialogue area, outside the visual line movement area, and inside the peripheral visual field of the first user. ### Claim 7 (depends on 2) The information processing device according to claim 2, wherein in a case where the first user and the second user use a common avatar, the avatar control unit sets a position of the common avatar in a common space between the peripheral visual field of the first user and the peripheral visual field of the second user. ### Claim 8 (depends on 2) The information processing device according to claim 2, wherein in a case where the first user and the second user use a common avatar, the avatar control unit selects either the peripheral visual field of the first user or the peripheral visual field of the second user, and sets the position of the common avatar in the selected peripheral visual field. ### Claim 9 (depends on 2) The information processing device according to claim 2, wherein in a case where the first user is having a dialogue with the second user without looking at the second user, the specification unit specifies a space between the first user and the second user as the voice dialogue area on a basis of the positional relationship. ### Claim 10 (depends on 9) The information processing device according to claim 9, wherein in a case where the first user and the second user are moving side by side in parallel, the avatar control unit sets the position of the avatar at a position outside a space in a moving direction of the first user and the second user. ### Claim 11 (depends on 9) The information processing device according to claim 9, wherein in a case where the first user is working, the avatar control unit sets the position of the avatar at a position outside a work area that is an area where the first user performs work. ### Claim 12 (depends on 1) The information processing device according to claim 1, wherein the specification unit specifies in advance an area in which it is estimated that the voice dialogue is possible to be performed between the first user and the second user as the voice dialogue area. ### Claim 13 (depends on 1) The information processing device according to claim 1, wherein the avatar control unit causes a position of the avatar to be presented by displaying an object or outputting a voice. ### Claim 14 (depends on 1) The information processing device according to claim 1, further comprising a recognition unit that performs voice recognition on a basis of the voice information, wherein the predetermined action is an action determined on a basis of a result of the voice recognition by the recognition unit. ### Claim 15 (depends on 1) The information processing device according to claim 1, wherein the acquisition unit includes a microphone that acquires the voice information of the first user and the second user, and acquires the position information of the second user on a basis of the voice information acquired by the microphone. ### Claim 16 (depends on 1) The information processing device according to claim 1, wherein the acquisition unit includes an imaging device capable of imaging a periphery of the first user, and acquires the position information of the second user on a basis of an image captured by the imaging device. ### Claim 17 (depends on 1) The information processing device according to claim 1, further comprising a communication unit, wherein the avatar control unit acquires information regarding a position where it is possible to display the avatar for the second user from an external device via communication by the communication unit. ### Claim 18 (depends on 1) The information processing device according to claim 1, further comprising a display unit, wherein the display unit displays the avatar at the position set by the avatar control unit. ### Claim 19 An information processing method comprising: acquiring position information, visual line detection information, and voice information of a first user, and position information of a second user; determining a positional relationship between the first user and the second user on a basis of the position information of the first user and the position information of the second user; and specifying a voice dialogue area in which a voice dialogue is possible to be performed between the first user and the second user on a basis of the determined positional relationship; and setting a position of an avatar outside the specified voice dialogue area, wherein in a case where the voice information of the first user is acquired when the first user is looking outside the voice dialogue area, the setting causes the avatar to execute a predetermined action, and in a case where the voice information of the first user is acquired when the first user is looking inside the voice dialogue area, the setting suppresses the avatar from executing the predetermined action. ### Claim 20 A recording medium recording a program causing a computer to function as: an acquisition unit that acquires position information, visual line detection information, and voice information of a first user, and position information of a second user; a determination unit that determines a positional relationship between the first user and the second user on a basis of the position information of the first user and the position information of the second user; a specification unit that specifies a voice dialogue area in which a voice dialogue is possible to be performed between the first user and the second user on a basis of the determined positional relationship; and an avatar control unit that sets a position of an avatar outside the specified voice dialogue area, wherein in a case where the voice information of the first user is acquired when the first user is looking outside the voice dialogue area, the avatar control unit causes the avatar to execute a predetermined action, and in a case where the voice information of the first user is acquired when the first user is looking inside the voice dialogue area, the avatar control unit suppresses the avatar from executing the predetermined action.