# US 11,907,434 B2 · Information processing apparatus, information processing system, and information processing method > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 11,907,434 B2 - Document ID: US11907434B2 - Status: Granted - Kind: B2 - Application number: 17/790,413 - Filed: 2021-01-21 - Published: 2023-03-09 - Granted: 2024-02-20 - Earliest claimed: 2020-01-29 - Assignees: Sony Group Corporation - Applicants: Sony Group Corporation - Inventors: Takayoshi Shimizu; Kazuma Takahashi - Web page: https://patentgrep.com/patent/US11907434B2 ## CPC - G06F 3/017: {Gesture based interaction, e.g. based on a set of recognized hand gestures (interaction based on gestures traced on a digitiser G06F3/04883)} - G06F 3/0482: Interaction with lists of selectable items, e.g. menus - G06F 3/011: {Arrangements for interaction with the human body, e.g. for user immersion in virtual reality (blind teaching G09B21/00)} - G06F 3/016: {Input arrangements with force or tactile feedback as computer generated output to the user} - G06F 3/0481: based on specific properties of the displayed interaction object or a metaphor-based environment, e.g. interaction with desktop elements like windows or icons, or assisted by a cursor's changing behaviour or appearance - H04N 13/243: using three or more two-dimensional [2D] image sensors - H04N 13/344: with head-mounted left-right displays - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) - H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION ## Abstract To provide an information processing apparatus (100) including: a detection unit (110) that detects an object selected by a user with a finger-pointing gesture as a selected object, in which the detection unit detects the selected object on the basis of a first extension line extending from a first line segment connecting a first reference point indicating a position of a wrist of a hand of the user corresponding to the finger-pointing gesture and a second reference point corresponding to a height of the wrist on a first perpendicular line perpendicularly extending from a base of an arm of the user corresponding to the finger-pointing gesture to a floor surface toward a wrist side. ## Claims ### Claim 1 An information processing apparatus comprising: a detection unit that detects an object selected by a user with a finger-pointing gesture as a selected object, wherein the detection unit detects the selected object on a basis of a first extension line extending from a first line segment connecting a first reference point indicating a position of a wrist of a hand of the user corresponding to the finger-pointing gesture and a second reference point corresponding to a height of the wrist on a first perpendicular line perpendicularly extending from a base of an arm of the user corresponding to the finger-pointing gesture to a floor surface toward a wrist side. ### Claim 2 (depends on 1) The information processing apparatus according to claim 1, wherein the detection unit detects the object corresponding to a point on the first extension line as the selected object. ### Claim 3 (depends on 1) The information processing apparatus according to claim 1, wherein the detection unit detects the object corresponding to a second perpendicular line perpendicularly extending from a point located on the first extension line to the floor surface as the selected object. ### Claim 4 (depends on 1) The information processing apparatus according to claim 1, wherein the detection unit detects the object corresponding to a space having a conical shape having the first extension line as an axis and having the first reference point as a vertex as the selected object. ### Claim 5 (depends on 1) The information processing apparatus according to claim 1, wherein the detection unit detects a plurality of the selected objects. ### Claim 6 (depends on 1) The information processing apparatus according to claim 1, wherein a detection mode of the detection unit is switched according to an angle of a corner formed by a second line segment connecting a third reference point indicating a position of a base of an arm of the user corresponding to the finger-pointing gesture and the first reference point and the first perpendicular line on a plane including the second line segment and the first perpendicular line. ### Claim 7 (depends on 6) The information processing apparatus according to claim 6, wherein in a case where the angle of the corner is larger than or equal to a first angle and less than or equal to a second angle, the detection unit executes a first detection mode of detecting the selected object on a basis of the first extension line. ### Claim 8 (depends on 7) The information processing apparatus according to claim 7, wherein in a case where the angle of the corner is larger than the second angle, the detection unit executes a second detection mode of detecting the selected object on a basis of a second extension line extending from the second line segment toward the wrist side. ### Claim 9 (depends on 7) The information processing apparatus according to claim 7, wherein in a case where the angle of the corner is smaller than the first angle, the detection unit executes a third detection mode of not detecting the selected object. ### Claim 10 (depends on 6) The information processing apparatus according to claim 6, further comprising: a determination unit that determines whether a pointing action of selecting the selected object by the user with the finger-pointing gesture satisfies a predetermined stability condition, wherein the determination unit determines that the pointing action satisfies the stability condition in a case where the angle of the corner or a distance from a spatial origin to the wrist of the hand of the user corresponding to the finger-pointing gesture is within a predetermined range within a predetermined time, and the detection unit detects the selected object on a basis of a determination result. ### Claim 11 (depends on 10) The information processing apparatus according to claim 10, wherein the determination unit acquires a plurality of values of the angle of the corner or the distance, calculates a statistical value on a basis of the acquired values, and determines whether or not the pointing action satisfies the stability condition on a basis of the calculated statistical value. ### Claim 12 (depends on 1) The information processing apparatus according to claim 1, wherein the object is a real object on a real space. ### Claim 13 (depends on 12) The information processing apparatus according to claim 12, wherein the real object includes at least one of a human, an animal, a still object, a robot, a lighting, a communication device, and a display. ### Claim 14 (depends on 13) The information processing apparatus according to claim 13, further comprising: a control unit that controls at least one of the robot, the lighting, the communication device, and the display to perform a predetermined operation. ### Claim 15 (depends on 12) The information processing apparatus according to claim 12, further comprising: a display unit that superimposes and displays a predetermined image on the detected selected object. ### Claim 16 (depends on 1) The information processing apparatus according to claim 1, wherein the object is a virtual object on a virtual space. ### Claim 17 (depends on 1) The information processing apparatus according to claim 1, wherein the user and the object are located on a same real space. ### Claim 18 (depends on 1) The information processing apparatus according to claim 1, wherein the user and the object are located on different spaces. ### Claim 19 An information processing system comprising: a detection unit that detects an object selected by a user with a finger-pointing gesture as a selected object; wherein the detection unit detects the selected object on a basis of a first extension line extending from a first line segment connecting a first reference point indicating a position of a wrist of a hand of the user corresponding to the finger-pointing gesture and a second reference point corresponding to a height of the wrist on a first perpendicular line perpendicularly extending from a base of an arm of the user corresponding to the finger-pointing gesture to a floor surface toward a wrist side. ### Claim 20 An information processing method for detecting an object selected by a user with a finger-pointing gesture as a selected object, the method comprising: detecting, by an information processing apparatus, the selected object on a basis of a first extension line extending from a first line segment connecting a first reference point indicating a position of a wrist of a hand of the user corresponding to the finger-pointing gesture and a second reference point corresponding to a height of the wrist on a first perpendicular line perpendicularly extending from a base of an arm of the user corresponding to the finger-pointing gesture to a floor surface toward a wrist side.