# US 11,307,670 B2 · Information processing apparatus, information processing method, and program > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 11,307,670 B2 - Document ID: US11307670B2 - Status: Granted - Kind: B2 - Application number: 17/057,902 - Filed: 2019-03-22 - Published: 2021-07-08 - Granted: 2022-04-19 - Earliest claimed: 2018-05-31 - Assignees: Sony Corporation - Applicants: Sony Corporation - Inventors: Ryusei Koike; Kae Nagano; Koji Sato; Koji Furusawa - Web page: https://patentgrep.com/patent/US11307670B2 ## CPC - G06F 3/017: {Gesture based interaction, e.g. based on a set of recognized hand gestures (interaction based on gestures traced on a digitiser G06F3/04883)} - G06F 3/0486: Drag-and-drop - G06F 3/0481: based on specific properties of the displayed interaction object or a metaphor-based environment, e.g. interaction with desktop elements like windows or icons, or assisted by a cursor's changing behaviour or appearance - G06F 3/04883: for inputting data by handwriting, e.g. gesture or text - G06F 3/16: Sound input; Sound output (speech processing G10L) - H04N 7/142: {Constructional details of the terminal equipment, e.g. arrangements of the camera and the display} - H04N 7/147: {Communication arrangements, e.g. identifying the communication as a video-communication, intermediate storage of the signals (selecting H04Q)} - G06Q 10/103: {Workflow collaboration or project management} - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) - H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION - G06Q: INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR ## Abstract An information processing apparatus, an information processing method, and a program that allow an intuitive grasp of an object indicated by a communication partner in communication through an image are provided. The information processing apparatus includes a control unit that controls interactive communication to display, on a display unit, a captured image of a space on a communication partner side acquired from the communication partner side, the control unit controlling to display an image linking a pointer used by a person on one side to an object indicated by the person. ## Claims ### Claim 1 An information processing apparatus comprising a control unit configured to control interactive communication to display, on a display unit, a captured image of a space on a communication partner side acquired from the communication partner side, and control to display a pointer used by a person on one side to an object indicated by the person and a linking image linking the pointer to the object, the linking image being different than the pointer, wherein the control unit and the display unit are each implemented via at least one processor. ### Claim 2 (depends on 1) The information processing apparatus according to claim 1, wherein the linking image is a linear image connecting the pointer and the object. ### Claim 3 (depends on 1) The information processing apparatus according to claim 1, wherein the pointer is a finger of the person. ### Claim 4 (depends on 1) The information processing apparatus according to claim 1, wherein the linking image is a linear image connecting a fingertip pointed by a communication partner shown on the display unit displaying the captured image, and the object indicated by the communication partner. ### Claim 5 (depends on 1) The information processing apparatus according to claim 1, wherein the object is a real object, and the linking image is a linear image connecting the pointer and the real object across a second display unit on which the real object is placed and a first display unit on which the captured image is displayed. ### Claim 6 (depends on 1) The information processing apparatus according to claim 1, wherein the object is a real object, and the control unit is further configured to perform control to project, by a projector, a linear projection image connecting a place on which the real object is placed to a linear display image connecting the pointer and the object on the display unit on which the captured image is displayed. ### Claim 7 (depends on 1) The information processing apparatus according to claim 1, further comprising a first display unit, and a second display unit, wherein the object is a virtual object, the linking image is a linear image connecting the pointer and the virtual object across the second display unit on which the virtual object is displayed and the first display unit on which the captured image is displayed, wherein the first display unit and the second display unit are each implemented via at least one processor. ### Claim 8 (depends on 1) The information processing apparatus according to claim 1, wherein the control unit is further configured to identify an object located on a local side indicated by a communication partner shown on the display unit, on a basis of indicated object information received from the communication partner side. ### Claim 9 (depends on 1) The information processing apparatus according to claim 1, wherein the control unit is further configured to identify an object located on a local side indicated by a communication partner shown on the display unit, on a basis of indicated object information received from the communication partner side, and perform control, when the pointer is not shown on the display unit, to superimpose and display a virtual image of the pointer, and display a linear image connecting the pointer of the virtual image and the identified object. ### Claim 10 (depends on 1) The information processing apparatus according to claim 1, wherein the control unit is further configured to superimpose, on an object on the communication partner side shown on the display unit, a virtual image showing hidden surfaces of the object for display, and identify a surface indicated by a user on a basis of the user's indication motion toward the virtual image. ### Claim 11 (depends on 1) The information processing apparatus according to claim 1, wherein the control unit is further configured to identify a surface of an object located on a local side indicated by a communication partner shown on the display unit, on a basis of indicated object information received from the communication partner side, and the linking image is a linear image connecting the pointer used by the communication partner shown on the display unit and the identified surface of the object. ### Claim 12 (depends on 1) The information processing apparatus according to claim 1, wherein the control unit is further configured to identify an object indicated by a user on a basis of the user's indication motion. ### Claim 13 (depends on 12) The information processing apparatus according to claim 12, wherein the display unit is further configured to perform display to clearly show the identified object. ### Claim 14 (depends on 12) The information processing apparatus according to claim 12, wherein the display unit is further configured to identify an object area indicated by the user on a basis of the user's voice. ### Claim 15 (depends on 1) The information processing apparatus according to claim 1, wherein only a first portion of the linking image is displayed on a second display unit on which the object is placed and only a second portion of the linking image is displayed on a first display unit on which the captured image is displayed, the first portion being different than the second portion, and wherein the first display unit and the second display unit are each implemented via at least one processor. ### Claim 16 An information processing method comprising: controlling, by a processor, interactive communication to display, on a display, a captured image of a space on a communication partner side acquired from the communication partner side; and controlling, by the processor, to display a pointer used by a person on one side to an object indicated by the person and a linking image linking the pointer to the object, the linking image being different than the pointer. ### Claim 17 A non-transitory computer-readable medium having embodied thereon a program, which when executed by a computer causes the computer to execute an information processing method, the method comprising: controlling interactive communication to display, on a display, a captured image of a space on a communication partner side acquired from the communication partner side; and controlling to display a pointer used by a person on one side to an object indicated by the person and a linking image linking the pointer to the object, the linking image being different than the pointer.