# US 12,217,343 B2 · Information processing device, information processing method, and program for causing a virtual object to perform movement according to a sound reproduced from a real sound source > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 12,217,343 B2 - Document ID: US12217343B2 - Status: Granted - Kind: B2 - Application number: 17/774,681 - Filed: 2020-09-04 - Published: 2022-12-29 - Granted: 2025-02-04 - Earliest claimed: 2019-11-12 - Assignees: Sony Group Corporation - Applicants: Sony Group Corporation - Inventors: Takaomi Kimura; Yasuko Ishihara; Hajime Wakabayashi; Atsushi Ishihara; Masayuki Inoue; Hirotake Ichikawa; Hidenori Aoki; Ryo Fukazawa - Web page: https://patentgrep.com/patent/US12217343B2 ## CPC - G06T 13/205: {driven by audio data} - G06T 19/00: Manipulating three-dimensional [3D] models or images for computer graphics - G06T 19/003: {Navigation within 3D models or images} - G06T 19/006: {Mixed reality (object pose determination, tracking or camera calibration for mixed reality G06T7/00)} - G09G 5/00: Control arrangements or circuits for visual indicators common to cathode-ray tube indicators and other visual indicators (image data processing or generation, in general G06T) - G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL - G09G: ARRANGEMENTS OR CIRCUITS FOR CONTROL OF INDICATING DEVICES USING STATIC MEANS TO PRESENT VARIABLE INFORMATION (arrangements for transferring data between digital computers and displays G06F3/14; static indicating arrangements comprising an association of a number of separate sources or light control cells G09F9/00; static indicating arrangements comprising integral associations of a number of light sources H01J, H01K, H10H, H05B33/12; scanning, transmission or reproduction of documents or the like, e.g. facsimile transmission, details thereof H04N1/00) ## Abstract An information processing device including a control unit that displays a virtual object on a three-dimensional coordinate space associated with a real space and causes the virtual object to perform a predetermined movement according to a predetermined sound reproduced from a real sound source disposed in the real space, in which the control unit performs delay amount setting processing of increasing a delay amount of movement start timing at which the virtual object is caused to perform the predetermined movement with respect to reproduction timing at which the real sound source reproduces the predetermined sound, as a position of the real sound source in the real space and a position of the virtual object on the three-dimensional coordinate space are separated from each other. ## Claims ### Claim 1 An information processing device comprising a control unit configured to initiate display of a virtual object on a three-dimensional coordinate space, the three-dimensional coordinate space being associated with a real space, cause the virtual object to perform a predetermined movement according to a predetermined sound reproduced from a real sound source disposed in the real space, and perform delay amount setting processing that includes increasing a delay amount between a movement start timing at which the virtual object is caused to perform the predetermined movement and a reproduction timing at which the real sound source reproduces the predetermined sound, as a distance between a position of the real sound source on the three-dimensional coordinate space associated with the real space and a position of the virtual object on the three-dimensional coordinate space associated with the real space increases, wherein the control unit is implemented via at least one processor. ### Claim 2 (depends on 1) The information processing device according to claim 1, wherein the control unit is further configured to acquire the position of the virtual object on the associated three-dimensional coordinate space using a relative position of the virtual object defined with respect to a position of the information processing device on the three-dimensional coordinate space associated with the real space. ### Claim 3 (depends on 2) The information processing device according to claim 2, wherein the control unit is further configured to calculate the distance between the position of the real sound source and the position of the virtual object on the associated three-dimensional coordinate space, and set the delay amount on a basis of a result of the calculation. ### Claim 4 (depends on 3) The information processing device according to claim 3, wherein the position of the real sound source in the real space is supplied from an external device. ### Claim 5 (depends on 2) The information processing device according to claim 2, wherein the position of the information processing device is variable. ### Claim 6 (depends on 2) The information processing device according to claim 2 further comprising: a position information acquisition unit configured to acquire the position of the information processing device, wherein the position information acquisition unit is implemented via at least one processor. ### Claim 7 (depends on 1) The information processing device according to claim 1, wherein the control unit is further configured to perform synchronization processing of setting the movement start timing to predetermined timing regardless of the delay amount. ### Claim 8 (depends on 7) The information processing device according to claim 7, wherein the control unit is further configured to determine the movement start timing on a basis of the synchronization processing in a case where synchronization is set as predetermined synchronization information, and determine the movement start timing on a basis of the delay amount setting processing in a case where asynchronization is set as predetermined synchronization information. ### Claim 9 (depends on 8) The information processing device according to claim 8, wherein the synchronization information is set for each of a plurality of virtual objects. ### Claim 10 (depends on 7) The information processing device according to claim 7, wherein the predetermined timing is common timing to another information processing device operated by a same application. ### Claim 11 (depends on 7) The information processing device according to claim 7, wherein in a case where the synchronization processing is performed, the control unit displays information corresponding to the synchronization processing. ### Claim 12 (depends on 1) The information processing device according to claim 1, wherein in a case where share is set as predetermined share information, the control unit displays the virtual object so as to correspond to a positional relation between a position of the information processing device on the three-dimensional coordinate space associated with the real space and the position of the virtual object on the associated three-dimensional coordinate space, and wherein in a case where not-share is set as predetermined share information, the control unit displays the virtual object so as to correspond to a positional relation between a position of each of a plurality of information processing devices on the three-dimensional coordinate space associated with the real space and the position of the virtual object on the associated three-dimensional coordinate space. ### Claim 13 (depends on 12) The information processing device according to claim 12, wherein the share information is set for each of a plurality of virtual objects. ### Claim 14 (depends on 1) The information processing device according to claim 1, wherein the control unit performs the delay amount setting processing for each of a plurality of virtual objects to be displayed. ### Claim 15 (depends on 1) The information processing device according to claim 1, wherein the information processing device includes a portable wearable display that displays the virtual object. ### Claim 16 (depends on 1) The information processing device according to claim 1, wherein the delay amount setting processing is performed in each of a plurality of information processing devices operated by a same application. ### Claim 17 An information processing method performed by a control unit implemented via at least one processor, the method comprising: displaying a virtual object on a three-dimensional coordinate space, the three-dimensional coordinate space being associated with a real space; causing the virtual object to perform a predetermined movement according to a predetermined sound reproduced from a real sound source disposed in the real space; and performing delay amount setting processing that includes increasing a delay amount between a movement start timing at which the virtual object is caused to perform the predetermined movement and a reproduction timing at which the real sound source reproduces the predetermined sound, as a position of the real sound source on the three-dimensional coordinate space associated with the real space and a position of the virtual object on the three-dimensional coordinate space associated with the real space increases. ### Claim 18 A non-transitory computer-readable storage medium having embodied thereon a program, which when executed by a computer causes the computer to execute an information processing method, the method comprising: displaying a virtual object on a three-dimensional coordinate space, the three-dimensional coordinate space being associated with a real space; causing the virtual object to perform a predetermined movement according to a predetermined sound reproduced from a real sound source disposed in the real space; and performing delay amount setting processing that includes increasing a delay amount between a movement start timing at which the virtual object is caused to perform the predetermined movement a reproduction timing at which the real sound source reproduces the predetermined sound, as a position of the real sound source on the three-dimensional coordinate space associated with the real space and a position of the virtual object on the three-dimensional coordinate space associated with the real space increases.