Image display system and information processing apparatus and control methods thereof
AssigneeCanon Kabushiki Kaisha
InventorNorihito Aoki
Life of this patent
Estimating the term…
Abstract
This invention suppresses the narrowing of the communication band between a device and an information processing apparatus that form an image display system. Hence, an HMD of a user includes a sensor that detects a position and orientation, a communication interface that transmits, to the information processing apparatus, position and orientation information representing the detected position and orientation and receives a CG command from the information processing apparatus, a rendering unit that renders a CG based on the received CG command, and a display control unit that displays the rendered CG on a display unit.
Claims9
An image display system comprising:
- a computer graphics (CG) displaying device including a display device that displays an image to a user;
- and
- an information processing apparatus that transmits information for displaying CG on the display device,
- wherein the displaying device comprises:
- an image capturing device configured to capture an image of a physical space;
- and
- a position and orientation detecting sensor that detects a position and orientation of the displaying device;
- a first communication interface that communicates with the information processing apparatus;
- at least one first processor or circuit configured to implement first instructions to execute a first plurality of tasks, including:
- a first transmitting task that transmits, via the first communication interface, to the information processing apparatus, position and orientation information related to the position and orientation detected by the position and orientation detecting sensor and specific information for specifying a frame captured by the image capturing device;
- a first receiving task that receives a CG command from the information processing apparatus;
- a rendering task that renders CG based on the CG command received by the first receiving task;
- a synthesizing task that synthesizes the CG on to the captured image of the physical space in a case where the specific information of the CG rendered by the rendering task and the specific information of a synthesizing target image match;
- and
- a display control task that controls to the display device to display the CG rendered by the rendering task and an image synthesized by the synthesizing task, and
- wherein the information processing apparatus comprises:
- a second communication interface for communicating with the displaying device;
- at least one second processor or circuit configured to implement second instructions to execute a second plurality of tasks, including:
- a second receiving task that receives the position and orientation information and the specific information from the displaying device;
- a CG command generating task that generates the CG command based on the position and orientation information and the specific information received from the displaying device;
- and
- a second transmitting task that transmits the generated CG command to the displaying device via the second communication interface.
A control method for an image display system comprising a computer graphics (CG) displaying device that includes an image capturing device that captures an image of a physical space and a display device that displays an image to a user, and an information processing apparatus that transmits information for displaying CG on the display device, the method comprising:
- a detecting step of detecting, with the displaying device, via a sensor provided in the displaying device, a position and orientation of the displaying device;
- a first transmitting step of transmitting, to the information processing apparatus, with the displaying device, via a first communication interface provided in the displaying device, position and orientation information related to the position and orientation detected in the detecting step and specific information for specifying a frame captured by the image capturing device;
- a first receiving step of receiving, with the displaying device, via the first communication interface, a CG command from the information processing apparatus;
- a rendering step of rendering, with the displaying device, CG based on the CG command received in the receiving step;
- a synthesizing step of synthesizing the CG on to the captured image of the physical space in a case where the specific information of the CG rendered in the rendering step and the specific information of a synthesizing target image match;
- a displaying step of displaying, with the displaying device, the rendered CG rendered in the rendering step and an image synthesized in the synthesizing step;
- a second receiving step of receiving, from the displaying device, with the information processing apparatus, via a second communication interface provided in the image processing apparatus, the position and orientation information and the specific information;
- a CG command generating step of generating, with the information processing apparatus, the CG command based on the position and orientation information and the specific information received in the second receiving step;
- and
- a second transmitting step of transmitting, with the information processing apparatus, via the second communication interface, the generated CG command to the displaying device.
A non-transitory computer-readable storage medium storing a program executable by at least one processor, of an image display system comprising a computer graphics (CG) displaying device that includes an image capturing device that captures an image of a physical space and a display device that displays an image to a user, and an information processing apparatus that transmits information for displaying CG on the display device, to execute a method comprising:
- a detecting step of detecting, with the displaying device, via a sensor provided in the displaying device, a position and orientation of the displaying device;
- a first transmitting step of transmitting, to the information processing apparatus, with the displaying device, via a first communication interface provided in the displaying device, position and orientation information related to the position and orientation detected in the detecting step and specific information for specifying a frame captured by the image capturing device;
- a first receiving step of receiving, with the displaying device, via the first communication interface, a CG command from the information processing apparatus;
- a rendering step of rendering, with the displaying device, CG based on the CG command received in the receiving step;
- a synthesizing step of synthesizing the CG on to the captured image of the physical space in a case where the specific information of the CG rendered in the rendering step and the specific information of a synthesizing target image match;
- a displaying step of displaying, with the displaying device, the rendered CG rendered in the rendering step and an image synthesized in the synthesizing step;
- a second receiving step of receiving, from the displaying device, with the information processing apparatus, via a second communication interface provided in the image processing apparatus, the position and orientation information and the specific information;
- a CG command generating step of generating, with the information processing apparatus, the CG command based on the position and orientation information and the specific information received in the receiving step;
- and
- a second transmitting step of transmitting, with the information processing apparatus, via the second communication interface, the generated CG command to the displaying device via the communication interface.
Dependencies are inferred from the claim text.