Information processing apparatus, information processing method, computer program, and image processing system

An information processing apparatus comprising:

• a receiver configured to receive first information on a posture of a head of an observer and second information on a posture of a body other than the head of the observer;
• an image rendering processing unit configured to generate a display image corresponding to a posture of the observer based on the first information and the second information;
• and
• a calibration processing unit configured to acquire a calibration parameter for mutually converting a coordinate system in which the posture of the head of the observer is indicated in the first information and a coordinate system in which the posture of the body other than the head of the observer is indicated in the second information,
• wherein the image rendering processing unit performs image generation using posture information corrected by the calibration parameter, and
• wherein the calibration processing unit, based on head posture information obtained from the first information and second posture information obtained from the second information, calculates a calibration parameter used to correct the second posture information and corrects the second posture information using the calibration parameter.

An information processing apparatus comprising:

• a receiver configured to receive first information on a posture of a head of an observer and second information on a posture of a body other than the head of the observer;
• an image rendering processing unit configured to generate a display image corresponding to a posture of the observer based on the first information and the second information;
• and
• a calibration processing unit configured to acquire a calibration parameter,
• wherein the image rendering processing unit performs image generation using posture information corrected by the calibration parameter, and
• wherein the calibration processing unit estimates a calibration parameter used to correct the posture information based on rotational displacement for a certain period of a coordinate system of a head posture obtained from the first information and a coordinate system of a posture obtained from the second information and a property of a human body.

An information processing apparatus comprising:

• a receiver configured to receive first information on a posture of a head of an observer and second information on a posture of a body other than the head of the observer;
• an image rendering processing unit configured to generate a display image corresponding to a posture of the observer based on the first information and the second information;
• and
• a calibration processing unit configured to acquire a calibration parameter,
• wherein the image rendering processing unit performs image generation using posture information corrected by the calibration parameter, and
• wherein the calibration processing unit sets a time-series average of calibration parameters calculated for a certain period based on head posture information obtained from the first information and second posture information obtained from the second information to be a final calibration parameter.

An information processing apparatus comprising:

• a receiver configured to receive first information on a posture of a head of an observer and second information on a posture of a body other than the head of the observer;
• an image rendering processing unit configured to generate a display image corresponding to a posture of the observer based on the first information and the second information,
• wherein the receiver receives at least a posture of a trunk of the observer as the second information,
• wherein the image rendering processing unit generates a free viewpoint image that tracks a posture of the head of the observer by orienting a line-of-sight direction of the observer in a free viewpoint space based on the first information and by orienting an orientation (viewpoint position) of the body of the observer in the free viewpoint space based on a posture of the trunk of the observer obtained from the second information, and
• wherein the image rendering processing unit generates a free viewpoint image by correcting a world coordinate system in a manner that a difference between a head posture and a trunk posture in the free viewpoint space decreases under a biosafety support mode.

An information processing apparatus comprising:

• a receiver configured to receive first information on a posture of a head of an observer and second information on a posture of a body other than the head of the observer;
• and
• an image rendering processing unit configured to generate a display image corresponding to a posture of the observer based on the first information and the second information,
• wherein the receiver receives at least a posture of a trunk of the observer as the second information,
• wherein the image rendering processing unit generates a free viewpoint image that tracks a posture of the head of the observer by orienting a line-of-sight direction of the observer in a free viewpoint space based on the first information and by orienting an orientation (viewpoint position) of the body of the observer in the free viewpoint space based on a posture of the trunk of the observer obtained from the second information, and
• wherein the image rendering processing unit generates a free viewpoint image by fixing a head posture in a free viewpoint space at a point of time when a biosafety support mode is set and by changing a trunk posture in a free viewpoint space depending on a head posture detected by ahead posture detection unit under the biosafety support mode.

An information processing method comprising:

• a receiving step of receiving first information on a posture of a head of an observer and second information on a posture of a body other than the head of the observer;
• and
• an image rendering processing step of generating a display image corresponding to a posture of the observer based on the first information and the second information,
• wherein at least a posture of a trunk of the observer as the second information is received in the receiving step, and
• wherein, in the image rendering processing step,
• a free viewpoint image is generated that tracks a posture of the head of the observer by orienting a line-of-sight direction of the observer in a free viewpoint space based on the first information and by orienting an orientation (viewpoint position) of the body of the observer in the free viewpoint space based on a posture of the trunk of the observer obtained from the second information, and
• when a control signal used is received to instruct the observer to move in the free viewpoint space as an input, a movement amount in a front direction of the trunk of the observer in the free viewpoint space is determined based on the posture of the trunk of the observer indicated in the second information and by recognizing the orientation of the body of the observer as a position of a front of the observer.

A non-transitory storage medium configured to store a computer program written in a computer readable format to cause a computer to function as:

• a head posture operation unit configured to calculate posture information of a head of an observer based on a result obtained by detection in a head posture detection unit configured to detect a posture of the head of the observer;
• a second posture operation unit configured to calculate posture information of a second part of a body other than the head of the observer based on a result obtained by detection in a second posture detection unit configured to detect a posture of one or more of the second part;
• and
• an image rendering processing unit configured to process an image to be displayed on a display unit based on a posture of the head of the observer and a posture of the second part, the display unit being fixed to the head or a face of the observer,
• wherein the detection in the second posture detection unit detects at least a posture of a trunk of the observer as the one or more of the second part, and
• wherein the image rendering processing unit is configured to
• generate a free viewpoint image that tracks a posture of the head of the observer by orienting a line-of-sight direction of the observer in a free viewpoint space based on the posture information of the head of the observer calculated in the head posture operation unit and by orienting an orientation (viewpoint position) of the body other than the head of the observer in the free viewpoint space based on a posture of the trunk of the observer obtained from the posture information of the second part of the body other than the head calculated in the second posture operation unit, and
• when receiving a control signal used to instruct the observer to move in the free viewpoint space as an input, determine a movement amount in a front direction of the trunk of the observer in the free viewpoint space based on the posture of the trunk of the observer indicated in the second information and by recognizing the orientation of the body of the observer as a position of a front of the observer.

An image processing system comprising:

• a display unit fixed to a head or a face of an observer;
• a head posture detection unit configured to detect a posture of the head of the observer;
• a second posture detection unit configured to detect a posture of one or more second parts of a body other than the head of the observer;
• and
• an image rendering processing unit configured to process a display image on the display unit based on a posture of the head of the observer and a posture of each of the second parts,
• wherein detection in the second posture detection unit detects at least a posture of a trunk of the observer as the one or more second parts, and
• wherein the image rendering processing unit is configured to
• generate a free viewpoint image that tracks a posture of the head of the observer by orienting a line-of-sight direction of the observer in a free viewpoint space based on detection of the posture of the head of the observer by the head posture detection unit and by orienting an orientation (viewpoint position) of the one or more second parts of the body other than the head of the observer in the free viewpoint space based on a posture of the trunk of the observer obtained from the detection in the second posture detection unit, and
• when receiving a control signal used to instruct the observer to move in the free viewpoint space as an input, determine a movement amount in a front direction of the trunk of the observer in the free viewpoint space based on the posture of the trunk of the observer indicated in the second information and by recognizing the orientation of the body of the observer as a position of a front of the observer.