Image composition system, image composition method, and image composition apparatus

An information processing method for an information processing apparatus comprising the steps of:

• acquiring a sense image of an image sensing unit with the information processing apparatus;
• acquiring coordinate values of registered indices with the information processing apparatus;
• detecting image positions of the registered indices in the sense image with the information processing apparatus;
• calculating position and orientation of the image sensing unit on the basis of the detected image positions of the registered indices and the coordinate values of the registered indices with the information processing apparatus;
• detecting an image position of an additional index in the sense image in accordance with a user's instruction with the information processing apparatus;
• calculating a coordinate value of the additional index using the image positions of the registered indices, the coordinate values of the registered indices, the position and orientation of the image sensing unit, and the image position of the additional index with the information processing apparatus;
• calculating a feature amount of the additional index from the sense image with the information processing apparatus;
• and
• registering the coordinate value and the feature amount of the additional index with the information processing apparatus,
• wherein when the coordinate value of the additional index cannot be calculated due to insufficient information for calculating the coordinate value, the additional index is temporarily registered as a vague index,
• wherein when the coordinate value of the additional index can be calculated due to sufficient information for calculating the coordinate value, the additional index is regularly registered as a regular index, and
• wherein when the coordinate value of the vague index can be calculated due to sufficient information for calculating the coordinate value, the vague index is re-registered as a regular index.

An information processing apparatus comprising:

• first acquisition means for acquiring a sense image of an image sensing unit;
• second acquisition means for acquiring coordinate values of registered indices;
• detecting means for detecting image positions of the registered indices in the sense image;
• calculating means for calculating position and orientation of the image sensing unit on the basis of the detected image positions of the registered indices and the coordinate values of the registered indices;
• detection means for detecting an image position of an additional index in the sense image in accordance with a user's instruction;
• calculation means for calculating a coordinate value of the additional index using image positions of the registered indices, the coordinate values of the registered indices, the position and orientation of the image sensing unit, and the image position of the additional index;
• and
• registration means for registering the calculated coordinate value of the additional index as a registered index,
• wherein when the coordinate value of the additional index cannot be calculated due to insufficient information for calculating the coordinate value, the additional index is temporarily registered as a vague index,
• wherein when the coordinate value of the additional index can be calculated due to sufficient information for calculating the coordinate value, the additional index is regularly registered as a regular index, and
• wherein when the coordinate value of the vague index can be calculated due to sufficient information for calculating the coordinate value, the vague index is re-registered as a regular index.

An information processing apparatus comprising:

• first acquisition means for acquiring a sense image of an image sensing unit;
• second acquisition means for acquiring coordinate values of registered indices;
• detecting means for detecting image positions of the registered indices in the sense image;
• calculating means for calculating position and orientation of the image sensing unit on the basis of the detected image positions of the registered indices and the coordinate values of the registered indices;
• detection means for detecting an image position of an additional index in the sense image in accordance with a user's information;
• first calculation means for calculating a coordinate value of the additional index using image positions of the registered indices, the coordinate values of the registered indices, the position and orientation of the image sensing unit, and the image position of the additional index;
• second calculation means for calculating a feature amount of the additional index from the sense image;
• and
• registration means for registering the coordinate value and the feature amount of the additional index,
• wherein when the coordinate value of the additional index cannot be calculated due to insufficient information for calculating the coordinate value, the additional index is temporarily registered as a vague index,
• wherein when the coordinate value of the additional index can be calculated due to sufficient information for calculating the coordinate value, the additional index is regularly registered as a regular index, and
• wherein when the coordinate value of the vague index can be calculated due to sufficient information for calculating the coordinate value, the vague index is re-registered as a regular index.

A computer-readable medium encoded with a computer program for controlling an information processing apparatus, comprising:

• a program code of a first acquisition step of acquiring a sense image of an image sensing unit;
• a program code of a second acquisition step of acquiring coordinate values of registered indices;
• a program code of a detecting step of detecting image positions of the registered indices in the sense image;
• a program code of a calculating step of calculating position and orientation of the image sensing unit on the basis of the detected image positions of the registered indices and the coordinate values of the registered indices;
• a program code of a detection step of detecting an image position of an additional index in the sense image in accordance with a user's instruction;
• a program code of a calculation step of calculating a coordinate value of the additional index using the image positions of the registered indices, the coordinate values of the registered indices, the position and orientation of the image sensing unit, and the image position of the additional index;
• and
• a program code of a registration step of registering the calculated coordinate value of the additional index as a registered index,
• wherein when the coordinate value of the additional index cannot be calculated due to insufficient information for calculating the coordinate value, the additional index is temporarily registered as a vague index,
• wherein when the coordinate value of the additional index can be calculated due to sufficient information for calculating the coordinate value, the additional index is regularly registered as a regular index, and
• wherein when the coordinate value of the vague index can be calculated due to sufficient information for calculating the coordinate value, the vague index is re-registered as a regular index.

A computer-readable medium encoded with a computer program for controlling an information processing apparatus, comprising:

• a program code of a first acquisition step of acquiring a sense image of an image sensing unit;
• a program code of a second acquisition step of acquiring coordinate values of registered indices;
• a program code of a detecting step of detecting image positions of the registered indices in the sense image;
• a program code of a calculating step of calculating position and orientation of the image sensing unit on the basis of the detected image positions of the registered indices and the coordinate values of the registered indices;
• a program code of a detection step of detecting an image position of an additional index in the sense image in accordance with a user's instruction;
• a program code of a first calculation step of calculating a coordinate value of the additional index using the image positions of the registered indices, the coordinate values of the registered indices, the position and orientation of the image sensing unit, and the image position of the additional index;
• a program code of a second calculation step of calculating a feature amount of the additional index from the sense image;
• and
• a program code of a registration step of registering the coordinate value and the feature amount of the additional index,
• wherein when the coordinate value of the additional index cannot be calculated due to insufficient information for calculating the coordinate value, the additional index is temporarily registered as a vague index,
• wherein when the coordinate value of the additional index can be calculated due to sufficient information for calculating the coordinate value, the additional index is regularly registered as a regular index, and
• wherein when the coordinate value of the vague index can be calculated due to sufficient information for calculating the coordinate value, the vague index is re-registered as a regular index.