Image processing apparatus, image processing method and program

An image processing method performed by an image processing device which includes a feature data storage unit which stores feature data indicating a feature of appearance of an object, a superimposition display data storage unit which stores superimposition display data which is to be a source of an image that is superimposed on a visual field of a user and an item position which is a position of an item forming the superimposition display data, an environment map generating unit, a superimposition display position determining unit, an image superimposing unit, an operating object recognizing unit, and a process executing unit, the image processing method comprising the steps of:

• generating, by the environment map generating unit, an environment map representing a position of one or more objects existing in a real space based on an input image obtained by imaging the real space using an imaging device and the feature data stored in the feature data storage unit;
• determining, by the superimposition display position determining unit, a position of an object having a predetermined flat surface or curved surface out of an object imaged in the input image based on the environment map;
• generating, by the superimposition display image generating unit, a superimposition display image by setting the superimposition display data at the position of the object determined by the superimposition display position determining unit;
• superimposing, by the image superimposing unit, the superimposition display image on the visual field of the user;
• recognizing, by the operating object recognizing unit, an operating object imaged in the input image;
• and
• executing, by the process executing unit, a process corresponding to the item selected based on a position of the operating object recognized by the operating object recognizing unit.

A program for causing a computer to function as an image processing device which includes

• a feature data storage unit which stores feature data indicating a feature of appearance of an object,
• a superimposition display data storage unit which stores superimposition display data which is to be a source of an image that is superimposed on a visual field of a user and an item position which is a position of an item forming the superimposition display data,
• an environment map generating unit which generates an environment map representing a position of one or more objects existing in a real space based on an input image obtained by imaging the real space using an imaging device and the feature data stored in the feature data storage unit,
• a superimposition display position determining unit which determines a position of an object having a predetermined flat surface or curved surface out of an object imaged in the input image based on the environment map,
• a superimposition display image generating unit which generates a superimposition display image by setting the superimposition display data at the position of the object determined by the superimposition display position determining unit,
• an image superimposing unit which superimposes the superimposition display image on the visual field of the user,
• an operating object recognizing unit which recognizes an operating object imaged in the input image, and
• a process executing unit which executes a process corresponding to the item selected based on a position of the operating object recognized by the operating object recognizing unit.

An image processing device comprising:

• a feature data storage unit which stores feature data indicating a feature of appearance of an object;
• a superimposition display data storage unit which stores superimposition display data which is to be a source of an image that is superimposed on a visual field of a user;
• an environment map generating unit which generates an environment map representing a position of one or more objects existing in a real space based on an input image obtained by imaging the real space using an imaging device and the feature data stored in the feature data storage unit;
• a superimposition display position determining unit which determines a position of an object having a predetermined flat surface or curved surface out of an object imaged in the input image based on the environment map;
• a superimposition display image generating unit which generates a superimposition display image by setting the superimposition display data at the position of the object determined by the superimposition display position determining unit;
• and
• an image superimposing unit which superimposes the superimposition display image on the visual field of the user and causes a display unit to display the superimposition display image.

An image processing method performed by an image processing device which includes a feature data storage unit which stores feature data indicating a feature of appearance of an object, a superimposition display data storage unit which stores superimposition display data which is to be a source of an image that is superimposed on a visual field of a user, an environment map generating unit, a superimposition display position determining unit, a superimposition display image generating unit, and an image superimposing unit, the image processing method comprising the steps of:

• generating, by the environment map generating unit, an environment map representing a position of one or more objects existing in a real space based on an input image obtained by imaging the real space using an imaging device and the feature data stored in the feature data storage unit;
• determining, by the superimposition display position determining unit, a position of an object having a predetermined flat surface or curved surface out of an object imaged in the input image based on the environment map;
• generating, by the superimposition display image generating unit, a superimposition display image by setting the superimposition display data at the position of the object determined by the superimposition display position determining unit;
• and
• superimposing, by the image superimposing unit, the superimposition display image on the visual field of the user, and causing, by the image superimposing unit, a display unit to display the superimposition display image.

A program for causing a computer to function as an image processing device which includes

• a feature data storage unit which stores feature data indicating a feature of appearance of an object,
• a superimposition display data storage unit which stores superimposition display data which is to be a source of an image that is superimposed on a visual field of a user,
• an environment map generating unit which generates an environment map representing a position of one or more objects existing in a real space based on an input image obtained by imaging the real space using an imaging device and the feature data stored in the feature data storage unit,
• a superimposition display position determining unit which determines a position of an object having a predetermined flat surface or curved surface out of an object imaged in the input image based on the environment map,
• a superimposition display image generating unit which generates a superimposition display image by setting the superimposition display data at the position of the object determined by the superimposition display position determining unit, and
• an image superimposing unit which superimposes the superimposition display image on the visual field of the user and causes a display unit to display the superimposition display image.