Image processing device, image data generation device, image processing method, image data generation method, and data structure of image file

An image data generation device for generating data for an image capturing an object, comprising:

• a measured image acquisition unit configured to acquire a plurality of measured images capturing the object in different directions of a light source;
• a color map creation unit configured to create a color map from one of the measured images captured when the light source is in a predefined direction, the color map containing color information for each pixel;
• a normal map creation unit configured to analyze the measured images according to the photometric stereo method and to create a normal map representing normal vectors of the object in association with each pixel contained within the image;
• a height map creation unit configured to create a height map representing the respective height of the object in association with each of the pixels, by scanning the entirety of the normal map in a predefined direction and by integrating the inclination of the object identified in the predefined direction by referring to the respective normal vectors;
• and
• a storage unit configured to store the color map, the normal map, and the height map as the image data.

An image processing method comprising:

• identifying frame coordinates of an area that is located in an image to be displayed and that is to be newly displayed in response to a request input by a user to move a viewpoint;
• reading at least one of a plurality of items of image data stored in a storage device and respectively used for a plurality of processes for rendering an image to be displayed, and using the read data to render an area in a displayed image corresponding to the frame coordinates;
• and
• outputting data for the rendered area to a display device,
• wherein the rendering comprises determining which of one or more processes to perform among the plurality of processes that respectively use the plurality of items of image data, depending on the height of a viewpoint responsive to the request to move the viewpoint.

An image data generation method for generating data for an image capturing an object, comprising:

• acquiring a plurality of measured images capturing the object in different directions of a light source and storing the images in a memory;
• creating a color map from one of the measured images read from the memory and captured when the light source is in a predefined direction, the color map containing color information for each pixel;
• analyzing the measured image read from the memory according to the photometric stereo method and creating a normal map representing normal vectors of the object in association with each pixel contained within the image;
• creating a height map representing the respective height of the object in association with each of the pixels, by scanning the entirety of the normal map in a predefined direction and by integrating the inclination of the object identified in the predefined direction by referring to the respective normal vectors;
• and
• storing the color map, the normal map, and the height map in a storage device as the image data.

A computer program embedded on a non-transitory computer-readable recording medium, comprising:

• a module configured to identify frame coordinates of an area that is located in an image to be displayed and that it to be newly displayed in response to a request input by a user to move a viewpoint;
• a module configured to read at least one of a plurality of items of image data stored in a storage device and respectively used for a plurality of processes for rendering an image to be displayed, and use the read data to render an area in a displayed image corresponding to the frame coordinates;
• and
• a module configured to output data for the rendered displayed image to a display device,
• wherein the module for rendering determines which of one or more processes to perform among the plurality of processes that respectively use the plurality of items of image data, depending on the height of a viewpoint responsive to the request to move the viewpoint.

A computer program embedded on a non-transitory computer-readable recording medium to implement on a computer a function of generating data for an image capturing an object, comprising:

• a module acquiring a plurality of measured images capturing the object in different directions of a light source and storing the images in a memory;
• a module configured to create a color map from one of the measured images read from the memory and captured when the light source is in a predefined direction, the color map containing color information for each pixel;
• a module configured to analyze the measured image read from the memory according to the photometric stereo method and create a normal map representing normal vectors of the object in association with each pixel contained within the image;
• a module configured to create a height map representing the respective height of the object in association with each of the pixels, by scanning the entirety of the normal map in a predefined direction and by integrating the inclination of the object identified in the predefined direction by referring to the respective normal vectors;
• and
• a module configured to store the color map, the normal map, and the height map in a storage device as the image data.

A non-transitory computer-readable recording medium having embodied thereon a computer program comprising:

• a module configured to identify frame coordinates of an area that is located in an image to be displayed and that is to be newly displayed in response to a request input by a user to move a viewpoint;
• a module configured to read at least one of a plurality of items of image data stored in a storage device and respectively used for a plurality of processes for rendering an image to be displayed, and use the read data to render an area in a displayed image corresponding to the frame coordinates;
• and
• a module configured to output data for the rendered displayed image to a display device,
• wherein the module for rendering determines which of one or more processes to perform among the plurality of processes that respectively use the plurality of items of image data, depending on the height of a viewpoint responsive to the request to move the viewpoint.

A non-transitory computer-readable recording medium having embodied thereon a computer program for implementing on a computer a function of generating data for an image capturing an object, the computer program comprising:

• a module acquiring a plurality of measured images capturing the object in different directions of a light source and storing the images in a memory;
• a module configured to create a color map from one of the measured images read from the memory and captured when the light source is in a predefined direction, the color map containing color information for each pixel;
• a module configured to analyze the measured image read from the memory according to the photometric stereo method and create a normal map representing normal vectors of the object in association with each pixel contained within the image;
• a module configured to create a height map representing the respective height of the object in association with each of the pixels, by scanning the entirety of the normal map in a predefined direction and by integrating the inclination of the object identified in the predefined direction by referring to the respective normal vectors;
• and
• a module configured to store the color map, the normal map, and the height map in a storage device as the image data.

A data structure of an image file read from a storage device to display at least a part of an image on a display, the data structure mapping the following to each other:

• a color map used to represent a color in a displayed image and maintaining color information for each pixel;
• a height map used to represent parallax relative to an object in an image and maintaining, for each pixel, information on the height of the object, respectively, relative to an image plane;
• a normal map used to represent surface roughness of an object in an image and maintaining, for each pixel, information on a normal vector on the surface of the object;
• and
• a horizon map used to cast a shadow corresponding to a light source of a predefined direction and maintaining, for each pixel, information on a direction of a light source when a shadow of an object is cast on the pixel.