Back to results
US 9,870,644 B2Granted
Apparatus and method for image processing
AssigneeSamsung Electronics Co., Ltd.
InventorsIn Woo Ha, Tae Hyun Rhee
Life of this patent
Estimating the term…
Abstract
Image processing apparatus and method. A modeling unit may generate a three-dimensional (3D) model corresponding to an actual object from an input color image and an input depth image corresponding to the input color image. A calculator may perform photon-based rendering with respect to an input virtual object and the 3D model based on input light environment information, and may generate a difference image comprising color distortion information occurring by inserting the virtual object into the 3D model. A rendering unit may generate a result image comprising the virtual object by synthesizing the difference image and the input color image.
Claims17
An image processing apparatus, comprising:
- a modeling processor to generate a three-dimensional (3D) model corresponding to an actual object using an input color image and an input depth image corresponding to the input color image;
- an illumination map calculating processor to generate an illumination map of the 3D model corresponding to the actual object by performing rendering based on the input light environment information with assuming the input depth image as a constant material;
- and
- a material map calculating processor to generate a material map of the 3D model by performing a pixel-by-pixel divide operation with respect to the illumination map in the input color image and distinguish a color of the actual object from a background color;
- a calculating processor to perform photon-based rendering with respect to an input virtual object and the 3D model based on the input light environment information, to generate an irradiance map comprising illumination difference occurring by inserting the virtual object into the 3D model, and to generate a difference image comprising color distortion information in RGB color values of respective pixels occurring by inserting the virtual object into the 3D model using the irradiance map;
- and
- a rendering processor to generate a result image comprising the virtual object by synthesizing the difference image and the input color image.
4 more elements
An image processing apparatus, comprising:
- a modeling processor to generate a three-dimensional (3D) model using an input color image and an input depth image corresponding to the input color image;
- an illumination map calculating processor to generate an illumination map of the 3D model corresponding to an actual object by performing rendering based on the input light environment information with assuming the input depth image as a constant material;
- and
- a material map calculating processor to generate a material map of the 3D model by performing a pixel-by-pixel divide operation with respect to the illumination map in the input color image and distinguish a color of the actual object from a background color;
- a calculating processor to perform photon-based rendering with respect to an input virtual object and the 3D model based on the input light environment information, to generate an irradiance map comprising illumination difference occurring by inserting the virtual object into the 3D model, and to generate a difference image comprising color distortion information in RGB color values of respective pixels between a first image and a second image, the first image acquired by rendering the 3D model at a camera view based on the input light environment information and the second image using the irradiance map acquired by rendering the 3D model and an input virtual object at the camera view based on the input light environment information;
- and
- a rendering processor to generate a result image comprising the virtual object by synthesizing the difference image and the input color image.
4 more elements
An image processing apparatus, comprising:
- a modeling processor to generate a three-dimensional (3D) model corresponding to an actual object using an input color image and a corresponding input depth image;
- an illumination map calculating processor to generate an illumination map of an object associated with an input depth image by performing rendering based on input light environment information with assuming the input depth image as a constant material, perform photon-based rendering with respect to an input virtual object and the 3D model based on the input light environment information, and to generate an irradiance map comprising illumination difference occurring by inserting the virtual object into the 3D model;
- a material map calculating processor to generate a material map of the object by performing a pixel-by-pixel divide operation with respect to the illumination map in the input color image and distinguish a color of the actual object from a background color;
- and
- a difference image calculating processor configured to generate a difference image comprising color distortion information in RGB color values of respective pixels occurring by inserting the virtual object into the 3D model;
- and
- a rendering processor to generate a result image comprising the virtual object by synthesizing the difference image and the input color image.
4 more elements
An image processing method, comprising:
- generating a three-dimensional (3D) model corresponding to an actual object using an input color image and an input depth image corresponding to the input color image;
- generating an illumination map of the 3D model corresponding to the actual object by performing rendering based on the input light environment information with assuming the input depth image as a constant material;
- generating a material map of the 3D model by performing a pixel-by-pixel divide operation with respect to the illumination map in the input color image and distinguishing a color of the actual object from a background color;
- performing photon-based rendering with respect to an input virtual object and the 3D model based on the input light environment information, to generate an irradiance map comprising illumination difference occurring by inserting the virtual object into the 3D model, and to generate a difference image comprising color distortion information in RGB color values of respective pixels occurring by inserting the virtual object into the 3D model using the irradiance map;
- generating a result image comprising the virtual object by synthesizing the difference image and the input color image.
2 more elements
An image processing method, comprising:
- generating a three-dimensional (3D) model using an input color image and an input depth image corresponding to the input color image;
- generating an illumination map of the 3D model corresponding to an actual object by performing rendering based on the input light environment information with assuming the input depth image as a constant material, and
- generating a material map of the 3D model by performing a pixel-by-pixel divide operating with respect to the illumination map in the input color image and distinguishing a color of the actual object from a background color;
- performing photon-based rendering with respect to an input virtual object and the 3D model based on the input light environment information;
- generating an irradiance map comprising illumination difference occurring by inserting the virtual object into the 3D model;
- generating a difference image comprising color distortion information in RGB color values of respective pixels between a first image and a second image using the irradiance map, the first image acquired by rendering the 3D model at a camera view based on input light environment information and the second image acquired by rendering the 3D model and an input virtual object at the camera view based on the input light environment information, and to generate an irradiance map of the 3D model based on a difference between a photon and an anti-photon;
- and
- generating a result image comprising the virtual object by synthesizing the difference image and the input color image.
5 more elements
A non-transitory computer-readable medium comprising a program for instructing a computer to perform an image processing method including:
- generating a three-dimensional (3D) model corresponding to an actual object using an input color image and an input depth image corresponding to the input color image;
- generating an illumination map of the 3D model corresponding to the actual object by performing rendering based on the input light environment information with assuming the input depth image as a constant material;
- and
- generating a material map of the 3D mode by performing a pixel-by-pixel divide operation with respect to the illumination map in the input color image and distinguishing a color of the actual object from a background color;
- performing photon-based rendering with respect to an input virtual object and the 3D model based on the input light environment information, to generate an irradiance map comprising illumination difference occurring by inserting the virtual object into the 3D model, and to generate a difference image comprising color distortion information in RGB color values of respective pixels occurring by inserting the virtual object into the 3D model using the irradiance map;
- and
- generating a result image comprising the virtual object by synthesizing the difference image and the input color image.
4 more elements
A non-transitory computer-readable medium comprising a program for instructing a computer to perform an image processing method including:
- generating a three-dimensional (3D) model using an input color image and an input depth image corresponding to the input color image;
- generating an illumination map of the 3D model corresponding to an actual object by performing rendering based on the input light environment information with assuming the input depth image as a constant material,
- generating a material map of the 3D model by performing a pixel-by-pixel divide operation with respect to the illumination map in the input color image and distinguishing a color of the actual object from a background color;
- performing photon-based rendering with respect to an input virtual object and the 3D model based on the input light environment information;
- generating an irradiance map comprising illumination difference occurring by inserting the virtual object into the 3D model;
- generating a difference image comprising color distortion information in RGB color values of respective pixels between a first image and a second image using the irradiance map, the first image acquired by rendering the 3D model at a camera view based on the input light environment information and the second image acquired by rendering the 3D model and an input virtual object at the camera view based on the input light environment information;
- and
- generating a result image comprising the virtual object by synthesizing the difference image and the input color image.
5 more elements
Dependencies are inferred from the claim text.