Image processing apparatus, image processing method, and storage medium

An image processing method, which determines, for a combined image obtained by combining pixels of a given first image and pixels of an unknown second image either translucently or non-translucently using an unknown coefficient indicating a transparency, whether at least one pixel included in the combined image is a translucently combined pixel, the method comprising:

• calculating, from pixel values of the combined image and pixel values of the first image of respective pixels in a predetermined area including one pixel, pixel values of an image corresponding to the second image when each of a plurality of values is used as the coefficient, and calculating a total of differences between the pixel values of the one pixel and other pixels included in the predetermined area in the image corresponding to the second image;
• estimating one of the plurality of values of the coefficient that has a total of differences, which is the smallest among totals of the differences corresponding to the plurality of values of the coefficient, as the coefficient used for the one pixel to obtain the combined image;
• and
• determining that the one pixel is a pixel obtained by translucently combining of the first image and of the second image when the estimated value of the coefficient is larger than a predetermined value.

A non-transitory computer-readable storage medium storing a computer program for causing a computer, included in an image processing apparatus which determines, for a combined image obtained by combining pixels of a given first image and pixels of an unknown second image either translucently or non-translucently using an unknown coefficient indicating a transparency, whether at least one pixel included in the combined image is a translucently combined pixel, to:

• calculate, from pixel values of the combined image and pixel values of the first image of respective pixels in a predetermined area including one pixel, pixel values of an image corresponding to the second image when each of a plurality of values is used as the coefficient, and calculate a total of differences between the pixel values of the one pixel and other pixels included in the predetermined area in the image corresponding to the second image;
• estimate one of the plurality of values of the coefficient that has a total of differences, which is the smallest among totals of the differences corresponding to the plurality of values of the coefficient, as the coefficient used for the one pixel to obtain the combined image;
• and
• determine that the one pixel is a pixel obtained by translucently combining the first image and the second image when the estimated value of the coefficient is larger than a predetermined value.

An image processing apparatus comprising:

• a receiving unit configured to receive a captured image obtained by capturing an image of an external world from an image capturing unit;
• a storage unit configured to store the captured image received by the receiving unit;
• a reduction unit configured to reduce an information volume of the captured image received by the receiving unit;
• an acquisition unit configured to acquire a combined image obtained by combining the captured image, the information volume of which is reduced by the reduction unit, and a computer graphics (CG) image;
• an identifying unit configured to identify a translucent area, which was translucently combined, from the combined image acquired by the acquisition unit;
• a separation unit configured to separate the captured image, the information volume of which is reduced, and the CG image in the translucent area identified by the identifying unit;
• and
• a recombining unit configured to recombine an area of the captured image separated by the separation unit and an area corresponding to the captured image of an area which is not identified as the translucent area by the identifying unit by replacing the areas by the captured image stored in the storage unit,
• wherein the separation unit is configured to calculate a transparency in the translucent area using a target pixel in the translucent area and surrounding pixels which surround the target pixel, and to separate the captured image, the information volume of which is reduced, and the CG image based on the calculated transparency.

An image processing apparatus comprising:

• a receiving unit configured to receive a captured image obtained by capturing an image of an external world from an image capturing unit;
• a storage unit configured to store the captured image received by the receiving unit;
• an acquisition unit configured to acquire an image combined with a computer graphics (CG) image from an external apparatus;
• an identifying unit configured to identify a translucent area, which was translucently combined, from the combined image acquired by the acquisition unit;
• a separation unit configured to separate the CG image and an image other than the CG image in the translucent area identified by the identifying unit;
• and
• a recombining unit configured to recombine an area of the image other than the CG image separated by the separation unit and an area which does not correspond to the CG image of an area which is not identified as the translucent area by the identifying unit by replacing the areas by the captured image stored in the storage unit,
• wherein the separation unit is configured to calculate a transparency in the translucent area using a target pixel in the translucent area and surrounding pixels which surround the target pixel, and to separate the CG image and the image other than the CG image based on the calculated transparency.

An image processing method for an image processing apparatus, which includes a storage unit configured to store a captured image, the method comprising:

• receiving a captured image obtained by capturing an image of an external world from an image capturing unit;
• saving the captured image received in the receiving to the storage unit;
• reducing an information volume of the captured image received in the receiving;
• acquiring a combined image obtained by combining the captured image, the information volume of which is reduced in the reducing, and a computer graphics (CG) image;
• identifying a translucent area, which was translucently combined, from the combined image acquired in the acquiring;
• separating the captured image, the information volume of which is reduced, and the CG image in the translucent area identified in the identifying;
• and
• recombining an area of the captured image separated in the separating and an area corresponding to the captured image of an area which is not identified as the translucent area in the identifying by replacing these areas by the captured image saved to the storage unit,
• wherein in the separating, a transparency in the translucent area is calculated using a target pixel in the translucent area and surrounding pixels which surround the target pixel, and the captured image, the information volume of which is reduced, and the CG image are separated based on the calculated transparency.

An image processing method for an image processing apparatus, which includes a storage unit configured to store a captured image, the method comprising:

• receiving a captured image obtained by capturing an image of an external world from an image capturing unit;
• saving the captured image received in the receiving to the storage unit;
• acquiring a combined image combined with a computer graphics (CG) image from an external apparatus;
• identifying a translucent area, which was translucently combined, from the combined image acquired in the acquiring;
• separating the CG image and an image other than the CG image in the transparent area identified in the identifying;
• and
• recombining an area of the image other than the CG image separated in the separating and an area which does not correspond to the CG image of an area which is not identified as the translucent area in the identifying by replacing these areas by the captured image saved to the storage unit,
• wherein in the separating, a transparency in the translucent area is calculated using a target pixel in the translucent area and surrounding pixels which surround the target pixel, and the CG image and the image other than the CG image are separated based on the calculated transparency.

A non-transitory computer-readable storage medium storing a computer program for causing a computer, included in an image processing apparatus that includes a storage unit configured to store a captured image, to:

• receive a captured image obtained by capturing an image of an external world from an image capturing unit;
• save the received captured image to the storage unit;
• reduce an information volume of the received captured image;
• acquire a combined image obtained by combining the captured image, the information volume of which is reduced, and a computer graphics (CG) image;
• identify a translucent area, which was translucently combined, from the acquired combined image;
• separate the captured image, the information volume of which is reduced, and the CG image in the identified translucent area;
• and
• recombine an area of the separated captured image and an area corresponding to the captured image of an area which is not identified as the translucent area by replacing these areas by the captured image saved to the storage unit,
• wherein a transparency in the translucent area is calculated using a target pixel in the translucent area and surrounding pixels which surround the target pixel, and the captured image, the information volume of which is reduced, and the CG image are separated based on the calculated transparency.

A non-transitory computer-readable storage medium storing a computer program for causing a computer, included in an image processing apparatus that includes a storage unit configured to store a captured image, to:

• receive a captured image obtained by capturing an image of an external world from an image capturing unit;
• save the received captured image to the storage unit;
• acquire a combined image combined with a computer graphics (CG) image from an external apparatus;
• identify a translucent area, which was translucently combined, from the combined image acquired in the acquiring;
• separate the CG image and an image other than the CG image in the identified transparent area;
• and
• recombine an area of the image other than the separated CG image and an area which does not correspond to the CG image of an area which is not identified as the translucent area by replacing these areas by the captured image saved to the storage unit,
• wherein a transparency in the translucent area is calculated using a target pixel in the translucent area and surrounding pixels which surround the target pixel, and the CG image and the image other than the CG image are separated based on the calculated transparency.

An information processing apparatus, which determines whether or not at least one pixel included in a combined image obtained by pixels of a given first image and pixels of an unknown second image using an unknown coefficient indicating a transparency is a pixel that is combined using both the first image and the second image, the apparatus comprising:

• a calculation unit configured to calculate, from pixel values of the combined image and pixel values of the first image of respective pixels in a predetermined area including one pixel, pixel values of an image corresponding to the second image when each of a plurality of values is used as the coefficient, and to calculate a total of differences between the pixel values of the one pixel and other pixels included in the predetermined area in the image corresponding to the second image;
• an estimation unit configured to estimate one of the plurality of values of the coefficient that has a total of differences, which is the smallest among totals of the differences corresponding to the plurality of values of the coefficient, as the coefficient used for the one pixel to obtain the combined image;
• and
• a determination unit configured to determine that the one pixel is a pixel which is combined using both the pixel values of the first image and the pixel values of the second image when the estimated value of the coefficient is larger than a predetermined value.