Image processing apparatus, integrated circuit, program, imaging apparatus, and display apparatus

An integrated circuit comprising:

• a separation unit configured to separate a captured image into a plurality of regions including a first region and a second region that is shallower than the first region in terms of depth indicated by depth information of the captured image;
• an extension unit configured to duplicate pixels that constitute the first region and are positioned in a neighborhood of a boundary between the first region and the second region, onto the neighborhood of the boundary outside the first region, thereby generating an extended region;
• a blur processing unit configured to perform blur processing on the extended region and the second region based on the depth of the second region indicated by the depth information;
• and
• a composition unit configured to, after the blur processing, composite a value of each of pixels constituting the extended region and a value of one of pixels constituting the second region that correspond in position with each other.

A non-transitory computer readable recording medium having recorded therein a program that causes a computer to perform blur effect processing, the blur effect processing comprising:

• a separation step of separating a captured image into a plurality of regions including a first region and a second region that is shallower than the first region in terms of depth indicated by depth information of the captured image;
• an extension step of duplicating pixels that constitute the first region and are positioned in a neighborhood of a boundary between the first region and the second region, onto the neighborhood of the boundary outside the first region, thereby generating an extended region;
• a blur processing step of performing blur processing on the extended region and the second region based on the depth of the second region indicated by the depth information;
• and
• a composition step of, after the blur processing, compositing a value of each of pixels constituting the extended region and a value of one of pixels constituting the second region that correspond in position with each other.

An imaging apparatus comprising:

• an imaging unit having a stereo camera;
• a display unit;
• a memory configured to store therein a program for blur effect processing and a left image and a right image captured by the stereo camera;
• a DSP configured to perform generation processing of generating a depth image indicating depth information for each pixel based on the captured left and right images stored in the memory, and perform the blur effect processing on at least one of the captured left and right images;
• and
• a CPU configured to control processing of storing the left image and the right image captured by the stereo camera in the memory, control processing of causing the DSP to perform the generation processing and the blur effect processing, and control processing of causing the display unit to display a result of the blur effect processing, wherein
• the blur effect processing comprises:
• a separation step of separating a target image into a plurality of regions including a first region and a second region that is shallower than the first region in terms of depth indicated by depth information of the target image;
• an extension step of duplicating pixels that constitute the first region and are positioned in a neighborhood of a boundary between the first region and the second region, onto the neighborhood of the boundary outside the first region, thereby generating an extended region;
• a blur processing step of performing blur processing on the extended region and the second region based on the depth of the second region indicated by the depth information;
• and
• a composition step of, after the blur processing, compositing a value of each of pixels constituting the extended region and a value of one of pixels constituting the second region that correspond in position with each other.

An imaging apparatus comprising:

• an imaging unit having a monocular camera;
• a display unit;
• a distance sensor unit configured to measure, for each pixel, a distance to a subject captured by the imaging unit;
• a memory configured to store therein a program for blur effect processing, an image captured by the monocular camera, and the distance measured for each pixel by the distance sensor unit;
• a DSP configured to perform generation processing of generating a depth image indicating depth information for each pixel based on the captured image stored in the memory and the measured distance for each pixel, and perform the blur effect processing on the captured image;
• and
• a CPU configured to control processing of storing the left image and the right image captured by the monocular camera in the memory, control processing of causing the DSP to perform the generation processing and the blur effect processing, and control processing of causing the display unit to display a result of the blur effect processing, wherein
• the blur effect processing comprises:
• a separation step of separating a target image into a plurality of regions including a first region and a second region that is shallower than the first region in terms of depth indicated by depth information of the target image;
• an extension step of duplicating pixels that constitute the first region and are positioned in a neighborhood of a boundary between the first region and the second region, onto the neighborhood of the boundary outside the first region, thereby generating an extended region;
• a blur processing step of performing blur processing on the extended region and the second region based on the depth of the second region indicated by the depth information;
• and
• a composition step of, after the blur processing, compositing a value of each of pixels constituting the extended region and a value of one of pixels constituting the second region that correspond in position with each other.

A display apparatus comprising:

• a display unit;
• a memory configured to store therein a program for blur effect processing, a captured image, and a depth image indicating depth information for each pixel of the captured image;
• a DSP configured to perform the blur effect processing on the captured image;
• and
• a CPU configured to control processing of causing the DSP to perform the blur effect processing, and control processing of causing the display unit to display a result of the blur effect processing, wherein
• the blur effect processing comprises:
• a separation step of separating a target image into a plurality of regions including a first region and a second region that is shallower than the first region in terms of depth indicated by depth information of the target image;
• an extension step of duplicating pixels that constitute the first region and are positioned in a neighborhood of a boundary between the first region and the second region, onto the neighborhood of the boundary outside the first region, thereby generating an extended region;
• a blur processing step of performing blur processing on the extended region and the second region based on the depth of the second region indicated by the depth information;
• and
• a composition step of, after the blur processing, compositing a value of each of pixels constituting the extended region and a value of one of pixels constituting the second region that correspond in position with each other.

An image processing apparatus comprising:

• a blur processing unit configured to, based on a captured image and a depth image indicating depth information of the captured image, perform blur processing on the captured image, thereby generating a blurred image;
• and
• a repetition unit configured to, based on the generated blurred image and the depth information, repeatedly perform the blur effect processing on the generated blurred image to generate a new blurred image until a number of repetitions of the blur effect processing reaches a predetermined number, wherein
• the blur effect processing unit includes:
• a separation unit configured to separate a target image into a plurality of regions including a first region and a second region that is shallower than the first region in terms of depth indicated by depth information of the target image;
• an extension unit configured to duplicate pixels that constitute the first region and are positioned in a neighborhood of a boundary between the first region and the second region, onto the neighborhood of the boundary outside the first region, thereby generating an extended region;
• a blur processing unit configured to perform blur processing on the extended region and the second region based on the depth of the second region indicated by the depth information;
• and
• a composition unit configured to, after the blur processing, composite a value of each of pixels constituting the extended region and a value of one of pixels constituting the second region that correspond in position with each other.

A non-transitory computer readable recording medium having recorded therein a program that causes a computer to perform predetermined image processing, the predetermined image processing comprising:

• a blur effect processing step of, based on a captured image and a depth image indicating depth information of the captured image, performing blur effect processing on the captured image, thereby generating a blurred image;
• and
• a repetition step of, based on the generated blurred image and the depth information, repeatedly performing the blur effect processing on the generated blurred image to generate a new blurred image until a number of repetitions of the blur effect processing reaches a predetermined number, wherein
• the blur effect processing comprises:
• a separation step of separating a target image into a plurality of regions including a first region and a second region that is shallower than the first region in terms of depth indicated by depth information of the target image;
• an extension step of duplicating pixels that constitute the first region and are positioned in a neighborhood of a boundary between the first region and the second region, onto the neighborhood of the boundary outside the first region, thereby generating an extended region;
• a blur processing step of performing blur processing on the extended region and the second region based on the depth of the second region indicated by the depth information;
• and
• a composition step of, after the blur processing, compositing a value of each of pixels constituting the extended region and a value of one of pixels constituting the second region that correspond in position with each other.