Generating an improved depth map using a multi-aperture imaging system

A non-transitory computer-readable storage medium storing executable computer program instructions for processing depth information, the instructions executable by a processor and causing the processor to perform a method comprising:

• capturing first raw image data associated with a first image of a scene, the first raw image data captured using a first imaging system characterized by a first point spread function;
• capturing second raw image data associated with a second image of the scene, the second raw image data captured using a second imaging system characterized by a second point spread function that varies as a function of depth differently than the first point spread function;
• generating high-frequency image data using the first raw image data and the second raw image data;
• identifying edges using normalized derivative values of the high-frequency image data;
• determining edge depth information for the identified edges using a bank of blur kernels;
• determining fill depth information for image components other than the identified edges;
• and
• generating a depth map of the scene using the edge depth information and the fill depth information.

A method for generating depth information, the method comprising:

• capturing a series of raw image data of a scene, the series of raw image data being representative of a series of image frames, the series of raw image data including:
• a first raw image data associated with a first image of the scene, the first raw image data captured using a first imaging system characterized by a first point spread function, and
• a second raw image data associated with a second image of the scene, the second raw image data captured using a second imaging system characterized by a second point spread function that varies as a function of depth differently than the first point spread function;
• determining that a region in at least two of the series of image frames is moving with respect to other regions in the at least two image frames,
• generating high-frequency image data using the first raw image data and the second raw image data using a high pass filter on the identified region;
• identifying edges using normalized derivative values of the high-frequency image data;
• and
• determining edge depth information the identified edges using a bank of blur kernels.

A method for generating depth information, the method comprising:

• capturing first raw image data associated with a first image of a scene, the first raw image data captured using a first imaging system characterized by a first point spread function;
• capturing second raw image data associated with a second image of the scene, the second raw image data captured using a second imaging system characterized by a second point spread function that varies as a function of depth differently than the first point spread function, the first imaging system and the second imaging system using the same detector and the first raw image data and the second raw image data captured as part of a first frame;
• generating high-frequency image data using the first raw image data and the second raw image data;
• identifying edges using normalized derivative values of the high-frequency image data;
• determining edge depth information for the identified edges using a bank of blur kernels;
• illuminating, via an infrared illumination source, the scene with a first infrared pulse of structured light;
• capturing third raw image data corresponding a second frame of the illuminated scene;
• determining a first partial fill depth information using a structured depth model and the third raw image data of the illuminated scene;
• illuminating, via the infrared illumination source, the scene with a second infrared pulse of light;
• capturing fourth raw image data of the scene illuminated using the first pulse of infrared light as part of a third image frame, wherein the second pulse of infrared light is offset from an electronic shutter associated with the image sensor;
• determining a second partial fill depth information for the scene using the first raw image data and the second raw image data;
• determining fill depth information using the first partial depth information and the second partial depth information;
• and
• generating a depth map of the scene using the edge depth information and the fill depth information.