Back to results
US 9,432,563 B2Granted
Three-dimensional imaging through multi-image processing
AssigneeBroadcom Corporation
InventorIlya Blayvas
Life of this patent
Estimating the term…
Abstract
Embodiments of imaging devices of the present disclosure automatically utilize sequential image captures in an image processing pipeline. In one embodiment, control processing circuitry captures a plurality of sub-frames, each of the sub-frames comprising a representation of a scene captured at a different focus position of a lens. A depth map of the scene may be generated comprising a depth of the portion of the scene based at least upon the focus position at which the one of the sub-frames was captured. An output frame of the scene may also be generated by utilizing pixel values of the sub-frame which are determined to be at optimal focus positions.
Claims20
An imaging device, comprising:
- an optical lens adapted to form a focused image at a plurality of focus distances;
- an image sensor adapted to transform an optical image formed by the optical lens into a digital signal representing the optical image;
- a memory that stores acquired sub-frames, wherein each of the acquired sub-frames comprises a digital signal representation of an optical image of an object from the image sensor, the optical image being captured at a focus distance from the optical lens corresponding to a focus position of the optical lens, the focus position being different for each of the acquired sub-frames;
- and
- processing circuitry configured to:
- process the acquired sub-frames and form a single output frame from the acquired sub-frames acquired at different optical lens focus positions by identifying, for each pixel in the single output frame, a value of a pixel in one of the acquired sub-frames that has an optimal focus value as compared to corresponding pixel values in remaining ones of the acquired sub-frames and using the identified value of the pixel in the single output frame, and
- calculate a depth map for the single output frame based at least in part on the focus positions of the acquired sub-frames, the depth map representing, for each pixel selected for the single output frame, a corresponding focus distance of the pixel, the corresponding focus distance of at least one pixel being interpolated based at least in part on another focus value of another pixel in another one of the acquired sub-frames that has a second most optimal focus value as compared to the corresponding pixel values in the remaining ones of the acquired sub-frames.
4 more elements
A method, comprising:
- capturing, in an imaging device, a plurality of sub-frames at an input frame rate, each of the plurality of sub-frames comprising a representation of an object captured at a focus position of a lens, the focus position being different for each of the plurality of sub-frames;
- identifying, in the imaging device, one of the plurality of sub-frames having a portion of the object captured in a region, the region having a focus accuracy at least as great as a corresponding region of remaining ones of the plurality of sub-frames used to capture the portion;
- calculating, in the imaging device, a depth map stream for the object, at least one of the depth maps of the depth map stream being based at least upon the focus position at which the one of the plurality of sub-frames was captured, the at least one of the depth maps comprising a focus distance of the portion of the object;
- and
- generating a stream of output frames comprising the object at an output frame rate utilizing regions of the plurality of sub-frames which are determined to be at optimal focus positions, the at least one of the output frames based at least upon the region of the one of the plurality of sub-frames and the at least one of the depth maps, wherein the output frames comprise stereoscopic images having pixels that are offset based at least on the depth maps of the depth map stream.
2 more elements
A computer program product comprising instructions stored in a non-transitory computer-readable storage medium, the instructions comprising:
- instructions to capture of a plurality of sub-frames at an input frame rate by an optical lens, each of the plurality of sub-frames comprising a representation of an object captured at a focus position of the optical lens, the focus position being different for each of the plurality of sub-frames;
- instructions to identify one of the plurality of sub-frames having a portion of the object captured in a region, the region having a focus accuracy at least as great as a corresponding region of remaining ones of the plurality of sub-frames used to capture the portion;
- instructions to compile a depth map of the object based at least upon the focus position at which the one of the plurality of sub-frames was captured, the depth map comprising a focus distance of the portion of the object;
- and
- instructions to combine the depth map with a single output frame to generate a stereoscopic image, wherein pixels of the stereoscopic image are offset based at least on the depth map that comprises the focus distance of the portion of the object.
2 more elements
Dependencies are inferred from the claim text.