Back to results
US 2022/0067997 A1Published application
Systems, methods, and media for high dynamic range quanta burst imaging
AssigneeWisconsin Alumni Research Foundation
InventorsMohit Gupta, Sizhuo Ma
Life of this patent
Estimating the term…
Abstract
In accordance with some embodiments, systems, methods and media for high dynamic range quanta burst imaging are provided. In some embodiments, the system comprises: an image sensor comprising single photon detectors in an array; a processor programmed to: generate a sequence of binary images representing a scene; divide the sequence of binary images into blocks; generate block-sum images from the blocks; determine alignments between the block-sum images and a reference block-sum image; warp the sequence of binary images based on the alignments; generate warped block-sum images using warped binary images; merge the warped block-sum images; display a final image of the scene based on the merged warped block-sum images.
Claims20
A system for generating digital images, comprising:
- an image sensor comprising a plurality of detectors configured to detect arrival of individual photons, the plurality of detectors arranged in an array;
- a display;
- at least one processor that is programmed to:
- divide a sequence of images generated by the image sensor into a plurality of blocks, including a reference block, a first auxiliary block, and a second auxiliary block, such that a set of images is associated with each of the plurality of blocks, wherein each of the images comprises a plurality of pixels;
- generate a reference block-sum image based on a sum of the set of images associated with the reference block, such that each pixel of the reference block-sum image is associated with a value corresponding to a number of photon detections at that pixel in the set of images;
- generate a first auxiliary block-sum image based on a sum of the set of images associated with the first auxiliary block, such that each pixel of the first auxiliary block-sum image is associated with a value corresponding to a number of photon detections at that pixel in the set of images;
- generate a second auxiliary block-sum image based on a sum of the set of images associated with the second auxiliary block, such that each pixel of the second auxiliary block-sum image is associated with a value corresponding to a number of photon detections at that pixel in the set of images;
- determine an alignment between the first auxiliary block-sum image and the reference block-sum image;
- determine an alignment between the second auxiliary block-sum image and the reference block-sum image;
- generate a first set of warped images by warping at least a first subset of images in the set of images associated with the first auxiliary block based on the alignment between the first auxiliary image and the reference block-sum image;
- generate a first warped block-sum image based on the first set of warped images;
- generate a second set of warped images by warping at least a second subset of images in the set of images associated with the second auxiliary block based on the alignment between the second auxiliary image and the reference block-sum image;
- generate a second warped block-sum image based on the second set of warped images;
- generate a third set of warped images by warping at least a third subset of images in the set of images associated with the reference block based on an alignment between the reference block-sum image and an adjacent block-sum image;
- generate a third warped block-sum image based on the third set of warped images;
- merge a plurality of warped block-sum images, including at least the first warped block-sum image, the second warped block-sum image, and the third warped block-sum image;
- and
- cause the display to present a final image of the scene based on the merged plurality of warped block-sum images.
15 more elements
A method for generating digital images, comprising:
- dividing a sequence of images generated by an image sensor into a plurality of blocks, including a reference block, a first auxiliary block, and a second auxiliary block, such that a set of images is associated with each of the plurality of blocks, wherein the image sensor comprises a plurality of detectors configured to detect arrival of individual photons, the plurality of detectors arranged in an array, and wherein each of the images comprises a plurality of pixels;
- generating a reference block-sum image based on a sum of the set of images associated with the reference block, such that each pixel of the reference block-sum image is associated with a value corresponding to a number of photon detections at that pixel in the set of images;
- generating a first auxiliary block-sum image based on a sum of the set of images associated with the first auxiliary block, such that each pixel of the first auxiliary block-sum image is associated with a value corresponding to a number of photon detections at that pixel in the set of images;
- generating a second auxiliary block-sum image based on a sum of the set of images associated with the second auxiliary block, such that each pixel of the second auxiliary block-sum image is associated with a value corresponding to a number of photon detections at that pixel in the set of images;
- determining an alignment between the first auxiliary block-sum image and the reference block-sum image;
- determining an alignment between the second auxiliary block-sum image and the reference block-sum image;
- generating a first set of warped images by warping at least a first subset of images in the set of images associated with the first auxiliary block based on the alignment between the first auxiliary image and the reference block-sum image;
- generating a first warped block-sum image based on the first set of warped images;
- generating a second set of warped images by warping at least a second subset of images in the set of images associated with the second auxiliary block based on the alignment between the second auxiliary image and the reference block-sum image;
- generating a second warped block-sum image based on the second set of warped images;
- generating a third set of warped images by warping at least a third subset of images in the set of images associated with the reference block based on an alignment between the reference block-sum image and an adjacent block-sum image;
- generating a third warped block-sum image based on the third set of warped images;
- merging a plurality of warped block-sum images, including at least the first warped block-sum image, the second warped block-sum image, and the third warped block-sum image;
- and
- causing a display to present a final image of the scene based on the merged plurality of warped block-sum images.
12 more elements
A non-transitory computer readable medium containing computer executable instructions that, when executed by a processor, cause the processor to perform a method for generating digital images, the method comprising:
- dividing a sequence of images generated by an image sensor into a plurality of blocks, including a reference block, a first auxiliary block, and a second auxiliary block, such that a set of images is associated with each of the plurality of blocks, wherein the image sensor comprises a plurality of detectors configured to detect arrival of individual photons, the plurality of detectors arranged in an array, and wherein each of the images comprises a plurality of pixels;
- generating a reference block-sum image based on a sum of the set of images associated with the reference block, such that each pixel of the reference block-sum image is associated with a value corresponding to a number of photon detections at that pixel in the set of images;
- generating a first auxiliary block-sum image based on a sum of the set of images associated with the first auxiliary block, such that each pixel of the first auxiliary block-sum image is associated with a value corresponding to a number of photon detections at that pixel in the set of images;
- generating a second auxiliary block-sum image based on a sum of the set of images associated with the second auxiliary block, such that each pixel of the second auxiliary block-sum image is associated with a value corresponding to a number of photon detections at that pixel in the set of images;
- determining an alignment between the first auxiliary block-sum image and the reference block-sum image;
- determining an alignment between the second auxiliary block-sum image and the reference block-sum image;
- generating a first set of warped images by warping at least a first subset of images in the set of images associated with the first auxiliary block based on the alignment between the first auxiliary image and the reference block-sum image;
- generating a first warped block-sum image based on the first set of warped images;
- generating a second set of warped images by warping at least a second subset of images in the set of images associated with the second auxiliary block based on the alignment between the second auxiliary image and the reference block-sum image;
- generating a second warped block-sum image based on the second set of warped images;
- generating a third set of warped images by warping at least a third subset of images in the set of images associated with the reference block based on an alignment between the reference block-sum image and an adjacent block-sum image;
- generating a third warped block-sum image based on the third set of warped images;
- merging a plurality of warped block-sum images, including at least the first warped block-sum image, the second warped block-sum image, and the third warped block-sum image;
- and
- causing a display to present a final image of the scene based on the merged plurality of warped block-sum images.
12 more elements
Dependencies are inferred from the claim text.