Apparatus and method for capturing digital images

A digital image capturing apparatus, comprising:

• an image pickup unit obtaining light field data to change a view of a captured image and refocus the captured image;
• and
• a data processor generating a refocusing image using the light field data,
• wherein the image pickup unit performs optical modulation on light beams to obtain light field data of the modulated light beams so that the light beams are individually identified when two or more of the light beams overlap, and the data processor demodulates the light field data to obtain demodulated light field data of the light beams not subjected to the optical modulation, further comprising:
• a main lens;
• a modulator performing the optical modulation on the light beams;
• a photo sensor array capturing the light beams;
• and
• a microlens array disposed between the main lens and the photo sensor array,
• wherein the microlens array orients light beams collected from the main lens toward the photo sensor array through the microlens array, and wherein the modulator is positioned to modulate light before entering the microlens array or after orientation of light beams by the microlens array,
• wherein a plurality of the overlapping light beams converge on a same portion of the photo sensor array and include directional resolution information,
• the modulator includes a 2-dimensional orthogonal mask to modulate the light beams using a first function applied to light beams passing through an upper portion of the mask and using a second function applied to light beams passing through a lower portion of the mask, the first and second functions being mathematically orthogonal to one another,
• the photo sensor array obtains an image by summing a first view representing light beams passing through the upper portion of the mask convolved with the first function, with a second view representing light beams passing through the lower portion of the mask convolved with the second function, and
• the directional resolution of the photo sensor array is greater than or equal to twice a spatial sampling rate of the photo sensor array.

A method of capturing digital images, comprising:

• through plural redirections of at least two respective incident light beams, incident to a respective separation light element of plural light separation elements from different directions, each redirection redirecting the at least two respective incident light beams toward a same portion of a photo sensor array so the at least two incident light beams are overlapping at the same portion, wherein the same portion is different for each of the redirection by each of the light separation elements;
• performing optical modulation on light beams so that the overlapped light beams of each of the same portions of the photo sensor array is separately identifiable based on the modulation;
• obtaining light field data of the modulated light beams to refocus an image represented by the modulated light beams;
• demodulating the light field data and generating demodulated light field data of the light beams by separating each of the respective overlapped light beams of each same portion of the photo sensor into separate identified light beams based on the modulation and identifying different incident directions of each of the separated light beams relative to the respective light separation element and at least one input aperture directing the at least two respective incident light beams toward the plural light separation elements;
• and
• generating a refocused image using the generated light field data,
• wherein the plural light separation elements include a main lens and a microlens array disposed between the main lens and the photo sensor array capturing the light beams, wherein the microlens array orients light beams collected from the main lens toward the photo sensor array through the microlens array,
• wherein a plurality of the overlapping light beams converge on the same portion of the photo sensor array and include directional resolution information, and
• the performing optical modulation includes:
• modulating the light beams by using a 2-dimensional orthogonal mask to modulate the light beams using a first function applied to light beams passing through an upper portion of the mask and using a second function applied to light beams passing through a lower portion of the mask, the first and second functions being mathematically orthogonal to one another,
• obtaining an image by summing a first view representing light beams passing through the upper portion of the mask convolved with the first function, with a second view representing light beams passing through the lower portion of the mask convolved with the second function, and
• the directional resolution of the photo sensor array is greater than or equal to twice a spatial sampling rate of the photo sensor array.

A digital image capturing apparatus, comprising:

• an image pickup unit obtaining light field data, including a microlens array disposed between a lens and a photo sensor array, with each microlens of the microlens array redirecting respective light directed from the lens based upon direction of the respective light directed from at least one aperture of the lens, and with each of the microlenses of the microlens array redirecting the respective light toward a same portion of the photo sensor array to be overlapped on the same portion, wherein the same portion is different for each microlens;
• and
• a data processor generating a view depth of an object using the light field data;
• wherein the image pickup unit performs optical modulation on light beams to obtain light field data by the photo sensor array, of the modulated light beams, so that respective overlapped light beams incident to each same portion of the photo sensor array are separable and individually identifiable based on the modulation,
• wherein the image pickup unit is configured so that the respective overlapped light beams are incident to the lens from different directions, and wherein the data processor separates the respective overlapped light beams into separate light beams and performs the individual identification of the respective separated light beams,
• wherein a plurality of the overlapping light beams converge on the same portion of the photo sensor array and include directional resolution information,
• the image pickup unit performs optical modulation using a 2-dimensional orthogonal mask to modulate the light beams using a first function applied to light beams passing through an upper portion of the mask and using a second function applied to light beams passing through a lower portion of the mask, the first and second functions being mathematically orthogonal to one another,
• the photo sensor array obtains an image by summing a first view representing light beams passing through the upper portion of the mask convolved with the first function, with a second view representing light beams passing through the lower portion of the mask convolved with the second function, and
• the directional resolution of the photo sensor array is greater than or equal to twice a spatial sampling rate of the photo sensor array.

A method of capturing digital images, comprising:

• through plural redirections of different light beams directed from at least one aperture of a lens, respectively redirecting respective light beams directed from the at least one aperture of the lens to overlap on a same portion of a photo sensor array, wherein the same portion is different for each of the respective redirections;
• performing optical modulation on light beams so that respective overlapped light beams incident to each same portion of the photo sensor array are separable and individually identifiable based on the modulation;
• and
• demodulating light field data of the modulated light beams and generating demodulated light field data of the light beams, including separating each of the respective overlapped light beams into separate light beams based on the modulation, identifying each of the respective separated light beams as being a respective one of the different light beams directed from the at least one aperture, and identifying respective incident directions of each of the respective light beams directed from the at least one aperture before the redirection of the respective light beams to overlap on the same portion of the photo sensor array,
• wherein the redirecting respective light beams directed from the at least one aperture of the lens to overlap on a same portion of a photo sensor array further comprises orienting light beams collected from the lens toward the photo sensor array through a microlens array disposed between the lens and the photo sensor array capturing the light beams,
• wherein a plurality of the overlapping light beams converge on the same portion of the photo sensor array and include directional resolution information, and
• the performing optical modulation includes:
• modulating the light beams by using a 2-dimensional orthogonal mask to modulate the light beams using a first function applied to light beams passing through an upper portion of the mask and using a second function applied to light beams passing through a lower portion of the mask, the first and second functions being mathematically orthogonal to one another, and
• obtaining an image by summing a first view representing light beams passing through the upper portion of the mask convolved with the first function, with a second view representing light beams passing through the lower portion of the mask convolved with the second function, and
• the directional resolution of the photo sensor array is greater than or equal to twice a spatial sampling rate of the photo sensor array.

A digital image capturing apparatus, comprising:

• a lens including at least one aperture;
• a modulator performing the optical modulation on incident light beams;
• a microlens array disposed between the lens and the photo sensor array, each respective microlens of the microlens array being configured to respectively redirect at least two light beams incident to the respective microlens to a same portion of a photo sensor array as respective light beams overlapping the same portion, wherein the same portion is different for each microlens, and wherein the digital image capturing apparatus is configured so the at least two light beams incident to the respective microlens are directed from the at least one aperture of the lens toward the respective microlens so the respective microlens produces the respective overlapped light beams;
• and
• the photo sensor array, wherein the photo sensor array is configured relative to the microlens array so that each same portion of the photo sensor array captures each of the respective overlapped light beams,
• wherein the modulator is positioned to modulate the incident light beams before being incident to the respective microlens or after production of the respective overlapped light beams by the respective microlens, such that the modulator is configured relative to the microlens array so that each of the captured respective overlapped light beams by each respective same portion of the photo sensor array are separately identifiable by a data processing device and so that a respective incidence direction of each separately identified light beam relative to the lens is identifiable by the data processing apparatus for identifying a focal plane of an object from which the object light beams are directed to the digital image capturing apparatus,
• wherein a plurality of the overlapping light beams converge on the same portion of the photo sensor array and include directional resolution information,
• the modulator includes a 2-dimensional orthogonal mask to modulate the light beams using a first function applied to light beams passing through an upper portion of the mask and using a second function applied to light beams passing through a lower portion of the mask, the first and second functions being mathematically orthogonal to one another,
• the photo sensor array obtains an image by summing a first view representing light beams passing through the upper portion of the mask convolved with the first function, with a second view representing light beams passing through the lower portion of the mask convolved with the second function, and
• the directional resolution of the photo sensor array is greater than or equal to twice a spatial sampling rate of the photo sensor array.