Edge angle and baseline angle correction in depth imaging

A depth imaging system, comprising:

• an image sensor comprising a pixel array having a first pixel axis and a second pixel axis orthogonal to each other;
• an angle-sensitive optical encoder disposed over the image sensor;
• and
• a computer device operatively coupled to the image sensor and comprising a processor and a non-transitory computer readable storage medium having stored thereon computer readable instructions that, when executed by the processor, cause the processor to perform operations,
• wherein the image sensor is configured to capture image data from a scene by detecting, with the pixel array, light incident from the scene having passed through the angle-sensitive optical encoder, wherein the image data comprises a first set of pixel responses corresponding to a first set of pixels of the pixel array and a second set of pixel responses corresponding to a second set of pixels of the pixel array, wherein the first set of pixel responses form a first image of the scene and the second set of pixel responses form a second image of the scene, and wherein the first image and the second image represent two different viewpoints of the scene separated from each other by an effective baseline defined by the angle-sensitive optical encoder and oriented at a baseline angle that is obliquely offset with respect to a nominal baseline direction parallel to the first pixel axis,
• wherein the angle-sensitive optical encoder is configured to modulate the incident light prior to detection by the pixel array in accordance with an angle of incidence of the incident light such that the first set of pixel responses and the second set of pixel responses vary differently from each other as a function of angle of incidence, and
• wherein the operations performed by the processor comprise:
• receiving the image data from the scene captured by the image sensor;
• identifying an edge present in the first image and the second image;
• determining an edge angle associated with the edge;
• determining a parallel disparity representing a distance in image space between the edge as viewed in the first image and the edge as viewed in the second image, wherein the parallel disparity is measured along a disparity axis parallel to the first pixel axis;
• and
• determining depth information about the edge based on the determined parallel disparity, the determined edge angle, and calibration data relating vectorial disparity information along and transverse to the nominal baseline direction to object distance information and edge angle information.

A depth imaging method, comprising:

• receiving image data from a scene captured with a depth imaging system comprising (i) an image sensor configured to detect light incident from the scene and (ii) an angle-sensitive optical encoder interposed between the image sensor and the scene, the image sensor comprising a pixel array having a first pixel axis and a second pixel axis orthogonal to each other, and the angle-sensitive optical encoder being configured to modulate the incident light prior to detection by the pixel array in accordance with an angle of incidence of the incident light, wherein the image data comprises a first set of pixel responses and a second set of pixel responses corresponding to a first set of pixels and a second set of pixels of the pixel array, respectively, wherein the first set of pixel responses and the second set of pixel responses vary differently from each other as a function of angle of incidence, wherein the first set of pixel responses and the second set of pixel responses form a first image and a second image of the scene, respectively, and wherein the first image and the second image represent two different viewpoints of the scene separated from each other by an effective baseline defined by the angle-sensitive optical encoder and oriented at a baseline angle that is obliquely offset with respect to a nominal baseline direction parallel to the first pixel axis;
• performing an image transformation operation on the image data, wherein the image transformation operation comprises applying an image rotation operation to each of the first image and the second image in a direction toward the first pixel axis by a rotation angle related to the baseline angle, thereby obtaining a baseline-angle-corrected first image and a baseline-angle-corrected second image;
• determining a parallel disparity representing a distance in image space between a scene feature as viewed in the baseline-angle-corrected first image and the scene feature as viewed in the baseline-angle-corrected second image, wherein the parallel disparity is measured along a disparity axis parallel to the first pixel axis;
• and
• determining depth information about the edge based on the parallel disparity and calibration data relating disparity information along the nominal baseline direction to object distance information.

A depth imaging system, comprising:

• an image sensor comprising a pixel array having a first pixel axis and a second pixel axis orthogonal to each other;
• an angle-sensitive optical encoder disposed over the image sensor;
• and
• a computer device operatively coupled to the image sensor and comprising a processor and a non-transitory computer readable storage medium having stored thereon computer readable instructions that, when executed by the processor, cause the processor to perform operations,
• wherein the image sensor is configured to capture image data from a scene by detecting, with the pixel array, light incident from the scene having passed through the angle-sensitive optical encoder, wherein the image data comprises a first set of pixel responses corresponding to a first set of pixels of the pixel array and a second set of pixel responses corresponding to a second set of pixels of the pixel array, wherein the first set of pixel responses form a first image of the scene and the second set of pixel responses form a second image of the scene, and wherein the first image and the second image represent two different viewpoints of the scene separated from each other by an effective baseline defined by the angle-sensitive optical encoder and oriented at a baseline angle that is obliquely offset with respect to a nominal baseline direction parallel to the first pixel axis,
• wherein the angle-sensitive optical encoder is configured to modulate the incident light prior to detection by the pixel array in accordance with an angle of incidence of the incident light such that the first set of pixel responses and the second set of pixel responses vary differently from each other as a function of angle of incidence, and
• wherein the operations performed by the processor comprise:
• receiving the image data from the scene captured by the image sensor;
• performing an image transformation operation on the image data, wherein the image transformation operation comprises applying an image rotation operation to each of the first image and the second image in a direction toward the first pixel axis by a rotation angle equal to the baseline angle, thereby obtaining a baseline-angle-corrected first image and a baseline-angle-corrected second image;
• determining a parallel disparity representing a distance in image space between a scene feature as viewed in the baseline-angle-corrected first image and the scene feature as viewed in the baseline-angle-corrected second image, wherein the parallel disparity is measured along a disparity axis parallel to the first pixel axis;
• and
• determining depth information about the scene feature based on the parallel disparity and calibration data relating disparity information along the nominal baseline direction to object distance information.