# US 9,883,169 B2 · Optical system, apparatus and method for operating an apparatus using helmholtz reciprocity > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 9,883,169 B2 - Document ID: US9883169B2 - Status: Granted - Kind: B2 - Application number: 14/666,817 - Filed: 2015-03-24 - Published: 2015-10-01 - Granted: 2018-01-30 - Earliest claimed: 2014-03-31 - Assignees: Sony Corporation - Applicants: Sony Corporation - Inventors: Roderick Koehle; Markus Kamm - Web page: https://patentgrep.com/patent/US9883169B2 ## CPC - H04N 13/254: in combination with electromagnetic radiation sources for illuminating objects - G06T 7/593: from stereo images - H04N 13/207: using a single two-dimensional [2D] image sensor - H04N 2013/0081: {Depth or disparity estimation from stereoscopic image signals} - H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION - G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL ## Abstract An optical system includes an illumination unit and an imaging unit configured to image a scene including at least one object into a first image and a second image by using Helmholtz reciprocity. The illumination unit is configured to emit light into at least one light emitting solid angle and the imaging unit is configured to receive light from a light receiving solid angle. The light receiving solid angle is at least as large as each of the light emitting solid angles. ## Claims ### Claim 1 An apparatus comprising an optical system according with an illumination unit and an imaging unit configured to image a scene including at least one object into at least a first image and a second image by using Helmholtz reciprocity; and a processor configured to obtain disparity information concerning the first image and the second image on the basis of a comparison of intensity sum values along corresponding lines in the first image and the second image and configured to retrieve depth information concerning the object from the disparity information, wherein the processor is configured to obtain the disparity information by: integrating in the first image an intensity of the captured light along the corresponding lines, to obtain intensity sum profiles of the first image; integrating in the second image the intensity of the captured light along the corresponding lines, to obtain intensity sum profiles of the second image; selecting for each pair of corresponding lines of the first image and the second image the corresponding intensity sum profile of the first image and the corresponding intensity sum profile of the second image; determining from the selected intensity sum profile of the first image for each intensity sum value a first position on the integration path within the first image at which the intensity sum value is reached; determining from the selected intensity sum profile of the second image for each intensity sum value a second position on the integration path within the second image at which the intensity sum value is reached; and determining for each intensity sum value the disparity from the first position and the second position. ### Claim 2 (depends on 1) The apparatus according to claim 1 , wherein the illumination unit comprises at least one aperture whose opening defines a light emitting solid angle. ### Claim 3 (depends on 1) The apparatus according to claim 1 wherein light emitting solid angles of the illumination unit are obtained by emitting light through a variable blocking structure. ### Claim 4 (depends on 1) The apparatus according to claim 1, wherein an optical device comprises the illumination unit and the imaging unit. ### Claim 5 (depends on 4) The apparatus according to claim 4, wherein an optical element within the optical device is configured to align an optical axis of the illumination unit and an optical axis of the imaging unit. ### Claim 6 (depends on 1) The apparatus according to claim 1, further comprising a plurality of illumination units arranged symmetrically around the imaging unit. ### Claim 7 (depends on 3) The apparatus according to claim 3, wherein the variable blocking structure is a liquid crystal display device. ### Claim 8 (depends on 1) The apparatus according to claim 1, wherein the corresponding lines are corresponding epipolar lines. ### Claim 9 (depends on 1) The apparatus according to claim 1, wherein at least one end of the corresponding lines within the first image and the second image is predetermined by a first illumination unit and a second illumination unit included in the illumination unit. ### Claim 10 A method for operating an apparatus including an optical system, the method comprising: imaging a scene including at least one object into at least a first image and a second image by using Helmholtz reciprocity; obtaining disparity information concerning the first image and the second image on the basis of a comparison of intensity sum values along corresponding lines in the first image and the second image; and retrieving depth information concerning the object from the disparity information, wherein obtaining the disparity information comprises integrating in the first image an intensity along the corresponding lines to obtain intensity sum profiles of the first image; integrating in the second image the intensity along the corresponding lines to obtain intensity sum profiles of the second image selecting for each pair of corresponding lines of the first image and the second image the corresponding intensity sum profile of the first image and the corresponding intensity sum profile of the second image; determining from the selected intensity sum profile of the first image for each value of integrated intensity a first position on the integration path within the first image at which the value of integrated intensity is reached; determining from the selected intensity sum profile of the second image for each value of integrated intensity a second position on the integration path within the second image at which the value of integrated intensity is reached; and determining for each value of integrated intensity the disparity from the first position and the second position. ### Claim 11 (depends on 10) The method according to claim 10, wherein the corresponding lines are corresponding epipolar lines. ### Claim 12 (depends on 10) The method according to claim 10, wherein at least one end of the corresponding lines within the first image and the second image is provided by illuminating the object. ### Claim 13 (depends on 10) The method according to claim 10, wherein imaging the scene comprises: emitting light into at least one light emitting solid angle to illuminate the object; and receiving light reflected from the object from a light receiving solid angle, which is at least as large as each of the light emitting solid angles; wherein at least one end of the corresponding lines within the first image and the second image is predetermined by a boundary of the at least one light emitting solid angle mapped to the light receiving solid angle.