# US 10,502,556 B2 · Systems and methods for 3D surface measurements > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 10,502,556 B2 - Document ID: US10502556B2 - Status: Granted - Kind: B2 - Application number: 15/655,656 - Filed: 2017-07-20 - Published: 2018-01-25 - Granted: 2019-12-10 - Earliest claimed: 2016-07-20 - Assignees: Mura Inc. - Applicants: Mura Inc. - Inventors: Lingfei Meng; Jun Jiang - Web page: https://patentgrep.com/patent/US10502556B2 ## CPC - G01B 11/25: by projecting a pattern, e.g. {one or more lines,} moiré fringes on the object (G01B11/255 takes precedence {; image analysis for depth or shape recovery G06T7/50}) - G01B 11/303: {using photoelectric detection means} - H04N 13/271: wherein the generated image signals comprise depth maps or disparity maps - H04N 13/296: Synchronisation thereof; Control thereof - G01B: MEASURING LENGTH, THICKNESS OR SIMILAR LINEAR DIMENSIONS; MEASURING ANGLES; MEASURING AREAS; MEASURING IRREGULARITIES OF SURFACES OR CONTOURS - H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION ## Abstract Certain embodiments are directed to 3D surface measurement systems and methods configured to direct engineered illumination of beams in one or more ray bundles to N illumination directions incident a sample surface. The systems and methods can measure a map of surface normals, a depth map, and a map of surface properties using intensity images captured while the engineered illumination is directed to the sample surface. ## Claims ### Claim 1 A 3D surface measurement system comprising: an engineered illumination system configured to provide at least one ray bundle sequentially at N illumination directions incident a surface of a sample being imaged, wherein each ray bundle comprises illumination beams of various intensities, the illumination beams of each ray bundle configured in different directions converging toward the surface of the sample during operation; a first camera with an imaging lens and at least one sensor configured to acquire intensity images based on light received from the illuminated sample, each intensity image acquired while the at least one ray bundle is directed at one of the N illumination directions; and a controller configured to execute instructions to: determine a sensor response at each sensor pixel of a plurality of sensor pixels of the at least one sensor, the sensor response determined from the intensity images corresponding to the N illumination directions; match the sensor response at each sensor pixel to one of a plurality of predetermined sensor responses to determine a surface normal at each sensor pixel; and construct a map of surface normals of the surface of the sample by combining the determined surface normals of the plurality of sensor pixels of the at least one sensor. ### Claim 2 (depends on 1) The 3D surface measurement system of claim 1, wherein the engineered illumination system comprises: an illumination device with at least one light source; and one or more optical elements configured to generate and propagate the at least one ray bundle based on light from the at least one light source to the surface of the sample and to propagate light from the illuminated surface to the first camera. ### Claim 3 (depends on 2) The 3D surface measurement system of claim 2, wherein the one or more optical elements comprise a diffuser configured to spread illumination from the at least one light source to generate the illumination beams of various intensities of each ray bundle according to a transmission profile of the diffuser. ### Claim 4 (depends on 3) The 3D surface measurement system of claim 3, wherein the diffuser transmission profile is configured to spread the illumination beams of various intensities to different angles according to a Gaussian or Gaussian-like distribution. ### Claim 5 (depends on 2) The 3D surface measurement system of claim 2, wherein the one or more optical elements are further configured to distribute the illumination beams of various intensities at different angles according to a linear gradient. ### Claim 6 (depends on 2) The 3D surface measurement system of claim 2, wherein the one or more optical elements are further configured to distribute the illumination beams of various intensities at different angles according to a random distribution. ### Claim 7 (depends on 2) The 3D surface measurement system of claim 2, further comprising an additional illumination source configured to direct illumination incident the surface of the sample at an oblique angle. ### Claim 8 (depends on 2) The 3D surface measurement system of claim 2, wherein the one or more optical elements comprise: a mirror configured to receive illumination beams propagated from the at least one light source; and a beam splitter configured to receive the illumination beams from the mirror and reflecting the illumination beams to the surface of the sample and configured to transmit light reflected from the surface of the sample. ### Claim 9 (depends on 2) The 3D surface measurement system of claim 2, wherein the one or more optical elements comprise: a combination filter of a plurality of neutral density filters, the combination filter configured to engineer illumination from the at least one light source into the illumination beams of various intensities of each of the at least one ray bundle, each neutral density filter associated with one illumination beam; and a collimator configured to collimate the illumination beams propagated from the combination filter. ### Claim 10 (depends on 1) The 3D surface measurement system of claim 1, wherein the engineered illumination system comprises: an illumination array having a plurality of illumination elements, the illumination array configured to activate one of the illumination elements at a single exposure time; and a collimator configured to collimate illumination received from the one activated illumination element; and a diffuser configured to spread the collimated illumination from the collimator to the illumination beams of various intensities of each ray bundle according to a transmission profile of the diffuser. ### Claim 11 (depends on 1) The 3D surface measurement system of claim 1, further comprising a motion control device configured to manipulate the sample and/or the engineered illumination and the camera together. ### Claim 12 (depends on 11) The 3D surface measurement system of claim 11, wherein the motion control device is configured to manipulate the sample and/or the engineered illumination and the camera together to shift the illumination beams to different regions of the surface of the sample. ### Claim 13 (depends on 11) The 3D surface measurement system of claim 11, wherein the motion control device is configured to manipulate the sample and/or the engineered illumination and the camera together to direct the illumination beams in the N illumination directions. ### Claim 14 (depends on 2) The 3D surface measurement system of claim 2, further comprising a wavelength and/or polarization control device configured to alter the wavelength and/or polarization of the at least one ray bundle or alter the wavelength and/or polarization of reflected light from the surface of the sample. ### Claim 15 (depends on 2) The 3D surface measurement system of claim 2, further comprising a first wavelength and/or polarization control device configured to alter the wavelength and/or polarization of the at least one ray bundle and a second wavelength and/or polarization control device configured to alter the wavelength and/or polarization of the reflected light from the surface of the sample. ### Claim 16 (depends on 2) The 3D surface measurement system of claim 2, wherein the controller further configured to execute instructions to determine a depth map and/or surface properties of the surface of the sample. ### Claim 17 (depends on 2) The 3D surface measurement system of claim 2, further comprising a second camera with an imaging lens and at least one sensor, the second camera configured to capture intensity images at the N illumination directions based on light received from the illuminated sample, wherein the second camera captures intensity images from a different angle than the first camera. ### Claim 18 (depends on 1) The 3D surface measurement system of claim 1, wherein the illumination beams of each ray bundle converge to a point. ### Claim 19 A 3D surface measurement method comprising: receiving a plurality of intensity images of a sample in a signal from at least one sensor of a camera, the plurality of intensity images captured at a plurality of exposure times, each intensity image acquired while one or more ray bundles is directed at one of N illumination directions, each ray bundle comprising illumination beams of various intensities, the illumination beams of each ray bundle directed in different directions converging toward the surface of the sample; determining a sensor response at each sensor pixel of a plurality of sensor pixels of the at least one sensor, the sensor response determined from the intensity images at the N illumination directions; matching a sensor response at each sensor pixel to one of a plurality of predetermined sensor responses to determine a surface normal at each sensor pixel; and construct a map of surface normals of the surface of the sample by combining the determined surface normals of the plurality of sensor pixels of the at least one sensor. ### Claim 20 (depends on 19) The 3D surface measurement method of claim 19, further comprising determining a depth map of the surface from the map of surface normals. ### Claim 21 (depends on 19) The 3D surface measurement method of claim 19, further comprising determining a map of surface properties of the surface based on the map of surface normals. ### Claim 22 (depends on 21) The 3D surface measurement method of claim 21, wherein the surface properties comprise one or more of a reflectance property, an emissivity property, and a translucency property. ### Claim 23 (depends on 19) The 3D surface measurement method of claim 19, further comprising determining a defect in the surface of the sample based a map of surface properties and/or a depth map of the surface, wherein each map is determined from the map of surface normals. ### Claim 24 (depends on 19) The 3D surface measurement method of claim 19, further comprising: determining a depth map of the surface from the map of surface normals; determining one or more maps of surface properties of the surface based on the map of surface normals; and generating a physically-based rendering based on the map of surface normals, the depth map, and the at least one of the one or more maps of surface properties of the surface. ### Claim 25 A 3D surface measurement method comprising: a) engineering at least one ray bundle directed sequentially to N illumination directions incident a surface a sample being imaged, each ray bundle comprising illumination beams of various intensities, the illumination beams of each ray bundle directed in different directions converging toward the surface of the sample; b) acquiring, using at least one sensor of a camera, intensity images at a plurality of exposure times based on light from the illuminated sample, each intensity image acquired while the one or more ray bundles is directed at one of the N illumination directions; and c) communicating the intensity images to one or more processors, wherein the one or more processors determine a sensor response at each sensor pixel of a plurality of sensor pixels of the at least one sensor, the sensor response determined from the intensity images at the N illumination directions, match the sensor response at each sensor pixel to one of a plurality of predetermined sensor responses to determine a surface normal at each sensor pixel, and construct a map of surface normals of the surface of the sample by combining the determined surface normals of the plurality of sensor pixels of the at least one sensor. ### Claim 26 (depends on 24) The 3D surface measurement method of claim 24, further comprising changing wavelength or polarization of the illumination beams of the at least one ray bundle and/or light from the illuminated sample, and then repeating a) and b).