# US 2008/0304081 A1 · Acquisition of Surface Normal Maps from Spherical Gradient Illumination

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2008/0304081 A1
- Document ID: US20080304081A1
- Status: Published application
- Kind: A1
- Application number: 12/105,141
- Filed: 2008-04-17
- Published: 2008-12-11
- Earliest claimed: 2007-04-17
- Assignees: University of Southern California
- Inventors: Paul E. Debevec; Wan-Chun Ma; Timothy Hawkins
- Web page: https://patentgrep.com/patent/US20080304081A1

## CPC

- G01B 11/25: by projecting a pattern, e.g. {one or more lines,} moiré fringes on the object (G01B11/255 takes precedence {; image analysis for depth or shape recovery G06T7/50})
- G01B: MEASURING LENGTH, THICKNESS OR SIMILAR LINEAR DIMENSIONS; MEASURING ANGLES; MEASURING AREAS; MEASURING IRREGULARITIES OF SURFACES OR CONTOURS

## Abstract

An apparatus for generating a surface normal map of an object may include a plurality of light sources having intensities that are controllable so as to generate one or more gradient illumination patterns. The light sources are configured and arranged to illuminate the surface of the object with the gradient illumination patterns. A camera may receive light reflected from the illuminated surface of the object, and generate data representative of the reflected light. A processing system may process the data so as to estimate the surface normal map of the surface of the object. A specular normal map and a diffuse normal map of the surface of the object may be generated separately, by placing polarizers on the light sources and in front of the camera so as to illuminate the surface of the object with polarized spherical gradient illumination patterns.

## Claims

### Claim 1

An apparatus for generating a surface normal map of an object, comprising:
a plurality of light sources having intensities that are controllable so as to generate one or more gradient illumination patterns, the light sources configured and arranged to illuminate the surface of the object with the gradient illumination patterns;
an optical imaging system configured to receive light reflected from the illuminated surface of the object, and to generate data representative of the reflected light; and
a processing system configured to process the data so as to estimate the surface normal map of the object.

### Claim 2 (depends on 1)

The apparatus of claim 1, wherein the gradient illumination patterns are substantially spherical in their angular extent surrounding the object.

### Claim 3 (depends on 1)

The apparatus of claim 1, wherein the data representative of the reflected light comprise data representative of specular reflectance of the surface of the object.

### Claim 4 (depends on 1)

The apparatus of claim 1, wherein the data representative of the reflected light comprise data representative of diffuse reflectance of the surface of the object.

### Claim 5 (depends on 1)

The apparatus of claim 1, wherein the data representative of the reflected light comprise data representative of subsurface reflectance of the object.

### Claim 6 (depends on 1)

The apparatus of claim 1, wherein the plurality of light sources comprise LEDs.

### Claim 7 (depends on 1)

The apparatus of claim 1, wherein the plurality of light sources comprise reflected light from a screen illuminated by a video projector.

### Claim 8 (depends on 1)

The apparatus of claim 1, wherein the plurality of light sources comprise light from pixels of a computer monitor.

### Claim 9 (depends on 1)

The apparatus of claim 1, further comprising a controller configured to individually control the intensities of the light sources so as to generate the gradient illumination patterns and to drive the plurality of light sources so as to illuminate the surface of the object with the gradient illumination patterns.

### Claim 10 (depends on 1)

The apparatus of claim 1, wherein the gradient illumination patterns comprise:
a substantially constant spherical illumination pattern;
a substantially linear gradient illumination pattern along an x-axis;
a substantially linear gradient illumination pattern along a y-axis; and
a substantially linear gradient illumination pattern along a z-axis.

### Claim 11 (depends on 1)

The apparatus of claim 1, wherein the data representative of the reflected light comprise a plurality of pixels; and
wherein the surface normal map provides a surface normal value for each pixel.

### Claim 12 (depends on 1)

The apparatus of claim 1, wherein the plurality of light sources are arranged in a substantially spherical configuration around the object.

### Claim 13

A method of generating a surface normal map of an object, comprising:
illuminating a surface of the object with one or more gradient illumination patterns;
generating data representative of the light reflected from the illuminated surface of the object; and
processing the data so as to estimate the surface normal map of the object.

### Claim 14 (depends on 13)

The method of claim 13, wherein the gradient illumination patterns are substantially spherical in their angular extent surrounding the object.

### Claim 15 (depends on 13)

The method of claim 13, wherein the data representative of the reflected light comprise data representative of at least one of:
specular reflectance of the surface of the object;
diffuse reflectance of the surface of the object; and
subsurface reflectance of the object.

### Claim 16

An apparatus for separately generating a specular normal map and a diffuse normal map of a surface of an object, comprising:
a plurality of light sources having intensities that are controllable so as to generate one or more gradient illumination patterns;
a set of polarizers adapted to be positioned relative to the light sources so as to polarize light from the light sources;
wherein the light sources and the set of polarizers are arranged to illuminate the surface of the object with one or more polarized gradient illumination patterns;
a camera configured to receive light that is reflected from the illuminated surface of the object, and to generate from the reflected light photographic data of the surface of the object;
a camera polarizer configured to polarize the reflected light in a way that specularly reflected light is separated from diffusely reflected light, before the reflected light is received by camera; and
a processing system configured to generate, from the photographic data, specular reflectance data representative of the specularly reflected light and diffuse reflectance data representative of the diffusely reflected light, the processing system further configured to estimate a specular normal map from the specular reflectance data and a diffuse normal map from the diffuse reflectance data.

### Claim 17 (depends on 16)

The apparatus of claim 16, wherein the diffuse normal map comprises a first diffuse normal map for a red color channel in the photographic data, a second diffuse normal map for a blue color channel in the photographic data, and a third diffuse normal map for a green color channel in the photographic data.

### Claim 18 (depends on 16)

The apparatus of claim 16, wherein the set of polarizers and the camera polarizer comprise linear polarizers.

### Claim 19 (depends on 16)

The apparatus of claim 16, wherein the set of polarizers comprise linear polarizers oriented so as to polarize the light from the light sources so that after reflection of the light by the object toward the camera the specularly reflected light is polarized in a consistent direction, and wherein the camera polarizer comprises a linear polarizer adapted to be oriented to attenuate polarized specular light reflected by the object.

### Claim 20 (depends on 16)

The apparatus of claim 16, wherein the plurality of polarizers and the camera polarizer comprise circular polarizers.

### Claim 21 (depends on 16)

The apparatus of claim 16, wherein the plurality of polarizers are configured to circularly polarize light along a first direction, and the camera polarizer is configured to circularly polarize light along a second direction; and wherein the first direction and the second direction are the same.

### Claim 22 (depends on 16)

The apparatus of claim 16, wherein the plurality of polarizers and the camera polarizer comprise individually tunable polarizers, and wherein the plurality of polarizers and the camera polarizer are tunable so as to minimize observed specular reflection from the object as viewed through the camera polarizer.

### Claim 23 (depends on 16)

The apparatus of claim 16, further comprising a controller configured to individually control the intensities of the light sources so that the light sources and the polarizers generate the polarized spherical gradient illumination patterns, the controller further configured to drive the plurality of light sources so that the light sources and the polarizers illuminate the surface of the object with the polarized spherical gradient illumination patterns.

### Claim 24

A method of separately generating a specular normal map and a diffuse normal map of a surface of an object, comprising:
illuminating the surface of the object with one or more polarized spherical gradient illumination patterns;
polarizing light reflected from the illuminated surface of the object in a way that separates specularly reflected light from diffusely reflected light;
generating specular reflectance data representative of the specularly reflected light and diffuse reflectance data representative of the diffusely reflected light;
estimating a specular normal map from the specular reflectance data; and
estimating a diffuse normal map from the diffuse reflectance data.

### Claim 25 (depends on 24)

The method of claim 24, wherein the act of illuminating the surface of the object with polarized spherical gradient illumination patterns comprises placing a set of polarizers on a plurality of controllable light sources, and driving the light sources so as to generate the polarized spherical gradient illumination patterns.

### Claim 26 (depends on 24)

The method of claim 24, wherein the act of polarizing light reflected from the illuminated surface of the object in a way that separates specularly reflected light from diffusely reflected light comprises placing a camera polarizer in front of a camera that receives light reflected from the illuminated surface of the object, and tuning the set of polarizers and the camera polarizer so as to minimize the specular reflection from the object as viewed through the camera polarizer.

### Claim 27

An image rendering system for rendering an image of an object, the image comprising a plurality of pixels, the image rendering system comprising:
a plurality of light sources having intensities that are controllable so as to generate one or more spherical gradient illumination patterns;
a set of polarizers adapted to be positioned relative to the light sources so as to polarize light from the light sources;
wherein the light sources and the set of polarizers are arranged to illuminate the surface of the object with one or more polarized spherical gradient illumination patterns;
a camera configured to receive light that is reflected from the illuminated surface of the object, and to generate from the reflected light photographic data of the surface of the object;
a camera polarizer configured to polarize the reflected light, the camera polarizer having an orientation relative to the set of polarizers that is adjustable so as to separate specularly reflected light from diffusely reflected light, before the reflected light is received by the camera; and
a processing system configured to generate specular reflectance data for the specularly reflected light and diffuse reflectance data for the diffusely reflected light, and to estimate a specular normal map from the specular reflectance data and a diffuse normal map from the diffuse reflectance data;
the processing system further configured to render a specular reflectance component of each of the pixels in the image using the specular normal map estimated from the specular reflectance data, and to render a diffuse reflectance component of each of the pixels in the image using the diffuse normal map estimated from the diffuse reflectance data.

### Claim 28

A method of rendering an image of an object, the image comprising a plurality of pixels, the method comprising:
illuminating the surface of the object with one or more polarized spherical gradient illumination patterns;
polarizing light reflected from the illuminated surface of the object in a way that separates specularly reflected light from diffusely reflected light;
generating specular reflectance data representative of the specularly reflected light and diffuse reflectance data representative of the diffusely reflected light;
estimating a specular normal map from the specular reflectance data, and separately estimating a diffuse normal map from the diffuse reflectance data;
rendering a specular reflectance component of each of the pixels in the image using the specular normal map estimated from the specular reflectance data; and
rendering a diffuse reflectance component of each of the pixels in the image using the diffuse normal map estimated from the diffuse reflectance data.

### Claim 29

A high resolution scanning system, comprising:
a plurality of light sources having intensities that are controllable so as to generate one or more spherical gradient illumination patterns;
a set of polarizers adapted to be positioned relative to the light sources so as to polarize light from the light sources;
wherein the light sources and the set of polarizers are arranged to illuminate the surface of the object with one or more polarized spherical gradient illumination patterns;
a three-dimensional scanning system configured to obtain a three-dimensional scan of the illuminated object's geometry, the three dimensional scan having a substantially medium resolution;
another polarizer configured and arranged to polarize the reflected light so that specularly reflected light is separated from diffusely reflected light, before the reflected light is received by the scanning system;
a processing system configured to generate specular reflectance data representative of the specularly reflected light and to estimate a specular normal map from the specular reflectance data, the processing system further configured to increase resolution of the geometry obtained from the three-dimensional scanning system, using the specular normal map.

### Claim 30

A high resolution scanning system, comprising:
a plurality of light sources having intensities that are controllable so as to generate one or more spherical gradient illumination patterns;
a set of polarizers adapted to be positioned relative to the light sources so as to polarize light from the light sources;
wherein the light sources and the set of polarizers are arranged to illuminate the surface of the object with one or more polarized spherical gradient illumination patterns;
a video projector configured to project one or more structured light patterns onto the illuminated surface of the object;
one or more cameras configured to receive light that is reflected from the illuminated surface of the object on which the structured light patterns have been projected, and to generate photographic data from the received light;
a camera polarizer configured and arranged to polarize the reflected light so that specularly reflected light is separated from diffusely reflected light, before the reflected light is received by camera;
a processing system configured to generate specular reflectance data representative of the specularly reflected light and to estimate a specular normal map from the specular reflectance data, the processing system further configured to increase the resolution of the photographic data using the specular normal map.
