# US 2012/0075534 A1 · Integrated low power depth camera and projection device

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2012/0075534 A1
- Document ID: US20120075534A1
- Status: Published application
- Kind: A1
- Application number: 12/892,589
- Filed: 2010-09-28
- Published: 2012-03-29
- Earliest claimed: 2010-09-28
- Assignees: Microsoft Corporation
- Inventors: Sagi Katz; Shlomo Felzenshtein; Avishai Adler; Giora Yahav
- Web page: https://patentgrep.com/patent/US20120075534A1

## CPC

- H04N 9/3194: {including sensor feedback}
- G01S 17/894: Three-dimensional [3D] imaging with simultaneous measurement of time-of-flight at a two-dimensional [2D] array of receiver pixels, e.g. time-of-flight cameras or flash lidar
- H04N 9/3182: {Colour adjustment, e.g. white balance, shading or gamut (white balance per se H04N9/73; control of amplitude of colour signals H04N9/68; colour control circuits for displays in general G09G5/02, G09G3/2003, G09G3/3607)}
- G01S 17/18: wherein range gates are used
- H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION
- G01S: RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES

## Abstract

A video projector device includes a visible light projector to project an image on a surface or object, and a visible light sensor, which can be used to obtain depth data regarding the object using a time-of-flight principle. The sensor can be a charge-coupled device which obtains color images as well as obtaining depth data. The projected light can be provided in successive frames. A frame can include a gated sub-frame of pulsed light followed by continuous light, while the sensor is gated, to obtain time of flight data, an ungated sub-frame of pulsed light followed by continuous light, while the sensor is ungated, to obtain reflectivity data and a background sub-frame of no light followed by continuous light, while the sensor is gated, to determine a level of background light. A color sub-frame projects continuous light, while the sensor is active.

## Claims

### Claim 1

A video projector device, comprising:
a light source which emits visible light;
a driver which drives the light source;
at least one optical component which projects the visible light in a field of view;
a sensor which senses visible light, including visible light which is reflected from at least one object in the field of view, the sensor comprises a plurality of pixels; and
at least one control circuit which provides a gated sub-frame in which the light source is driven in a pulsed mode while the sensor is operated in a gated mode, followed by the light source being driven in a continuous mode while the sensor is read to obtain light intensity values, where the at least one control circuit derives depth data regarding the at least one object in the field of view based on the light intensity values, using a time-of-flight principle.

### Claim 2 (depends on 1)

The video projector device of claim 1, wherein:
the at least one control circuit provides an ungated sub-frame in which the light source is driven in the pulsed mode while the sensor is operated in an ungated mode, followed by the light source being driven in the continuous mode while the sensor is read to obtain light intensity values, where the at least one control circuit adjusts the light intensity values obtained during the gated sub-frame based on the light intensity values obtained during the ungated sub-frame, to compensate for reflectivity of the at least one object in the field of view.

### Claim 3 (depends on 1)

The video projector device of claim 1, wherein:
the at least one control circuit provides at least one background sub-frame in which no visible light is emitted while the sensor is operated in a gated mode, followed by the light source being driven in the continuous mode while the sensor is read to obtain light intensity values, where the at least one control circuit adjusts the light intensity values obtained during the gated sub-frame based on the light intensity values obtained during the background sub-frame, to compensate for background light in the field of view.

### Claim 4 (depends on 3)

The video projector device of claim 3, wherein:
the at least one control circuit provides two of the background sub-frames in a frame.

### Claim 5 (depends on 1)

The video projector device of claim 1, wherein:
the at least one control circuit provides a color sub-frame in which the light source is driven in the continuous mode while the sensor is operated in a standard color sensing mode, followed by the light source being driven in the continuous mode while the sensor is read to obtain light intensity values which include color data from the field of view.

### Claim 6 (depends on 1)

The video projector device of claim 1, wherein:
the at least one control circuit provides a sub-frame in which the light source is driven in the continuous mode while the sensor is active.

### Claim 7 (depends on 1)

The video projector device of claim 1, wherein:
the at least one control circuit drives the light source using triangular shaped pulses during the gated sub-frame.

### Claim 8 (depends on 1)

The video projector device of claim 1, wherein:
the at least one control circuit drives the light source using pulses which each have at least one of: (a) a leading edge which transitions higher at a rate which is below a step up rate of the light source, and (b) a trailing edge which transitions lower at a rate which is below a step down rate of the light source.

### Claim 9 (depends on 1)

The video projector device of claim 1, wherein:
the light source is a backlight of the video projector device.

### Claim 10 (depends on 1)

The video projector device of claim 1, wherein:
the light source, the light source, at least one optical component, the at least one control circuit and the sensor are provided in a common housing.

### Claim 11 (depends on 1)

The video projector device of claim 1, further comprising:
at least one light-transmissive LCD panel which encodes video information onto the visible light, the at least one light-transmissive LCD panel comprises a plurality of pixels.

### Claim 12 (depends on 1)

The video projector device of claim 1, wherein:
the at least one control circuit focuses the at least one optical component based on the depth data.

### Claim 13

A video projector device, comprising:
a light source which emits visible light;
a driver which drives the light source;
at least one optical component which projects the modulated visible light in a field of view;
a sensor which senses visible light, including visible light which is reflected from at least one object in the field of view, the sensor comprises a plurality of pixels; and
at least one control circuit which provides a gated sub-frame in which the light source is driven in a pulsed mode so that pulsed visible light is emitted while the sensor is operated in a gated mode, followed by no visible light being emitted while the sensor is read to obtain light intensity values, where the at least one control circuit derives depth data regarding the at least one object in the field of view based on the light intensity values, using a time-of-flight principle.

### Claim 14 (depends on 13)

The video projector device of claim 13, wherein:
the at least one control circuit provides an ungated sub-frame in which the light source is driven in the pulsed mode so that pulsed visible light is emitted while the sensor is operated in an ungated mode, followed by no visible light being emitted while the sensor is read to obtain light intensity values, where the at least one control circuit normalizes the light intensity values obtained during the gated sub-frame based on the light intensity values obtained during the ungated sub-frame, to compensate for reflectivity of the at least one object in the field of view.

### Claim 15 (depends on 13)

The video projector device of claim 13, wherein:
no visible light is emitted at a time other than during the gated sub-frame and the ungated sub-frame.

### Claim 16 (depends on 15)

The video projector device of claim 15, wherein:
the light source is turned off to cause no visible light to be emitted.

### Claim 17

A video projector device, comprising:
a light source which emits visible light;
a driver which drives the light source;
at least one optical component which projects the visible light in a field of view;
a sensor which senses visible light, including visible light which is reflected from at least one object in the field of view, the sensor comprises a plurality of pixels; and
at least one control circuit which provides, at different times: (a) a depth-sensing frame in which the at least one control circuit derives depth data regarding the at least one object in the field of view based on light intensity values which are read from the sensor, using a time-of-flight principle, and (b) a non-depth-sensing frame in which the at least one control circuit does not derive depth data regarding the at least one object in the field of view.

### Claim 18 (depends on 16)

The video projector device of claim 16, wherein:
the at least one control circuit causes the light source to emit visible light at a higher illumination power during the depth-sensing frame than during the non-depth-sensing frame.

### Claim 19 (depends on 16)

The video projector device of claim 16, wherein:
the at least one control circuit adjusts a frame period so that the depth-sensing frame is longer than the non-depth-sensing frame.

### Claim 20 (depends on 16)

The video projector device of claim 16, wherein:
the at least one control circuit is controllable to selectively operate in a first mode in which the at least one control circuit uses a limited number of one or more frames to obtain depth data regarding the at least one object in the field of view, and in a second mode in which the at least one control circuit continuously obtains depth data regarding at least one object in the field of view.
