# US 2016/0366347 A1 · Using Infrared Images of a Monitored Scene to Identify Windows

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2016/0366347 A1
- Document ID: US20160366347A1
- Status: Published application
- Kind: A1
- Application number: 14/745,230
- Filed: 2015-06-19
- Published: 2016-12-15
- Earliest claimed: 2015-06-12
- Assignees: Google Inc.
- Applicants: Google Inc.
- Inventors: DONGEEK SHIN; MICHAEL DIXON
- Web page: https://patentgrep.com/patent/US20160366347A1

## CPC

- G08B 13/19602: {Image analysis to detect motion of the intruder, e.g. by frame subtraction}
- G06T 7/20: Analysis of motion (motion estimation for coding, decoding, compressing or decompressing digital video signals H04N19/43, H04N19/51)
- G06T 7/254: involving subtraction of images
- G06T 7/514: from specularities
- G06T 7/586: from multiple light sources, e.g. photometric stereo
- G06V 10/145: Illumination specially adapted for pattern recognition, e.g. using gratings
- G06V 20/52: Surveillance or monitoring of activities, e.g. for recognising suspicious objects (recognising microscopic objects G06V20/69)
- G08B 13/19617: {Surveillance camera constructional details}
- G08B 13/19656: {Network used to communicate with a camera, e.g. WAN, LAN, Internet}
- G08B 13/19658: {Telephone systems used to communicate with a camera, e.g. PSTN, GSM, POTS}
- G08B 17/10: Actuation by presence of smoke or gases {, e.g. automatic alarm devices for analysing flowing fluid materials by the use of optical means}
- H04N 13/20: Image signal generators
- H04N 23/11: for generating image signals from visible and infrared light wavelengths
- H04N 23/20: for generating image signals from infrared radiation only
- H04N 23/45: for generating image signals from two or more image sensors being of different type or operating in different modes, e.g. with a CMOS sensor for moving images in combination with a charge-coupled device [CCD] for still images
- H04N 23/56: provided with illuminating means
- H04N 23/661: Transmitting camera control signals through networks, e.g. control via the Internet
- H04N 23/74: by influencing the scene brightness using illuminating means
- G06T 2207/10016: Video; Image sequence
- G06T 2207/10024: Color image
- G06T 2207/10048: Infrared image
- G06T 2207/10152: Varying illumination
- G06T 2207/30232: Surveillance
- G06V 2201/121: using special illumination
- H04N 13/271: wherein the generated image signals comprise depth maps or disparity maps
- H04N 23/667: Camera operation mode switching, e.g. between still and video, sport and normal or high- and low-resolution modes
- H04N 23/90: Arrangement of cameras or camera modules, e.g. multiple cameras in TV studios or sports stadiums
- G08B: SIGNALLING SYSTEMS, e.g. PERSONAL CALLING SYSTEMS; ORDER TELEGRAPHS; ALARM SYSTEMS
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION

## Abstract

A camera system includes memory, a lens assembly arranged to direct light from a scene onto an image sensing element, an image sensing element configured to receive light from the scene via the lens assembly, at least one infrared illuminator configured to transmit infrared light, and a processor, coupled to the image sensing element and the illuminators. The processor is configured to operate the illuminators and the image sensing element in a first mode whereby infrared light transmitted by the illuminators and reflected from the scene is used to generate a two-dimensional image of the scene. The processor is also configured to operate the illuminators and the image sensing element in a second mode whereby infrared light transmitted by the illuminators and reflected from the scene is used to detect a first region, located within the scene, having a specific material.

## Claims

### Claim 1

A camera system, comprising:
a memory configured to store information;
a lens assembly arranged to direct light from a scene onto an image sensing element;
an image sensing element configured to receive light from the scene via the lens assembly;
at least one infrared illuminator configured to transmit infrared light; and
a processor, coupled to the image sensing element and the at least one infrared illuminator and configured to:
operate the at least one infrared illuminator and the image sensing element in a first mode whereby infrared light transmitted by the at least one infrared illuminator and reflected from the scene is used to generate a two-dimensional image of the scene; and
operate the at least one infrared illuminator and the image sensing element in a second mode whereby infrared light transmitted by the at least one infrared illuminator and reflected from the scene is used to detect a first region, located within the scene, having a specific material.

### Claim 2 (depends on 1)

The camera system of claim 1, wherein the first region is transparent and the first region is determined to be made of glass.

### Claim 3 (depends on 1)

The camera system of claim 1, wherein the first mode is operated at night.

### Claim 4 (depends on 1)

The camera system of claim 1, wherein the camera system comprises a plurality of infrared illuminators, and wherein in the first mode all of infrared illuminators are activated to generate the two-dimensional image.

### Claim 5 (depends on 1)

The camera system of claim 1, wherein the second mode is a specularity detection mode.

### Claim 6 (depends on 1)

The camera system of claim 1, wherein the camera system comprises a plurality of infrared illuminators, and wherein at least some of the infrared illuminators are activated at different times to identify the first region.

### Claim 7 (depends on 6)

The camera system of claim 6, wherein a plurality of subsets of the infrared illuminators are activated in sequence, each subset illuminating a different portion of the scene.

### Claim 8 (depends on 6)

The camera system of claim 6, wherein a plurality of subsets of the infrared illuminators are activated in sequence and each of the subsets of illuminators is configured at a different angle relative to the image sensing element.

### Claim 9 (depends on 1)

The camera system of claim 1, wherein in the second mode the processor is configured to:
capture a first image of a scene while one or more of the illuminators are activated;
capture a second image of the scene is captured while none of the illuminators are activated;
construct a light intensity map for the scene by subtracting light intensity values from the second image from corresponding light intensity values in the first image; and
storing to the memory the light intensity map.

### Claim 10 (depends on 9)

The camera system of claim 9, wherein the first and second images are IR images.

### Claim 11 (depends on 9)

The camera system of claim 9, wherein the processor is further configured to:
transmit the stored light intensity map to a window detection module configured to identify windows in the scene.

### Claim 12 (depends on 9)

The camera system of claim 9, wherein the light intensity map comprises a plurality of pixels.

### Claim 13 (depends on 9)

The camera system of claim 9, wherein the processor is further configured to:
use the light intensity map to identify a plurality of pixels whose corresponding intensity values are within a predefined intensity range; and
cluster the identified plurality of pixels into one or more regions including the first region.

### Claim 14 (depends on 13)

The camera system of claim 13, wherein clustering the identified plurality of pixels into one or more regions further comprises using a depth map that is constructed using the image sensor array.

### Claim 15 (depends on 13)

The camera system of claim 13, wherein clustering the identified plurality of pixels into one or more regions further comprises using an RGB image of the scene captured using the image sensor array.

### Claim 16 (depends on 13)

The camera system of claim 13, wherein the predefined intensity range comprises all intensity values below a threshold value, and the specific material is glass, the method thereby determining that the first region corresponds to a window in the scene.

### Claim 17 (depends on 16)

The camera system of claim 16, wherein determining that the first region corresponds to a window comprises determining that the first region is substantially a quadrilateral.

### Claim 18 (depends on 1)

The camera system of claim 1, wherein the processor is further configured to:
receive a video stream of the scene from the camera system;
review the video stream to detect movement in the scene, wherein the first region is excluded from movement detection; and
generate a motion alert when there is motion detected at the scene outside of the first region.

### Claim 19

A method, comprising:
at a camera system having a memory configured to store information, a lens assembly arranged to direct light from a scene onto an image sensing element, an image sensing element configured to receive light from the scene via the lens assembly, at least one infrared illuminator configured to transmit infrared light, and a processor, coupled to the image sensing element and the at least one infrared illuminator:
operating the at least one infrared illuminator and the image sensing element in a first mode whereby infrared light transmitted by the at least one infrared illuminator and reflected from the scene is used to generate a two-dimensional image of the scene; and
operating the at least one infrared illuminator and the image sensing element in a second mode whereby infrared light transmitted by the at least one infrared illuminator and reflected from the scene is used to detect a first region, located within the scene, having a specific material.

### Claim 20

A non-transitory computer readable storage medium storing one or more programs configured for execution by a camera system having a memory configured to store information, a lens assembly arranged to direct light from a scene onto an image sensing element, an image sensing element configured to receive light from the scene via the lens assembly, at least one infrared illuminator configured to transmit infrared light, and a processor, coupled to the image sensing element and the at least one infrared illuminator, wherein the one or more programs comprise instructions for:
operating the at least one infrared illuminator and the image sensing element in a first mode whereby infrared light transmitted by the at least one infrared illuminator and reflected from the scene is used to generate a two-dimensional image of the scene; and
operating the at least one infrared illuminator and the image sensing element in a second mode whereby infrared light transmitted by the at least one infrared illuminator and reflected from the scene is used to detect a first region, located within the scene, having a specific material.