# US 2016/0366346 A1 · Using infrared images of a monitored scene to identify windows > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 2016/0366346 A1 - Document ID: US20160366346A1 - Status: Published application - Kind: A1 - Application number: 14/738,806 - Filed: 2015-06-12 - Published: 2016-12-15 - Earliest claimed: 2015-06-12 - Assignees: Google Inc. - Applicants: Google Inc. - Inventors: DONGEEK SHIN; MICHAEL DIXON - Web page: https://patentgrep.com/patent/US20160366346A1 ## CPC - G08B 13/19602: {Image analysis to detect motion of the intruder, e.g. by frame subtraction} - G06T 7/20: Analysis of motion (motion estimation for coding, decoding, compressing or decompressing digital video signals H04N19/43, H04N19/51) - G06T 7/254: involving subtraction of images - G06T 7/514: from specularities - G06T 7/586: from multiple light sources, e.g. photometric stereo - G06V 10/145: Illumination specially adapted for pattern recognition, e.g. using gratings - G06V 20/52: Surveillance or monitoring of activities, e.g. for recognising suspicious objects (recognising microscopic objects G06V20/69) - G08B 13/19617: {Surveillance camera constructional details} - G08B 13/19656: {Network used to communicate with a camera, e.g. WAN, LAN, Internet} - G08B 13/19658: {Telephone systems used to communicate with a camera, e.g. PSTN, GSM, POTS} - G08B 17/10: Actuation by presence of smoke or gases {, e.g. automatic alarm devices for analysing flowing fluid materials by the use of optical means} - H04N 13/20: Image signal generators - H04N 23/11: for generating image signals from visible and infrared light wavelengths - H04N 23/20: for generating image signals from infrared radiation only - H04N 23/45: for generating image signals from two or more image sensors being of different type or operating in different modes, e.g. with a CMOS sensor for moving images in combination with a charge-coupled device [CCD] for still images - H04N 23/56: provided with illuminating means - H04N 23/661: Transmitting camera control signals through networks, e.g. control via the Internet - H04N 23/74: by influencing the scene brightness using illuminating means - G06T 2207/10016: Video; Image sequence - G06T 2207/10024: Color image - G06T 2207/10048: Infrared image - G06T 2207/10152: Varying illumination - G06T 2207/30232: Surveillance - G06V 2201/121: using special illumination - H04N 13/271: wherein the generated image signals comprise depth maps or disparity maps - H04N 23/667: Camera operation mode switching, e.g. between still and video, sport and normal or high- and low-resolution modes - H04N 23/90: Arrangement of cameras or camera modules, e.g. multiple cameras in TV studios or sports stadiums - G08B: SIGNALLING SYSTEMS, e.g. PERSONAL CALLING SYSTEMS; ORDER TELEGRAPHS; ALARM SYSTEMS - G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL - G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING - H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION ## Abstract A process classifies objects in a scene. The process receives a captured IR image of a scene taken by a 2-dimensional image sensor array of a camera system while one or more IR illuminators of the camera system are emitting IR light, thereby forming an IR intensity map of the scene with a respective intensity value determined for each pixel of the IR image. The process uses the IR intensity map to identify a plurality of pixels whose corresponding intensity values are within a predefined intensity range, and clusters the identified plurality of pixels into one or more regions that are substantially contiguous. The process determines that a first region of the one or more regions corresponds to a specific material based, at least in part, on the intensity values of the pixels in the first region. The process then stores information in the memory that identifies the first region. ## Claims ### Claim 1 A method of classifying objects in a scene, comprising: at a computing device having one or more processors, and memory storing one or more programs configured for execution by the one or more processors: receiving a captured IR image of a scene taken by a 2-dimensional image sensor array of a camera system while one or more IR illuminators of the camera system are emitting IR light, thereby forming an IR intensity map of the scene with a respective intensity value determined for each pixel of the IR image; using the IR intensity map to identify a plurality of pixels whose corresponding intensity values are below a threshold value corresponding to glass; clustering the identified plurality of pixels into one or more regions that are substantially contiguous; determining that a first region of the one or more regions corresponds to a glass surface based, at least in part, on a difference in area between the first region and a geometrical shape containing the first region being less than a threshold percentage of the geometrical shape's area; storing information in the memory that identifies the first region; receiving a video stream of the scene from the camera system; reviewing the video stream to detect movement in the scene, wherein the first region is excluded from movement detection; and generating a motion alert when there is motion detected at the scene outside of the first region. ### Claim 2 (depends on 1) The method of claim 1, wherein each pixel of the IR image corresponds to a unique respective image sensor in the image sensor array. ### Claim 3 (depends on 1) The method of claim 1, wherein the pixels of the IR image form a partition of the image sensors in the image sensor array and wherein at least one pixel corresponds to a plurality of image sensors in the image sensor array. ### Claim 4 (depends on 1) The method of claim 1, wherein the computing device is a server distinct from the camera system. ### Claim 5 (depends on 1) The method of claim 1, wherein the computing device is included in the camera system. ### Claim 6 (depends on 1) The method of claim 1, wherein the camera system has a plurality of IR illuminators, and forming an IR intensity map of the scene comprises: for each of a plurality of distinct subsets of IR illuminators of the camera system, receiving a respective IR sub-image of the scene while the respective subset of IR illuminators are emitting IR light and the IR illuminators not in the respective subset are not emitting IR light; wherein determining the respective intensity value for a respective pixel comprises computing an average of intensity values at the pixel in each of the sub-images. ### Claim 7 (canceled) ### Claim 8 (depends on 1) The method of claim 1, wherein clustering the identified plurality of pixels into one or more regions further comprises using an RGB image of the scene captured using the image sensor array. ### Claim 9 (canceled) ### Claim 10 (canceled) ### Claim 11 (canceled) ### Claim 12 (canceled) ### Claim 13 A computing device, comprising: one or more processors; memory; and one or more programs stored in the memory configured for execution by the one or more processors, the one or more programs comprising instructions for: receiving a captured IR image of a scene taken by a 2-dimensional image sensor array of a camera system while one or more IR illuminators of the camera system are emitting IR light, thereby forming an IR intensity map of the scene with a respective intensity value determined for each pixel of the IR image; using the IR intensity map to identify a plurality of pixels whose corresponding intensity values are below a threshold value corresponding to glass; clustering the identified plurality of pixels into one or more regions that are substantially contiguous; determining that a first region of the one or more regions corresponds to a glass surface based, at least in part, on a difference in area between the first region and a geometrical shape containing the first region being less than a threshold percentage of the geometrical shape's area; storing information in the memory that identifies the first region; receiving a video stream of the scene from the camera system; reviewing the video stream to detect movement in the scene, wherein the first region is excluded from movement detection; and generating a motion alert when there is motion detected at the scene outside of the first region. ### Claim 14 (depends on 13) The computing device of claim 13, wherein the computing device is a server distinct from the camera system. ### Claim 15 (depends on 13) The computing device of claim 13, wherein the camera system has a plurality of IR illuminators, and forming an IR intensity map of the scene comprises: for each of a plurality of distinct subsets of IR illuminators of the camera system, receiving a respective IR sub-image of the scene while the respective subset of IR illuminators are emitting IR light and the IR illuminators not in the respective subset are not emitting IR light; wherein determining the respective intensity value for a respective pixel comprises computing an average of intensity values at the pixel in each of the sub-images. ### Claim 16 (depends on 13) The computing device of claim 13, wherein clustering the identified plurality of pixels into one or more regions further comprises using a depth map that was constructed using the image sensor array. ### Claim 17 (canceled) ### Claim 18 (canceled) ### Claim 19 (canceled) ### Claim 20 A non-transitory computer readable storage medium storing one or more programs configured for execution by a computing device having one or more processors and memory, the one or more programs comprising instructions for: receiving a captured IR image of a scene taken by a 2-dimensional image sensor array of a camera system while one or more IR illuminators of the camera system are emitting IR light, thereby forming an IR intensity map of the scene with a respective intensity value determined for each pixel of the IR image; using the IR intensity map to identify a plurality of pixels whose corresponding intensity values are below a threshold value corresponding to glass; clustering the identified plurality of pixels into one or more regions that are substantially contiguous; determining that a first region of the one or more regions corresponds to a glass surface based, at least in part, on a difference in area between the first region and a geometrical shape containing the first region being less than a threshold percentage of the geometrical shape's area; storing information in the memory that identifies the first region; receiving a video stream of the scene from the camera system; reviewing the video stream to detect movement in the scene, wherein the first region is excluded from movement detection; and generating a motion alert when there is motion detected at the scene outside of the first region. ### Claim 21 (depends on 20) The computer readable storage medium of claim 20, wherein the camera system has a plurality of IR illuminators, and forming an IR intensity map of the scene comprises: for each of a plurality of distinct subsets of IR illuminators of the camera system, receiving a respective IR sub-image of the scene while the respective subset of IR illuminators are emitting IR light and the IR illuminators not in the respective subset are not emitting IR light; wherein determining the respective intensity value for a respective pixel comprises computing an average of intensity values at the pixel in each of the sub-images. ### Claim 22 (depends on 20) The computer readable storage medium of claim 20, wherein clustering the identified plurality of pixels into one or more regions further comprises using a depth map that was constructed using the image sensor array. ### Claim 23 (depends on 20) The computer readable storage medium of claim 20, wherein clustering the identified plurality of pixels into one or more regions further comprises using an RGB image of the scene captured using the image sensor array. ### Claim 24 (depends on 20) The computer readable storage medium of claim 20, wherein each pixel of the IR image corresponds to a unique respective image sensor in the image sensor array. ### Claim 25 (depends on 20) The computer readable storage medium of claim 20, wherein the pixels of the IR image form a partition of the image sensors in the image sensor array and wherein at least one pixel corresponds to a plurality of image sensors in the image sensor array. ### Claim 26 (depends on 13) The computing device of claim 13, wherein the pixels of the IR image form a partition of the image sensors in the image sensor array and wherein at least one pixel corresponds to a plurality of image sensors in the image sensor array. ### Claim 27 (depends on 13) The computing device of claim 13, wherein the computing device is a server distinct from the camera system. ### Claim 28 (depends on 13) The computing device of claim 13, wherein the computing device is included in the camera system.