# US 11,227,165 B2 · Automatic lighting and security device

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 11,227,165 B2
- Document ID: US11227165B2
- Status: Granted
- Kind: B2
- Application number: 16/067,920
- Filed: 2017-01-04
- Published: 2020-08-27
- Granted: 2022-01-18
- Earliest claimed: 2016-01-04
- Assignees: Netatmo
- Applicants: Netatmo
- Inventors: David Elvira; Mehdi Felhi; Frédéric Potter; Pauline Luc; Kelvin Moutet
- Web page: https://patentgrep.com/patent/US11227165B2

## CPC

- H05B 47/125: by using cameras
- G06F 18/22: Matching criteria, e.g. proximity measures
- G06F 18/2431: Multiple classes
- G06T 7/246: using feature-based methods, e.g. the tracking of corners or segments
- G06V 10/143: Sensing or illuminating at different wavelengths
- G06V 10/25: Determination of region of interest [ROI] or a volume of interest [VOI]
- G06V 20/52: Surveillance or monitoring of activities, e.g. for recognising suspicious objects (recognising microscopic objects G06V20/69)
- G06V 20/56: exterior to a vehicle by using sensors mounted on the vehicle
- G08B 13/19606: {Discriminating between target movement or movement in an area of interest and other non-signicative movements, e.g. target movements induced by camera shake or movements of pets, falling leaves, rotating fan}
- H04N 7/183: {for receiving images from a single remote source}
- H04N 23/56: provided with illuminating means
- G06T 2207/10048: Infrared image
- G06T 2207/20084: Artificial neural networks [ANN]
- G06T 2207/30232: Surveillance
- Y02B 20/40: Control techniques providing energy savings, e.g. smart controller or presence detection
- H05B: ELECTRIC HEATING; ELECTRIC LIGHT SOURCES NOT OTHERWISE PROVIDED FOR; CIRCUIT ARRANGEMENTS FOR ELECTRIC LIGHT SOURCES, IN GENERAL
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G08B: SIGNALLING SYSTEMS, e.g. PERSONAL CALLING SYSTEMS; ORDER TELEGRAPHS; ALARM SYSTEMS
- H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION
- Y02B: CLIMATE CHANGE MITIGATION TECHNOLOGIES RELATED TO BUILDINGS, e.g. HOUSING, HOUSE APPLIANCES OR RELATED END-USER APPLICATIONS

## Abstract

In a number of aspects, the invention discloses a device (100) comprising one or more infrared light sources (40, 41); one or more visible light sources (50, 51); an image sensor (10); a processing unit (20) configured to analyze a series of images of a region of interest output by the image sensor; a control unit (30) configured to generate one or more of an activation of the one or more visible light sources or an alarm based on a command received from the processing unit; wherein the analyze comprises detecting a moving foreground in the series of images of the region of interest, tracking one or more characterizing features in the moving foreground and classifying the one or more characterizing features into two of more types of objects of interest, a type of an object of interest determining a command sent by the processing unit to the control unit.

## Claims

### Claim 1

A device comprising:
one or more infrared light sources;
one or more visible light sources;
an image sensor;
a processing unit configured to analyze a series of images of a region of interest output by the image sensor;
a control unit configured to generate one or more of an activation of the one or more visible light sources or an alarm based on a command received from the processing unit;
wherein the analyzing comprises detecting a moving foreground in the series of images of the region of interest, tracking one or more characterizing features in the moving foreground and classifying the one or more characterizing features into two of more types of moving objects of interest, a type of a moving object of interest determining a command sent by the processing unit to the control unit; and
wherein the detecting comprises comparing difference images between the series of images of the region of interest and images of a Background Model of said region of interest to threshold images, the threshold images being calculated dynamically using a shift-variance, and the tracking comprises allocating a Good Feature value to a characterizing feature in Bounded Blocks output by the detecting if a counter of correlated movements of the characterizing feature is increased and a counter of immobility is equal to zero.

### Claim 2 (depends on 1)

The device of claim 1, further connected to a network through a communication link, wherein the alarm is sent to a user device on the communication link.

### Claim 3 (depends on 2)

The device of claim 2, wherein the control unit further triggers images of the moving object of interest to be sent to the user device on the communication link.

### Claim 4 (depends on 1)

The device of claim 1, wherein the detecting uses a Y luminance value in a YUV encoding of pixels in the series of images.

### Claim 5 (depends on 1)

The device of claim 1, wherein the tracking further comprises creating a vignette with a Bounded Block having a number of Good Features higher than a threshold.

### Claim 6 (depends on 5)

The device of claim 5, wherein the classifying comprises using one or more of a Neural Network, a History of Oriented Gradient or a Support Vector Machine classifier.

### Claim 7 (depends on 6)

The device of claim 6, wherein the Neural Network is a Convolutional Neural Network classifier and comprises one or more Fully Connected Layers.

### Claim 8 (depends on 1)

The device of claim 1, wherein:
the camera comprises a lens and an infrared light and visible light sensor,
the at least one infrared light source has a cone of emission of infrared light,
the at least one visible light source has a cone of emission of visible light,
there is no intersection between the lens of the camera on the one hand and the cones of emission of infrared light and of visible light on the other hand.

### Claim 9 (depends on 8)

The device of claim 8, wherein the lens has an axis of revolution A and comprises a first face having a normal vector oriented towards the sensor and a second face having a normal vector oriented towards the environment, and in which a first plane tangential to the first face and at right angles to the axis of revolution A defines a first half-space to which the sensor belongs and a second half-space to which the second face of the lens and the environment belong, and:
the cone of emission of the at least one infrared light source has a vertex arranged in the second half-space;
the cone of emission of the at least one visible light source has a vertex arranged in the second half-space.

### Claim 10 (depends on 9)

The device of claim 9, wherein:
the at least one infrared light source has a cone of emission of axis A40,
the at least one visible light source has a cone of emission of axis A50, and
the camera has a cone of absorption of axis A,
in the second half-space, the distance between the axis A40 and the axis A is constant or increasing when moving away from the first plane, and
in the second half-space, the distance between the axis A50 and the axis A is constant or increasing when moving away from the first plane.

### Claim 11 (depends on 5)

The device of claim 5, comprising a plurality of visible light sources, each visible light source having a cone of emission having an axis, wherein in the first half-space, for each visible light source of the plurality of visible light sources, the distance between the axis of the cone of emission of said source and the axis of the cone of emission of each other source is constant or increasing when moving away from the first plane.

### Claim 12 (depends on 5)

The device of claim 5, wherein the lens has an axis of revolution A, further comprising a protection element for the at least one infrared light source, for the at least one visible light source and for the lens of the camera, the protection element being transparent to the infrared light and to the visible light, the protection element extending substantially along a plane at right angles to the axis of revolution A.

### Claim 13

A method of monitoring a region of interest comprising:
lighting the region of interest with one or more infrared light sources;
capturing series of images of the region of interest by an image sensor;
analyzing by a processing unit the series of images of the region of interest output by the image sensor;
generating by a control unit one or more of an activation of one or more visible light sources or an alarm based on a command received from the processing unit;
wherein the analyzing comprises detecting a moving foreground in the series of images of the region of interest, tracking one or more characterizing features in the moving foreground and classifying the one or more characterizing features into two of more types of moving objects of interest, a type of a moving object of interest determining a command sent by the processing unit to the control unit; and
wherein the detecting comprises comparing difference images between the series of images of the region of interest and images of a Background Model of said region of interest to threshold images, the threshold images being calculated dynamically using a shift-variance, and the tracking comprises allocating a Good Feature value to a characterizing feature in Bounded Blocks output by the detecting if a counter of correlated movements of the characterizing feature is increased and a counter of immobility is equal to zero.

### Claim 14

A device comprising:
one or more infrared light sources;
one or more visible light sources;
an image sensor;
a processing unit configured to analyze a series of images of a region of interest output by the image sensor;
a control unit configured to generate one or more of an activation of the one or more visible light sources or an alarm based on a command received from the processing unit;
wherein the analyzing comprises detecting a moving foreground in the series of images of the region of interest, tracking one or more characterizing features in the moving foreground and classifying the one or more characterizing features into two of more types of moving objects of interest, a type of a moving object of interest determining a command sent by the processing unit to the control unit, wherein:
the camera comprises a lens and an infrared light and visible light sensor,
the at least one infrared light source has a cone of emission of infrared light,
the at least one visible light source has a cone of emission of visible light,
there is no intersection between the lens of the camera on the one hand and the cones of emission of infrared light and of visible light on the other hand.