# US 2020/0394804 A1 · Analysis and deep learning modeling of sensor-based object detection data in bounded aquatic environments

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2020/0394804 A1
- Document ID: US20200394804A1
- Status: Published application
- Kind: A1
- Application number: 16/443,866
- Filed: 2019-06-17
- Published: 2020-12-17
- Earliest claimed: 2019-06-17
- Assignees: Guard, Inc.
- Inventors: Chris Barton; Srinivasa Narasimhan; Nigel Morris; Nichole Suzanne Rouillac; Robin Nicholas Hubbard; Jonathan Chei-Feung Lau
- Web page: https://patentgrep.com/patent/US20200394804A1

## CPC

- G06T 7/80: Analysis of captured images to determine intrinsic or extrinsic camera parameters, i.e. camera calibration
- G06N 3/0464: Convolutional networks [CNN, ConvNet]
- G06N 3/08: Learning methods
- G06N 3/09: Supervised learning
- G06T 7/11: Region-based segmentation
- G06T 7/194: involving foreground-background segmentation
- G06T 7/20: Analysis of motion (motion estimation for coding, decoding, compressing or decompressing digital video signals H04N19/43, H04N19/51)
- G06T 7/521: from laser ranging, e.g. using interferometry; from the projection of structured light
- G06T 7/70: Determining position or orientation of objects or cameras (camera calibration G06T7/80)
- G06V 10/145: Illumination specially adapted for pattern recognition, e.g. using gratings
- G06V 10/764: using classification, e.g. of video objects
- G06V 10/82: using neural networks
- G06V 20/52: Surveillance or monitoring of activities, e.g. for recognising suspicious objects (recognising microscopic objects G06V20/69)
- G06V 40/20: Movements or behaviour, e.g. gesture recognition (recognition of facial expressions G06V40/16)
- G08B 21/08: responsive to the presence of persons in a body of water, e.g. a swimming pool; responsive to an abnormal condition of a body of water
- G06T 2207/10052: Images from lightfield camera
- G06T 2207/30196: Human being; Person
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G08B: SIGNALLING SYSTEMS, e.g. PERSONAL CALLING SYSTEMS; ORDER TELEGRAPHS; ALARM SYSTEMS

## Abstract

Techniques for analysis and deep learning modeling of sensor-based object detection data in bounded aquatic environments are described, including capturing an image from a sensor disposed substantially above a waterline, the sensor being housed in a structure electrically coupled to a light housing, converting the image into data, the data being digitally encoded, evaluating the data to separate background data from foreground data, generating tracking data from the data after the background data is removed, the tracking data being evaluated to determine whether a head or a body are detected by comparing the tracking data to classifier data, tracking the head or the body relative to the waterline if the head or the body are detected in the tracking data, and determining a state associated with the head or the body.

## Claims

### Claim 1

A method, comprising:
capturing an image from a sensor disposed proximate to a waterline, the sensor being housed in a structure electrically coupled to a light;
converting the image into data, the data being digitally encoded by a processor in electronic communication with the sensor;
evaluating the data to separate background data from foreground data;
generating tracking data from the data after the background data is removed, the tracking data being evaluated to determine whether a head or a body are detected by comparing the tracking data to classifier data;
tracking the head or the body relative to the waterline if the head or the body are detected in the tracking data; and
determining a state associated with the head or the body, if the head or the body is detected, the state being associated with state data, the state data being used to determine a drowning state.

### Claim 2 (depends on 1)

The method of claim 1, wherein the background data is associated with one or more boundaries of a body of water in which the sensor is disposed proximate to the waterline.

### Claim 3 (depends on 1)

The method of claim 1, further comprising calibrating the sensor to align a field of view associated with the sensor.

### Claim 4 (depends on 1)

The method of claim 1, further comprising performing an initial calibration of the sensor, the initial calibration being performed to identify one or more image regions of a body of water.

### Claim 5 (depends on 1)

The method of claim 1, wherein the classifier data is generated by a model comprising a plurality of datasets collected from one or more data sources.

### Claim 6 (depends on 1)

The method of claim 1, wherein the sensor comprises a camera.

### Claim 7 (depends on 1)

The method of claim 1, wherein the sensor comprises a camera and a polarization filter.

### Claim 8 (depends on 1)

The method of claim 1, wherein the sensor uses lidar to capture the image.

### Claim 9 (depends on 1)

The method of claim 1, wherein an optical feature is implemented with a camera configured to capture the image.

### Claim 10 (depends on 1)

The method of claim 1, wherein the image is captured using a camera implemented with a spectral filter.

### Claim 11 (depends on 1)

The method of claim 1, wherein the image is captured using a camera implemented with a polarization filter.

### Claim 12 (depends on 1)

The method of claim 1, wherein the sensor comprises a transmitter configured to project a light curtain over a wide angle field above the waterline.

### Claim 13 (depends on 1)

The method of claim 1, wherein the sensor is configured to detect an object if one or more beams associated with a light curtain are interrupted.

### Claim 14 (depends on 1)

The method of claim 1, wherein a timer is started when the head is detected, based on the comparing the tracking data to the classifier data, to be substantially below the waterline.

### Claim 15 (depends on 1)

The method of claim 1, wherein the drowning state is determined when the head or the body is substantially floating and no motion has been detected during a period of time.

### Claim 16

A system, comprising:
a database configured to store data associated with an image captured by a sensor disposed proximate to a waterline associated with a body of water; and
a processor configured to capture an image from the sensor, the sensor being housed in a structure electrically coupled to a light, to convert the image into data, the data being digitally encoded by a processor in electronic communication with the sensor, to evaluate the data to separate background data from foreground data, to generate tracking data from the data after the background data is removed, the tracking data being evaluated to determine whether a head or a body are detected by comparing the tracking data to classifier data, to track the head or the body relative to the waterline if the head or the body are detected in the tracking data, and to determine a state associated with the head or the body, if the head or the body is detected, the state being associated with state data, the state data being used to determine a drowning state.

### Claim 17 (depends on 16)

The system of claim 16, wherein the sensor is in data communication with a hub.

### Claim 18 (depends on 16)

The system of claim 16, wherein the sensor is in data communication with a hub, the hub being configured to generate an alarm if the drowning state is determined.

### Claim 19 (depends on 16)

The system of claim 16, wherein the sensor is configured to sense a field of view of substantially 180 degrees relative to a vertical axis and 180 degrees relative to a horizontal axis above and below the waterline, respectively.

### Claim 20

A non-transitory computer readable medium having one or more computer program instructions configured to perform a method, the method comprising:
capturing an image from a sensor disposed proximate to a waterline, the sensor being housed in a structure electrically coupled to a light;
converting the image into data, the data being digitally encoded by a processor in electronic communication with the sensor;
evaluating the data to separate background data from foreground data;
generating tracking data from the data after the background data is removed, the tracking data being evaluated to determine whether a head or a body are detected by comparing the tracking data to classifier data;
tracking the head or the body relative to the waterline if the head or the body are detected in the tracking data; and
determining a state associated with the head or the body, if the head or the body is detected, the state being associated with state data, the state data being used to determine a drowning state.
