# US 2022/0277162 A1 · Optical person recognition techniques for social distancing

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2022/0277162 A1
- Document ID: US20220277162A1
- Status: Published application
- Kind: A1
- Application number: 17/187,306
- Filed: 2021-02-26
- Published: 2022-09-01
- Earliest claimed: 2021-02-26
- Assignees: Kogniz, Inc.
- Applicants: Kogniz, Inc.
- Inventors: Jed Putterman; Daniel Putterman; Bill Klein
- Web page: https://patentgrep.com/patent/US20220277162A1

## CPC

- G06T 7/246: using feature-based methods, e.g. the tracking of corners or segments
- G06V 20/53: {Recognition of crowd images, e.g. recognition of crowd congestion}
- G01J 5/0025: {Living bodies (ear thermometers G01J5/0011; detecting, measuring or recording for diagnostic purposes A61B5/00)}
- G06N 3/0442: characterised by memory or gating, e.g. long short-term memory [LSTM] or gated recurrent units [GRU]
- G06N 3/045: Combinations of networks
- G06N 3/0464: Convolutional networks [CNN, ConvNet]
- G06N 3/09: Supervised learning
- G06T 7/251: {involving models}
- G06T 7/73: using feature-based methods
- G06V 40/173: {face re-identification, e.g. recognising unknown faces across different face tracks}
- G06V 40/23: {Recognition of whole body movements, e.g. for sport training}
- G01J 2005/0077: {Imaging}
- G06T 2207/10016: Video; Image sequence
- G06T 2207/10021: Stereoscopic video; Stereoscopic image sequence
- G06T 2207/30196: Human being; Person
- G06T 2207/30201: Face
- G06T 2207/30232: Surveillance
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G01J: MEASUREMENT OF INTENSITY, VELOCITY, SPECTRAL CONTENT, POLARISATION, PHASE OR PULSE CHARACTERISTICS OF INFRARED, VISIBLE OR ULTRAVIOLET LIGHT; COLORIMETRY; RADIATION PYROMETRY (light sources F21, H01J, H01K, H05B; investigating properties of materials by optical means G01N)
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS

## Abstract

Methods, systems, and devices for data processing and artificial intelligence (AI) techniques are described. A system may support person detection using an optical camera. For example, the optical camera may detect motion and the system may implement a neural network to determine if the motion corresponds to a person's body. If the system detects a person, the system may assign a tracker identifier (ID) to the person's body and convert the position of the body in the view of the camera to a position in a horizontal plane. Based on the horizontal view, the system may spatially track the person in an environment, including determining if the person interacts with other people within a threshold distance, whether there are any congestion points in the environment, or both. The system may trigger alert procedures based on the interactions, congestion points, temperature readings for one or more people, or any combination thereof.

## Claims

### Claim 1

A method for person detection and tracking, comprising:
detecting motion at an optical camera, the motion corresponding to a subset of pixels of a set of pixels for a view of the optical camera;
inputting information corresponding to at least the subset of pixels into a trained neural network;
obtaining, as an output of the trained neural network, an indication that at least a portion of the subset of pixels corresponds to a body;
assigning, in memory, a tracker identifier to the body;
converting from a first positioning of the body in the view of the optical camera to a second positioning of the body in a horizontal plane based at least in part on the subset of pixels and a non-linear matrix calculation; and
storing, in the memory and with an association to the tracker identifier, the second positioning of the body in the horizontal plane.

### Claim 2 (depends on 1)

The method of claim 1, further comprising:
determining a distance from the optical camera to the body, wherein converting from the first positioning of the body in the view of the optical camera to the second positioning of the body in the horizontal plane is based at least in part on the distance.

### Claim 3 (depends on 2)

The method of claim 2, wherein the distance is determined based at least in part on a first bounding box of pixels for a reference object in the view of the optical camera and a second bounding box of pixels for the body or a portion of the body, wherein the reference object has a first dimension and a second dimension defined in the memory.

### Claim 4 (depends on 2)

The method of claim 2, wherein the distance is determined by a stereoscopic sensor.

### Claim 5 (depends on 1)

The method of claim 1, further comprising:
determining a direction of the motion for the body across a plurality of frames of the optical camera; and
storing, in the memory and with a second association to the tracker identifier, the direction of the motion for the body.

### Claim 6 (depends on 5)

The method of claim 5, further comprising:
dynamically adjusting a storage granularity for the tracker identifier based at least in part on a rate of change for the second positioning of the body in the horizontal plane across the plurality of frames; and
storing, in the memory and with the association to the tracker identifier, a plurality of positions of the body in the horizontal plane according to the storage granularity for the tracker identifier, wherein the storage granularity corresponds to a number of data points that is less than a number of frames in the plurality of frames.

### Claim 7 (depends on 5)

The method of claim 5, further comprising:
determining a location in the horizontal plane associated with a number of bodies greater than a threshold number of bodies based at least in part on the stored second positioning of the body in the horizontal plane, the stored direction of the motion for the body, or both; and
sending, for display in a user interface, an indication of the location with a high traffic alert indicator.

### Claim 8 (depends on 7)

The method of claim 7, further comprising:
determining a rerouting suggestion for the horizontal plane based at least in part on the location and the high traffic alert indicator.

### Claim 9 (depends on 1)

The method of claim 1, further comprising:
generating a circle of a specific radius centered around the body in the horizontal plane; and
determining if a positioning of a second body corresponding to a second tracker identifier in the horizontal plane is within the generated circle for the body.

### Claim 10 (depends on 9)

The method of claim 9, further comprising:
triggering an alert procedure based at least in part on the positioning of the second body being within the generated circle for the body.

### Claim 11 (depends on 9)

The method of claim 9, further comprising:
sending, for display in a user interface, the second positioning of the body in the horizontal plane and the generated circle for the body.

### Claim 12 (depends on 1)

The method of claim 1, further comprising:
sending, for display in a user interface, the view of the optical camera;
receiving, from the user interface, a user input indicating an area of importance, a boundary threshold, or both in the view of the optical camera; and
triggering an action based at least in part on the body being within the area of importance, the body crossing the boundary threshold, or both.

### Claim 13 (depends on 12)

The method of claim 12, wherein triggering the action comprises:
performing a temperature reading on at least a portion of the body using a thermal sensor aligned with the optical camera.

### Claim 14 (depends on 1)

The method of claim 1, further comprising:
inputting second information corresponding to at least a second subset of the subset of pixels into a second trained neural network, wherein the second subset of the subset of pixels is based at least in part on a portion of the body; and
obtaining, as a second output of the second trained neural network, a second indication that at least a portion of the second subset of the subset of pixels corresponds to a face.

### Claim 15 (depends on 14)

The method of claim 14, further comprising:
classifying one or more features of the face based at least in part on one or more additional neural networks, wherein the one or more features of the face comprise whether the face is wearing a mask, whether the face is wearing glasses, whether the face is wearing a hat, whether the face corresponds to a known face stored in the memory, or a combination thereof.

### Claim 16

An apparatus for person detection and tracking, comprising:
a processor;
memory coupled with the processor; and
instructions stored in the memory and executable by the processor to cause the apparatus to:
detect motion at an optical camera, the motion corresponding to a subset of pixels of a set of pixels for a view of the optical camera;
input information corresponding to at least the subset of pixels into a trained neural network;
obtain, as an output of the trained neural network, an indication that at least a portion of the subset of pixels corresponds to a body;
assign, in the memory, a tracker identifier to the body;
convert from a first positioning of the body in the view of the optical camera to a second positioning of the body in a horizontal plane based at least in part on the subset of pixels and a non-linear matrix calculation; and
store, in the memory and with an association to the tracker identifier, the second positioning of the body in the horizontal plane.

### Claim 17 (depends on 16)

The apparatus of claim 16, wherein the instructions are further executable by the processor to cause the apparatus to:
determine a distance from the optical camera to the body, wherein converting from the first positioning of the body in the view of the optical camera to the second positioning of the body in the horizontal plane is based at least in part on the distance.

### Claim 18 (depends on 16)

The apparatus of claim 16, wherein the instructions are further executable by the processor to cause the apparatus to:
determine a direction of the motion for the body across a plurality of frames of the optical camera; and
store, in the memory and with a second association to the tracker identifier, the direction of the motion for the body.

### Claim 19 (depends on 18)

The apparatus of claim 18, wherein the instructions are further executable by the processor to cause the apparatus to:
dynamically adjust a storage granularity for the tracker identifier based at least in part on a rate of change for the second positioning of the body in the horizontal plane across the plurality of frames; and
store, in the memory and with the association to the tracker identifier, a plurality of positions of the body in the horizontal plane according to the storage granularity for the tracker identifier, wherein the storage granularity corresponds to a number of data points that is less than a number of frames in the plurality of frames.

### Claim 20

A non-transitory computer-readable medium storing code for person detection and tracking, the code comprising instructions executable by a processor to:
detect motion at an optical camera, the motion corresponding to a subset of pixels of a set of pixels for a view of the optical camera;
input information corresponding to at least the subset of pixels into a trained neural network;
obtain, as an output of the trained neural network, an indication that at least a portion of the subset of pixels corresponds to a body;
assign, in memory, a tracker identifier to the body;
convert from a first positioning of the body in the view of the optical camera to a second positioning of the body in a horizontal plane based at least in part on the subset of pixels and a non-linear matrix calculation; and
store, in the memory and with an association to the tracker identifier, the second positioning of the body in the horizontal plane.