# US 2020/0356826 A1 · Individual identification and tracking via combined video and lidar systems

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2020/0356826 A1
- Document ID: US20200356826A1
- Status: Published application
- Kind: A1
- Application number: 16/941,214
- Filed: 2020-07-28
- Published: 2020-11-12
- Earliest claimed: 2018-08-05
- Assignees: COM-IoT Technologies
- Applicants: COM-IoT Technologies
- Inventors: Haitham Khedr; Ahmed Madkor
- Web page: https://patentgrep.com/patent/US20200356826A1

## CPC

- G01S 17/89: for mapping or imaging
- G06F 18/251: {of input or preprocessed data}
- G06T 7/292: Multi-camera tracking
- G06T 7/521: from laser ranging, e.g. using interferometry; from the projection of structured light
- G06T 7/80: Analysis of captured images to determine intrinsic or extrinsic camera parameters, i.e. camera calibration
- G06V 20/52: Surveillance or monitoring of activities, e.g. for recognising suspicious objects (recognising microscopic objects G06V20/69)
- G06V 20/64: Three-dimensional [3D] objects
- G06V 40/103: {Static body considered as a whole, e.g. static pedestrian or occupant recognition}
- G06V 40/161: {Detection; Localisation; Normalisation}
- G06V 40/172: {Classification, e.g. identification}
- G06T 2207/10028: Range image; Depth image; 3D point clouds
- G06T 2207/30196: Human being; Person
- G06T 2207/30201: Face
- G01S: RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING

## Abstract

Individual identification and tracking are provided via combined video and LiDAR systems. In various embodiments, a virtual plane may be generated. A video frame including the virtual plane is recorded via a first imaging modality. One or more objects are detected in the video frame when the one or more objects enters the virtual plane. An identifier is assigned to each of the one or more objects. One or more three-dimensional shapes are detected at the second imaging modality when the one or more three-dimensional shapes enters the virtual plane. For each of the one or more objects, a corresponding shape of the one or more three-dimensional shapes is determined. Each identifier is assigned to the respective corresponding shape of the one or more three-dimensional shapes. After assigning each identifier, a plurality of positional data is recorded for each of the one or more three-dimensional shapes.

## Claims

### Claim 1

A system for data continuity across multiple imaging modalities, the system comprising:
a first imaging modality;
a second imaging modality;
a computing node comprising a computer readable storage medium having program instructions embodied therewith, the program instructions executable by a processor of the computing node to cause the processor to perform a method comprising:
generating a virtual plane disposed at a first distance from the first imaging modality and a second distance from the second imaging modality;
recording a video frame via the first imaging modality, the video frame comprising the virtual plane;
detecting, at the first imaging modality, one or more objects in the video frame when the one or more objects enters the virtual plane;
assigning an identifier to each of the one or more objects;
detecting, at the second imaging modality, one or more three-dimensional shapes when the one or more three-dimensional shapes enters the virtual plane;
for each of the one or more objects, determining a corresponding shape of the one or more three-dimensional shapes;
assigning each identifier to the respective corresponding shape of the one or more three-dimensional shapes; and
after assigning each identifier, recording a plurality of positional data for each of the one or more three-dimensional shapes.

### Claim 2 (depends on 1)

The system according to claim 1, wherein the first imaging modality comprises a digital camera.

### Claim 3 (depends on 1)

The system according to claim 1, wherein the second imaging modality comprises LiDAR.

### Claim 4 (depends on 1)

The system according to claim 1, wherein first distance is greater than the second distance.

### Claim 5 (depends on 1)

The system according to claim 1, wherein first distance is less than the second distance.

### Claim 6 (depends on 1)

The system according to claim 1, wherein first and second distances are the same.

### Claim 7 (depends on 1)

The system according to claim 1, wherein at least one of the first distance and the second distance is adjustable.

### Claim 8 (depends on 1)

The system according to claim 1, wherein at least one of the first distance and the second distance is fixed.

### Claim 9 (depends on 1)

The system according to claim 1, wherein the virtual plane is a two-dimensional plane.

### Claim 10 (depends on 1)

The system according to claim 1, wherein the virtual plane is a three-dimensional surface.

### Claim 11 (depends on 1)

The system according to claim 1, wherein virtual plane is substantially perpendicular to the ground.

### Claim 12 (depends on 1)

The system according to claim 1, wherein each identifier comprises a name.

### Claim 13 (depends on 1)

The system according to claim 1, wherein each identifier is anonymized.

### Claim 14 (depends on 1)

The system according to claim 1, wherein each identifier comprises a serial number.

### Claim 15 (depends on 1)

The system according to claim 1, further comprising calibrating the first imaging modality and the second imaging modality based on the first distance and the second distance.

### Claim 16 (depends on 1)

The system according to claim 1, wherein the one or more objects are detected using a facial recognition algorithm.

### Claim 17 (depends on 1)

The system according to claim 1, wherein the detecting one or more objects and detecting one or more three-dimensional shapes is performed substantially simultaneously.

### Claim 18

A method for data continuity across multiple imaging modalities, the method comprising:
generating a virtual plane disposed at a first distance from a first imaging modality and a second distance from a second imaging modality;
recording a video frame via the first imaging modality, the video frame comprising the virtual plane;
detecting, at the first imaging modality, one or more objects in the video frame when the one or more objects enters the virtual plane;
assigning an identifier to each of the one or more objects;
detecting, at the second imaging modality, one or more three-dimensional shapes when the one or more three-dimensional shapes enters the virtual plane;
for each of the one or more objects, determining a corresponding shape of the one or more three-dimensional shapes;
assigning each identifier to the respective corresponding shape of the one or more three-dimensional shapes; and
after assigning each identifier, recording a plurality of positional data for each of the one or more three-dimensional shapes.

### Claim 19

A computer program product for data continuity across multiple imaging modalities, the computer program product comprising a computer readable storage medium having program instructions embodied therewith, the program instructions executable by a processor to cause the processor to perform a method comprising:
generating a virtual plane disposed at a first distance from a first imaging modality and a second distance from a second imaging modality;
recording a video frame via the first imaging modality, the video frame comprising the virtual plane;
detecting, at the first imaging modality, one or more objects in the video frame when the one or more objects enters the virtual plane;
assigning an identifier to each of the one or more objects;
detecting, at the second imaging modality, one or more three-dimensional shapes when the one or more three-dimensional shapes enters the virtual plane;
for each of the one or more objects, determining a corresponding shape of the one or more three-dimensional shapes;
assigning each identifier to the respective corresponding shape of the one or more three-dimensional shapes; and
after assigning each identifier, recording a plurality of positional data for each of the one or more three-dimensional shapes.
