# US 2019/0377066 A1 · Tracking objects using video images and lidar

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2019/0377066 A1
- Document ID: US20190377066A1
- Status: Published application
- Kind: A1
- Application number: 16/431,493
- Filed: 2019-06-04
- Published: 2019-12-12
- Earliest claimed: 2018-06-06
- Assignees: DSCG Solutions, Inc.
- Applicants: Dscg Solutions, Inc.
- Inventors: Kendall Belsley; Richard Sebastian
- Web page: https://patentgrep.com/patent/US20190377066A1

## CPC

- G01S 7/4808: {Evaluating distance, position or velocity data}
- G01S 17/66: Tracking systems using electromagnetic waves other than radio waves
- G01S 7/4802: {using analysis of echo signal for target characterisation; Target signature; Target cross-section}
- G01S 7/4815: {using multiple transmitters}
- G01S 17/42: Simultaneous measurement of distance and other co-ordinates (indirect measurement G01S17/46)
- G01S 17/58: Velocity or trajectory determination systems; Sense-of-movement determination systems
- G01S 17/86: Combinations of lidar systems with systems other than lidar, radar or sonar, e.g. with direction finders
- G01S: RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES

## Abstract

A technique of monitoring and/or tracking an object includes scanning a set of beams of electromagnetic radiation in lines over the object; obtaining data indicating a velocity of the object and position of the object out of the plane as the beam of electromagnetic radiation is scanned over the object; and performing a correction operation to produce a correction to the absolute position in the plane of the object at a later time after an initial time, the correction being based on the obtained data.

## Claims

### Claim 1

A method comprising:
receiving, by processing circuitry, a video frame including (i) an image of an object located at an absolute position in a plane and (ii) a timestamp indicating an initial time at which the object was located at the absolute position in the plane;
scanning, by the processing circuitry, a set of beams of electromagnetic radiation in lines over the object;
obtaining, by the processing circuitry, data indicating a velocity of the object and position of the object out of the plane as the beam of electromagnetic radiation is scanned over the object;
performing, by the processing circuitry, a correction operation to produce a correction to the absolute position in the plane of the object at a later time after the initial time, the correction being based on the obtained data.

### Claim 2 (depends on 1)

The method as in claim 1,
wherein scanning the beam of electromagnetic radiation in lines over the object includes:
performing a first line scan in a first direction in the plane at a first time after the initial time;
performing a second line scan at a position normal to the first scan line in the plane, in a second direction at a second time, the second direction being opposite to the first direction; and
performing a third line scan at a position normal to the second line scan, in the first direction at a third time.

### Claim 3 (depends on 2)

The method as in claim 2,
wherein performing the correction operation includes:
performing a prediction operation on the first line scan to produce a predicted third line scan at the third time;
generating differences in position normal to the plane between points on the third line scan at the third time and a point on the predicted third line scan at the third time; and
producing, as the correction, the position along the first direction at which a difference in position along the beam direction is optimal.

### Claim 4 (depends on 3)

The method as in claim 3,
wherein producing the position along the first direction at which a difference in position normal to the plane is optimal includes:
performing an interpolation operation on the differences in position normal to the plane between points on the third line scan at the third time and a point on the predicted third line scan at the third time to produce a minimum difference and a position along the first direction at which the minimum difference is achieved.

### Claim 5 (depends on 3)

The method as in claim 3,
wherein performing the prediction operation includes:
producing a rotational speed of the object at a point on the first line scan normal to the plane;
generating a speed of the object at the point on the first line scan along the first direction based on the rotational speed of the object at the point; and
producing, as a point on the predicted third line scan, a sum of a position of the point on the first line scan plus a correction based on the speed of the object and a difference between the first time and the third time.

### Claim 6 (depends on 1)

The method as in claim 1, further comprising:
at a new time after the later time, receiving another video frame including (i) another image of an object located at another absolute position in another plane and (ii) a timestamp indicating the new time at which the object was located at the other absolute position in the other plane; and
forming a trajectory of the object in space based on the positions of the object at the at the initial time, the later time, and the new time.

### Claim 7

A computer program product comprising a nontransitory storage medium, the computer program product including code that, when executed by processing circuitry of a computer, causes the processing circuitry to perform a method, the method comprising:
receiving a video frame including (i) an image of an object located at an absolute position in a plane and (ii) a timestamp indicating an initial time at which the object was located at the absolute position in the plane;
scanning a set of beams of electromagnetic radiation in lines over the object;
obtaining data indicating data indicating a velocity of the object and position of the object out of the plane as the beam of electromagnetic radiation is scanned over the object;
performing a correction operation to produce a correction to the absolute position in the plane of the object at a later time after the initial time, the correction being based on the obtained data.

### Claim 8 (depends on 7)

The computer program product as in claim 7,
wherein scanning the beam of electromagnetic radiation in lines over the object includes:
performing a first line scan in a first direction in the plane at a first time after the initial time;
performing a second line scan at a position normal to the first scan line in the plane, in a second direction at a second time, the second direction being opposite to the first direction; and
performing a third line scan at a position normal to the second line scan, in the first direction at a third time.

### Claim 9 (depends on 8)

The computer program product as in claim 8,
wherein performing the correction operation includes:
performing a prediction operation on the first line scan to produce a predicted third line scan at the third time;
generating differences in position normal to the plane between points on the third line scan at the third time and a point on the predicted third line scan at the third time; and
producing, as the correction, the position along the first direction at which a difference in position along the beam direction is optimal.

### Claim 10 (depends on 9)

The computer program product as in claim 9,
wherein producing the position along the first direction at which a difference in position normal to the plane is optimal includes:
performing an interpolation operation on the differences in position normal to the plane between points on the third line scan at the third time and a point on the predicted third line scan at the third time to produce a minimum difference and a position along the first direction at which the minimum difference is achieved.

### Claim 11 (depends on 9)

The computer program product as in claim 9,
wherein performing the prediction operation includes:
producing a rotational speed of the object at a point on the first line scan normal to the plane;
generating a speed of the object at the point on the first line scan along the first direction based on the rotational speed of the object at the point; and
producing, as a point on the predicted third line scan, a sum of a position of the point on the first line scan plus a correction based on the speed of the object and a difference between the first time and the third time.

### Claim 12 (depends on 7)

The computer program product as in claim 7, wherein the method further comprises:
at a new time after the later time, receiving another video frame including (i) another image of an object located at another absolute position in another plane and (ii) a timestamp indicating the new time at which the object was located at the other absolute position in the other plane; and
forming a trajectory of the object in space based on the positions of the object at the at the initial time, the later time, and the new time.

### Claim 13

An electronic apparatus, comprising:
memory; and
controlling circuitry coupled to the memory, the controlling circuitry being configured to:
receive a video frame including (i) an image of an object located at an absolute position in a plane and (ii) a timestamp indicating an initial time at which the object was located at the absolute position in the plane;
scan a set of beams of electromagnetic radiation in lines over the object;
obtain data indicating data indicating a velocity of the object and position of the object out of the plane as the beam of electromagnetic radiation is scanned over the object;
perform a correction operation to produce a correction to the absolute position in the plane of the object at a later time after the initial time, the correction being based on the obtained data.

### Claim 14 (depends on 13)

The electronic apparatus as in claim 13,
wherein the controlling circuitry configured to scan the beam of electromagnetic radiation in lines over the object is further configured to:
perform a first line scan in a first direction in the plane at a first time after the initial time;
perform a second line scan at a position normal to the first scan line in the plane, in a second direction at a second time, the second direction being opposite to the first direction; and
perform a third line scan at a position normal to the second line scan, in the first direction at a third time.

### Claim 15 (depends on 14)

The electronic apparatus as in claim 14,
wherein the controlling circuitry configured to perform the correction operation is further configured to:
perform a prediction operation on the first line scan to produce a predicted third line scan at the third time;
generate differences in position normal to the plane between points on the third line scan at the third time and a point on the predicted third line scan at the third time; and
produce, as the correction, the position along the first direction at which a difference in position along the beam direction is optimal.

### Claim 16 (depends on 15)

The electronic apparatus as in claim 15,
wherein the controlling circuitry configured to produce the position along the first direction at which a difference in position normal to the plane is optimal is further configured to:
perform an interpolation operation on the differences in position normal to the plane between points on the third line scan at the third time and a point on the predicted third line scan at the third time to produce a minimum difference and a position along the first direction at which the minimum difference is achieved.

### Claim 17 (depends on 15)

The electronic apparatus as in claim 15,
wherein the controlling circuitry configured to perform the prediction operation is further configured to:
produce a rotational speed of the object at a point on the first line scan normal to the plane;
generate a speed of the object at the point on the first line scan along the first direction based on the rotational speed of the object at the point; and
produce, as a point on the predicted third line scan, a sum of a position of the point on the first line scan plus a correction based on the speed of the object and a difference between the first time and the third time.

### Claim 18 (depends on 13)

The electronic apparatus as in claim 13, wherein the controlling circuitry is further configured to:
at a new time after the later time, receive another video frame including (i) another image of an object located at another absolute position in another plane and (ii) a timestamp indicating the new time at which the object was located at the other absolute position in the other plane; and
form a trajectory of the object in space based on the positions of the object at the at the initial time, the later time, and the new time.
