# US 2021/0342602 A1 · System and method for lateral vehicle detection

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2021/0342602 A1
- Document ID: US20210342602A1
- Status: Published application
- Kind: A1
- Application number: 17/377,206
- Filed: 2021-07-15
- Published: 2021-11-04
- Earliest claimed: 2018-03-18
- Assignees: Tusimple, Inc.
- Applicants: Tusimple, Inc.
- Inventors: Zhipeng YAN; Lingting GE; Pengfei CHEN; Panqu WANG
- Web page: https://patentgrep.com/patent/US20210342602A1

## CPC

- G08G 1/166: {for active traffic, e.g. moving vehicles, pedestrians, bikes}
- G06T 3/18: Image warping, e.g. rearranging pixels individually
- G06T 7/73: using feature-based methods
- G06V 10/443: {by matching or filtering}
- G06V 10/7515: {Shifting the patterns to accommodate for positional errors}
- G06V 10/757: {Matching configurations of points or features}
- G06V 20/56: exterior to a vehicle by using sensors mounted on the vehicle
- G06V 20/58: Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads
- G08G 1/04: using optical or ultrasonic detectors
- G08G 1/167: {Driving aids for lane monitoring, lane changing, e.g. blind spot detection}
- G05D 1/0055: {with safety arrangements}
- G05D 1/0246: {using a video camera in combination with image processing means}
- G05D 1/617: Safety or protection, e.g. defining protection zones around obstacles or avoiding hazards (arrangements for controlling the position or course of two or more vehicles for avoiding collisions therebetween G05D1/693; arrangements for reacting to or preventing system or operator failure G05D1/80)
- G05D 2101/20: using external object recognition
- G06T 2207/30248: Vehicle exterior or interior
- G08G: TRAFFIC CONTROL SYSTEMS (guiding railway traffic, ensuring the safety of railway traffic B61L; arrangement of road signs or traffic signals E01F9/00; radar or analogous systems, sonar systems, lidar systems specially adapted for traffic control G01S13/91, G01S15/88, G01S17/88; {radar or analogous systems, sonar systems, lidar systems specially adapted for anti-collision purposes G01S13/93, G01S15/93, G01S17/93})
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G05D: SYSTEMS FOR CONTROLLING OR REGULATING NON-ELECTRIC VARIABLES

## Abstract

A system and method for lateral vehicle detection is disclosed. A particular embodiment can be configured to: receive lateral image data from at least one laterally-facing camera associated with an autonomous vehicle; warp the lateral image data based on a line parallel to a side of the autonomous vehicle; perform object extraction on the warped lateral image data to identify extracted objects in the warped lateral image data; and apply bounding boxes around the extracted objects.

## Claims

### Claim 1

A system comprising:
a data processor; and
a memory for storing an autonomous lateral vehicle detection system, executable by the data processor, the autonomous lateral vehicle detection system being configured to:
receive image data from at least one camera installed on an autonomous vehicle;
warp the image data based on an orientation of the autonomous vehicle and an installation orientation of the at least one camera relative to the orientation of the autonomous vehicle; and
extract an object from the warped image data.

### Claim 2 (depends on 1)

The system of claim 1 wherein the at least one camera comprises a plurality of cameras, wherein the autonomous lateral vehicle system is further configured to:
identify matching portions of extracted features from the warped image data from different ones of the plurality of cameras;
stitch together the warped image data based on the matching portions of the extracted features;
form a bounding box based on the matching portions of the extracted object; and
apply the bounding box around the extracted object in the warped image data.

### Claim 3 (depends on 2)

The system of claim 2 wherein applying the bounding box around the extracted object in the warped image data comprises:
applying portions of the bounding box respectively, on the warped image data from different ones of the plurality of cameras, wherein
forming the bounding box based on the matching portions of the extracted object comprises:
stitching together the portions of the bounding box based on the matching portions of the extracted object.

### Claim 4 (depends on 1)

The system of claim 1 wherein the at least one camera is at least one laterally-facing camera associated with the autonomous vehicle, wherein the at least one laterally-facing camera is from the group consisting of: a forward laterally-facing camera and a backward laterally-facing camera.

### Claim 5 (depends on 1)

The system of claim 1 wherein the autonomous lateral vehicle system is further configured to warp the image data based on a line defined with configuration parameters corresponding to the installation orientation of the at least one camera relative to the autonomous vehicle.

### Claim 6 (depends on 5)

The system of claim 5 wherein the line is parallel to a side of the autonomous vehicle defined with the configuration parameters.

### Claim 7 (depends on 6)

The system of claim 6 wherein the line parallel to the side of the autonomous vehicle is configured based on an installation of the at least one camera, wherein the configuration parameters further correspond to an orientation of a bottom edge of the image data.

### Claim 8

A method comprising:
receiving image data from at least one camera installed on an autonomous vehicle;
warping the image data based on an orientation of the autonomous vehicle and an installation orientation of the at least one camera relative to the orientation of the autonomous vehicle; and
extracting an object from the warped image data.

### Claim 9 (depends on 8)

The method of claim 8 wherein the warped image data creates a trapezoidal image.

### Claim 10 (depends on 8)

The method of claim 8 further comprising:
applying a rectangular bounding box around the extracted object in the warped image data.

### Claim 11 (depends on 8)

The method of claim 8 wherein the at least one camera comprises a plurality of cameras, wherein the method further comprises:
identifying matching portions of extracted features from the warped image data from different ones of the plurality of cameras; and
stitching together the warped image data based on the matching portions of the extracted features, wherein the stitched image data forms a single contiguous image representing a stitched image from the plurality of cameras.

### Claim 12 (depends on 11)

The method of claim 11 further comprising:
forming a bounding box based on the matching portions of the extracted object; and
applying the bounding box around the extracted object in the warped image data.

### Claim 13 (depends on 8)

The method of claim 8 wherein the at least one camera comprises a plurality of cameras, wherein the method further comprises:
identifying matching portions of extracted features from the warped image data from different ones of the plurality of cameras; and
combining the warped image data based on the matching portions of the extracted features, wherein the combined image data forms a single contiguous image representing a combined image from the plurality of cameras.

### Claim 14 (depends on 8)

The method of claim 8 further comprising:
applying a bounding box around the extracted object in the warped image data, wherein a number of the extracted object is at least two, wherein a number of the bounding box is at least two, wherein each of the bounding boxes corresponds to each of the extracted objects respectively.

### Claim 15

A non-transitory machine-useable storage medium embodying instructions which, when executed by a machine, cause the machine to:
receive image data from at least one camera installed on an autonomous vehicle;
warp the image data based on an orientation of the autonomous vehicle and an installation orientation of the at least one camera relative to the orientation of the autonomous vehicle; and
extract an object from the warped image data.

### Claim 16 (depends on 15)

The non-transitory machine-useable storage medium of claim 15 wherein the object in the warped image data is aligned with the orientation of the autonomous vehicle.

### Claim 17 (depends on 15)

The non-transitory machine-useable storage medium of claim 15 wherein the at least one camera comprises a plurality of cameras, wherein the non-transitory machine-useable storage medium embodying instruction further causes the machine to:
stitch together the warped image data from different ones of the plurality of cameras to form a single image, wherein a bounding box is applied around the extracted object in the single image.

### Claim 18 (depends on 17)

The non-transitory machine-useable storage medium of claim 17 wherein the bounding box is trapezoidal.

### Claim 19 (depends on 17)

The non-transitory machine-useable storage medium of claim 17 wherein the single image is modified to be rectangular.

### Claim 20 (depends on 15)

The non-transitory machine-useable storage medium of claim 15 wherein a two-dimensional (2D) bounding box is applied around the extracted object in the warped image data.
