# US 2021/0241637 A1 · Collision awareness using cameras mounted on a vehicle

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2021/0241637 A1
- Document ID: US20210241637A1
- Status: Published application
- Kind: A1
- Application number: 16/778,109
- Filed: 2020-01-31
- Published: 2021-08-05
- Earliest claimed: 2020-01-31
- Assignees: Honeywell International Inc.
- Applicants: Honeywell International Inc.
- Inventors: Sunitha Panchangam; Anoop Surendran; Nathan Krishnamoorthy; Lawrence J. Surace
- Web page: https://patentgrep.com/patent/US20210241637A1

## CPC

- G08G 5/80: Anti-collision systems
- B64D 45/00: Aircraft indicators or protectors not otherwise provided for
- G06V 20/52: Surveillance or monitoring of activities, e.g. for recognising suspicious objects (recognising microscopic objects G06V20/69)
- G08G 5/25: Transmission of traffic-related information between aircraft
- G08G 5/51: for control when on the ground, e.g. taxiing or rolling
- G08G 5/723: {from the aircraft}
- H04N 23/54: Mounting of pick-up tubes, electronic image sensors, deviation or focusing coils
- G08G: TRAFFIC CONTROL SYSTEMS (guiding railway traffic, ensuring the safety of railway traffic B61L; arrangement of road signs or traffic signals E01F9/00; radar or analogous systems, sonar systems, lidar systems specially adapted for traffic control G01S13/91, G01S15/88, G01S17/88; {radar or analogous systems, sonar systems, lidar systems specially adapted for anti-collision purposes G01S13/93, G01S15/93, G01S17/93})
- B64D: EQUIPMENT FOR FITTING IN OR TO AIRCRAFT; FLIGHT SUITS; PARACHUTES; ARRANGEMENT OR MOUNTING OF POWER PLANTS OR PROPULSION TRANSMISSIONS IN AIRCRAFT
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION

## Abstract

In some examples, a collision awareness system includes two cameras mounted on portions of a vehicle and processing circuitry configured to determine a position of an object based on an image captured by a first camera when the object is within a field of view of the first camera. The processing circuitry is further configured to determine the position of the object based on an image captured by a second camera when the object is within a field of view of the second camera. The processing circuitry is also configured to determine whether a distance between a future position of the vehicle and a current or future position of the object is less than a threshold level. In response to determine that the distance is less than the threshold level, the processing circuitry can generate an alert.

## Claims

### Claim 1

A collision awareness system for predicting a collision between an ownship aircraft and a second aircraft during ground operations, the collision awareness system comprising:
a first camera mounted on a first portion of the ownship aircraft;
a second camera mounted on a second portion of the ownship aircraft; and
processing circuitry configured to:
in response to determining that the second aircraft is within a field of view of the first camera, determine a velocity of the second aircraft and a current position of the second aircraft based on one or more images captured by the first camera;
in response to determining that the second aircraft is within a field of view of the second camera and not within the field of view of the first camera, determine the velocity of the second aircraft and the current position of the second aircraft based on one or more images captured by the second camera;
in response to determining that the second aircraft is not within the field of view of the first camera or the field of view of the second camera, determine the current position of the second aircraft based on a surveillance message;
determine a future position of the second aircraft based on the velocity of the second aircraft and the current position of the second aircraft;
determine that a distance between a future position of the ownship aircraft and the future position of the second aircraft is less than a threshold level; and
generate an alert in response to determining that the distance is less than the threshold level.

### Claim 2 (depends on 1)

The collision awareness system of claim 1, wherein the processing circuitry is configured to:
determine the current position of the second aircraft based on a data source;
determine a confidence factor based on the data source; and
determine the threshold level for the distance based on the confidence factor.

### Claim 3 (depends on 2)

The collision awareness system of claim 2, wherein the processing circuitry is configured to determine the confidence factor by:
determining the confidence factor having a first value in response to determining that the second aircraft is within the field of view of the first camera, and
determining the confidence factor having a second value in response to determining that the second aircraft is not within the field of view of the first camera or the field of view of the second camera, the second value indicating a lower level of confidence than the first value.

### Claim 4 (depends on 3)

The collision awareness system of claim 3, wherein the processing circuitry is further configured to generate the alert by:
generating a first audio alert or presenting a first visual alert based on the confidence factor having the first value; and
generating a second audio alert or presenting a second visual alert based on the confidence factor having the second value,
a tone of the second audio alert being different than a tone of the first audio alert, and a color of the second visual alert being different than a color of the first visual alert.

### Claim 5 (depends on 1)

The collision awareness system of claim 1, wherein the processing circuitry is further configured to suppress generation of the alert in response to determining that the distance is not less than the threshold level.

### Claim 6 (depends on 1)

The collision awareness system of claim 1,
wherein the processing circuitry is configured to receive the surveillance message from the second aircraft, and
wherein the surveillance message indicates the current position of the second aircraft, the velocity of the second aircraft, and a type of the second aircraft.

### Claim 7 (depends on 1)

The collision awareness system of claim 1, wherein the processing circuitry is further configured to determine the threshold level based on a size of the ownship aircraft and further based on a size of the second aircraft.

### Claim 8 (depends on 7)

The collision awareness system of claim 7, wherein the processing circuitry is further configured to determine the size of the second aircraft based on a surveillance message received from the second aircraft.

### Claim 9 (depends on 1)

The collision awareness system of claim 1,
wherein the processing circuitry includes:
first processing circuitry colocated with the first camera on the first portion of the ownship aircraft;
second processing circuitry colocated with the second camera on the second portion of the ownship aircraft; and
third processing circuitry located in a cockpit of the ownship aircraft, wherein the first processing circuitry is configured to:
determine the current position of the second aircraft based on one or more images captured by the first camera in response to determining that the second aircraft is within the field of view of the first camera; and
cause a first transmitter colocated with the first camera to transmit the current position of the second aircraft to the third processing circuitry, and wherein the second processing circuitry is configured to:
determine the current position of the second aircraft based on one or more images captured by the second camera in response to determining that the second aircraft is within the field of view of the second camera; and
cause a second transmitter colocated with the second camera to transmit the current position of the second aircraft to the third processing circuitry.

### Claim 10 (depends on 1)

The collision awareness system of claim 1, wherein the processing circuitry is configured to generate the alert by presenting the current position of the second aircraft to a flight crew of the aircraft.

### Claim 11 (depends on 1)

The collision awareness system of claim 1,
wherein the processing circuitry is further configured to determine a predicted time to collision between the ownship aircraft and the second aircraft, and
wherein the processing circuitry is configured to generate the alert by presenting the predicted time to collision to a flight crew of the aircraft.

### Claim 12 (depends on 1)

The collision awareness system of claim 1, wherein the processing circuitry is configured to determine the future position of the second aircraft by:
determining that the second aircraft has started a turning maneuver based on the one or more images captured by the first camera or the second camera;
determining a predicted extent of the turning maneuver based on the one or more images captured by the first camera or the second camera; and
determining the future position of the second aircraft based on the predicted extent of the turning maneuver.

### Claim 13

A collision awareness method for predicting a collision between an ownship aircraft and a second aircraft during ground operations, the collision awareness method comprising:
determining, in a first instance, that the other aircraft is within a field of view of a first camera mounted on a first portion of the ownship aircraft;
determining, in the first instance and in response to determining that the second aircraft is within the field of view of the first camera, a velocity of the second aircraft and a current position of the second aircraft based on one or more images captured by the first camera;
determining, in a second instance, that the other aircraft is within a field of view of a second camera mounted on a second portion of the ownship aircraft and not within the field of view of the first camera;
determining, in the second instance and in response to determining that the second aircraft is within the field of view of the second camera and not within the field of view of the first camera, the velocity of the second aircraft and the current position of the second aircraft based on one or more images captured by the second camera;
determining, in a third instance, that the second aircraft is not within the field of view of the first camera or the field of view of the second camera;
determining, in the third instance and in response to determining that the second aircraft is not within the field of view of the first camera or the field of view of the second camera, the current position of the second aircraft based on a surveillance message;
determining a future position of the second aircraft based on the velocity of the second aircraft and the current position of the second aircraft;
determining that a distance between a future position of the ownship aircraft and the future position of the second aircraft is less than a threshold level; and
generating an alert in response to determining that the distance is less than the threshold level.

### Claim 14 (depends on 13)

The collision awareness method of claim 13, wherein determining the current position of the second aircraft comprises determining the current position of the second aircraft based on a data source, the collision awareness method further comprising:
determining a confidence factor based on the data source; and
determining the threshold level for the distance based on the confidence factor.

### Claim 15 (depends on 14)

The collision awareness method of claim 14, wherein determining the confidence factor comprises:
determining the confidence factor having a first value in response to determining that the second aircraft is within the field of view of the first camera; and
determining the confidence factor having a second value in response to determining that the second aircraft is not within the field of view of the first camera or the field of view of the second camera, the second value indicating a lower level of confidence than the first value.

### Claim 16 (depends on 13)

The collision awareness method of claim 13, further comprising suppressing generation of the alert in response to determining that the distance is not less than the threshold level.

### Claim 17 (depends on 13)

The collision awareness method of claim 13, further comprising:
receiving the surveillance message from the second aircraft, wherein the surveillance message indicates the current position of the second aircraft, the velocity of the second aircraft, and a type of the second aircraft; and
determining the threshold level based on a size of the ownship aircraft and further based on the size of the second aircraft.

### Claim 18 (depends on 13)

The collision awareness method of claim 13, further comprising determining a predicted time to collision based on the current position of the ownship aircraft and the current position of the second aircraft,
wherein generating the alert comprises presenting the predicted time to collision to a flight crew of the aircraft.

### Claim 19

A collision awareness system for predicting a collision between an ownship aircraft and an object during ground operations, the collision awareness system comprising:
a first camera mounted on a first portion of the ownship aircraft;
a second camera mounted on a second portion of the ownship aircraft; and
processing circuitry configured to:
determine that the object is within a field of view of the first camera;
determine a current position of the object based on one or more images captured by the first camera in response to determining that the object is within the field of view of the first camera;
determine that the object is within a field of view of the second camera and not within the field of view of the first camera;
determine the current position of the object based on one or more images captured by the second camera in response to determining that the object is within the field of view of the second camera and not within the field of view of the first camera;
determine that a distance between a future position of the ownship aircraft and the current position of the object is less than a threshold level; and
generate an alert in response to determining that the distance is less than the threshold level.

### Claim 20 (depends on 19)

The collision awareness system of claim 19,
wherein the processing circuitry is configured to determine the current position of the object based on a data source, and
wherein the processing circuitry is further configured to:
determine a confidence factor having a first value in response to determining that the second aircraft is within the field of view of the first camera;
determine the confidence factor having a second value in response to determining that the second aircraft is not within the field of view of the first camera or the field of view of the second camera, the second value indicating a lower level of confidence than the first value; and
determine the threshold level for the distance based on the confidence factor.