# US 2010/0305857 A1 · Method and System for Visual Collision Detection and Estimation

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2010/0305857 A1
- Document ID: US20100305857A1
- Status: Published application
- Kind: A1
- Application number: 12/776,202
- Filed: 2010-05-07
- Published: 2010-12-02
- Earliest claimed: 2009-05-08
- Inventors: Jeffrey Byrne; Raman K. Mehra
- Web page: https://patentgrep.com/patent/US20100305857A1

## CPC

- G06T 7/73: using feature-based methods
- G06T 2207/30248: Vehicle exterior or interior
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL

## Abstract

Collision detection and estimation from a monocular visual sensor is an important enabling technology for safe navigation of small or micro air vehicles in near earth flight. In this paper, we introduce a new approach called expansion segmentation, which simultaneously detects “collision danger regions” of significant positive divergence in inertial aided video, and estimates maximum likelihood time to collision (TTC) in a correspondenceless framework within the danger regions. This approach was motivated from a literature review which showed that existing approaches make strong assumptions about scene structure or camera motion, or pose collision detection without determining obstacle boundaries, both of which limit the operational envelope of a deployable system. Expansion segmentation is based on a new formulation of 6-DOF inertial aided TTC estimation, and a new derivation of a first order TTC uncertainty model due to subpixel quantization error and epipolar geometry uncertainty. Proof of concept results are shown in a custom designed urban flight simulator and on operational flight data from a small air vehicle.

## Claims

### Claim 1

A system for collision detection and estimation in a moving vehicle with respect to stationary objects, the system comprising:
an image source providing a plurality of images in a direction of motion;
an inertial information source providing positional and directional information about the vehicle; and
a computer system having at least one processing unit and associated memory, the computer system being connected to the image source to receive image data and to the inertial information source to receive positional and orientation information;
wherein the memory includes computer programs adapted for use by the computer system to process image data for a first image and a second image and positional and orientation information associated with the image data to determine a time to collision value for at least one pixel in said second image as a function of the first image data, the second image data, the positional information and the orientation information.

### Claim 2 (depends on 1)

The system according to claim 1 wherein at least one computer program includes a phase correlation module and the computer system uses the phase correlation module to indentify pixels in the second image that correspond to pixels in the first image.

### Claim 3 (depends on 1)

The system according to claim 1 wherein at least one computer program includes a feature detection module and a phase correlation module and the computer system uses the feature detection module to detect features in the first and second images and uses the phase correlation module to indentify pixels in the second image that correspond to pixels in the first image using feature detection information.

### Claim 4 (depends on 1)

The system according to claim 1 wherein at least one computer program includes a convolution module and a phase correlation module and the computer system uses the convolution module to detect features in the first and second images and uses the phase correlation module to indentify pixels in the second image that correspond to pixels in the first image using convolution information.

### Claim 5 (depends on 1)

The system according to claim 1 wherein computer system determines a time to collision uncertainty for at least one pixel in the second image.

### Claim 6 (depends on 1)

The system according to claim 1 wherein computer system determines a time to collision value for every pixel in the second image.

### Claim 7 (depends on 1)

The system according to claim 1 wherein computer system determines a time to collision value and a time to collision uncertainty for every pixel in the second image.

### Claim 8 (depends on 1)

The system according to claim 1 wherein computer system determines a time to collision value for every pixel in the second image and associates each pixel with a collision probability value as a function of the time to collision value for each pixel and a predetermined threshold.

### Claim 9 (depends on 1)

The system according to claim 1 wherein computer system determines a time to collision value for every pixel in the second image and associates each pixel with a binary collision value as a function of the time to collision value for each pixel and a predetermined threshold.

### Claim 10 (depends on 1)

The system according to claim 1 wherein computer system determines a time to collision value for every pixel in the second image and associates each pixel with a binary collision value as a function of the time to collision value for each pixel and a predetermined threshold and wherein at least one computer program includes an expansion segmentation module and the computer system uses the expansion segmentation module to group pixels in the second image according into one of two groups as a function of the binary collision value.

### Claim 11 (depends on 10)

The system according to claim 10 wherein the expansion segmentation module uses a Markov Random Field analysis to group the pixels in the second image.

### Claim 12 (depends on 1)

The system according to claim 1 further comprising a collision avoidance system, the collision avoidance being adapted and configured to change the movement of the vehicle in at least one dimension as a function of the time to collision value for at least one pixel.

### Claim 13 (depends on 1)

The system according to claim 1 wherein the computer system determines a collision value for at least one pixel as a function of the time to collision value and wherein the direction of motion of the vehicle is changed as function of the collision value for at least one pixel.

### Claim 14

A method of collision detection and estimate for a moving vehicle, the vehicle including an image source providing a plurality of images in a direction of motion, an inertial information source providing positional and orientation information about the vehicle and a system for processing image data, positional and orientation information, the method comprising:
retrieving first image data corresponding to a first image and positional and orientation information associated with the first image;
retrieving second image data corresponding to a second image and positional and orientation information associated with the second image; and
determining a time to collision value for at least one pixel in the second image as a function of the first image data and associated positional and orientation information and the second image data and associated positional and orientation information.

### Claim 15 (depends on 14)

The method according to claim 14 further comprising:
identifying at least one pixel in the second image that corresponds to at least one pixel in the first image using phase correlation.

### Claim 16 (depends on 14)

The method according to claim 14 further comprising:
detecting features in at least one image using image convolution; and
identifying at least one pixel in the second image that corresponds to at least one pixel in the first image using phase correlation and image convolution information.

### Claim 17 (depends on 14)

The method according to claim 14 further comprising:
determining a time to collision uncertainty value for at least one pixel in the second image as a function of the first image data and associated positional and orientation information and the second image data and associated positional and orientation information.

### Claim 18 (depends on 14)

The method according to claim 14 further comprising:
determining a time to collision value for each pixel in the second image as a function of the first image data and associated positional and orientation information and the second image data and associated positional and orientation information.

### Claim 19 (depends on 14)

The method according to claim 14 further comprising:
determining a time to collision value and a time to collision uncertainty value for each pixel in the second image as a function of the first image data and associated positional and orientation information and the second image data and associated positional and orientation information.

### Claim 20 (depends on 14)

The method according to claim 14 further comprising:
determining a collision probability value for each pixel in the second image as a function of the time to collision value for each pixel and a predetermined threshold.

### Claim 21 (depends on 14)

The method according to claim 14 further comprising:
determining a binary collision value for each pixel in the second image as a function of the time to collision value for each pixel and a predetermined threshold.

### Claim 22 (depends on 14)

The method according to claim 14 further comprising:
determining a binary collision value for each pixel in the second image as a function of the time to collision value for each pixel and a predetermined threshold and
grouping, using expansion segmentation, each pixel in the second image into one of two groups as a function of the binary collision value.

### Claim 23 (depends on 22)

The method according to claim 22 further comprising using Markov Random Field analysis to group each pixel in the second image into one of two groups.

### Claim 24 (depends on 14)

The method according to claim 14 further comprising:
changing a direction of movement of the vehicle as a function of the time to collision values for at least one pixel.

### Claim 25 (depends on 14)

The method according to claim 14 further comprising:
determining a collision value for at least one pixel as a function of the time to collision value for at least one pixel and
changing a direction of motion of the vehicle as a function of the collision value for at least one pixel.
