US 5,559,695 A · Apparatus and method for self-calibrating visual time-to-contact sensor

Abstract

The invention relates to a method and a self-calibrating visual time-to-contact (TTC) sensor for a mobile manned or autonomous unmanned vehicle which operates in a high speed manner and which permits the continuous, adaptive motion of the vehicle through the vehicle's environment. More specifically, the invention relates to an apparatus and method for novel use of active sensor control for aligning a camera, mounted on the vehicle, to track the direction of motion of the vehicle in order to successfully navigate in complex environments and to simplify difficult processing steps previously attempted through algorithmic means. Rather than attempting to explicitly find a focus of expansion (FOE) from image data, the invention continuously calibrates the sensor to point in the direction of the FOE, using weighted global average of the horizontal and vertical component of the optical flow. The pan and tilt angles of the visual TTC sensor are iteratively changed by the scanning mechanism in the opposite direction by small increments, biased by the global magnitude of the optical flow. In this way, the visual TTC sensor is not constrained to point in any direction at the start, and after several iterations it will point near the FOE and continue to do so as the vehicle moves. By actively centering the FOE in the image sequence, better accuracy is possible. By avoiding the focus of expansion calculation, the method becomes self-calibrating and it is faster and more robust to vibrations and sensor misalignments occurring in changing and complex indoor or outdoor environments.

Claims21

Dependent claims

A self-calibrating visual sensor system, mounted on a mobile vehicle, comprising:

a visual time-to-contact (TTC) sensor that determines a time before said sensor will contact an object in its path, said TTC sensor generating a plurality of image frames descriptive of a travel area in the direction of movement of the mobile vehicle;
an image sensor processor, coupled to said visual TTC sensor, comprising an image memory for storing at least two image frames outputted by the visual TTC sensor;
an image processing unit comprising an image grabber, coupled to the image memory and operable for reading the stored image frames from the image memory, and a visual control means for continuous calibration of the visual TTC sensor to point in the direction of focus of expansion (FOE) by adjusting pan and tilt angles of the visual TTC sensor;
and
a video bus between the image processing unit and the image sensor processor for transfer of image frames.

2 more elements

A method of self-calibrating a visual time-to-contact (TTC) sensor that determines a time before said sensor will contact an object in its path, said TTC sensor mounted on a mobile vehicle, for continuous calibration of the visual TTC sensor to point in the direction of focus of expansion (FOE), without explicit calculation of the FOE, comprising the following steps:

(a) grabbing an image at time t1;
(b) moving the mobile vehicle forward with approximately linear motion;
(c) grabbing second image at time t2;
(d) computing an average optical flow by calculating a weighted global average of the horizontal and vertical components of the optical flow between time t1 and t2;
(e) computing pan and tilt angles to point the visual TTC sensor closer to the focus of expansion (FOE);
(f) moving the visual TTC sensor with the calculated pan and tilt angles toward the FOE;
(g) computing the TTC value for the image pair for time t1 and t2 and calculating the average of the most recent TTC values, if the FOE is centered in the image;
and
(i) iteratively repeating the steps (a) to (g) for adjusting the course of movement of the mobile vehicle until the desired position is achieved.

6 more elements

Dependencies are inferred from the claim text.

Apparatus and method for self-calibrating visual time-to-contact sensor

Life of this patent

Apparatus and method for self-calibrating visual time-to-contact sensor

Life of this patent

Abstract

Claims21