US 6,608,910 B1Granted

Computer vision method and apparatus for imaging sensors for recognizing and tracking occupants in fixed environments under variable illumination

AssigneeHRL Laboratories, LLC

InventorsNarayan Srinivasa, Yuri Owechko

Life of this patent

Estimating the term…

Loading patent…

Abstract

An computer vision method and system for recognizing and tracking occupants in a fixed space under variable illumination. The system utilizes a camera to capture an initial image of the unoccupied fixed space and subsequently captures images of the occupied fixed space. The edge maps of the current estimate of the unoccupied fixed space including illumination variations and the occupied fixed space are computed. The edge map of the current estimate of the unoccupied fixed space is then subtracted from the edge map of the occupied fixed space to yield a residual edge map, which is then processed to extract the image of the occupant. At least one equivalent rectangle is then computed from the two-dimensional moments of the image of the occupant. The equivalent rectangles are then used to determine the occupant type and position and to track changes in real-time. This method and system is generally designed for use with automobile safety systems such as “smart” airbags. However, it may be tailored to many applications such as computer gaming, adaptive automotive controls, and “smart” homes, among others.

Claims12

Dependent claims

A computer vision apparatus for recognizing and tracking occupants in a fixed space including:

at least one sensor fixedly positioned to capture light from a fixed space beginning at a time prior to an occupant's entrance and continuing subsequent to an occupant's entrance to form a series of N images having pixels with pixel intensities, the series of N images including a current image Nc and a previous image Nc−1, with the previous image Nc−1 being the image immediately prior to the current image Nc in the series of N subsequent images, the sensor converting the series of N images into electronic data;
a processor positioned to receive electronic data from the sensor and to compare the current image Nc with the previous image Nc−1 in order to dynamically determine the occupant type and occupant position;
computing an low-level vision-based feature map of the previous image Nc −1of the fixed space;
computing an low-level vision-based feature map of the current image Nc of the fixed space;
computing a residual low-level vision-based feature map having connected regions withsizes, by calculating the difference between low-level vision-based feature map of the current image Nc of the fixed space and the previous image Nc−1 of the fixed space, where the pixels corresponding to the occupant in the previous subsequent image have been replaced by corresponding pixels of the current image Nc of the fixed space;
applying a high pass size filter to remove connected regions of the residual low-level vision-based feature map below a specified size, resulting in a size-filtered residual low-level-vision-based feature map including gaps and a bounded area;
filling the bounded area of the size-filtered residual low-level vision-based feature map, resulting in a filled low-level vision-based feature area;
calculating two-dimensional moments of the filled low-level vision-based feature area and using the two-dimensional moments to construct an equivalent rectangle with moments equal to that of the filled low-level vision-based feature area;
and
determining the centroid, orientation, size and aspect ratio of the equivalent rectangle.

7 more elements

A computer vision method for recognizing and tracking occupants in a fixed space including the steps of:

(a) providing at least one sensor fixedly positioned to capture light reflected from a fixed space beginning at a time prior an occupant's entrance and continuing subsequent to an occupant's entrance to form a series of N images having pixels with pixel intensities, the series of N images including a current image Nc and a previous image Nc−1, with the previous image Nc−1 being the image immediately prior to the current image Nc in the series of N subsequent images, the sensor converting the series of N images into electronic data;
and
(b) processing the electronic data with a processing system positioned to receive electronic data from the sensor and to compare current image Nc with the previous image Nc−1 in order to dynamically determine the occupant type and occupant position.

A computer vision method for recognizing and tracking occupants in a fixed space including the steps of:

(a) providing at least one sensor fixedly positioned to capture light reflected from a fixed space beginning at a time prior to an occupant's entrance and continuing subsequent to an occupant's to form a series of N images having pixels with pixel intensities, the series of N images including a current image Nc and a previous image Nc−1, with the previous image Nc−1 being the image immediately prior to the current image Nc in the series of N subsequent images, the sensor converting the series of N images into electronic data;
and
(b) processing the electronic data with a processing system positioned to receive electronic data from the sensor and to:
i. compute an edge map of the previous image Nc−1 of the fixed space;
ii. compute an edge map of the current image Nc of the fixed space;
iii. compute a residual edge map having connected regions with sizes, from the difference between the edge map of the current image Nc of the fixed space and the previous image Nc−1 of the fixed space, where the pixels corresponding to the occupant in the previous subsequent image have been replaced by corresponding pixels of the current image Nc of the fixed space;
iv. apply a high pass size filter to remove connected regions of the residual edge map below a specified size, resulting in a size-filtered residual edge map including gaps;
v. apply an edge dilation operator to the size-filtered residual edge map to dilate the size-filtered residual edge map to eliminate gaps, resulting in a dilated residual edge map including edge pixels;
vi. vertically fill the area between the edge pixels in the dilated residual edge map using the pixel intensity from the subsequent images resulting in a filled edge area;
vii. calculate two-dimensional moments of the filled edge area and using the two-dimensional moments to construct an equivalent rectangle with moments equal to that of the filled edge area;
viii. determine the centroid, orientation, size, and aspect ratio of the equivalent rectangle;
ix. partitioning the filled edge area into a plurality N of sub-filled edge areas;
x. calculating the two-dimensional moments of each of the N sub-filled edge areas and using the two dimensional moments to construct an equivalent rectangle for each of the N sub-filled edge areas;
and
xi. determining the centroid, orientation, size, and aspect ratio of each of the equivalent rectangles for each of the N sub-filled edge areas;
and
(c) providing the centroid, orientation, size, and aspect ratio of the equivalent rectangle and the centroid, orientation, size, and aspect ratio of each of the equivalent rectangles for each of the N sub-filled edge areas as output.

14 more elements

Dependencies are inferred from the claim text.