US 2010/0246901 A1Published application

Operation Support System, Vehicle, And Method For Estimating Three-Dimensional Object Area

AssigneeSanyo Electric Co., Ltd.

InventorChanghui Yang

Life of this patent

Estimating the term…

Loading patent…

Abstract

Camera images at times (t1 and t2) are acquired from a camera installed on a vehicle, and the camera images are converted to bird's-eye view images at the times (t1 and t2). A plurality of feature points are extracted from the camera image at the time (t1) and the movement vectors of the respective feature points between first and second camera images are detected. Then, the respective feature points and the respective movement vectors are mapped onto a bird's-eye image coordinate plane. Two or more feature points on the bird's-eye view images are permitted to be targets, the positions and the movement vectors of the target feature points are applied to an equation of constraint which ground surface feature points have to satisfy, thereby discriminating whether the target feature points are the ground surface feature points. Then, from position information and movement vector information on the two or more feature points discriminated to be the ground surface feature points, movement information on the vehicle is obtained and used for taking the difference between the bird's-eye images at the times (t1 and t2) to estimate a three-dimensional object area.

Claims20

Dependent claims

An operation support system which is provided with a camera fitted to a mobile object to shoot surroundings of the mobile object, and which estimates, based on camera images on a camera coordinate plane obtained from the camera, a three-dimensional object area in an image based on the camera image, the operation support system comprising:

an image acquisition portion which acquires first and second camera images shot by the camera at first and second time points, respectively, while the mobile object is moving, the first and second time points being different from each other;
a movement vector detection portion which extracts n feature points (where n is an integer of 2 or more) from the first camera image, and which also detects movement vectors, on the camera coordinate plane, of the feature points between the first and second camera images;
a bird's eye conversion portion which projects the camera images, and the feature points and the movement vectors on the camera coordinate plane onto a bird's eye view coordinate plane which is parallel to ground to thereby convert the first and second camera images into first and second bird's eye view images, respectively, and detect positions of the feature points on the first bird's eye view image and movement vectors of the feature points on the bird's eye view coordinate plane between the first and second bird's eye view images;
a determination portion which determines, by use of a restraint condition for a ground feature point located on the ground to satisfy, whether or not a target feature point on the first bird's eye view image is the ground feature point;
a movement information estimation portion which estimates movement information of the mobile object between the first and second time points based on positions on the first bird's eye view image, and movement vectors on the bird's eye view coordinate plane, of two or more feature points which are each judged as the ground feature point;
and
a three-dimensional object area estimation portion which estimates the three-dimensional object area based on the first and second bird's eye view images and the movement information.

4 more elements

A three-dimensional object area estimation method for estimating, based on a camera image on a camera coordinate plane obtained from a camera fitted to a mobile object to shoot surroundings of the mobile object, a three-dimensional object area in an image based on the camera image, the three-dimensional object area estimation method comprising:

an image acquisition step for acquiring first and second camera images shot by the camera at first and second time points, respectively, while the mobile object is moving, the first and second time points being different from each other;
a movement vector detection step for extracting n feature points (where n is an integer of 2 or more) from the first camera image, and detecting movement vectors of the feature points on the camera coordinate plane between the first and second camera images;
a bird's eye conversion step for projecting the camera images, and the feature points and the movement vectors on the camera coordinate plane onto a bird's eye view coordinate plane which is parallel to ground, to thereby convert the first and second camera images into first and second bird's eye view images, respectively, and detect positions of the feature points on the first bird's eye view image and movement vectors of the feature points on the bird's eye view coordinate plane between the first and second bird's eye view images;
a determination step for determining, by use of a restraint condition for a ground feature point located on the ground to satisfy, whether or not a target feature point on the first bird's eye view image is the ground feature point;
a movement information estimation step for estimating movement information of the mobile object between the first and second time points based on positions on the first bird's eye view image, and movement vectors on the bird's eye view coordinate plane, of two or more feature points which are each judged as the ground feature point;
and
a three-dimensional object area estimation step for estimating the three-dimensional object area based on the first and second bird's eye view images and the movement information.

4 more elements

Dependencies are inferred from the claim text.