Moving object localization in 3D using a single camera
AssigneeNEC Laboratories America, Inc.
InventorsManmohan Chandraker, Shiyu Song, Yuanqing Lin, Xiaoyu Wang
Life of this patent
Estimating the term…
Abstract
Systems and methods are disclosed for autonomous driving with only a single camera by moving object localization in 3D with a real-time framework that harnesses object detection and monocular structure from motion (SFM) through the ground plane estimation; tracking feature points on moving cars a real-time framework to and use the feature points for 3D orientation estimation; and correcting scale drift with ground plane estimation that combines cues from sparse features and dense stereo visual data.
Claims16
A computer vision method for autonomous driving that uses only a single camera, comprising:
- moving object localization in 3D with a real-time framework that harnesses object detection and monocular structure from motion (SFM) through a ground plane estimation;
- tracking feature points on moving cars a real-time framework to and use the feature points for 3D orientation estimation;
- correcting scale drift with ground plane estimation that combines cues from sparse features and dense stereo visual data;
- and
- applying a Kalman filter framework that adapts the fusion observation covariances at every frame to reflect the relative uncertainty of each cue, the Kalman filter comprising a model of state of evolution as
- xk=Axk-1+wk-1,p(w):N(0,Q),
- zk=Hxk-1+vk-1,p(v):N(0,U),
- where x represents a state variable, z an observation, Q and U represent covariances of process and observation noise and are assumed to be zero mean multivariate normal distributions as indicated by p(w):N(0,Q) and p(v):N(0,U), A represents state transition, w a process noise, H an observation that maps true state space into observed state space, and v an observation noise.
A computer vision system for autonomous driving that uses only a single camera, comprising:
- a real-time framework for moving object localization in 3D that harnesses object detection and monocular structure from motion (SFM) through the ground plane estimation,
- a real-time framework to track feature points on moving cars and use them for 3D orientation estimation,
- computer code to correct scale drift using ground plane estimation that combines cues from sparse features and dense stereo visual data, and
- a Kalman filter framework that adapts the fusion observation covariances at every frame to reflect the relative uncertainty of each cue, the Kalman filter comprising a model of state of evolution as
- xk=Axk-1+wk-1,p(w):N(0,Q),
- zk=Hxk-1+vk-1,p(v):N(0,U),
- where x represents a state variable, z an observation, Q and U represent covariances of process and observation noise and are assumed to be zero mean multivariate normal distributions as indicated by p(w):N(0,Q) and p(v):N(0,U), A represents state transition, w a process noise, H an observation that maps true state space into observed state space, and v an observation noise;
- wherein the system includes a processor for processing the real time framework and the computer code.
Dependencies are inferred from the claim text.