US 9,959,595 B2Granted

Dense structure from motion

AssigneeMobileye Vision Technologies Ltd.

InventorsHarel Livyatan, Oded Berberian, Barak Cohen, Gideon Stein

Life of this patent

Estimating the term…

Loading patent…

Abstract

Determining three-dimensional structure in a road environment using a system mountable in a host vehicle including a camera connectable to a processor. Multiple image frames are captured in the field of view of the camera. In the image frames, a line is selected below which the road is imaged. The line separates between upper images essentially excluding images of the road and lower images essentially including images of the road. One or more of the lower images is warped, according to a road homography to produce at least one warped lower image. The three-dimensional structure may be provided from motion of a matching feature within the upper images or from motion of a matching feature within at least one of the lower images and at least one warped lower image.

Claims20

Dependent claims

A method for determining a three-dimensional structure in an environment, the method performed by a system mountable in a host vehicle, wherein the system includes a camera operatively connectable to a processor, the method comprising:

capturing a plurality of image frames in a field of view of the camera, wherein the plurality of image frames includes a first image frame and a second image frame;
identifying a road of the environment in the first image frame;
in the first image frame, selecting a line, wherein the line separates a first portion of the first image frame determined to include the road of the environment and a second portion of the first image frame determined to exclude the road of the environment;
detecting a feature in the first portion of the first image frame;
matching the feature in the first portion of the first image frame with image data in the second image frame to produce a first matching;
warping the first portion of the first image frame according to a road homography and ego-motion of the host vehicle to produce a warped portion of the image frame, wherein the ego-motion is based on at least one of speed, inertia, and steering angle of the host vehicle;
detecting the feature in the warped portion of the first image frame;
matching the feature in the warped portion of the first image frame with the image data in the second image frame to produce a second matching;
and
selecting from among the first matching and the second matching to produce a selected matching.

7 more elements

A system for determining three-dimensional structure in a road environment, the system, mountable in a host vehicle, including a camera operatively connectible to a processor, the processor configured to:

capture a plurality of image frames in a field of view of the camera, wherein the plurality of image frames includes a first image frame and a second image frame;
identify a road of the environment in the first image frame;
in the first image frame, select a line, wherein the line separates a first portion of the first image frame determined to include the road of the environment and a second portion of the first image frame determined to exclude the road of the environment;
detect a feature in the first portion of the first image frame;
match the feature in the first portion of the first image frame with image data in the second image frame to produce a first matching;
warp the first portion of the first image frame according to a road homography and ego-motion of the host vehicle to produce a warped portion of the image frame, wherein the ego-motion is based on at least one of speed, inertia, and steering angle of the host vehicle;
detect the feature in the warped portion of the first image frame;
match the feature in the warped portion of the first image frame with the image data in the second image frame to produce a second matching;
and
select from among the first matching and the second matching to produce a selected matching.

7 more elements

Dependencies are inferred from the claim text.

Facts

Application: 14/182,715Filed 18 Feb 2014 · continuation in part
Priority: 21 Sep 2010
Published: US 2014/0161323 A112 Jun 2014
Granted: 1 May 2018
Term ends: Not estimatedThe source records don’t support an estimate.

Classification10 codes

G06V 20/588
Exterior to a vehicle by using sensors mounted on the vehicle: recognition of the road, e.g. of lane markings; Recognition of the vehicle driving…
G06F 18/22
Analysing: matching criteria, e.g. proximity measures
G06T 3/18
Geometric image transformations in the plane of the image: image warping, e.g. rearranging pixels individually
G06T 7/20
Image analysis: analysis of motion

Family

Application61/385,122Us provisional application · filed 21 Sep 2010
Application13/237,163Parent of this continuation-in-part · filed 20 Sep 2011
Application61/567,132Us provisional application · filed 6 Dec 2011
Application61/727,722Us provisional application · filed 18 Nov 2012
Application61/727,755Us provisional application · filed 19 Nov 2012
Application13/693,713Parent of this continuation-in-part · filed 4 Dec 2012
Application61/766,738Us provisional application · filed 20 Feb 2013
Application61/767,321Us provisional application · filed 21 Feb 2013
US 2014/0161323 A1Pre-grant publication · published 12 Jun 2014
US 9,118,816 B2Parent of this continuation-in-part · filed 25 Aug 2015
US 9,280,711 B2Parent of this continuation-in-part · filed 8 Mar 2016
US 2017/0287108 A9Pre-grant publication · published 5 Oct 2017
US 2012/0069185 A1Published application
US 2013/0141580 A1Published application
US 9,959,595 B2This patent

From the US filing history. Foreign family members aren’t listed.