Fusion of vision and depth sensors for navigation in complex environments
AssigneeLuftronix, Inc.
InventorsRoman Pavlyuk, Volodymyr Kozak, Klaus Sonnenleiter, Denise Spell
Life of this patent
Estimating the term…
Abstract
A system and a method for autonomous localization of a ground, surface or aerial vehicle with respect to a surface of a complex shape is disclosed. Localization is used by way of an optical flow method. Sensed optical (electro-optical, infrared, thermal) images are augmented by depth data, i.e., distance from the distance sensor to points on the sensed surface object. A modification of the optical flow method is applied to sequence of frames obtained in such a way to compute displacement of the vehicle with respect to the surface object.
Claims20
A close range autonomous flying device navigation system, comprising:
- an autonomous flying device body;
- a flying mechanism attached to said autonomous flying device body, adapted to cause said autonomous flying device to move in three dimensions;
- a directional-energy source pointed in at least a first direction;
- a distance-measuring device which determines a distance to a closest object in said first direction based on a received reflection of said directional energy;
- a camera pointed in said first direction;
- a contrast determination system which determines a plurality of points of contrast within images generated in a plurality of frames from output of said camera;
- and
- a position determination system which determines a location of said autonomous flying device in three dimensions based on a combination of:
- a) movement of each point of contrast of said plurality of points of contrast from one frame to at least one additional frame in said plurality of frames;
- and
- b) distance to an object represented by each said point of contrast from said first frame to said at least one additional frame as determined by said distance measuring device.
An autonomous flying device, configured to travel over a flight path from a starting location to an ending location, said autonomous flying device configured to carry out the following steps:
- receive visual images of surroundings of said autonomous flying device using a camera in the form of multiple frames;
- determine, in each frame of said multiple frames, points of greatest contrast;
- track movement of said points of greatest contrast across at least some of said multiple frames;
- emit directional energy at each of said points of greatest contrast in a field of view of said camera and determine distance of objects from said autonomous flying device based on a received response or detection of said directional energy;
- determine a position of said autonomous flying device in three dimensions, based on movement of said points of greatest contrast across said at least some of said multiple frames and said distance of said objects, wherein each object of said objects is correlated to a point of great contrast of said points of greatest contrast.
Dependencies are inferred from the claim text.