Back to results
US 11,307,595 B2Granted
Apparatus for acquisition of distance for all directions of moving body and method thereof
AssigneeMultipleye Co., Ltd.
InventorJongwoo Lim
Life of this patent
Estimating the term…
Abstract
A vision apparatus for a moving body is provided. The vision apparatus for a moving body includes a plurality of cameras that are arranged to be distanced from one another, and are arranged in a diagonal direction to the moving direction of the moving body, and a processor that receives images photographed at each of the plurality of cameras, and stereo-matches the plurality of received images and generates distance information for all directions of the moving body.
Claims18
An image processing device for acquiring a distance from a moving body to at least one object located in any direction of the moving body, comprising:
- at least one memory that stores instructions;
- and
- at least one processor configured to execute the instructions to perform or support another device to perform:
- on condition that a plurality of cameras, having ultra-wide field of views (FOVs) and capable of covering all directions of the moving body by using each of their respective FOVs, have been arranged to be apart from one another on the moving body, in response to acquiring a plurality of images generated by the cameras, (I) a process of inputting the images to a spherical sweep network and allowing the spherical sweep network to (i) project a plurality of pixels included in the images onto a virtual spherical coordinate system to thereby generate a plurality of spherical images, wherein the virtual spherical coordinate system is configured to include virtual n spheres, each of whose radii corresponds to each of sampling distances from a reference point and wherein the pixels included in the images are sampled for each of the sampling distances and (ii) apply a three-dimensional (3D) concatenating operation to the spherical images to thereby generate a four-dimensional (4D) initial cost volume, (II) a process of inputting the 4D initial cost volume to a cost volume computation network, including a plurality of 3D convolution layers and their corresponding 3D deconvolution layers, to thereby generate a 3D final cost volume, and (III) a process of (i) generating inverse depth indices respectively corresponding to inverse depths of the virtual n spheres by using the 3D final cost volume, wherein each of the inverse depths represent each of inverse values of each of the radii of the virtual n spheres, and (ii) extracting the radii by referring to the inverse depths derived from the inverse depth indices, to thereby acquire the distance from the moving body to the at least one object located in any direction of the moving body.
1 more element
A method for acquiring a distance from a moving body to at least one object located in any direction of the moving body, comprising:
- acquiring at an image processing device a plurality of images generated by a plurality of cameras spaced apart from one another on the moving body, each of the plurality of cameras having ultra-wide field of views and capable of covering all directions of the moving body by using each of their respective FOVs;
- (a) inputting, by the image processing device, the images to a spherical sweep network, the spherical sweep network
- (i) projecting a plurality of pixels included in the images onto a virtual spherical coordinate system to generate a plurality of spherical images, wherein the virtual spherical coordinate system is configured to include virtual n spheres, each of whose radii corresponds to each of sampling distances from a reference point, wherein the pixels included in the images having been sampled for each of the sampling distances, and
- (ii) applying a three-dimensional (3D) concatenating operation to the spherical images to generate a four-dimensional (4D) initial cost volume;
- (b) inputting, by the image processing device, the 4D initial cost volume to a cost volume computation network, including a plurality of 3D convolution layers and corresponding 3D deconvolution layers, and generating a 3D final cost volume;
- and
- (c) the image processing device
- (i) generating inverse depth indices respectively corresponding to inverse depths of the virtual n spheres by using the 3D final cost volume, wherein each of the inverse depths represent each of inverse values of each of the radii of the virtual n spheres, and
- (ii) extracting the radii by referring to the inverse depths derived from the inverse depth indices, to thereby acquire the distance from the moving body to the at least one object located in any direction of the moving body.
6 more elements
Dependencies are inferred from the claim text.