# US 12,217,444 B2 · Method for measuring the topography of an environment > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 12,217,444 B2 - Document ID: US12217444B2 - Status: Granted - Kind: B2 - Application number: 17/784,896 - Filed: 2020-12-08 - Published: 2023-11-02 - Granted: 2025-02-04 - Earliest claimed: 2019-12-13 - Assignees: Connaught Electronics Ltd. - Applicants: Connaught Electronics Ltd. - Inventors: Ciaran Hughes; Pantelis Ermilios; Matthias Lenk; Sunil Chandra - Web page: https://patentgrep.com/patent/US12217444B2 ## CPC - G06T 7/579: from motion - G06T 7/593: from stereo images - H04N 13/221: using the relative movement between cameras and objects - H04N 13/243: using three or more two-dimensional [2D] image sensors - G06T 2207/10028: Range image; Depth image; 3D point clouds - G06T 2207/20021: Dividing image into blocks, subimages or windows - G06T 2207/30252: Vehicle exterior; Vicinity of vehicle - G06T 2207/30264: Parking - G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL - H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION ## Abstract A method for forming a point cloud corresponding to the topography of an imaged environment involves acquiring an image of the environment with a camera having a WFOV lens mounted on a vehicle; changing the camera pose by an adjustment greater than a threshold; and acquiring another image of the environment with the camera at the changed pose. The images are mapped onto respective surfaces to form respective mapped images, defined by the same nonplanar geometry. One of the mapped images is divided into blocks of pixels. For each block, a depth map is formed by performing a search through the other mapped image to evaluate a disparity in the position of the location of the block of pixels in each of the mapped images. The depth map is converted into a partial point cloud corresponding to the local topography of the environment surrounding the vehicle as it moves through the environment. ## Claims ### Claim 1 A method for forming a point cloud corresponding to the topography of an imaged environment comprising: acquiring a first image of the environment with a camera having a wide field of view lens mounted on a vehicle; changing the camera pose by an adjustment that is more than a threshold; acquiring a second image of the environment with the camera at the changed pose; mapping the first image onto a first surface to form a first mapped image; mapping the second image onto a second surface to form a second mapped image, the first surface and the second surface being defined by the same non-planar geometry, wherein the non-planar geometry defines a cylindrical surface and is symmetric about an optical axis of the camera, wherein the non-planar geometry comprises: a first non-planar surface; and a second non-planar surface, the first non-planar surface extending from the camera to the second non-planar surface, and the second non-planar surface converging from the first non-planar surface towards the optical axis; dividing the one of the first or second mapped images into blocks of pixels; for each block of pixels, forming a depth map by performing a search through the other of the first or second mapped images to evaluate a disparity in the position of the location of the block of pixels in each of the mapped images; converting the depth map into a portion of a point cloud corresponding to the local topography of the environment surrounding the vehicle as it moves through the environment; and scaling the point cloud according to the adjustment in camera pose. ### Claim 2 (depends on 1) The method of claim 1, wherein performing a search comprises performing a one-dimensional search of image data along rectified epipolar lines. ### Claim 3 (depends on 1) The method of claim 1, further comprising assessing the change in camera pose with odometry information supplied from vehicle odometry sensors. ### Claim 4 (depends on 3) The method of claim 3, wherein the wide field of view lens is a fisheye lens. ### Claim 5 (depends on 4) The method of claim 4, wherein the non-planar geometry defines a conical surface. ### Claim 6 (depends on 1) The method of claim 1, wherein the non-planar geometry defines a spherical surface. ### Claim 7 (depends on 1) The method of claim 1, wherein the second non-planar surface defines a frusto-conical surface. ### Claim 8 (depends on 1) The method of claim 1, wherein the second non-planar surface defines a hemispherical surface. ### Claim 9 (depends on 8) The method of claim 8, wherein the mapping is performed by building a sparse look-up table for a subset of pixels in the mapped images and storing the corresponding camera image coordinates for that pixel with decimal precision. ### Claim 10 (depends on 1) A vehicle for movement in a forward direction, the vehicle comprising: a camera with an optical axis aligned to the forward direction; and a camera with an optical axis aligned in a rearward direction, the rearward direction opposing the forward direction, wherein each camera operates according to the method of claim 1. ### Claim 11 (depends on 1, 10) The vehicle of claim 10, wherein the vehicle further comprises: two cameras aligned in opposing directions that are perpendicular to the forward direction, wherein the two cameras operate according to the method of claim 1. ### Claim 12 (depends on 10) The vehicle of claim 10, wherein the cameras provide information to a system for automated parking or autonomous driving of the vehicle. ### Claim 13 (depends on 1) A method for displaying the scene around a vehicle comprising: acquiring images from a plurality of cameras located around the periphery of the vehicle with overlapping fields of view; stitching the images into a stitched image; and mapping the stitched image onto a surface defined by the point cloud produced by the method of claim 1. ### Claim 14 A method for forming a point cloud corresponding to the topography of an imaged environment comprising: acquiring a first image of the environment with a camera having a wide field of view lens mounted on a vehicle; changing the camera pose by an adjustment that is more than a threshold; acquiring a second image of the environment with the camera at the changed pose; mapping the first image onto a first surface to form a first mapped image; mapping the second image onto a second surface to form a second mapped image, the first surface and the second surface being defined by the same non-planar geometry, wherein the non-planar geometry defines a cylindrical surface and is symmetric about an optical axis of the camera, wherein the non-planar geometry comprises: a first non-planar surface; and a second non-planar surface, the first non-planar surface extending from the camera to the second non-planar surface, and the second non-planar surface converging from the first non-planar surface towards the optical axis, wherein the second non-planar surface defines a frusto-conical surface; dividing the one of the first or second mapped images into blocks of pixels; for each block of pixels, forming a depth map by performing a search through the other of the first or second mapped images to evaluate a disparity in the position of the location of the block of pixels in each of the mapped images; converting the depth map into a portion of a point cloud corresponding to the local topography of the environment surrounding the vehicle as it moves through the environment; and scaling the point cloud according to the adjustment in camera pose. ### Claim 15 A method for forming a point cloud corresponding to the topography of an imaged environment comprising: acquiring a first image of the environment with a camera having a wide field of view lens mounted on a vehicle; changing the camera pose by an adjustment that is more than a threshold; acquiring a second image of the environment with the camera at the changed pose; mapping the first image onto a first surface to form a first mapped image; mapping the second image onto a second surface to form a second mapped image, the first surface and the second surface being defined by the same non-planar geometry, wherein the non-planar geometry defines a cylindrical surface, wherein the non-planar geometry is symmetric about an optical axis of the camera, wherein the non-planar geometry comprises: a first non-planar surface; and a second non-planar surface, the first non-planar surface extending from the camera to the second non-planar surface, and the second non-planar surface converging from the first non-planar surface towards the optical axis, wherein the second non-planar surface defines a hemispherical surface; dividing the one of the first or second mapped images into blocks of pixels; for each block of pixels, forming a depth map by performing a search through the other of the first or second mapped images to evaluate a disparity in the position of the location of the block of pixels in each of the mapped images; converting the depth map into a portion of a point cloud corresponding to the local topography of the environment surrounding the vehicle as it moves through the environment; and scaling the point cloud according to the adjustment in camera pose. ### Claim 16 (depends on 15) The method of claim 15, wherein the mapping is performed by building a sparse look-up table for a subset of pixels in the mapped images and storing the corresponding camera image coordinates for that pixel with decimal precision.