# US 2021/0241514 A1 · Techniques for real-time mapping in a movable object environment > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 2021/0241514 A1 - Document ID: US20210241514A1 - Status: Published application - Kind: A1 - Application number: 17/229,409 - Filed: 2021-04-13 - Published: 2021-08-05 - Earliest claimed: 2018-10-29 - Assignees: DJI Technology, Inc. - Applicants: DJI Technology, Inc. - Inventors: Arjun Sukumar MENON; Jifei XU; Rohit SANT; Zhiyuan LI; Blake KARWOSKI; Joshua ACOSTA; Arnaud THIERCELIN; Weifeng LIU - Web page: https://patentgrep.com/patent/US20210241514A1 ## CPC - G01S 7/51: Display arrangements - B64D 47/00: Equipment not otherwise provided for - B64U 10/14: with four distinct rotor axes, e.g. quadcopters - G01C 11/025: {by scanning the object} - G01C 21/20: Instruments for performing navigational calculations (G01C21/24, G01C21/26 take precedence) - G01C 23/00: Combined instruments indicating more than one navigational value, e.g. for aircraft; Combined measuring devices for measuring two or more variables of movement, e.g. distance, speed or acceleration - G01S 7/4813: {Housing arrangements} - G01S 17/42: Simultaneous measurement of distance and other co-ordinates (indirect measurement G01S17/46) - G01S 17/86: Combinations of lidar systems with systems other than lidar, radar or sonar, e.g. with direction finders - G01S 17/89: for mapping or imaging - G05D 1/0038: {by providing the operator with simple or augmented images from one or more cameras located onboard the vehicle, e.g. tele-operation (images analyzed by a computer and used for automatic navigation G05D1/0246)} - G05D 1/0094: {involving pointing a payload, e.g. camera, weapon, sensor, towards a fixed or moving target} - G05D 1/101: {specially adapted for aircraft} - G05D 1/689: Pointing payloads towards fixed or moving targets (positioning towed, pushed or suspended implements G05D1/672) - G06F 3/04845: for image manipulation, e.g. dragging, rotation, expansion or change of colour - G06T 15/00: Three-dimensional [3D] image rendering - G06T 15/08: Volume rendering - B64U 2101/30: for imaging, photography or videography - G06T 2200/24: involving graphical user interfaces [GUIs] - G06T 2210/56: Particle system, point based geometry or rendering - G01S: RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES - B64D: EQUIPMENT FOR FITTING IN OR TO AIRCRAFT; FLIGHT SUITS; PARACHUTES; ARRANGEMENT OR MOUNTING OF POWER PLANTS OR PROPULSION TRANSMISSIONS IN AIRCRAFT - B64U: UNMANNED AERIAL VEHICLES [UAV]; EQUIPMENT THEREFOR - G01C: MEASURING DISTANCES, LEVELS OR BEARINGS; SURVEYING; NAVIGATION; GYROSCOPIC INSTRUMENTS; PHOTOGRAMMETRY OR VIDEOGRAMMETRY (measuring liquid level G01F; radio navigation, determining distance or velocity by use of propagation effects, e.g. Doppler effects, propagation time, of radio waves, analogous arrangements using other waves G01S) - G05D: SYSTEMS FOR CONTROLLING OR REGULATING NON-ELECTRIC VARIABLES - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) - G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL ## Abstract Techniques are disclosed for real-time mapping in a movable object environment. A real-time mapping system can include at least one movable object including a computing device, a scanning sensor electronically coupled to the computing device, and a positioning sensor electronically coupled to the computing device. The computing device can include at least one processor and a mapping manager, the mapping manager may be configured to obtain mapping data from the scanning sensor and obtain positioning data from the positioning sensor. The mapping manager can associate the mapping data with the positioning data based at least on time data associated with the mapping data and the positioning data, and then generate a map in a first coordinate system based at least on the associated mapping data and positioning data. ## Claims ### Claim 1 A system for real-time mapping in a movable object environment, comprising: at least one movable object including a computing device; a scanning sensor electronically coupled to the computing device; a positioning sensor electronically coupled to the computing device; the computing device including at least one processor and a mapping manager, the mapping manager including instructions which, when executed by the processor, cause the mapping manager to: obtain mapping data from the scanning sensor; obtain positioning data from the positioning sensor; associate the mapping data with the positioning data based at least on time data associated with the mapping data and the positioning data; and generate a map in a first coordinate system based at least on the associated mapping data and positioning data, the map comprising a plurality of voxels and wherein an occupancy probability threshold value is used to determine whether a voxel includes a point in the map. ### Claim 2 (depends on 1) The system of claim 1, wherein the instructions to associate the mapping data with the positioning data based at least on the time data associated with the mapping data and the positioning data, when executed by the processor, further cause the mapping manager to: upsample the positioning data to include a number of positions equal to a number of points in the mapping data; and associate each point in the mapping data to a corresponding position in the upsampled positioning data. ### Claim 3 (depends on 2) The system of claim 2, further comprising: clock circuitry electronically coupled to the scanning sensor and the positioning sensor, to provide the time data associated with the mapping data and the positioning data. ### Claim 4 (depends on 1) The system of claim 1, wherein the instructions to generate the map comprising the plurality of voxels in the first coordinate system based at least on the associated mapping data and positioning data, when executed by the processor, further cause the mapping manager to: for each voxel of the plurality of voxels of the map: determine one or more points from the mapping data to be located in the voxel; and determine an occupancy probability for the voxel based at least on a number of points in that voxel. ### Claim 5 (depends on 4) The system of claim 4, wherein the occupancy probability is determined based on a variance of the positioning data associated with the one or more points located in the voxel. ### Claim 6 (depends on 4) The system of claim 4, wherein the instructions, when executed by the processor, further cause the mapping manager to: for each voxel having the occupancy probability greater than the occupancy probability threshold value: calculate an average position of the one or more points located in the voxel; and generate a point in the map at the average position. ### Claim 7 (depends on 6) The system of claim 6, wherein the instructions, when executed, further cause the mapping manager to: for each voxel having the occupancy probability greater than the occupancy probability threshold value: calculate an average intensity value of the one or more points located in the voxel; and associate the average intensity value with the generated point in the map. ### Claim 8 A method for real-time mapping in a movable object environment, comprising: obtaining mapping data from a scanning sensor supported by a movable object; obtaining positioning data from a positioning sensor supported by the movable object; associating the mapping data with the positioning data based at least on time data associated with the mapping data and the positioning data; and generating a map in a first coordinate system based at least on the associated mapping data and positioning data, the map comprising a plurality of voxels and wherein an occupancy probability threshold value is used to determine whether a voxel includes a point in the map. ### Claim 9 (depends on 8) The method of claim 8, wherein generating the map comprising the plurality of voxels in the first coordinate system based at least on the associated mapping data and positioning data, further comprises: for each voxel of the plurality of voxels of the map: determining one or more points from the mapping data to be located in the voxel; and determining an occupancy probability for the voxel based at least on a number of points in that voxel. ### Claim 10 (depends on 9) The method of claim 9, wherein the occupancy probability is determined based on a variance of the positioning data associated with the one or more points located in the voxel. ### Claim 11 (depends on 9) The method of claim 9, further comprising: for each voxel having the occupancy probability greater than the occupancy probability threshold value: calculating an average position of the one or more points in the voxel; and generating a point in the map at the average position. ### Claim 12 (depends on 11) The method of claim 11, further comprising: for each voxel having the occupancy probability greater than the occupancy probability threshold value: calculating an average intensity value of the one or more points in the voxel; and associating the average intensity value with the generated point in the map. ### Claim 13 (depends on 8) The method of claim 8, further comprising: determining a distribution of points in the mapping data, each point in the mapping data associated with a distance from a nearest neighboring point in the mapping data; and removing any points associated with a distance greater than a distance threshold value. ### Claim 14 (depends on 8) The method of claim 8, further comprising: downsampling the mapping data by a scaling factor; dividing the mapping data into the plurality of voxels; and outputting an average point from the downsampled mapping data for each of the plurality of voxels. ### Claim 15 A non-transitory computer readable storage medium including instructions stored thereon which, when executed by one or more processors, cause the one or more processors to: obtain mapping data from a scanning sensor supported by a movable object; obtain positioning data from a positioning sensor supported by the movable object; associate the mapping data with the positioning data based at least on time data associated with the mapping data and the positioning data; and generate a map in a first coordinate system based at least on the associated mapping data and positioning data, the map comprising a plurality of voxels and wherein an occupancy probability threshold value is used to determine whether a voxel includes a point in the map. ### Claim 16 (depends on 15) The non-transitory computer readable storage medium of claim 15, wherein the instructions to generate the map comprising the plurality of voxels in the first coordinate system based at least on the associated mapping data and positioning data, when executed, further cause the one or more processors to: for each voxel of the plurality of voxels of the map: determine one or more points from the mapping data to be located in the voxel; and determine an occupancy probability for the voxel based at least on a number of points in that voxel. ### Claim 17 (depends on 16) The non-transitory computer readable storage medium of claim 16, wherein the instructions, when executed, further cause the one or more processors to: for each voxel having the occupancy probability greater than the occupancy probability threshold value: calculate an average position of the one or more points located in the voxel; and generate a point in the map at the average position. ### Claim 18 (depends on 17) The non-transitory computer readable storage medium of claim 17, wherein the instructions, when executed, further cause the one or more processors to: for each voxel having the occupancy probability greater than the occupancy probability threshold value: calculate an average intensity value of the one or more points in the voxel; and associate the average intensity value with the generated point in the map. ### Claim 19 (depends on 15) The non-transitory computer readable storage medium of claim 15, wherein the instructions, when executed, further cause the one or more processors to: transform the map into a second coordinate system; and output the transformed map. ### Claim 20 (depends on 15) The non-transitory computer readable storage medium of claim 15, wherein the scanning sensor includes a light detection and ranging (LiDAR) sensor, and wherein the positioning sensor includes a real time kinematic (RTK) sensor or an inertial measurement unit (IMU) sensor.