# US 2019/0377346 A1 · Multi-terrain inspection robotic device and methods for configuring and guiding the same > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 2019/0377346 A1 - Document ID: US20190377346A1 - Status: Published application - Kind: A1 - Application number: 16/485,397 - Filed: 2018-02-09 - Published: 2019-12-12 - Earliest claimed: 2017-02-13 - Assignees: Vale S.A.; Serviço Nacional de Aprendizagem Industrial, Departamento Regional da Bahia - Applicants: Vale S.A.; Serviço Nacional de Aprendizagem Industrial, Departamento Regional da Bahia - Inventors: Gustavo Medeiros FREITAS; Filipe Augusto Santos ROCHA; Marcos Paulo TORRE; Angelo Frederico FONTES JUNIOR; Victória Regina RAMOS; Luiz Eduardo Da Costa NOGUEIRA; Alexandre Souza SANTOS; Eduardo COTA; Wilson MIOLA; Marco Antonio DOS REIS; Branilson Luiz Santos COSTA; Luis Carlos Moreno LEDEZMA; Ronaldo Pereira EVANGELISTA; Pedro Xavier ALCANTARA; Rebeca Tourinho LIMA; Tiago Pereira DE SOUZA; luri Viana BRANDI; Ramon Nunes Araújo; Márcio Fernando Mansur GOMES; Gabriel Carvalho GARCIA - Web page: https://patentgrep.com/patent/US20190377346A1 ## CPC - G05D 1/0094: {involving pointing a payload, e.g. camera, weapon, sensor, towards a fixed or moving target} - B62D 55/04: with tracks and alternative ground wheels, e.g. changeable from endless track vehicle into wheeled vehicle and vice versa {(for multi-purpose tractors B62D49/0635)} - B62D 55/06: with tracks without ground wheels - B62D 57/022: {consisting of members having both rotational and walking movements} - B62D 57/028: having wheels and mechanical legs (B62D57/024 takes precedence; ground-engaging vehicle fittings for supporting, lifting or manoeuvring the vehicle, wholly or in part B60S9/00) - B62D 57/032: with alternately or sequentially lifted supporting base and legs; with alternately or sequentially lifted feet or skid (B62D57/024 takes precedence) - G05D 1/0088: {characterized by the autonomous decision making process, e.g. artificial intelligence, predefined behaviours (using knowledge based models G06N5/00)} - G05D 1/0214: {in accordance with safety or protection criteria, e.g. avoiding hazardous areas (monitoring the location of vehicles within a certain area, e.g. forbidden or allowed areas, in traffic control systems for road vehicles G08G1/13)} - G05D 1/0274: {using mapping information stored in a memory device (navigation using map-matching G01C21/30)} - G05D: SYSTEMS FOR CONTROLLING OR REGULATING NON-ELECTRIC VARIABLES - B62D: MOTOR VEHICLES; TRAILERS (steering, or guiding on a desired track, of agricultural machines or implements A01B69/00; wheels, castors, axles, increasing wheel adhesion B60B; vehicle tyres, tyre inflation or tyre changing B60C; connections between vehicles of a train or the like B60D; vehicles for use on rail and road, amphibious or convertible vehicles B60F; suspension arrangements B60G; heating, cooling, ventilating or other air treating devices B60H; windows, windscreens, non-fixed roofs, doors or similar devices, protective coverings for vehicles not in use B60J; propulsion plant arrangements, auxiliary drives, transmissions, controls, instrumentation or dashboards B60K; electric equipment or propulsion of electrically-propelled vehicles B60L; power supply for electrically-propelled vehicles B60M; passenger accommodation not otherwise provided for B60N; adaptations for load transportation or to carry special loads or objects B60P; arrangement of signalling or lighting devices, the mounting or supporting thereof or circuits therefor, for vehicles in general B60Q; vehicles, vehicle fittings or vehicle parts not otherwise provided for B60R; servicing, cleaning, repairing, supporting, lifting, or manoeuvring, not otherwise provided for, B60S; brake arrangements, brake control systems or parts thereof B60T; air-cushion vehicles B60V; motor-cycles, accessories therefor B62J, B62K; testing of vehicles G01M) ## Abstract This disclosure presents a robotic device for multi-terrain inspection, composed by a robot body, a quick reconfigurable locomotion module and a mapping unit capable to model the inspected environment through a 3D colored point cloud. The robot has different locomotion mechanisms that can be quickly replaced, thereby changing the robot mobility characteristics. The device is controlled through teleoperation or autonomously. When in teleoperated mode, an operating assist module provides relevant locomotion information to the operator including a map that shows areas where the robot may not transpose or tip-over. This module also suggests to the operator other locomotion configurations to overcome obstacles presented in the map. When in autonomous mode, the navigation module provides a strategy to explore unknown environments and trace optimal locomotion path considering the traveled distance, tipping-over risk and energy consumption. Regarding the invention characteristics described above, the main objective is to perform inspections of confined and risk areas, i.e., caves, sewer and dam spillway galleries, and areas with risk of collapse. ## Claims ### Claim 1 A multi-terrain inspection robotic device, comprising: a robot frame, comprising a plurality of cameras with an illumination system; a mapping unit attached to the robot frame, comprising a plurality of sensors; a reconfigurable locomotion module configured to receive a plurality of locomotion mechanisms arranged according to selectable assembly patterns; an operating assist module configured to receive data from the plurality of sensors and the plurality of cameras and configured to identify an adequate assembly pattern of the plurality of locomotion mechanisms; and an autonomous navigation module configured to determine an optimal path planning for the multi-terrain inspection robotic device, wherein the autonomous navigation module is further configured to indicate an alternative assembly pattern of the plurality of locomotion mechanisms. ### Claim 2 (depends on 1) The multi-terrain inspection robotic device of claim 1, wherein the robot frame has a rectangular shape with six rotational joints, including three for each side. ### Claim 3 (depends on 2) The multi-terrain inspection robotic device of claim 2, wherein each joint has its own independent controllable motor. ### Claim 4 (depends on 1) The multi-terrain inspection robotic device of claim 1, wherein the plurality of locomotion mechanisms of the reconfigurable locomotion module are selected from a group comprising: common wheels; star-shaped wheels; arc-shaped legs; or steer tracks. ### Claim 5 (depends on 4) The multi-terrain inspection robotic device of claim 4, wherein the plurality of locomotion mechanisms of the reconfigurable locomotion module are promptly and unitarily replaced using a quick release pin. ### Claim 6 (depends on 4) The multi-terrain inspection robotic device of claim 4, wherein the reconfigurable locomotion module enables the assembly pattern of the plurality of locomotion mechanisms to be homogeneous, using only one type of locomotion mechanism of the plurality of locomotion mechanisms, or heterogeneous, using different types of the plurality of locomotion mechanisms together. ### Claim 7 (depends on 1) The multi-terrain inspection robotic device of claim 1, wherein the plurality of sensors of the mapping unit comprises: a three dimensional (3D) laser scanner, a full high definition (HD) color camera, a high brightness light emitting diode (LED) illuminator, and an Inertial Measurement Unit. ### Claim 8 (depends on 1) The multi-terrain inspection robotic device of claim 1, wherein the mapping unit has includes a laser scanner in a vertical position and a 360° rotational mechanism, which enables the sensor to scan surroundings of the multi-terrain inspection robotic device. ### Claim 9 (depends on 7) The multi-terrain inspection robotic device of claim 7, wherein the plurality of sensors of the mapping unit are configured to provide raw data, wherein the raw data includes 3D point cloud data and colored image data. ### Claim 10 (depends on 9) The multi-terrain inspection robotic device of claim 9, wherein the 3D point cloud data and the colored image data are fused to generate a colored 3D point cloud of an inspected area. ### Claim 11 (depends on 9) The multi-terrain inspection robotic device of claim 9, wherein the 3D point cloud data is used to generate a 3D mesh of an inspected area, wherein the 3D mesh defines a topographic map of surroundings of the multi-terrain inspection robotic device. ### Claim 12 (depends on 1) The multi-terrain inspection robotic device of claim 1, wherein the multi-terrain inspection robotic device is specifically designed for operating in mining activities. ### Claim 13 (depends on 1) The multi-terrain inspection robotic device of claim 1, wherein the multi-terrain inspection robotic device is specifically designed for operating in speleology activities. ### Claim 14 (depends on 1) The multi-terrain inspection robotic device of claim 1, wherein the multi-terrain inspection robotic device is specifically designed for cave inspections in speleology activities. ### Claim 15 (depends on 1) The multi-terrain inspection robotic device of claim 1, wherein the multi-terrain inspection robotic device is specifically designed for operating in confined areas. ### Claim 16 (depends on 1) The multi-terrain inspection robotic device of claim 1, wherein the multi-terrain inspection robotic device is specifically designed for operating in sewer and dam spillway galleries. ### Claim 17 (depends on 1) The multi-terrain inspection robotic device of claim 1, wherein the multi-terrain inspection robotic device is specifically designed for operating in areas with risk of collapse. ### Claim 18 (depends on 1) A method for guiding the multi-terrain inspection robotic device of claim 1, comprising: obtaining an updated topographic map of surroundings of the multi-terrain inspection robotic device; estimating a pose of the multi-terrain robotic device over numerous map locations for each of the plurality of locomotion mechanisms configurations; calculating stability metrics for all of the estimated poses; identifying areas in the updated topographic map for a locomotion mechanism in use of the plurality of locomotion mechanisms where a corresponding one of the stability metrics indicates that the multi-terrain inspection robotic device may tip-over with the locomotion mechanism in use, analyzing a corresponding one of the stability metrics of other ones of the plurality of locomotion mechanisms in the identified areas and identifying whether one or more of the other ones of the plurality of locomotion mechanisms do not tip-over while transposing the identified areas in order to define one or more transposable areas, tipping-over risk areas, or non-transposable areas; and generating a map with the one or more ones of the transposable areas, the tipping-over risk areas, or the non-transposable areas with the locomotion mechanism in use, also indicating alternative ones from the other ones of the plurality of locomotion mechanisms identified as being able to transpose the restricted areas. ### Claim 19 (depends on 18) The method of claim 18, further comprising determining a tipping-over risk estimation based on the stability metrics, the stability metrics being applied to the updated topographic map of the surroundings of the multi-terrain robotic device, considering a geometry of the multi-terrain robotic device according to the locomotion mechanism in use and the estimated poses and wherein the stability metrics include a force-angle measure of a tip-over stability margin and an energy stability margin. ### Claim 20 (depends on 1) A method for guiding the multi-terrain inspection robotic device of claim 1, comprising: obtaining an updated topographic map of surroundings of the multi-terrain inspection robotic device; obtaining gain values representing a mobility policy, which indicates which metric to optimize of at least one of: a shortest path, a minimum tipping-over risk, or a minimum energy consumption; obtaining a target point, which is either directly informed or obtained via an exploring algorithm; removing from the topographic map areas with a tipping-over risk and non-transposable areas by a locomotion mechanism in use to define a modified map; tracing, considering the modified map, the shortest path to the target point using a path planning algorithm; tracing, considering the modified map, a second path with less tipping-over risk to the target point as compared to the shortest path using a first path planning algorithm based on stability metrics, including force-angle measure of a tip-over stability margin or energy based stability margin; tracing, considering the modified map, an optimal energy consumption path to the target point using a second path planning algorithm based on the energy consumption of motors of the multi-terrain robotic device; choosing which one of the shortest path, the first path, or the second path is going to be used depending on a mobility policy defined for the autonomous navigation module; and commanding the multi-terrain robotic device to reach the goal point based on a chosen one of the shortest path, the first path, or the second path. ### Claim 21 (depends on 6) The multi-terrain inspection robotic device of claim 6, wherein, the assembly pattern is configured as one of: 6 common wheels; 6 star-shaped wheels; 6 arc-shaped legs; 4 common wheels on the corners and 2 central legs; 4 common wheels on the corners and 2 central star-shaped wheels; or 4 common wheels on the corners with steer tracks coupled thereto. ### Claim 22 (depends on 1) The method for guiding the multi-terrain inspection robotic device of claim 1, wherein the exploring algorithm comprises a cellular decomposition for complete coverage path planning to inspect the entire environment.