US 12,449,821 B2 · Autonomous source localization

Abstract

An autonomous system for detecting, localizing, and potentially deactivating chemical threats or emissions using multiple sensing modalities and reinforcement learning techniques. The system includes visual sensors (e.g., RGB, RGBD, LIDAR), non-visual sensors (e.g., gas concentration, airflow, GPS, RADAR), a neural network architecture and processor to fuse information from different sensors, a module based on deep reinforcement learning for decision making, and a robotic interface for executing actions. The neural network extracts relevant information from sensor streams and encodes them into a joint embedding space. The module considers the current observations, historical data, and previous actions to determine the optimal action for threat localization under partially observable conditions. The system is trained in simulated environments to minimize source localization time while accounting for various constraints. The autonomous system enables effective chemical threat detection and source localization in complex, dynamic environments without endangering human operators.

Claims21

Dependent claims

An autonomous source localization system comprising:

a robotic platform comprising a plurality of sensors that collect visual, non-visual, and gas concentration data of an environment being sensed by the plurality of sensors;
a module that integrates the visual, non-visual, and gas concentration data into a joint embedding space using a neural network architecture, wherein the neural network architecture is based on a transformer architecture;
a processor to:
receive the joint embedding space from the module;
receive a history of system states and actions of the robotic platform wherein the history of system states and actions comprises a location and orientation of the robotic platform and previous actions taken by the robotic platform;
and
select an operational action for the robotic platform based on the joint embedding space, the history of system states and actions, and neural network inputs into the processor;
a robotic interface that causes the robotic platform to perform the operational action selected by the processor.

5 more elements

A non-transitory computer-readable medium storing instructions for performing autonomous source localization in an environment being sensed, the instructions comprising:

collecting visual, non-visual, and gas concentration data of the environment being sensed by a plurality of sensors on a robotic platform;
integrating the visual, non-visual, and gas concentration data into a joint embedding space using a neural network architecture, wherein the neural network architecture is based on a transformer architecture;
receiving a history of system states and actions of the robotic platform, wherein the history of system states and actions comprises a location and orientation of the robotic platform and previous actions taken by the robotic platform;
selecting an operational action for the robotic platform based on the joint embedding space, the history of system states and actions, and neural network inputs;
and
causing the robotic platform to perform a selected operational action.

3 more elements

Dependencies are inferred from the claim text.

Autonomous source localization

Life of this patent

Autonomous source localization

Life of this patent

Abstract

Claims21