Telepresence robots having cognitive navigation capability

A telepresence robotic system comprising:

• a Web-Real Time Communication (WebRTC) to receive one or more audio signals from an operator using a speech based Human Robot Interface (HRI) and one or more visual snap shots from a current Field of View (FoV) of a robot to perform a plurality of tasks, wherein the one or more audio signals comprising a command instruction to interact with the robot and an ordinary speech to interact with remote audience through the robot;
• one or more hardware processors;
• a memory in communication with the one or more hardware processors, wherein the one or more hardware processors are configured to execute programmed instructions stored in the memory, to:
• process the received one or more audio signals to detect the command instruction using a Convolution Neural Network (CNN) based binary classifier;
• transmit the detected command instruction to a robot edge via a cloud and edge based hybrid communication architecture for translating the command instruction into a textual instruction using an automated speech recognition model, wherein the automated speech recognition model is trained using a transformer based deep neural network;
• identify the plurality of tasks from the textual instruction to formulate planning problem based on a Planning Domain Definition Language (PDDL) format, wherein each of the plurality of tasks comprises one or more sub-tasks;
• generate an execution plan to map the one or more sub-tasks using an fast-forward (FF)-planner and the received one or more visual snap shots for grounding a location;
• and
• execute the generated execution plan through low level application programming interface (API) calls to reach the grounded location.

A non-transitory computer readable medium storing one or more instructions which when executed by one or more processors on a telepresence robotic system, cause the one or more processors to perform method comprising:

• receiving, via a Web-Real Time Communication (WebRTC), one or more audio signals from an operator using a speech based Human Robot Interface (HRI), and one or more visual snap shots from a current Field of View (FoV) of a robot to perform a plurality of tasks, wherein the one or more audio signals comprising a command instruction to interact with the robot and an ordinary speech to interact with remote audience through the robot;
• processing, via a one or more hardware processors, the received one or more audio signals to detect the command instruction using a Convolution Neural Network (CNN) based binary classifier;
• transmitting, via the one or more hardware processors, the detected command instruction to a robot edge for translating the command instruction into a textual instruction using an automated speech recognition model, wherein the automated speech recognition model is trained using a transformer based deep neural network;
• identifying, via the one or more hardware processors, the plurality of tasks from the textual instruction to formulate planning problem based on a Planning Domain Definition Language (PDDL) format, wherein each of the plurality of tasks comprises one or more sub-tasks;
• generating, via the one or more hardware processors, an execution plan to map the one or more sub-tasks using an fast-forward (FF)-planner and the received one or more snap shots for grounding a location;
• and
• executing, via the one or more hardware processors, the generated execution plan through low level application programming interface (API) calls to reach the grounded location.