Robot systems, methods, control modules, and computer program products that leverage large language models
AssigneeSanctuary Cognitive Systems Corporation
InventorsGeordie Rose, Suzanne Gildert
Life of this patent
Estimating the term…
Abstract
Robot control systems, methods, control modules and computer program products that leverage one or more large language model(s) (LLMs) in order to achieve at least some degree of autonomy are described. Robot control parameters and/or instructions may advantageously be specified in natural language (NL) and communicated with the LLM via an NL prompt or query. The LLM module provides a task plan in NL, which can be evaluated for at least one fault or error. If at least one fault or error is identified, the LLM module can be queried to provide a resolution.
Claims20
A method of operation of a robot system, the method comprising:
- accessing a task plan by at least one processor of the robot system, the task plan expressed in a natural language (NL) and including a plurality of steps;
- determining, by the at least one processor, that the task plan includes at least one fault;
- generating, by the at least one processor, an NL description of the at least one fault;
- providing a NL query to a large language model (LLM) module, the NL query including at least the NL description of the at least one fault and an NL request for a resolution plan;
- receiving the resolution plan from the LLM module, the resolution plan expressed in NL;
- and
- executing an updated task plan by the robot system, the updated task plan based at least in part on the resolution plan.
A robot control module comprising at least one non-transitory processor-readable storage medium storing processor-executable instructions or data that, when executed by at least one processor of a processor-based system, cause the processor-based system to:
- access a task plan, the task plan expressed in a natural language (NL) and including a plurality of steps;
- determine, by the at least one processor, that the task plan includes at least one fault;
- generate, by the at least one processor, an NL description of the at least one fault;
- provide a NL query to a large language model (LLM) module, the NL query including at least the NL description of the at least one fault and an NL request for a resolution plan;
- receive the resolution plan from the LLM module, the resolution plan expressed in NL;
- and
- execute an updated task plan by the processor-based system, the updated task plan based at least in part on the resolution plan.
A robot system comprising:
- a robot body;
- at least one sensor;
- a robot controller including at least one processor and at least one non-transitory processor-readable storage medium, the at least one non-transitory processor-readable storage medium storing processor-executable instructions which when executed by the at least one processor cause the robot system to:
- access a task plan, the task plan expressed in a natural language (NL) and including a plurality of steps;
- determine, by the at least one processor, that the task plan includes at least one fault;
- generate, by the at least one processor, an NL description of the at least one fault;
- provide a NL query to a large language model (LLM) module, the NL query including at least the NL description of the at least one fault and an NL request for a resolution plan;
- receive the resolution plan from the LLM module, the resolution plan expressed in NL;
- and
- execute an updated task plan by the robot system, the updated task plan based at least in part on the resolution plan.
Dependencies are inferred from the claim text.