Controller, robot system, system, control method, and storage medium including control program

A control method for controlling a robot based on an instruction with a user, the control method comprising:

• receiving, from the user, an instruction for the robot;
• outputting, to the user, a reaction from the robot to the instruction;
• saving data of the instruction in an input buffer;
• storing a history log in a memory;
• generating a prompt to inquire about a next action of the robot;
• transmitting the prompt to a natural language processing system using a large language model;
• receiving a response from the natural language processing system;
• accumulating, in the history log, the data read from the input buffer and the response from the natural language processing system to the data;
• generating an action command for the robot based on the response;
• transmitting the action command to the robot to cause the robot to physically perform the next action;
• receiving an action result of the next action from the robot;
• overwriting the data in the input buffer with the action result;
• and
• generating a subsequent prompt including the action result read from the input buffer and the history log.

A non-transitory computer-readable storage medium comprising a control program configured to control a robot based on an interaction with a user, the control program being configured to, when executed by a computer, cause the computer to:

• receive, from the user, an instruction for the robot;
• output, to the user, a reaction from the robot to the instruction;
• save data of the instruction in an input buffer;
• store a history log in a memory;
• generate a prompt to inquire about a next action of the robot;
• transmit the prompt to a natural language processing system using a large language model;
• receive a response from the natural language processing system;
• accumulate, in the history log, the data read from the input buffer and the response from the natural language processing system to the data;
• generate an action command for the robot based on the response;
• transmit the action command to the robot to cause the robot to physically perform the next action;
• receive an action result of the next action from the robot;
• overwrite the data in the input buffer with the action result;
• and
• generate a subsequent prompt including the action result read from the input buffer and the history log.