Negotiation-based human-robot collaboration via augmented reality

A control planning system for an automaton, comprising:

• an automated planner configured to:
• automatically generate a plan for the automaton, comprising a sequence of actions and motion trajectory to be performed by the automaton to perform a task limited by a set of constraints;
• and
• automatically reschedule the sequence of actions and motion trajectory in dependence on a predicted waiting period to increase efficiency;
• a user interface, configured to present the plan to a human user;
• an input, configured to receive feedback from the human user relating to the presented plan;
• and
• a restrictor configured to automatically process the received feedback and present it to the automated planner as a constraint to update the set of constraints,
• wherein the automated planner is further configured to alter the plan for the automaton selectively dependent on the updated set of constraints.

A non-transitory computer readable medium, storing instructions for controlling an automaton, comprising:

• instructions for generating an efficient time-based plan for the automaton dependent on a set of constraints, comprising a sequence of actions and motion trajectory to be performed by the automaton;
• instructions for presenting the plan comprising the sequence of actions and motion trajectory through a human computer user interface for a human user;
• instructions for receiving feedback from the human user relating to the plan comprising sequence of actions and motion trajectory to be performed by the automaton;
• and
• instructions for altering the set of constraints on the plan in dependence on the received feedback from the human user and at least one predicted waiting period for the automaton to perform a respective action or motion trajectory.