Generating Physical Body Movements Based on Natural Language Inputs and Perceived Body Movements

A system for using perceived body movements to control generated body movements, the system comprising:

• at least one processing unit configured to perform operations, the operations comprise:
• accessing an artificial intelligence model;
• receiving audio data captured using at least one audio sensor, the audio data includes an input from an entity in a natural language, the input includes at least a first part and a second part, the second part differs from the first part;
• receiving image data captured using at least one image sensor, the image data depicts a particular movement, the particular movement is a movement of a particular portion of a particular body, the particular movement and the first part are concurrent, the particular body is associated with the entity;
• using the artificial intelligence model to analyze the audio data and the image data to determine a desired physical movement for a specific portion of a specific physical body of a robot, the desired physical movement is based on the input and the particular movement, the specific physical body differs from the particular body, the specific physical body of the robot includes one or more actuators;
• and
• generating digital signals, the digital signals are configured to control the one or more actuators to cause the desired physical movement to the specific portion of the specific physical body of the robot during an interaction with the entity.

A method for using perceived body movements to control generated body movements, the method comprising:

• accessing an artificial intelligence model;
• receiving audio data captured using at least one audio sensor, the audio data includes an input from an entity in a natural language, the input includes at least a first part and a second part, the second part differs from the first part;
• receiving image data captured using at least one image sensor, the image data depicts a particular movement, the particular movement is a movement of a particular portion of a particular body, the particular movement and the first part are concurrent, the particular body is associated with the entity;
• using the artificial intelligence model to analyze the audio data and the image data to determine a desired physical movement for a specific portion of a specific physical body of a robot, the desired physical movement is based on the input and the particular movement, the specific physical body differs from the particular body, the specific physical body of the robot includes one or more actuators;
• and
• generating digital signals, the digital signals are configured to control the one or more actuators to cause the desired physical movement to the specific portion of the specific physical body of the robot during an interaction with the entity.