Image analysis for personal interaction

A system for image analysis for personal interaction, the system comprising:

• at least one audio sensor;
• at least one image sensor;
• and
• at least one processing unit configured to perform operations, the operations comprise:
• accessing a conversational artificial intelligence model;
• capturing audio data using the at least one audio sensor, the audio data includes an input from an entity in a natural language, the input includes at least a first part and a second part, the second part differs from the first part;
• capturing image data using the at least one image sensor, the image data depicts a particular movement, the particular movement is a movement of a particular portion of a particular body, the particular movement and the first part are concurrent, the particular body is associated with the entity;
• calculating a first convolution of a fragment of the audio data associated with the first part to obtain a first plurality of numerical result values;
• calculating a second convolution of a fragment of the audio data associated with the second part to obtain a second plurality of numerical result values;
• calculating a third convolution of at least part of the image data to obtain a third plurality of numerical result values;
• calculating a function of the first plurality of numerical result values, the second plurality of numerical result values and the third plurality of numerical result values to obtain a specific mathematical object in a mathematical space;
• using the conversational artificial intelligence model to analyze the audio data and the image data to generate a response in the natural language to the input, the response is based on the specific mathematical object, the input and the particular movement;
• and
• providing the generated response to the entity.

A method for image analysis for personal interaction, the method comprising:

• accessing a conversational artificial intelligence model;
• receiving audio data, the audio data includes an input from an entity in a natural language, the input includes at least a first part and a second part, the second part differs from the first part;
• receiving image data, the image data depicts a particular movement, the particular movement is a movement of a particular portion of a particular body, the particular movement and the first part are concurrent, the particular body is associated with the entity;
• calculating a first convolution of a fragment of the audio data associated with the first part to obtain a first plurality of numerical result values;
• calculating a second convolution of a fragment of the audio data associated with the second part to obtain a second plurality of numerical result values;
• calculating a third convolution of at least part of the image data to obtain a third plurality of numerical result values;
• calculating a function of the first plurality of numerical result values, the second plurality of numerical result values and the third plurality of numerical result values to obtain a specific mathematical object in a mathematical space;
• using the conversational artificial intelligence model to analyze the audio data and the image data to generate a response in the natural language to the input, the response is based on the specific mathematical object, the input and the particular movement;
• and
• providing the generated response to the entity.