Socially assistive robot

A companion robot comprising:

• a head comprising a facial display;
• a neck mechanism mechanically coupled with the head;
• a facial camera;
• a game engine;
• a microphone configured to receive audio signals from the environment;
• and
• a processor electrically coupled with the facial display, the facial camera, the game engine, the neck mechanism, and the microphone;
• the processor configured to:
• receive facial images from the facial camera;
• receive speech input from the microphone;
• determine a user's mood based on the speech input and/or the facial images;
• select a robot facial image from a subset of facial images based on the user's mood;
• select a sequence of neck movements from a subset of neck movements based on the user's mood;
• move the head in accordance with the selected sequence of neck movements via the neck mechanism;
• determine a cognitive game based on the user's mood;
• present the cognitive game to the user via a user interface;
• produce an image of a face related to the cognitive game by the game engine and project it onto the facemask via the projector;
• receive a response to the cognitive game from the user via the user interface;
• and
• display the selected facial images on the facial display.

A method for controlling a companion robot, the method comprising:

• receiving facial images from a facial camera;
• receiving speech input from a microphone;
• determining, with a processor, a user's mood based on the speech input and/or the facial images;
• selecting a facial image from a subset of facial images based on the user's mood and a history of interactions between a user and the companion robot;
• selecting a sequence of neck movements from a subset of neck movements based on the user's mood;
• moving a head in accordance with the selected sequence of neck movements;
• determining a cognitive game based on the user's mood;
• presenting the cognitive game to the user via a user interface;
• producing an image of a face related to the cognitive game by the game engine and project it onto the facemask via the projector;
• receiving a response to the cognitive game from the user via the user interface;
• and
• displaying the selected facial image on a facial display disposed on the head.

A companion robot comprising:

• a head comprising:
• a facemask;
• and
• a projector configured to project facial images onto the facemask;
• motorized arms;
• a facial camera;
• a game engine;
• a microphone configured to receive audio signals from the environment;
• a speaker configured to output audio signals;
• and
• a processor electrically coupled with the projector, the motorized arms, the facial camera, the game engine, the microphone, and the speaker;
• the processor configured to:
• receive a user's speech input from the microphone;
• receive facial images from the facial camera;
• determine an audio output based on the user's speech input;
• determine a facial projection output based on the user's speech input;
• determine one or more arm motions based on the speech input;
• determine a cognitive game based on the user's speech input and facial images;
• present the cognitive game to the user via a user interface;
• produce an image for a face related to the cognitive game by the game engine and project it onto the facemask via the projector;
• receive a response to the cognitive game from the user via the user interface;
• output the audio output via the speaker;
• project the facial projection output on the facemask via the projector;
• and
• move the motorized arms as per the one or more arm motions.

A companion robot comprising:

• a face screen configured to display facial images;
• a microphone configured to receive audio signals from the environment;
• a speaker configured to output audio signals;
• a 3D camera;
• a graphical user interface;
• and
• a processor electrically coupled with the face screen, the microphone, the speaker, the 3D camera, and the graphical user interface;
• the processor configured to:
• receive speech input from the microphone;
• determine an audio output based on the speech input;
• receive 3D images from the 3D camera;
• determine a facial projection output based on the speech input;
• present a first cognitive exercise to a user based on the speech input via the graphical user interface;
• receive a first response to the first cognitive exercise from the user via the graphical user interface;
• determine an activity level of the user based on the 3D images and the first cognitive exercise;
• present a second cognitive exercise to the user via the graphical user interface if the determined activity level is low;
• output the audio output via the speaker;
• project the facial projection output on the face screen.