Apparatus and method for generating robot interaction behavior

An apparatus for generating robot interaction behavior, comprising:

• a memory for storing at least one program;
• and
• a processor for executing the program,
• wherein the program performs:
• generating a co-speech gesture of a robot corresponding to an utterance input of a user;
• generating a nonverbal behavior of the robot, that is a sequence of next joint positions of the robot, which are estimated from joint positions of the user and current joint positions of the robot based on a pre-trained neural network model for robot pose estimation;
• and
• generating a final behavior using at least one of the co-speech gesture and the nonverbal behavior,
• wherein the pre-trained neural network model for robot pose estimation includes an encoder, which receives the position values of joints of the user for respective sequences, extracted from a first-person view image of the robot, and processes the position values of the joints;
• and a decoder, which receives the output values of the encoder and the current pose of the robot and outputs the next positions of the joints of the robot.

A method for generating robot interaction behavior, comprising:

• generating a co-speech gesture of a robot corresponding to an utterance input of a user;
• generating a nonverbal behavior of the robot corresponding to a behavior input of the user based on a neural network model for robot pose estimation;
• and
• generating a final behavior using at least one of the co-speech gesture and the nonverbal behavior,
• wherein the robot pose estimation neural network model includes an encoder, which receives the position values of joints of the user for respective sequences, extracted from a first-person view image of the robot, and processes the position values of the joints;
• and a decoder, which receives the output values of the encoder and the current pose of the robot and outputs the next positions of the joints of the robot,
• when both of the co-speech gesture and the nonverbal behavior are generated,
• determining priorities of two behaviors including the co-speech gesture and the nonverbal behavior;
• calculating a motional variation of a behavior having a higher priority among the two behaviors;
• and
• outputting a new behavior based on the motional variation,
• wherein the new behavior is any one of the two behaviors or the two behaviors so as to overlap each other.