Non-transitory computer-readable recording medium, information processing method, and information processing apparatus

An information processing method executed by a computer, the information processing method comprising:

• acquiring video image data that includes target objects including a person and an object;
• specifying, by using graph data that defines a relationship between persons and/or a relationship between the person and the object, a relationship between each of the target objects included in the acquired video image data;
• specifying, by using a feature value including a facial expression and skeleton information indicating a joint position of the person included in the acquired video image data, transition of the skeleton information and facial expression information on the person from respective frames in the video data, and specifying transition of an elemental behavior associated with a basic action and the facial expression of the person from the specified transition of the skeleton information and the facial expression information on the person;
• specifying a current behavior of the person corresponding to the transition of the elemental behavior of the person specified in the video data, in accordance with a behavior identification rule associating the current behavior with the transition of the elemental behavior;
• and
• predicting, by inputting the specified current behavior of the person and the specified relationship to a probability model, a future behavior of the person performing the specified current behavior including a feeling and a state affected by inner emotions of the person, using a processor,
• wherein the probability model is a trained model that has learned i) a behavior transition probability as a probabilistic representation of a state transition indicating a possible future behavior after a certain behavior, and ii) a probability distribution obtained from observation information indicating the feature value and the relationship of the person in the video data.

An information processing apparatus comprising:

• a memory;
• and
• a processor coupled to the memory and configured to:
• acquire video image data that includes target objects including a person and an object;
• specify, by using graph data that defines a relationship between persons and/or a relationship between the person and the object, a relationship between each of the target objects included in the acquired video image data;
• specify, by using a feature value including a facial expression and skeleton information indicating a joint position of the person included in the acquired video image data, transition of the skeleton information and facial expression information on the person from respective frames in the video data, and specifying transition of an elemental behavior associated with a basic action and the facial expression of the person from the specified transition of the skeleton information and the facial expression information on the person;
• specify a current behavior of the person corresponding to the transition of the elemental behavior of the person specified in the video data, in accordance with a behavior identification rule associating the current behavior with the transition of the elemental behavior;
• and
• predict, by inputting the specified current behavior of the person and the specified relationship to a probability model, a future behavior of the person performing the specified current behavior including a feeling and a state affected by inner emotions of the person,
• wherein the probability model is a trained model that has learned i) a behavior transition probability as a probabilistic representation of a state transition indicating a possible future behavior after a certain behavior, and ii) a probability distribution obtained from observation information indicating the feature value and the relationship of the person in the video data.