Readable and editable NPC behavior creation using reinforcement learning

A computer implemented method comprising:

• generating, by one or more processors and based on user input, an initial behavior tree model for controlling behavior of a non-playable character in a computer game environment, wherein input to a behavior tree model is game environment data and output from the behavior tree model is non-playable character behavior data for causing the non-playable character in the computer game environment to perform one or more actions;
• determining, by the one or more processors and based on the behavior tree model, a behavior goal for a reinforcement-learning algorithm using an inverse reinforcement-learning technique;
• and
• training, by the one or more processors and based on the determined behavior goal and using the reinforcement-learning algorithm, a reinforcement-learning model for controlling behavior of the non-playable character in the computer game environment wherein input to the reinforcement-learning model is the game environment data and output from the reinforcement-learning model is the non-playable character behavior data for causing the non-playable character in the computer game environment to perform the one or more actions.

Apparatus comprising one or more processors and a memory, the memory comprising instructions that, when executed by the one or more processors, cause the apparatus to perform operations comprising:

• training, by the one or more processors and based on an initial behavior goal and using reinforcement-learning, a reinforcement-learning model for controlling behavior of a non-playable character in a computer game environment, wherein input to the reinforcement-learning model is game environment data and output from the reinforcement-learning model is non-playable character behavior data for causing the non-playable character in the computer game environment to perform one or more actions;
• converting, by the one or more processors, the trained reinforcement-learning model into a behavior tree model for controlling behavior of the non-playable character;
• editing, by the one or more processors and based on a user input, the behavior tree model to generate an updated behavior tree model for controlling behavior of the non-playable character;
• and
• outputting a final model for controlling non-player character behavior for use in the computer game environment, wherein the model of non-player character behavior is based at least in part on the updated behavior tree model, and wherein input to the final model for controlling non-player character behavior is game environment data and output from the final model for controlling non-player character behavior is non-playable character behavior data for causing the non-playable character in the computer game environment to perform the one or more actions.