Learning device, information processing device, and learned control model

An information processing device comprising:

• at least one memory configured to store program code;
• and at least one processor configured to access the program code and operate as instructed by the program code, the program code including:
• storage code configured to cause the at least one processor to store a learned machine learning model to control a flight of an aerial vehicle that inputs environmental information related to a surrounding environment of an aerial vehicle capable of flying unmanned and an environment of a planned flight area of the aerial vehicle, the environmental information comprising a three-dimensional map of the surrounding environment of the aerial vehicle based on Simultaneous Localization and Mapping (SLAM) and the environment of the planned flight area of the aerial vehicle, wherein the three-dimensional map incorporates weather information of at least one of the surrounding environment of the aerial vehicle and the environment of the planned flight area of the aerial vehicle, and wherein the weather information comprises wind information, and outputs control information for performing flight control of the aerial vehicle, the machine learning model to control the flight of the aerial vehicle being obtained by performing reinforcement learning using a reward representing an evaluation of a result of an action of the aerial vehicle when the aerial vehicle takes the action based on the control information;
• control information acquisition code configured to cause the at least one processor to acquire the control information output from the machine learning model to control the flight of the aerial vehicle by inputting the environmental information, as an environment state, including weather information to the machine learning model to control the flight of the aerial vehicle, the control information being a plurality of action value functions or a probability distribution corresponding to a plurality of different actions of the aerial vehicle;
• and
• selection code configured to cause the at least one processor to:
• based on the control information being the probability distribution, select an action of the aerial vehicle, from among the plurality of different actions, according to the probability distribution, or
• based on the control information being the plurality of action value functions, select the action of the aerial vehicle according to an action value function having a largest value from among the plurality of action value functions.

A non-transitory computer readable memory having stored thereon a learned machine learning model to control a flight of an aerial vehicle configured to cause a computer to function so as to output control information for performing flight control of an aerial vehicle capable of flying unmanned, based on environmental information on a surrounding environment of the aerial vehicle and an environment of a planned flight area of the aerial vehicle, the environmental information comprising a three-dimensional map of the surrounding environment of the aerial vehicle based on Simultaneous Localization and Mapping (SLAM) and the environment of the planned flight area of the aerial vehicle, wherein the three-dimensional map incorporates weather information of at least one of the surrounding environment of the aerial vehicle and the environment of the planned flight area of the aerial vehicle, and wherein the weather information comprises wind information, wherein

• the environmental information is input as an environment state to the learned machine learning model to control the flight of the aerial vehicle in time series,
• the learned machine learning model to control the flight of the aerial vehicle is reinforcement-learned using a reward representing an evaluation of a result of an action of the aerial vehicle when the aerial vehicle takes the action based on the control information output from the learned machine learning model to control the flight of the aerial vehicle, the control information being a plurality of action value functions or a probability distribution corresponding to a plurality of different actions of the aerial vehicle,
• the learned machine learning model to control the flight of the aerial vehicle causes a computer to function so as to perform an operation based on a function constituting the learned machine learning model to control the flight of the aerial vehicle with respect to the input environmental information and output the control information from the machine learning model to control the flight of the aerial vehicle, and
• the learned machine learning model causes a computer to function so as to:
• based on the control information being the probability distribution, select an action of the aerial vehicle, from among the plurality of different actions, according to the probability distribution, or
• based on the control information being the plurality of action value functions, select the action of the aerial vehicle according to an action value function having a largest value from among the plurality of action value functions.