Vehicle controller, vehicle control system, vehicle learning device, vehicle learning method, and memory medium

A vehicle control method that is performed by an execution device and a memory device, the vehicle control method comprising:

• using the memory device to store relationship defining data that defines a relationship between a state of a vehicle and an action variable related to an operation of an electronic device mounted on the vehicle;
• and
• using the execution device to
• obtain the state of the vehicle at each moment based on a detection value of a sensor at each moment,
• operate the electronic device based on a value of the action variable, the value of the action variable being determined by the relationship defining data and the obtained state of the vehicle,
• provide, based on the obtained state of the vehicle, a greater reward when a characteristic of the vehicle meets a standard than when the characteristic of the vehicle does not meet the standard,
• update the relationship defining data by inputting, to a predetermined update map, the obtained state of the vehicle, the value of the action variable used to operate the electronic device, and the reward corresponding to the operation of the electronic device,
• obtain a deterioration variable, the deterioration variable indicating a degree of deterioration of the vehicle, and
• define, as a return non-maximizing range, a range in which a value different from a value that maximizes an expected return related to the reward is used as the action variable, and
• in a case in which the degree of deterioration of the vehicle is greater than or equal to a predetermined degree, change the return non-maximizing range to a side on which the return non-maximizing range is expanded as compared to a case in which the degree of deterioration is less than the predetermined degree, and
• the update map outputs the updated relationship defining data so as to increase the expected return of a case in which the electronic device is operated in accordance with the relationship defining data.

A non-transitory computer readable medium that stores a vehicle control process executed by an execution device and a memory device, the vehicle control process comprises:

• using the memory device to store relationship defining data that defines a relationship between a state of a vehicle and an action variable related to an operation of an electronic device mounted on the vehicle;
• and
• using the execution device to:
• obtain the state of the vehicle at each moment based on a detection value of a sensor at each moment,
• operate the electronic device based on a value of the action variable, the value of the action variable being determined by the relationship defining data and the obtained state of the vehicle,
• provide, based on the obtained state of the vehicle, a greater reward when a characteristic of the vehicle meets a standard than when the characteristic of the vehicle does not meet the standard,
• update the relationship defining data by inputting, to a predetermined update map, the obtained state of the vehicle, the value of the action variable used to operate the electronic device, and the reward corresponding to the operation of the electronic device,
• obtain a deterioration variable, the deterioration variable indicating a degree of deterioration of the vehicle, and
• define, as a return non-maximizing range, a range in which a value different from a value that maximizes an expected return related to the reward is used as the action variable, and
• in a case in which the degree of deterioration of the vehicle is greater than or equal to a predetermined degree, change the return non-maximizing range to a side on which the return non-maximizing range is expanded as compared to a case in which the degree of deterioration is less than the predetermined degree, and
• wherein the update map outputs the updated relationship defining data so as to increase the expected return of a case in which the electronic device is operated in accordance with the relationship defining data.