Vehicle control data generating method, vehicle controller, vehicle control system, and vehicle learning device

A vehicle controller, comprising:

• a memory device and an execution device, wherein
• the vehicle controller
• stores, in the memory device, a deviation variable related to an amount of deviation of a road on which a vehicle is traveling from a reference road, and relationship-defining data that defines a relationship between a state of the vehicle and a gear ratio variable related to a gear ratio of a transmission mounted on the vehicle, and
• with the relationship-defining data stored in the memory device, causes the execution device to execute
• an operation process that operates the gear ratio of the transmission,
• an obtaining process that obtains the state of the vehicle and the deviation variable based on a detection value of a sensor,
• a reward calculating process that causes, on a basis of the state of the vehicle obtained by the state obtaining process, a reward, that is provided when a characteristic of the vehicle meets a predetermined standard, to be larger than a reward that is provided when the characteristic of the vehicle does not meet the predetermined standard, and
• an update process that updates the relationship-defining data by inputting, to a predetermined update map, the state of the vehicle and the deviation variable obtained by the state obtaining process, and the reward corresponding to the operation of the operation process,
• the update map outputs the relationship-defining data that has been updated to increase an expected return of the reward of a case in which the gear ratio is operated in accordance with the relationship-defining data, and
• the operation process includes a process that operates the transmission on a basis of the relationship-defining data and in accordance with a value of the gear ratio variable that corresponds to the state of the vehicle and the deviation variable.

A vehicle control system, comprising:

• an execution device and a memory device, wherein
• the vehicle control system
• stores, in the memory device, a deviation variable related to an amount of deviation of a road on which a vehicle is traveling from a reference road, and relationship-defining data that defines a relationship between a state of the vehicle and a gear ratio variable related to a gear ratio of a transmission mounted on the vehicle, and
• with the relationship-defining data stored in the memory device, causes the execution device to execute
• an operation process that operates the gear ratio of the transmission,
• an obtaining process that obtains the state of the vehicle and the deviation variable based on a detection value of a sensor,
• a reward calculating process that causes, on a basis of the state of the vehicle obtained by the state obtaining process, a reward, that is provided when a characteristic of the vehicle meets a predetermined standard, to be larger than a reward that is provided when the characteristic of the vehicle does not meet the predetermined standard, and
• an update process that updates the relationship-defining data by inputting, to a predetermined update map, the state of the vehicle and the deviation variable obtained by the state obtaining process, and the reward corresponding to the operation of the operation process,
• the update map outputs the relationship-defining data that has been updated to increase an expected return of the reward of a case in which the gear ratio is operated in accordance with the relationship-defining data,
• the operation process includes a process that operates the transmission on a basis of the relationship-defining data and in accordance with a value of the gear ratio variable that corresponds to the state of the vehicle and the deviation variable,
• the execution device includes a first execution device mounted on the vehicle and a second execution device that is an out-of-vehicle device,
• the first execution device executes at least the obtaining process and the operation process, and
• the second execution device executes at least the update process.

A vehicle controller employed in a vehicle control system, wherein

• the vehicle control system includes an execution device and a memory device,
• the vehicle controller
• stores, in the memory device, a deviation variable related to an amount of deviation of a road on which a vehicle is traveling from a reference road, and relationship-defining data that defines a relationship between a state of the vehicle and a gear ratio variable related to a gear ratio of a transmission mounted on the vehicle, and
• with the relationship-defining data stored in the memory device, causes the execution device to execute
• an operation process that operates the gear ratio of the transmission,
• an obtaining process that obtains the state of the vehicle and the deviation variable based on a detection value of a sensor,
• a reward calculating process that causes, on a basis of the state of the vehicle obtained by the state obtaining process, a reward, that is provided when a characteristic of the vehicle meets a predetermined standard, to be larger than a reward that is provided when the characteristic of the vehicle does not meet the predetermined standard, and
• an update process that updates the relationship-defining data by inputting, to a predetermined update map, the state of the vehicle and the deviation variable obtained by the state obtaining process, and the reward corresponding to the operation of the operation process,
• the update map outputs the relationship-defining data that has been updated to increase an expected return of the reward of a case in which the gear ratio is operated in accordance with the relationship-defining data,
• the operation process includes a process that operates the transmission on a basis of the relationship-defining data and in accordance with a value of the gear ratio variable that corresponds to the state of the vehicle and the deviation variable,
• the execution device includes a first execution device mounted on the vehicle and a second execution device that is an out-of-vehicle device,
• the first execution device executes at least the obtaining process and the operation process,
• the second execution device executes at least the update process, and
• the vehicle controller includes the first execution device.

A vehicle controller employed in a vehicle control system, wherein

• the vehicle control system includes an execution device and a memory device,
• the vehicle controller
• stores, in the memory device, a deviation variable related to an amount of deviation of a road on which a vehicle is traveling from a reference road, and relationship-defining data that defines a relationship between a state of the vehicle and a gear ratio variable related to a gear ratio of a transmission mounted on the vehicle, and
• with the relationship-defining data stored in the memory device, causes the execution device to execute
• an operation process that operates the gear ratio of the transmission,
• an obtaining process that obtains the state of the vehicle and the deviation variable based on a detection value of a sensor,
• a reward calculating process that causes, on a basis of the state of the vehicle obtained by the state obtaining process, a reward, that is provided when a characteristic of the vehicle meets a predetermined standard, to be larger than a reward that is provided when the characteristic of the vehicle does not meet the predetermined standard, and
• an update process that updates the relationship-defining data by inputting, to a predetermined update map, the state of the vehicle and the deviation variable obtained by the state obtaining process, and the reward corresponding to the operation of the operation process,
• the update map outputs the relationship-defining data that has been updated to increase an expected return of the reward of a case in which the gear ratio is operated in accordance with the relationship-defining data,
• the operation process includes a process that operates the transmission on a basis of the relationship-defining data and in accordance with a value of the gear ratio variable that corresponds to the state of the vehicle and the deviation variable,
• the execution device includes a first execution device mounted on the vehicle and a second execution device that is an out-of-vehicle device,
• the first execution device executes at least the obtaining process and the operation process,
• the second execution device executes at least the update process, and
• the vehicle controller includes the second execution device.