# US 2024/0200959 A1 · Apparatus for learning energy efficiency, a system having the same, and a method thereof > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 2024/0200959 A1 - Document ID: US20240200959A1 - Status: Published application - Kind: A1 - Application number: 18/217,646 - Filed: 2023-07-03 - Published: 2024-06-20 - Earliest claimed: 2022-12-14 - Assignees: Hyundai Motor Company; Kia Corporation - Applicants: Hyundai Motor Company; Kia Corporation - Inventors: Dong Hoon Jeong; Jung Hwan Bang; Hyung Seuk Ohn; Won Seok Jeon; Ki Sang Kim; Byeong Wook Jeon; Dong Hoon Won; Hee Yeon Nah - Web page: https://patentgrep.com/patent/US20240200959A1 ## CPC - G01C 21/3469: {Fuel consumption; Energy use; Emission aspects} - B60W 40/105: Speed - B60W 2510/083: Torque - G01C: MEASURING DISTANCES, LEVELS OR BEARINGS; SURVEYING; NAVIGATION; GYROSCOPIC INSTRUMENTS; PHOTOGRAMMETRY OR VIDEOGRAMMETRY (measuring liquid level G01F; radio navigation, determining distance or velocity by use of propagation effects, e.g. Doppler effects, propagation time, of radio waves, analogous arrangements using other waves G01S) - B60W: CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT ## Abstract An energy efficiency improvement learning apparatus includes a processor configured to perform learning for vehicle control to minimize vehicle energy consumption required for vehicle driving under random conditions based on a reinforcement learning algorithm, and provide learned parameters of the reinforcement learning algorithm to a vehicle. The energy efficiency improvement learning apparatus also includes a storage medium configured to store data and algorithms, and the reinforcement learning algorithm executable by the processor. ## Claims ### Claim 1 An energy efficiency improvement learning apparatus comprising: a processor configured to: perform learning for vehicle control to minimize vehicle energy consumption required for vehicle driving under random conditions based on a reinforcement learning algorithm, and provide learned parameters of the reinforcement learning algorithm to a vehicle; and a storage medium configured to store data and algorithms, wherein the storage medium stores the reinforcement learning algorithm executable by the processor. ### Claim 2 (depends on 1) The energy efficiency improvement learning apparatus of claim 1, wherein the processor is configured to: randomly generate at least one of an entire path from a starting point to a destination, a road shape, a target driving time, an initial vehicle speed, an average target speed, an upper speed limit, a lower speed limit, or any combination thereof, and start a reinforcement learning episode including a process of a state, an action, and a reward from an initial state to an end state. ### Claim 3 (depends on 2) The energy efficiency improvement learning apparatus of claim 2, wherein the action includes motor demand torque. ### Claim 4 (depends on 2) The energy efficiency improvement learning apparatus of claim 2, wherein the reward includes at least one of energy efficiency, a driving time, or a speed range. ### Claim 5 (depends on 1) The energy efficiency improvement learning apparatus of claim 1, wherein the processor is configured to: perform a first action according to a first state, determine a first reward according to the first action, and transition to a second state when the reinforcement learning algorithm is executed. ### Claim 6 (depends on 5) The energy efficiency improvement learning apparatus of claim 5, wherein the storage medium is configured to store the first state, the first action, the first reward, and the second state as first data. ### Claim 7 (depends on 6) The energy efficiency improvement learning apparatus of claim 6, wherein the processor is configured to learn a policy function for determining what action to take by using at least one piece of data storing a state, an action, a reward, and a new state. ### Claim 8 (depends on 1) The energy efficiency improvement learning apparatus of claim 1, wherein the processor is configured to determine a reward based on a predetermined speed constraint condition, a predetermined time constraint condition, and energy required for driving. ### Claim 9 (depends on 1) The energy efficiency improvement learning apparatus of claim 1, wherein the processor is configured to, after starting a reinforcement learning episode under the random conditions, compare a driving time during a predetermined learning period with a predetermined time constraint condition. ### Claim 10 (depends on 9) The energy efficiency improvement learning apparatus of claim 9, wherein the processor is configured to: end the reinforcement learning episode when the driving time during the predetermined learning period is greater than the predetermined time constraint condition, and calculate a reward based on a vehicle speed when the driving time during the predetermined learning period is smaller than the predetermined time constraint condition. ### Claim 11 (depends on 10) The energy efficiency improvement learning apparatus of claim 10, wherein the processor is configured to determine the reward based on the vehicle speed as a predetermined negative reward if the vehicle speed is lower than a predetermined lower limit or higher than a predetermined upper limit. ### Claim 12 (depends on 10) The energy efficiency improvement learning apparatus of claim 10, wherein the processor is configured to: when the vehicle speed is lower than a predetermined lower limit for ending learning or is higher than a predetermined upper limit for ending learning: determine the reward based on the vehicle speed as a predetermined maximum negative reward, and end the reinforcement learning episode. ### Claim 13 (depends on 12) The energy efficiency improvement learning apparatus of claim 12, wherein the processor is configured to, after ending the reinforcement learning episode, determine a final reward by calculating a reward based on a time and summing the reward based on the vehicle speed and the reward based on the time. ### Claim 14 (depends on 10) The energy efficiency improvement learning apparatus of claim 10, wherein the processor is configured to, when the vehicle speed is a value between a predetermined lower limit speed for ending learning and a predetermined upper limit speed for ending learning, determine a weight reward using a predetermined weight and required motor power. ### Claim 15 (depends on 14) The energy efficiency improvement learning apparatus of claim 14, wherein the processor is configured to: determine whether a driving distance during the predetermined learning period is smaller than a predetermined target driving distance, and when the driving distance during the predetermined learning period is smaller than the predetermined target driving distance, determine a final reward by summing the reward based on the vehicle speed and the weight reward. ### Claim 16 (depends on 14) The energy efficiency improvement learning apparatus of claim 14, wherein the processor is configured to: determine whether a driving distance during the predetermined learning period is smaller than a predetermined target driving distance, and when the driving distance during the predetermined learning period is equal to or greater than the predetermined target driving distance: end the reinforcement learning episode, calculate a time-based reward, and sum a vehicle speed-based reward, the time-based reward, and the weight reward to determine a final reward. ### Claim 17 (depends on 16) The energy efficiency improvement learning apparatus of claim 16, wherein the processor is configured to calculate the time-based reward by using a maximum time constraint reward value, a time factor, and a distance factor. ### Claim 18 (depends on 14) The energy efficiency improvement learning apparatus of claim 14, wherein the processor is configured to: linearly assign a weight to a ratio of a target time to an actual driving time, and exponentially assign a weight to the ratio of a target distance to the actual driving distance. ### Claim 19 An energy efficiency improvement system, comprising: an energy efficiency improvement learning apparatus configured to: perform learning for vehicle control to minimize vehicle energy consumption required for vehicle driving under random conditions based on a reinforcement learning algorithm, and provide learned parameters of the reinforcement learning algorithm to a vehicle; and a vehicle control apparatus configured to determine a vehicle control command by inputting vehicle state information to the reinforcement learning algorithm to which learned parameters received from the energy efficiency improvement learning apparatus are applied. ### Claim 20 An electrical energy efficiency improvement method for a vehicle, comprising: performing, by a processor, learning for vehicle control to minimize vehicle energy consumption required for vehicle driving under random conditions based on a reinforcement learning algorithm; and providing, by the processor, learned parameters of the reinforcement learning algorithm to a vehicle.