# US 12,179,776 B2 · Apparatus and method for determining optimal velocity of vehicle > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 12,179,776 B2 - Document ID: US12179776B2 - Status: Granted - Kind: B2 - Application number: 17/405,867 - Filed: 2021-08-18 - Published: 2022-06-30 - Granted: 2024-12-31 - Earliest claimed: 2020-12-29 - Assignees: Hyundai Motor Company; Kia Corporation - Applicants: Hyundai Motor Company; Kia Corporation - Inventors: Hee Yun Lee - Web page: https://patentgrep.com/patent/US12179776B2 ## CPC - B60W 30/143: {Speed control (B60W30/16 takes precedence)} - B60W 40/10: related to vehicle motion - B60W 40/04: Traffic conditions - B60W 60/0023: {in response to energy consumption} - G01R 31/382: Arrangements for monitoring battery or accumulator variables, e.g. SoC - G01S 19/01: Satellite radio beacon positioning systems transmitting time-stamped messages, e.g. GPS [Global Positioning System], GLONASS [Global Orbiting Navigation Satellite System] or GALILEO - B60W 2050/0025: {Transfer function weighting factor} - B60W 2510/244: Charge state - B60W 2520/00: Input parameters relating to overall vehicle dynamics - B60W 2520/10: Longitudinal speed - B60W 2552/00: Input parameters relating to infrastructure - B60W 2552/15: Road slope, i.e. the inclination of a road segment in the longitudinal direction - B60W 2554/406: Traffic density - B60W 2554/80: Spatial relation or speed relative to objects - B60W 2554/802: Longitudinal distance - B60W 2556/50: of positioning data, e.g. GPS [Global Positioning System] data - B60W 2720/103: Speed profile - B60W: CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT - G01R: MEASURING ELECTRIC VARIABLES; MEASURING MAGNETIC VARIABLES (indicating correct tuning of resonant circuits H03J3/12) - G01S: RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES ## Abstract An apparatus of determining an optimal velocity of a vehicle, may include an information receiving unit configured to receive and provide vehicle traveling information and traveling environment information which are state variables representing vehicle states required to determine a target velocity for optimizing vehicle fuel economy; and an optimal velocity determination unit configured to determine the target velocity in accordance with a vehicle traveling environment by use of a state variable and reward estimation model and a Q table having values according to the state variables and a control input, from the vehicle traveling information and the traveling environment information provided by the information receiving unit, and a method of determining an optimal velocity of a vehicle. ## Claims ### Claim 1 An apparatus for determining an optimal velocity of a vehicle, the apparatus comprising: an information receiving unit configured to receive and provide vehicle traveling information and traveling environment information which are state variables representing vehicle states required to determine a target velocity for optimizing vehicle fuel economy; an optimal velocity determination unit configured to determine the target velocity in accordance with a vehicle traveling environment by use of a state variable and reward estimation model and a Q table having values according the state variables and a control input, from the vehicle traveling information and the traveling environment information provided by the information receiving unit; and a control unit configured to control a vehicle velocity to follow the target velocity, wherein the control unit is configured to select a control input (u) having a smallest Q value among Q values (x, {u1, u2, u3 . . . }) of corresponding state variable and determine the selected control input as an optimal control input (u) to follow the target velocity, wherein the state variable and reward estimation model unit updates non-linear state variable estimation models through the state variables and a reward cost value, wherein the reward cost value (gk) is determined, by the following equation, as a value made by applying weighting to each of a function (d(rk+1)) in respect to a time (Δtime) for which the vehicle travels a predetermined unit distance and the distance (rk+1) from a preceding vehicle, and a function (vlimit(vk+1)) in respect to a target velocity deviation, and then summing up a change amount in battery SOC (ΔSOC) and the values to which the weighting is applied, g k + 1 = ( ∫ d k d k + 1 ⁢ S ⁢ O . ⁢ C v ⁡ ( d ) ⁢ dd + ∫ d k d k + 1 ⁢ 1 v ⁡ ( d ) ⁢ dd ) + η · d ⁡ ( r k + 1 ) + ξ · v limit ⁡ ( v k + 1 ) = Δ ⁢ ⁢ SOC + ω · Δ ⁢ ⁢ time + η · d ⁡ ( r k + 1 ) + ξ · v limit ⁡ ( v k + 1 ) ⁢ ⁢ wherein , d ⁡ ( r k + 1 ) = { 0 ⁢ ⁢ if ⁢ ⁢ r k + 1 > r safe 1 r k + 1 ⁢ ⁢ if ⁢ ⁢ r k + 1 ≤ r safe , v limit ⁡ ( v k + 1 ) = ( v k + 1 - v ref ) 2 , Equation and ω, η, ξ represent predetermined weighting constants, rsafe represents a safety distance from the preceding vehicle which is predetermined as a constant value, and vref represents a predetermined reference velocity. ### Claim 2 (depends on 1) The apparatus of claim 1, wherein the vehicle traveling information and the traveling environment information, which are the state variables, include one or more of vehicle position information, information related to a current velocity of the vehicle, information related to a gradient of a traveling road, a distance from the preceding vehicle, information related to real-time traffic on a road on a vehicle traveling route, and information related to an amount of remaining energy of the vehicle. ### Claim 3 (depends on 2) The apparatus of claim 2, wherein the information receiving unit includes: a Global Positioning System (GPS) receiving unit configured to receive vehicle position information; a traveling information receiving unit configured to receive the information related to the current velocity of the vehicle and the information related to the gradient of the traveling road; a sensor information receiving unit configured to receive the distance from the preceding vehicle; a traffic information receiving unit configured to receive the information related to the real-time traffic on the road on the vehicle traveling route; and a vehicle energy information receiving unit configured to the information related to the amount of the remaining energy of the vehicle. ### Claim 4 (depends on 3) The apparatus of claim 3, wherein the information related to the amount of remaining energy of the vehicle is information related to a state of charge (SOC) value of a battery that supplies operating power to a motor configured for driving the vehicle. ### Claim 5 (depends on 1) The apparatus of claim 1, wherein the optimal velocity determination unit includes: a calculation unit configured to estimate energy consumption according to a current velocity (vk) of the vehicle and the control input determined and provided by the control unit, and a velocity (vk+1) of a next step by use of the state variable and reward estimation model, from the vehicle traveling information and the traveling environment information, and to determine and update the Q table value by use of the estimated information; and the control unit configured to determine a target velocity and a control input having a smallest Q function value, the Q function value being a value of a Q table, by use of Q table information provided by the calculation unit or previously stored Q table information. ### Claim 6 (depends on 5) The apparatus of claim 5, wherein the control unit determines the target velocity by use of a predefined equation (f(xk, uk)), from the state variable (xk) and the determined control input (uk). ### Claim 7 (depends on 5) The apparatus of claim 5, wherein the calculation unit includes: a state variable and reward estimation model unit configured to estimate the energy consumption according to the current velocity (vk) of the vehicle and the control input, and the velocity (vk+1) of the next step by use of the state variable and reward estimation model, from the vehicle traveling information and the traveling environment information; and a Q table optimization calculation unit configured to determine and update the Q table value by use of the information estimated by the state variable and reward estimation model unit and provide the updated Q table value to the control unit. ### Claim 8 (depends on 7) The apparatus of claim 7, wherein the state variable and reward estimation model unit determines the current velocity (vk) of the vehicle, a gradient (θk) of a traveling road, a distance (rk) from the preceding vehicle, road traffic (ck), and a cost value (gk+1) of an objective function corresponding to an amount of remaining energy of the vehicle and a control input (uk), which are state variables observed in a current step (k), and the state variable and reward estimation model unit learns the cost value (gk+1) and an approximate model (ĝ(vk, θk, rk, ck, SOCk, uk)) matrix, which is the state variable and reward estimation model, by use of a predetermined learning rate (β). ### Claim 9 (depends on 7) The apparatus of claim 7, wherein the state variable and reward estimation model unit learns a vehicle velocity (vk+1) prediction model of the next step (k+1), which is a velocity estimation model ({circumflex over (v)}(vk, θk, uk)) configured for estimating the vehicle velocity (vk+1) of the next step from the current velocity (vk) of the vehicle, a gradient (θk) of the traveling road, and a control input (uk) of the current step, by use of a predetermined learning rate (β). ### Claim 10 (depends on 7) The apparatus of claim 7, wherein the Q table optimization calculation unit updates the Q table value representing a control policy according to the vehicle traveling information and the traveling environment information by use of a model-based reinforcement learning algorithm. ### Claim 11 A method of determining an optimal velocity of a vehicle, the method comprising: receiving, by an information receiving unit, vehicle traveling information and traveling environment information which are state variables representing vehicle states required to determine a target velocity for optimizing vehicle fuel economy; determining, by an optimal velocity determination unit, the target velocity in accordance with a vehicle traveling environment by use of a state variable and reward estimation model and a Q table having values according to the state variables and a control input, from the vehicle traveling information and the traveling environment information provided by the information receiving unit; and controlling, by a control unit, a vehicle velocity to follow the target velocity, wherein the control unit is configured to select a control input (u) having a smallest Q value among Q values (x, {u1, u2, u3 . . . }) of corresponding state variable and determine the selected control input as an optimal control input (u) to follow the target velocity, wherein the state variable and reward estimation model unit updates non-linear state variable estimation models through the state variables and a reward cost value, wherein the reward cost value (gk) is determined, by the following equation, as a value made by applying weighting to each of a function (d(rk+1)) in respect to a time (Δtime) for which the vehicle travels a predetermined unit distance and the distance (rk+1) from a preceding vehicle, and a function (vlimit(vk+1) in respect to a target velocity deviation, and then summing up a change amount in battery SOC (ΔSOC) and the values to which the weighting is applied, g k + 1 = ( ∫ d k d k + 1 ⁢ S ⁢ O . ⁢ C v ⁡ ( d ) ⁢ dd + ∫ d k d k + 1 ⁢ 1 v ⁡ ( d ) ⁢ dd ) + η · d ⁡ ( r k + 1 ) + ξ · v limit ⁡ ( v k + 1 ) = Δ ⁢ ⁢ SOC + ω · Δ ⁢ ⁢ time + η · d ⁡ ( r k + 1 ) + ξ · v limit ⁡ ( v k + 1 ) ⁢ ⁢ wherein , d ⁡ ( r k + 1 ) = { 0 ⁢ ⁢ if ⁢ ⁢ r k + 1 > r safe 1 r k + 1 ⁢ ⁢ if ⁢ ⁢ r k + 1 ≤ r safe , v limit ⁡ ( v k + 1 ) = ( v k + 1 - v ref ) 2 , Equation ω, η, ξ represent predetermined weighting constants, rsafe represents a safety distance from the preceding vehicle which is predetermined as a constant value, and vref represents a predetermined reference velocity. ### Claim 12 (depends on 11) The method of claim 11, wherein the vehicle traveling information and the traveling environment information, which are the state variables, include vehicle position information, information related to a current velocity of the vehicle, information related to a gradient of a traveling road, a distance from the preceding vehicle, information related to real-time traffic on a road on a vehicle traveling route, and information related to an amount of remaining energy of the vehicle. ### Claim 13 (depends on 11) The method of claim 11, wherein the optimal velocity determination unit includes: a calculation unit configured to estimate energy consumption according to a current velocity (vk) of the vehicle and the control input determined and provided by the control unit, and a velocity (vk+1) of a next step by use of the state variable and reward estimation model, from the vehicle traveling information and the traveling environment information, and to determine and update a Q table value by use of the estimated information; and the control unit configured to determine a target velocity and a control input having a smallest Q function value, the Q function value being a value of a Q table, by use of Q table information provided by the calculation unit or previously stored Q table information. ### Claim 14 (depends on 13) The method of claim 13, wherein the control unit determines the target velocity by use of a predefined equation (f(xk, uk)), from the state variable (xk) and the determined control input (uk). ### Claim 15 (depends on 13) The method of claim 13, wherein the calculation unit includes: a state variable and reward estimation model unit configured to estimate the energy consumption according to the current velocity (vk) of the vehicle and the control input, and the velocity (vk+1) of the next step by use of the state variable and reward estimation model, from the vehicle traveling information and the traveling environment information; and a Q table optimization calculation unit configured to determine and update the Q table value by use of the information estimated by the state variable and reward estimation model unit and provide the updated Q table value to the control unit. ### Claim 16 (depends on 15) The method of claim 15, wherein the state variable and reward estimation model unit determines a current velocity (vk) of the vehicle, a gradient (θk) of a traveling road, a distance (rk) from the preceding vehicle, road traffic (ck), and a cost value (gk+1) of an objective function corresponding to an amount of remaining energy of the vehicle and a control input (uk), which are state variables observed in a current step (k), and the state variable and reward estimation model unit learns the cost value (gk+1) and an approximate model (ĝ(vk, θk, rk, ck, SOCk, uk)) matrix, which is the state variable and reward estimation model, by use of a predetermined learning rate (β). ### Claim 17 (depends on 15) The method of claim 15, wherein the state variable and reward estimation model unit learns a vehicle velocity (vk+1) prediction model of the next step (k+1), which is a velocity estimation model ({circumflex over (v)}(vk, θk, uk)) configured for estimating the vehicle velocity (vk+1) of the next step from the current velocity (vk) of the vehicle, a gradient (θk) of a traveling road, and a control input (uk) of the current step, by use of a predetermined learning rate (β). ### Claim 18 (depends on 15) The method of claim 15, wherein the Q table optimization calculation unit updates the Q table value representing a control policy according to the vehicle traveling information and the traveling environment information by use of a model-based reinforcement learning algorithm.