Method and system for jointly controlling electric vehicle-heating, ventilation, and air conditioning system of building

A joint control system, comprising:

• a memory storing instructions;
• one or more communication interfaces;
• and
• one or more hardware processors coupled to the memory via the one or more communication interfaces, wherein the one or more hardware processors are configured by the instructions to:
• receive sensory information associated with a building, and pricing information from a building management system (BMS);
• determine an electric vehicle (EV) action to be performed on each EV of one or more EVs based, at least in part, on the received sensory information and the pricing information using a trained EV agent, wherein the EV action comprises one of a:
• charging action;
• and discharging action, wherein prior to determining the EV action to be performed on each EV of the one or more EVs based, at least in part, on the received sensory information and the pricing information using the trained EV agent, the one or more hardware processors are configured by the instructions to:
• determine whether the trained EV agent is available;
• upon determining that the trained EV agent is not available, train an EV agent based on the sensory information and the pricing information by:
• accessing the current SoC of each EV, the departure time of each EV, a required SoC at the departure time of each EV, the cost per unit of electricity at the current time and an EV proxy for a time-of-day pattern in arrival of EVs;
• computing a slack variable for each EV based, at least in part, on the departure time of the respective EV and the required SoC at the departure time for the respective EV;
• computing a state space value for each EV based, at least in part, on the computed slack variable of the respective EV, the current SoC of the respective EV, the departure time of the respective EV, the required SoC at the departure time of the respective EV, the cost per unit of electricity at the current time, and the proxy for the time-of-day pattern, wherein the state space value computed for each EV is used by the EV agent to decide an action value for the respective EV, and wherein the action value is one of:
• a charging rate, and a discharging rate;
• implementing an action on each EV based on the action value decided by the EV agent for the respective EV;
• computing a reward for each EV based on the action implemented on the respective EV using an EV reward algorithm, wherein the reward comprises one of:
• penalizing the EV agent in case slack variable constraints are not followed for the respective EV, and awarding the EV agent in case the slack variable constraints are followed for the respective EV, and wherein the awarding of the EV agent represents that the EV agent is learning to minimize total cost of power drawn by the EVs in the EV-HVAC system;
• and
• training the EV agent based on the computed reward using an EV training algorithm;
• determine a heating, ventilation and air conditioning (HVAC) action to be performed on an HVAC equipment based on the received sensory information using a trained HVAC agent, wherein the HVAC action comprises one of a:
• power to be drawn from a grid, and power to be drawn from at least one EV;
• and
• send the determined EV action to be performed on each EV of the one or more EVs, and the determined HVAC action to be performed on the HVAC equipment to the BMS, wherein the BMS implements the EV action determined for each EV on a respective EV and the HVAC action on the HVAC equipment to control the power consumption by the EV-HVAC system.

One or more non-transitory machine-readable information storage mediums comprising one or more instructions which when executed by one or more hardware processors cause:

• receiving, by a joint control system, sensory information associated with a building, and pricing information from a building management system (BMS);
• determining, by the joint control system, an electric vehicle (EV) action to be performed on each EV of one or more EVs based, at least in part, on the received sensory information and the pricing information using a trained EV agent, wherein the EV action comprises one of a:
• charging action;
• and discharging action, wherein the step of determining, by the joint control system, the EV action to be performed on each EV of the one or more EVs based, at least in part, on the received sensory information and the pricing information using the trained EV agent is preceded by:
• determining, by the joint control system, whether the trained EV agent is available;
• and
• upon determining that the trained EV agent is not available, training, by the joint control system, an EV agent based on the sensory information and the pricing information by:
• accessing, by the joint control system, the current SoC of each EV, the departure time of each EV, a required SoC at the departure time of each EV, the cost per unit of electricity at the current time and an EV proxy for a time-of-day pattern in arrival of EVs;
• computing, by the joint control system, a slack variable for each EV based, at least in part, on the departure time of the respective EV and the required SoC at the departure time for the respective EV;
• computing, by the joint control system, a state space value for each EV based, at least in part, on the computed slack variable of the respective EV, the current SoC of the respective EV, the departure time of the respective EV, the required SoC at the departure time of the respective EV, the cost per unit of electricity at the current time, and the proxy for the time-of-day pattern, wherein the state space value computed for each EV is used by the EV agent to decide an action value for the respective EV, and wherein the action value is one of:
• a charging rate, and a discharging rate;
• implementing, by the joint control system, an action on each EV based on the action value decided by the EV agent for the respective EV;
• computing, by the joint control system, a reward for each EV based on the action implemented on the respective EV using an EV reward algorithm, wherein the reward comprises one of:
• penalizing the EV agent in case slack variable constraints are not followed for the respective EV, and awarding the EV agent in case the slack variable constraints are followed for the respective EV, and wherein the awarding of the EV agent represents that the EV agent is learning to minimize total cost of power drawn by the EVs in the EV-HVAC system;
• and
• training, by the joint control system, the EV agent based on the computed reward using an EV training algorithm;
• determining, by the joint control system, a heating, ventilation and air conditioning (HVAC) action to be performed on an HVAC equipment based on the received sensory information using a trained HVAC agent, wherein the HVAC action comprises one of a:
• power to be drawn from a grid, and power to be drawn from at least one EV;
• and
• sending, by the joint control system, the determined EV action to be performed on each EV of the one or more EVs, and the determined HVAC action to be performed on the HVAC equipment to the BMS, wherein the BMS implements the EV action determined for each EV on a respective EV and the HVAC action on the HVAC equipment to control the power consumption by the EV-HVAC system.