Real-time carbon footprint reduction controller

A system for carbon footprint reduction of a system, comprising:

• a memory storing instructions;
• and
• one or more processors coupled to the memory and configured to execute the instructions to:
• obtain state data representative of states of the system, wherein the state data comprises internal states of the system and environmental states of an environment for the system, wherein the internal states comprise one or more of allocated workload, unal located workload, temperature of the system, energy consumption by the system, and energy stored by the system, and wherein the environmental states comprise one or more of grid carbon intensity, weather, and time, and wherein the system comprising a plurality of subsystems;
• receive reward data from a digital twin of the system that simulates operations of the system, the reward data comprising a plurality of rewards each associated with a subsystem of the plurality of subsystems;
• determine a plurality of actions, by a plurality of reinforcement learning agents, based on the state data and each of the plurality of rewards, wherein each reinforcement learning agent of the plurality of reinforcement learning agents is associated with a subsystem of the plurality of subsystems and assigns a weight to a reward of the plurality of rewards corresponding to the an associated subsystem of the plurality of subsystems that is greater than weights assigned to rewards corresponding to the other subsystems of the plurality of subsystems;
• and
• transition the system to updated states according to the plurality of actions.

A system, comprising:

• a hardware processor;
• a non-transitory machine-readable storage medium storing instructions that, when executed by the processor, cause the processor to:
• initiate a digital twin of a data center, the digital twin comprising a plurality of subsystem models corresponding to a plurality of subsystems of the data center and configured to simulate states of received environmental states of an environment for the data center, comprising one or more of grid carbon intensity, weather, or and time, for each subsystem model and determine rewards for each subsystem model, wherein the simulated states comprise one or more of allocated workload, unallocated workload, temperature of the data center, energy consumption by the data center, and energy stored by the data center;
• and
• initiate a plurality of reinforcement learning agents interfaced with the digital twin and configured to:
• receive the simulated states and the rewards from a digital twin, aggregate the rewards,
• determine actions for each subsystem model based on the aggregated rewards, and
• generate control signals for each subsystem according to the determined actions, wherein the data center is configured based on the control signals.