Autonomous distributed workload and infrastructure scheduling

A tangible, non-transitory, machine-readable medium storing instructions that when executed by one or more processors effectuate operations comprising:

• allocating, with one or more processors, resources in, or added to, a multi-tenant compute-cluster distributed across a plurality of different data centers in different geographic locations with a compute-cluster manager based on a plurality of different policies wherein,
• the compute-cluster manager is a distributed application executing on a plurality of computing devices and implementing a fault-tolerant protocol to select a master computing node of the compute-cluster manager from the plurality of computing devices,
• the resources include usage of a plurality of computing nodes in, or added to, the compute-cluster to execute one or more distributed workload applications, the distributed workload applications being configured to be executed on a plurality of different computing nodes of the compute-cluster, the usage of computing nodes in the compute-cluster comprising durations of time reserved on computing nodes to run tasks,
• the computing nodes include instances of micro-kernels, containers, virtual machines, or computing devices of the plurality of different data centers,
• each computing node of the compute-cluster has a different network, network address, or port from other computing nodes in the compute-cluster, and
• the compute-cluster manager is configured to allocate resources of a heterogeneous mix of computing devices in the plurality of different computing nodes to a heterogeneous mix of distributed workload applications executed concurrently;
• obtaining, with one or more processors, physical telemetry data of each of a plurality of at least some of the computing nodes, the physical telemetry data indicating attributes of a physical environment in which the respective computing node executes and being distinct from logical telemetry data indicative of logical attributes of computing nodes accessible via a respective operating system within which at least some of the computing nodes execute, the logical attributes including central processing unit utilization, memory utilization, and storage utilization;
• accessing, with one or more processors, a policy in the plurality of different policies that indicates how to allocate compute-cluster resources based on physical telemetry data, logical telemetry data, and workload;
• and
• allocating, with one or more processors, additional resources in, or added to, the compute-cluster to at least one of the distributed workload applications with the compute-cluster manager based on both the policy and the physical telemetry data, wherein the computer-cluster manager is configured to allocate resources to cause workloads to be scheduled based on amounts of computing resources needed to execute workloads, logical telemetry data of computing nodes, and physical telemetry data of computing nodes in accordance with one or more policies among the plurality of different policies, wherein:
• the policy and at least some other policies in the plurality of different policies are each associated with different tenant accounts in a data repository;
• the policy comprises a plurality of rules specifying resource allocation actions and criteria upon which determinations to perform the actions are based;
• the policy comprises weights by which performance metrics and physical telemetry data are combined in a plurality of weighted scores, each weighted score being associated with a different candidate resource allocation action;
• allocating additional resources comprises selecting a resource allocation action from the candidate resource allocation actions based on the scores;
• the physical telemetry data comprises a physical rack computing device location of a computing device in a rack;
• and
• the physical location is obtained by operations comprising:
• receiving a request for a rack computing device location, the rack computing device being mounted in the rack;
• wirelessly sensing a value indicative of the location based on physical proximity between a transmitter and a receiver, the proximity being caused by the rack computing device being mounted in one of a plurality of receptacles in the rack, each of the receptacles being associated with a different value indicative of a respective receptacle location;
• and
• generating, based on the wirelessly sensed value, output signals conveying location information related to the location of the rack-mounted computing device.