# US 2018/0321719 A1 · Thermal excursion detection in datacenter components

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2018/0321719 A1
- Document ID: US20180321719A1
- Status: Published application
- Kind: A1
- Application number: 15/587,016
- Filed: 2017-05-04
- Published: 2018-11-08
- Earliest claimed: 2017-05-04
- Assignees: Microsoft Technology Licensing, LLC
- Applicants: Microsoft Technology Licensing, LLC
- Inventors: Ashish MUNJAL; Lee W. PROGL; Prasad AYYALASOMAYAJULA; Uuganjargal KHANNA
- Web page: https://patentgrep.com/patent/US20180321719A1

## CPC

- G06F 1/206: {comprising thermal management}
- G06F 11/3006: {where the computing system is distributed, e.g. networked systems, clusters, multiprocessor systems (multiprogramming arrangements G06F9/46; allocation of resources G06F9/50)}
- G06F 11/3058: {Monitoring arrangements for monitoring environmental properties or parameters of the computing system or of the computing system component, e.g. monitoring of power, currents, temperature, humidity, position, vibrations (thermal management in cooling arrangements of a computing system G06F1/206)}
- G06F 11/3648: {using additional hardware}
- G06F 13/4068: {Electrical coupling}
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)

## Abstract

A fully out-of-band, automated process provides centralized, concurrent in-system cluster-level/hyper-scale CPU thermal excursion signature detection based on system event log (SEL) content. A chassis manager within a hyper-scale cluster operates as the initiator of a cluster-level SEL collection and analysis operation. The master chassis manager initiates a collect and analyze operation, which gets relayed to every chassis manager within the cluster. Every recipient chassis manager then further propagates the collect and analyze operation request to each of the server blades hosted within the chassis. A Baseboard Management Controller (BMC) on each server hosted within the chassis receives a cluster SEL read request from the chassis manager through out-of-band communication interfaces. The BMC forwards SEL data for the server back to the chassis manager, which analyzes the data received from all the servers on the chassis and then sends the results to next manager in the hierarchy.

## Claims

### Claim 1

A distributed computing system, comprising:
one or more groups each comprising a plurality of chassis, each chassis comprising a chassis manager and a plurality of servers, each server comprising a baseboard management controller and a plurality of processors;
each baseboard management controller configured to monitor the plurality of processors on its respective server and to store processor thermal events to a local system event log;
a chassis manager master configured to initiate a thermal event collection operation on all chassis within a group by sending a collection request to other chassis managers within the group; and
each chassis manager configured to receive thermal event data from the local system event logs on each baseboard management controller on its respective chassis and to forward the thermal event data to the chassis manager master.

### Claim 2 (depends on 1)

The distributed computing system of claim 1, further comprising:
each chassis manager configured to relay the collection request to each baseboard management controller on its respective chassis via out-of-band communication.

### Claim 3 (depends on 2)

The distributed computing system of claim 2, wherein the out-of-band communication is conveyed over an Intelligent Platform Management Interface (IPMI).

### Claim 4 (depends on 1)

The distributed computing system of claim 1, further comprising:
each chassis manager configured to analyze the thermal event data from the local system event logs and to initiate mitigation for thermal events on its respective chassis.

### Claim 5 (depends on 4)

The distributed computing system of claim 4, wherein the mitigation comprises controlling a cooling fan.

### Claim 6 (depends on 1)

The distributed computing system of claim 1, further comprising:
chassis manager master configured to analyze the thermal event data from the local system event logs from chassis within its group and to initiate mitigation for thermal events within the group.

### Claim 7 (depends on 1)

The distributed computing system of claim 1, further comprising:
chassis manager master configured to aggregate the thermal event data from the local system event logs from chassis within its group and to forward the aggregated thermal event data for the group to a higher-level controller within a datacenter hierarchy.

### Claim 8 (depends on 7)

The distributed computing system of claim 7, wherein the aggregated thermal event data is used to isolate a server problem at a blade, chassis, rack, colo, or datacenter level.

### Claim 9 (depends on 7)

The distributed computing system of claim 7, wherein the aggregated thermal event data is analyzed to correlate a hardware inventory to thermal events on a server.

### Claim 10 (depends on 7)

The distributed computing system of claim 7, wherein the aggregated thermal event data is analyzed to detect a scope of thermal excursions across regions of the datacenter.

### Claim 11 (depends on 10)

The distributed computing system of claim 10, wherein mitigations to address thermal excursions identified from the aggregated thermal event data are selected depending on the scope of thermal excursions.

### Claim 12

A method, comprising:
initiating, by a chassis manager master among a plurality of chassis managers, a thermal event collection operation on all chassis within a group by sending a collection request to the chassis managers within the group;
relaying, by each chassis manager, the collection request to each baseboard management controller on its respective chassis via out-of-band communication; and
forwarding, by each chassis manager, thermal event data received from local system event logs on each baseboard management controller to the chassis manager master.

### Claim 13 (depends on 12)

The method of claim 12, wherein the out-of-band communication is conveyed over an Intelligent Platform Management Interface (IPMI).

### Claim 14 (depends on 12)

The method of claim 12, further comprising:
analyzing, by each chassis manager, the thermal event data from the local system event logs and to initiate mitigation for thermal events on its respective chassis.

### Claim 15 (depends on 14)

The method of claim 14, wherein the mitigation comprises controlling a cooling fan.

### Claim 16 (depends on 12)

The method of claim 12, further comprising:
analyzing, by the chassis manager master, the thermal event data from the local system event logs; and
initiating mitigation for thermal events within the group.

### Claim 17 (depends on 12)

The method of claim 12, further comprising:
aggregating, by the chassis manager master, the thermal event data from the local system event logs; and
forwarding the aggregated thermal event data for the group to a higher-level controller within a datacenter hierarchy.

### Claim 18 (depends on 17)

The method of claim 17, further comprising:
using the aggregated thermal event data to isolate a server problem at a blade, chassis, rack, colo, or datacenter level.

### Claim 19 (depends on 17)

The method of claim 17, further comprising:
analyzing the aggregated thermal event data to correlate a hardware inventory to thermal events on a server.

### Claim 20 (depends on 17)

The method of claim 17, further comprising:
analyzing the aggregated thermal event data to detect a scope of thermal excursions across regions of the datacenter; and
selecting mitigations to address thermal excursions identified from the aggregated thermal event data based on the scope of thermal excursions.
