Automatic recovery of node resource memory devices

A computer-implemented method, comprising:

• collecting, by a platform basic input/output system (“BIOS”) of a node and from a node resource of the node, first information associated with operational states of a plurality of memory components of a memory device;
• determining, by the platform BIOS, whether at least one memory component among the plurality of memory components is undetected, by comparing the first information with second information associated with a resource inventory corresponding to the plurality of memory components of the memory device;
• based on a determination that at least one memory component is undetected, sending, by the platform BIOS, a first notification to a controller of the node, the first notification indicating that the at least one memory component is undetected;
• providing, by the controller, a first signal to a control plane fabric (“CPF”) agent in a control plane, the first signal being based on the first notification and indicating that the at least one memory component is undetected;
• receiving, by the controller, a first set of commands from the CPF agent, the first set of commands being based on a determination by the CPF agent regarding resolution to the at least one memory component being undetected, wherein the determination by the CPF agent regarding the resolution to the at least one memory component being undetected is based on identifying which resolution option among a plurality of resolution options to pursue, wherein the plurality of resolution options includes at least one of:
• causing the memory device to restart and to initiate an immediate reset of the plurality of memory components;
• causing the memory device to restart and to initiate an immediate retraining of the plurality of memory components;
• causing a firmware update of the memory device, followed by restarting of the memory device;
• causing the node resource to restart and to initiate an immediate reset of the node resource;
• causing the node resource to restart and to initiate an immediate retraining of the plurality of memory components coupled to the node resource;
• or
• causing a firmware update of the node resource, followed by restarting of the node resource;
• sending, by the controller, a second set of commands to the platform BIOS, based on the first set of commands;
• and
• initiating, by the platform BIOS, a recovery process for the plurality of memory components of the memory device, based on the second set of commands.

A system, comprising:

• a control plane fabric (“CPF”) agent in a control plane, the CPF agent executing code that causes the CPF agent to perform operations comprising:
• identifying a potential cause of at least one memory component being undetected among a plurality of memory components of a memory device, based on analysis of contents of a first signal that is provided by a controller of a node, the memory device being communicatively coupled to and controlled by a node resource of the node;
• checking a recovery catalog, the recovery catalog including a list of fault codes correlated with known faults and corresponding recovery actions;
• identifying a first fault code based on a comparison of the known faults listed in the recovery catalog with the identified potential cause;
• identifying a first resolution option, among a plurality of resolution options to pursue, based on a recovery action corresponding to the identified first fault code listed in the recovery catalog, wherein the plurality of resolution options includes at least one of:
• causing the memory device to restart and to initiate an immediate reset of the plurality of memory components;
• causing the memory device to restart and to initiate an immediate retraining of the plurality of memory components;
• causing a firmware update of the memory device, followed by restarting of the memory device;
• causing the node resource to restart and to initiate an immediate reset of the node resource;
• causing the node resource to restart and to initiate an immediate retraining of the plurality of memory components coupled to the node resource;
• or
• causing a firmware update of the node resource, followed by restarting of the node resource;
• and
• sending, to the controller, a first set of commands that cause the controller to instruct a platform basic input/output system (“BIOS”) of the node to initiate a recovery process for the plurality of memory components of the memory device, based on the first resolution option.