Cluster diagnostics data for distributed job execution

A system comprising:

• one or more processors of a machine;
• and
• a memory storing instructions that, when executed by the one or more processors, cause the machine to perform operations comprising:
• receiving, by a computing cluster, an application for processing by nodes of the computing cluster, the nodes including a driver node and a plurality of execution nodes for processing of tasks of the application;
• distributing, by the driver node, the tasks of the application for processing by the plurality of execution nodes;
• processing the tasks by the plurality of execution nodes;
• identifying, by the computing cluster, one or more errors in the application received by the computing cluster;
• in response to the one or more errors, re-processing the tasks of the application by the plurality of execution nodes with a telemetry setting active in the application, the driver node passing a log request to a log object of a wrapper to bypass a native logging service of the computing cluster in response to the telemetry setting being active, the driver node further receiving telemetry metadata for access to a telemetry network service of a distributed database that provides data to the computing cluster through a database connector, the driver node receiving the telemetry metadata through the database connector of the distributed database;
• distributing, by the driver node, the telemetry metadata to the plurality of execution nodes for purposes of re-processing;
• and
• transmitting, by the plurality of execution nodes, to the telemetry network service of the distributed database, log data that is generated by the plurality of execution nodes in re-processing the tasks of the application.

A non-transitory machine storage medium embodying instructions that, when executed by a machine, cause the machine to perform operations comprising:

• receiving, by a computing cluster, an application for processing by nodes of the computing cluster, the nodes including a driver node and a plurality of execution nodes for processing of tasks of the application;
• distributing, by the driver node, the tasks of the application for processing by the plurality of execution nodes;
• processing the tasks by the plurality of execution nodes;
• identifying, by the computing cluster, one or more errors in the application received by the computing cluster;
• in response to the one or more errors, re-processing the tasks of the application by the plurality of execution nodes with a telemetry setting active in the application, the driver node passing a log request to a log object of a wrapper to bypass a native logging service of the computing cluster in response to the telemetry setting being active, the driver node further receiving telemetry metadata for access to a telemetry network service of a distributed database that provides data to the computing cluster through a database connector, the driver node receiving the telemetry metadata through the database connector of the distributed database;
• distributing, by the driver node, the telemetry metadata to the plurality of execution nodes for re-processing;
• and
• transmitting, by the plurality of execution nodes, to the telemetry network service of the distributed database, log data that is generated by the plurality of execution nodes in re-processing the tasks of the application.