Calibrating a Machine-Learning Model in a Data Processing Environment

A data processing system for calibrating a machine learning model, including:

• one or more processing devices;
• and
• one or more machine-readable hardware storage devices storing instructions that are executable by the one or more processing devices to perform operations including:
• reading first input data structures structured with first fields specifying first values, wherein a first data structure includes a key field with a key value specifying a channel;
• executing first executable logic on the first values of the first fields of the first input data structures;
• based on the executing, outputting first output data structures specifying a plurality of channels specified by key values of the key fields and one or more values for each of the channels;
• reading second input data structures structured with second fields specifying second values, wherein a second data structure includes a first key field specifying a value of a Hypertext Transfer Protocol (HTTP) cookie and a second key field specifying a value representing a digital campaign;
• executing second executable logic on the second values of the second fields of the second input data structures and on the values of the first and second key fields;
• based on the executing, outputting second output data structures specifying attribution data for a plurality of digital channels;
• training a machine learning (ML) model to predict sales and/or return on investment by digital campaign, wherein the ML model is configured to receive as input the first and second data structures and to process the input to generate an output that specifies a predicted sale and/or a predicted return on investment, wherein the ML model comprises a plurality of nodes that are connected through edges and are aggregated into a plurality of layers comprising at least an input layer and an output layer, wherein each of the edges is configured to transmit a signal from one node to another node, and wherein an output of each of the plurality of nodes is computed based on inputs of the nodes in accordance with a plurality of weights;
• identifying an area of the machine learning model to be improved;
• generating one or more A/B tests that are related to the area to be improved, wherein an A/B test is configured to obtain further information that can be used to calibrate or refine the machine learning model in the area to be improved;
• determining one or more outcomes from executing the one or more A/B tests;
• and
• calibrating the ML model in accordance with the determined one or more outcomes.

One or more machine-readable hardware storage devices for calibrating a machine learning model, the one or more machine-readable hardware storage devices storing instructions that are executable by the one or more processing devices to perform operations including:

• reading first input data structures structured with first fields specifying first values, wherein a first data structure includes a key field with a key value specifying a channel;
• executing first executable logic on the first values of the first fields of the first input data structures;
• based on the executing, outputting first output data structures specifying a plurality of channels specified by key values of the key fields and one or more values for each of the channels;
• reading second input data structures structured with second fields specifying second values, wherein a second data structure includes a first key field specifying a value of a Hypertext Transfer Protocol (HTTP) cookie and a second key field specifying a value representing a digital campaign;
• executing second executable logic on the second values of the second fields of the second input data structures and on the values of the first and second key fields;
• based on the executing, outputting second output data structures specifying attribution data for a plurality of digital channels;
• training a machine learning (ML) model to predict sales and/or return on investment by digital campaign, wherein the ML model is configured to receive as input the first and second data structures and to process the input to generate an output that specifies a predicted sale and/or a predicted return on investment, wherein the ML model comprises a plurality of nodes that are connected through edges and are aggregated into a plurality of layers comprising at least an input layer and an output layer, wherein each of the edges is configured to transmit a signal from one node to another node, and wherein an output of each of the plurality of nodes is computed based on inputs of the nodes in accordance with a plurality of weights;
• identifying an area of the machine learning model to be improved;
• generating one or more A/B tests that are related to the area to be improved, wherein an A/B test is configured to obtain further information that can be used to calibrate or refine the machine learning model in the area to be improved;
• determining one or more outcomes from executing the one or more A/B tests;
• and
• calibrating the ML model in accordance with the determined one or more outcomes.