US 2026/0228620 A1Published application

Dynamic and unified performance evaluation in a machine-learning pipeline

AssigneeOracle International Corporation

InventorsRahul Yadav, Ashish Pandey, Seth Snider, Krishnadayalu Swaminathan

Life of this patent

Estimating the term…

Loading patent…

Abstract

The present disclosure relates to techniques for a dynamic and unified performance evaluation of a machine-learning (ML) pipeline, incorporating a configuration of ML models, based on a unified performance metric. The techniques involve accessing data comprising a set of data samples, where each data sample corresponds to one or more data types, including images, text, audio, or structured data elements. The data may be input into the ML pipeline, including ML models combined in the specific configuration (e.g., series, parallel), with model types corresponding to classification or regression. The decision boundary of each ML model is dynamically adjusted by evaluating relevant evaluation metrics. The ML pipeline may process data to generate a result based on the adjusted decision boundaries. The unified performance metric may be generated dynamically based on the model types, configuration, and their respective contributions. The result is evaluated based on the unified performance metric satisfying performance criteria.

Claims20

Dependent claims

A computer-implemented method including:

accessing data comprising a set of data samples, wherein each data sample of the set of data samples corresponds to one or more data types including an image, a text, an audio and a set of data elements;
inputting the data into a machine-learning pipeline comprising a first machine-learning model configured with a second machine-learning model to process the data in a specific configuration, wherein a model type associated with each machine-learning model of the machine-learning pipeline corresponds to a classification model and/or a regression model;
dynamically adjusting a decision boundary associated with each machine-learning model of the machine-learning pipeline by evaluating one or more evaluation metrics associated with each machine-learning model of the machine-learning pipeline;
generating, via the machine-learning pipeline, a result by processing the data based on the adjusted decision boundary associated with each machine-learning model of the machine-learning pipeline;
generating dynamically a unified performance metric that is adapted based on the specific configuration, the model type of each machine-learning model of the machine-learning pipeline and a contribution of each machine-learning model of the machine-learning pipeline;
evaluating the result of the machine-learning pipeline by calculating a value of the unified performance metric;
determining that the value of the unified performance metric satisfies a first predefined criterion;
and
outputting the result based on the determination.

6 more elements

A system comprising:

one or more data processors;
and
a non-transitory computer readable storage medium containing instruction which, when executed on the one or more data processors, cause the one or more data processors to perform a set of operations including:
accessing data comprising a set of data samples, wherein each data sample of the set of data samples corresponds to one or more data types including an image, a text, an audio and a set of data elements;
inputting the data into a machine-learning pipeline comprising a first machine-learning model configured with a second machine-learning model to process the data in a specific configuration, wherein a model type associated with each machine-learning model of the machine-learning pipeline corresponds to a classification model and/or a regression model;
dynamically adjusting a decision boundary associated with each machine-learning model of the machine-learning pipeline by evaluating one or more evaluation metrics associated with each machine-learning model of the machine-learning pipeline;
generating, via the machine-learning pipeline, a result by processing the data based on the adjusted decision boundary associated with each machine-learning model of the machine-learning pipeline;
generating dynamically a unified performance metric that is adapted based on the specific configuration, the model type of each machine-learning model in the machine-learning pipeline and a contribution of each machine-learning model in the machine-learning pipeline;
evaluating the result of the machine-learning pipeline by calculating a value of the unified performance metric;
determining that the value of the unified performance metric satisfies a first predefined criterion;
and
outputting the result based on the determination.

9 more elements

A computer-program product tangibly embodied in a non-transitory machine readable storage medium, including instructions configured to cause one or more data processors to perform to perform a set of operations comprising:

accessing data comprising a set of data samples, wherein each data sample of the set of data samples corresponds to one or more data types including an image, a text, an audio and a set of data elements;
inputting the data into a machine-learning pipeline comprising a first machine-learning model configured with a second machine-learning model to process the data in a specific configuration, wherein a model type associated with each machine-learning model of the machine-learning pipeline corresponds to a classification model and/or a regression model;
dynamically adjusting a decision boundary associated with each machine-learning model of the machine-learning pipeline by evaluating one or more evaluation metrics associated with each machine-learning model of the machine-learning pipeline;
generating, via the machine-learning pipeline, a result by processing the data based on the adjusted decision boundary associated with each machine-learning model of the machine-learning pipeline;
generating dynamically a unified performance metric that is adapted based on the specific configuration, the model type of each machine-learning model in the machine-learning pipeline and a contribution of each machine-learning model in the machine-learning pipeline;
evaluating the result of the machine-learning pipeline by calculating a value of the unified performance metric;
determining that the value of the unified performance metric satisfies a first predefined criterion;
and
outputting the result based on the determination.

6 more elements

Dependencies are inferred from the claim text.