US 2026/0065166 A1Published application

Model uninterrupted serving and evolution

AssigneeFeedzai—Consultadoria e Inovação Tecnológica, S.A.

InventorsNIKOLETTA MATSUR, ALBERTO EDUARDO ARTILHEIRO FERREIRA, AHMAD NASER EDDIN, JACOPO BONO, JOÃO LUCAS SILVA MARTINS, HUGO RICARDO COLAÇO FERREIRA, RICARDO PEREIRA, VLADIMIR BALAYAN, PEDRO GUSTAVO SANTOS RODRIGUES BIZARRO, ANA SOFIA LEAL GOMES

Life of this patent

Estimating the term…

patentgrep

Loading patent…

Abstract

Computer-implemented method and system for deployment of a first machine-learning model, and replacement, without service interruption, of a second machine-learning model in active on-line use, comprising: receiving, at a controller, a replacement request; in response to said replacement request, triggering the deployment of the first model and triggering the calculation of features to be used, collecting output data from the first model, fitting and inserting one or more calibration functions downstream from the first model, and routing inference requests to the first model instead of the second model; wherein the triggered deployment of the first model comprises preloading the first model into CPU or GPU memory, and making available the calculated features and the preloaded first model by CPU or GPU, respectively, before the inference requests are routed to the first model, thus enabling that no additional latency is added when traffic is rerouted to the first model.

Claims20

Dependent claims

A computer-implemented method for deployment of a first machine-learning model, and replacement, without service interruption, of a second machine-learning model in active on-line use, by the first machine-learning model, the method comprising:

receiving, at a controller configured by code executing therein, a replacement request determining a replacement of the second machine-learning model by the first machine-learning model, and
in response to said replacement request, the controller triggering the deployment of the first machine-learning model and triggering the calculation of features to be used by the first machine-learning model;
the method further comprising, carrying out by a computer system comprising said controller and configured by code executing therein, in response to said replacement request, to perform the steps of:
collecting output data from the first machine-learning model over a predetermined number of inference requests or events,
fitting one or more calibration functions to the collected output data to generate one or more calibrated outputs of the first machine-learning model,
inserting downstream from the first machine-learning model the one or more calibration functions, and
routing inference requests to the first machine-learning model instead of the second machine-learning model;
wherein the triggered deployment of the first machine-learning model comprises preloading the first machine-learning model into at least one of a CPU memory and a GPU memory;
wherein the method further comprises making available the calculated features to the preloaded first machine-learning model, and making available the preloaded first machine-learning model into the at least one of a CPU memory and a GPU memory by at least one of a CPU and a GPU, respectively, before routing inference requests to the first machine-learning model instead of the second machine-learning model.

6 more elements

A computer system for deployment of a first machine-learning model, and replacement, without service interruption, of a second machine-learning model in active on-line use, by the first machine-learning model, the system comprising a controller having a processor executing code that configures the processor for:

receiving a replacement request determining a replacement of the second machine-learning model by the first machine-learning model, and
in response to said replacement request, the controller triggering the deployment of the first machine-learning model and triggering the calculation of features to be used by the first machine-learning model,
the system having a further processor executing code that configures the system for, in response to said replacement request, to perform the steps of:
collecting output data from the first machine-learning model over a predetermined number of inference requests or events,
fitting one or more calibration functions to the collected output data to generate one or more calibrated outputs of the first machine-learning model,
inserting downstream from the first machine-learning model the one or more calibration functions, and
routing inference requests to the first machine-learning model instead of the second machine-learning model;
wherein the triggered deployment of the first machine-learning model comprises preloading the first machine-learning model into at least one of CPU memory and GPU memory,
the system being further configured for making available the calculated features to the preloaded first machine-learning model, and making available the preloaded first machine-learning model into the at least one of a CPU memory and a GPU memory by at least one of CPU and GPU, respectively, before routing inference requests to the first machine-learning model instead of the second machine-learning model.

6 more elements

Dependencies are inferred from the claim text.