# US 2019/0318240 A1 · Training machine learning models in distributed computing systems

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2019/0318240 A1
- Document ID: US20190318240A1
- Status: Published application
- Kind: A1
- Application number: 16/154,562
- Filed: 2018-10-08
- Published: 2019-10-17
- Earliest claimed: 2018-04-16
- Assignees: Kazuhm, Inc.
- Applicants: Kazuhm, Inc.
- Inventors: Rounak Prasad KULKARNI; Armin KADIYAN; Tim O'NEAL
- Web page: https://patentgrep.com/patent/US20190318240A1

## CPC

- G06F 9/5044: {considering hardware capabilities}
- G06F 8/61: Installation
- G06F 9/455: Emulation; Interpretation; Software simulation, e.g. virtualisation or emulation of application or operating system execution engines
- G06F 9/45558: {Hypervisor-specific management and integration aspects}
- G06F 9/5072: {Grid computing}
- G06F 9/5077: {Logical partitioning of resources; Management or configuration of virtualized resources (specific details on emulation or internal functioning of virtual machines G06F9/455)}
- G06F 9/546: {Message passing systems or structures, e.g. queues}
- G06F 18/214: Generating training patterns; Bootstrap methods, e.g. bagging or boosting
- G06N 3/045: Combinations of networks
- G06N 3/063: using electronic means
- G06N 3/08: Learning methods
- G06N 20/00: Machine learning
- H04L 41/046: comprising network management agents or mobile agents therefor
- H04L 43/0876: Network utilisation, e.g. volume of load or congestion level
- H04L 67/34: {involving the movement of software or configuration parameters (network booting or remote initial program loading [RIPL] G06F9/4416)}
- G06F 8/63: {Image based installation; Cloning; Build to order}
- G06F 2009/45562: {Creating, deleting, cloning virtual machine instances}
- G06F 2009/45587: {Isolation or security of virtual machine instances}
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- H04L: TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION (arrangements common to telegraphic and telephonic communication H04M)

## Abstract

Certain aspects of the present disclosure provide methods and systems for training a machine learning model, such as a neural network or deep learning model, in a distributed computing system. In some embodiments, aspects of the machine learning model are trained within containers distributed amongst nodes in the distributed computing environment.

## Claims

### Claim 1

A method for training a machine learning model in a distributed computing system:
receiving a model training request;
receiving a training data set;
determining a processing node available in a distributed computing system;
receiving static status information regarding the processing node;
causing a first container to be installed at the processing node based on the static status information, the first container being configured with a model training application;
causing a second container to be installed at the processing node based on the static status information, the second container being configured with the model training application;
assigning a first layer of a model to be trained by the model training application in the first container;
assigning a second layer of the model to be trained by the model training application in the second container;
receiving parameter data from the model training application in the first container, the model training application in the second container, and the model training application in the third container; and
calculating a model parameter based on the parameter data.

### Claim 2 (depends on 1)

The method of claim 1, further comprising:
assigning a first data subset to the model training application in the first container; and
assigning a second data subset to the model training application in the second container.

### Claim 3 (depends on 1)

The method of claim 1, further comprising:
assigning a first data subset to the model training application in the first container; and
assigning the first data subset to the model training application in the second container.

### Claim 4 (depends on 1)

The method of claim 1, further comprising:
causing a third container to be installed at the processing node based on the static status information, the third container being configured with the model training application;
assigning the first layer and the second layer to be trained by the model training application in the third container; and
receiving parameter data from the model training application in the third container.

### Claim 5 (depends on 1)

The method of claim 1, wherein:
the processing node comprises a local operating system, and
the model training application is configured to run on an operating system different from the local operating system.

### Claim 6 (depends on 5)

The method of claim 5, wherein the local operating is MICROSOFT WINDOWS®.

### Claim 7 (depends on 6)

The method of claim 6, wherein the application is configured to run on LINUX.

### Claim 8 (depends on 1)

The method of claim 1, wherein calculating the model parameter based on the parameter data comprises applying a parameter averaging method to the parameter data.

### Claim 9 (depends on 1)

The method of claim 1, wherein calculating the model parameter based on the parameter data comprises applying a gradient descent method to the parameter data.

### Claim 10

An apparatus for managing deployment of distributed computing resources, comprising:
a memory comprising computer-executable instructions; and
a processor in data communication with the memory and configured to execute the computer-executable instructions and cause the apparatus to perform a method for training a machine learning model in a distributed computing system, the method comprising:
receiving a model training request;
receiving a training data set;
determining a processing node available in a distributed computing system;
receiving static status information regarding the processing node;
causing a first container to be installed at the processing node based on the static status information, the first container being configured with a model training application;
causing a second container to be installed at the processing node based on the static status information, the second container being configured with the model training application;
assigning a first layer of a model to be trained by the model training application in the first container;
assigning a second layer of the model to be trained by the model training application in the second container;
receiving parameter data from the model training application in the first container and the model training application in the second container; and
calculating a model parameter based on the parameter data.

### Claim 11 (depends on 10)

The apparatus of claim 10, wherein the method further comprises:
assigning a first data subset to the model training application in the first container; and
assigning a second data subset to the model training application in the second container.

### Claim 12 (depends on 10)

The apparatus of claim 10, wherein the method further comprises:
assigning a first data subset to the model training application in the first container; and
assigning the first data subset to the model training application in the second container.

### Claim 13 (depends on 10)

The apparatus of claim 10, wherein the method further comprises:
causing a third container to be installed at the processing node based on the static status information, the third container being configured with the model training application;
assigning the first layer and the second layer to be trained by the model training application in the third container; and
receiving parameter data from the model training application in the third container.

### Claim 14 (depends on 10)

The apparatus of claim 10, wherein:
the processing node comprises a local operating system, and
the model training application is configured to run on an operating system different from the local operating system.

### Claim 15 (depends on 14)

The apparatus of claim 14, wherein the local operating is MICROSOFT WINDOWS®.

### Claim 16 (depends on 15)

The apparatus of claim 15, wherein the application is configured to run on LINUX.

### Claim 17 (depends on 10)

The apparatus of claim 10, wherein calculating the model parameter based on the parameter data comprises applying a parameter averaging method to the parameter data.

### Claim 18 (depends on 10)

The apparatus of claim 10, wherein calculating the model parameter based on the parameter data comprises applying a gradient descent method to the parameter data.

### Claim 19

A non-transitory computer-readable medium comprising instructions for performing a method for training a machine learning model in a distributed computing system, the method comprising:
receiving a model training request;
receiving a training data set;
determining a processing node available in a distributed computing system;
receiving static status information regarding the processing node;
causing a first container to be installed at the processing node based on the static status information, the first container being configured with a model training application;
causing a second container to be installed at the processing node based on the static status information, the second container being configured with the model training application;
assigning a first layer of a model to be trained by the model training application in the first container;
assigning a second layer of the model to be trained by the model training application in the second container;
receiving parameter data from the model training application in the first container, the model training application in the second container, and the model training application in the third container; and
calculating a model parameter based on the parameter data.

### Claim 20 (depends on 19)

The non-transitory computer-readable medium of claim 19, wherein:
the processing node comprises a local operating system, and
the model training application is configured to run on an operating system different from the local operating system.