# US 2023/0053575 A1 · Partitioning and placement of models

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2023/0053575 A1
- Document ID: US20230053575A1
- Status: Published application
- Kind: A1
- Application number: 17/578,872
- Filed: 2022-01-19
- Published: 2023-02-23
- Earliest claimed: 2021-08-18
- Assignees: Cisco Technology, Inc.
- Applicants: Cisco Technology, Inc.
- Inventors: Leo Marche; Thomas Michel-Ange Feltin; Andre Surcouf; Frank Brockners
- Web page: https://patentgrep.com/patent/US20230053575A1

## CPC

- G06F 9/5027: {the resource being a machine, e.g. CPUs, Servers, Terminals}
- G06F 9/505: {considering the load}
- G06N 3/045: Combinations of networks
- G06N 3/0464: Convolutional networks [CNN, ConvNet]
- G06N 3/08: Learning methods
- G06F 2209/501: Performance criteria
- G06F 2209/5017: Task decomposition
- G06F 2209/503: Resource availability
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS

## Abstract

This disclosure describes techniques and mechanisms for enabling a user to run heavy deep learning workloads on standard edge networks without off-loading computation to a cloud, leveraging the available edge computing resources, and efficiently partitioning and distributing a Deep Neural Network (DNN) over a network. The techniques enable the user to split a workload into multiple parts and process the workload on a set of smaller, less capable compute nodes in a distributed manner, without compromising on performance, and while meeting a Service Level Objective (SLO).

## Claims

### Claim 1

A method implemented by an orchestration system, the method comprising:
identifying a neural network model configured to process data generated by a computing device in a network;
analyzing the neural network model to determine a location in the neural network model at which to split the neural network model;
identifying a first computing device in the network at which to run a first workload associated with a first portion of the neural network model;
identifying a second computing device in the network at which to run a second workload associated with a second portion of the neural network model;
splitting the neural network model at the location to create the first workload and the second workload;
deploying the first workload to the first computing device; and
deploying the second workload to the second computing device.

### Claim 2 (depends on 1)

The method of claim 1, further comprising:
determining, based at least partly on monitoring the first computing device, that an event occurs that results in a deteriorated performance of the first computing device;
identifying a third computing device at which to run the first workload associated with the first portion of the neural network model; and
deploying the first workload to the third computing device.

### Claim 3 (depends on 2)

The method of claim 2, wherein the event comprises one of a CPU overload or a disconnect from the network.

### Claim 4 (depends on 1)

The method of claim 1, further comprising:
determining, based at least partly on monitoring the first computing device, that an event occurs that results in a deteriorated performance of the first computing device; and
identifying, based at least partly on the deteriorated performance, an optimized placement of the first workload associated with the first portion of the neural network model.

### Claim 5 (depends on 1)

The method of claim 1, wherein the neural network model comprises a deep learning neural network.

### Claim 6 (depends on 1)

The method of claim 1, wherein determining the location includes identifying a split location between one or more layers of the neural network model.

### Claim 7 (depends on 1)

The method of claim 1, wherein identifying the first computing device includes at least one of:
determining that an amount of central processing unit (CPU) available on the first computing device is sufficient to support the first workload; or
determining that an amount of bandwidth available to the first computing device is sufficient to receive data over the network to support the first workload.

### Claim 8 (depends on 1)

The method of claim 1, wherein identifying the first computing device is based at least in part on determining that a processor type or device type associated with the first computing device is optimized for running the first workload.

### Claim 9

A system comprising:
one or more processors; and
one or more non-transitory computer-readable media storing computer-executable instructions that, when executed by the one or more processors, cause the one or more processors to perform operations comprising:
identifying a neural network model configured to process data generated by a computing device in a network;
analyzing the neural network model to determine a location in the neural network model at which to split the neural network model;
identifying a first computing device in the network at which to run a first workload associated with a first portion of the neural network model;
identifying a second computing device in the network at which to run a second workload associated with a second portion of the neural network model;
splitting the neural network model at the location to create the first workload and the second workload;
deploying the first workload to the first computing device; and
deploying the second workload to the second computing device.

### Claim 10 (depends on 9)

The system of claim 9, the operations further comprising:
determining, based at least in part on monitoring the first computing device, that an event occurs that results in a deteriorated performance of the first computing device;
identifying a third computing device at which to run the first workload associated with the first portion of the neural network model; and
deploying the first workload to the third computing device.

### Claim 11 (depends on 10)

The system of claim 10, wherein the event comprises one of a CPU overload or a disconnect from the network.

### Claim 12

The system of the operations further comprising:
determining, based at least partly on monitoring the first computing device, that an event occurs that results in a deteriorated performance of the first computing device; and
identifying, based at least partly on the deteriorated performance, an optimized placement of the first workload associated with the first portion of the neural network model

### Claim 13 (depends on 9)

The system of claim 9, wherein the neural network model comprises a deep learning neural network.

### Claim 14 (depends on 9)

The system of claim 9, wherein determining the location includes identifying a split location between one or more layers of the neural network model, the split location being associated with optimizing throughput of the network.

### Claim 15 (depends on 9)

The system of claim 9, wherein identifying the first computing device or the second computing device is based at least in part on:
determining that an amount of central processing unit (CPU) available on the first computing device is sufficient to support the first workload; or
determining that an amount of bandwidth available to the first computing device is sufficient to receive data over the network to support the first workload.

### Claim 16 (depends on 9)

The system of claim 9, wherein identifying the first computing device is based at least in part on determining that a processor type or device type associated with the first computing device is optimized for running the first workload

### Claim 17

One or more non-transitory computer-readable media storing computer-readable instructions that, when executed by one or more processors, cause the one or more processors to perform operations comprising:
identifying a neural network model configured to process data generated by a computing device in a network;
analyzing the neural network model to determine a location in the neural network model at which to split the neural network model;
identifying a first computing device in the network at which to run a first workload associated with a first portion of the neural network model;
identifying a second computing device in the network at which to run a second workload associated with a second portion of the neural network model;
splitting the neural network model at the location to create the first workload and the second workload;
deploying the first workload to the first computing device; and
deploying the second workload to the second computing device.

### Claim 18 (depends on 17)

The one or more non-transitory computer-readable media of claim 17, the operations further comprising:
determining, based at least in part on monitoring the first computing device, that an event occurs that results in a deteriorated performance of the first computing device;
identifying a third computing device at which to run the first workload associated with the first portion of the neural network model; and
deploying the first workload to the third computing device.

### Claim 19 (depends on 18)

The one or more non-transitory computer-readable media of claim 18, wherein the event comprises one of a CPU overload or a disconnect from the network.

### Claim 20 (depends on 18)

The one or more non-transitory computer-readable media of claim 18, wherein determining the location includes identifying a split location between one or more layers of the neural network model, the split location being associated with optimizing throughput of the network.
