# US 2025/0156162 A1 · Resource constraint aware deep learning model optimization for serverless-based inference systems

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2025/0156162 A1
- Document ID: US20250156162A1
- Status: Published application
- Kind: A1
- Application number: 19/024,524
- Filed: 2025-01-16
- Published: 2025-05-15
- Earliest claimed: 2021-01-28
- Assignees: Red Hat, Inc.
- Applicants: Red Hat, Inc.
- Inventors: Huamin Chen; Jinjun Xiong; Roland Ludwig Huss
- Web page: https://patentgrep.com/patent/US20250156162A1

## CPC

- G06F 8/443: {Optimisation}
- G06F 8/4434: {Reducing the memory space required by the program code}
- G06F 8/4441: {Reducing the execution time required by the program code}
- G06F 9/45558: {Hypervisor-specific management and integration aspects}
- G06N 3/04: Architecture, e.g. interconnection topology
- G06F 2009/45579: {I/O management, e.g. providing access to device drivers or storage}
- G06N 3/045: Combinations of networks
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS

## Abstract

A method includes generating an optimized version of an inference serverless function using a graph compiler. The method further includes replacing a storage volume in an init container of the inference serverless function with a new storage volume comprising the optimized version of the inference serverless function.

## Claims

### Claim 1

A method, comprising:
generating an optimized version of an inference serverless function using a graph compiler; and
replacing, by a processing device of a webhook controller, a storage volume in an init container of the inference serverless function with a new storage volume comprising the optimized version of the inference serverless function.

### Claim 2 (depends on 1)

The method of claim 1, further comprising:
invoking a graph compiler image of the graph compiler;
providing a location of a deep learning model of the inference serverless function to the graph compiler; and
providing a resource claim and limit of the inference serverless function to the graph compiler.

### Claim 3 (depends on 1)

The method of claim 1, further comprising:
generating the new storage volume of the init container; and
storing the optimized version of the inference serverless function in the new storage volume of the init container.

### Claim 4 (depends on 1)

The method of claim 1, further comprising:
determining that a resource claim or limit of the inference serverless function has been modified since a prior invocation of the inference serverless function; and
providing an updated resource claim or limit to the graph compiler to generate the optimized version of the inference serverless function.

### Claim 5 (depends on 1)

The method of claim 1, further comprising determining that the inference serverless function can be optimized.

### Claim 6 (depends on 5)

The method of claim 5, wherein determining that the inference serverless function can be optimized comprises at least one of:
determining that the inference serverless function is being invoked for a first time,
determining that a resource claim or limit of the inference serverless function has been modified since a prior invocation of the inference serverless function, or
determining that a container volume associated with the inference serverless function has been modified since a prior invocation of the inference serverless function.

### Claim 7 (depends on 1)

The method of claim 1, further comprising generating the init container for the inference serverless function in response to determining that the inference serverless function can be optimized.

### Claim 8

A system, comprising:
a memory; and
a processing device operatively coupled to the memory, the processing device to:
generate an optimized version of an inference serverless function using a deep neural network optimizer; and
replace a storage volume in an init container of the inference serverless function with a new storage volume comprising the optimized version of the inference serverless function.

### Claim 9 (depends on 8)

The system of claim 8, the processing device further to:
invoke a deep neural network optimizer image of the deep neural network optimizer;
provide a location of a deep learning model of the inference serverless function to the deep neural network optimizer; and
provide a resource claim and limit of the inference serverless function to the deep neural network optimizer.

### Claim 10 (depends on 8)

The system of claim 8, the processing device further to:
generate the new storage volume of the init container; and
store the optimized version of the inference serverless function in the new storage volume of the init container.

### Claim 11 (depends on 8)

The system of claim 8, the processing device further to:
determine that a resource claim or limit of the inference serverless function has been modified since a prior invocation of the inference serverless function; and
provide an updated resource claim or limit to the deep neural network optimizer for generating the optimized version of the inference serverless function.

### Claim 12 (depends on 8)

The system of claim 8, the processing device further to provide a pre-configured cost model associated with the inference serverless function to the deep neural network optimizer for generating the optimized version of the inference serverless function.

### Claim 13 (depends on 8)

The system of claim 8, wherein to determine that the inference serverless function can be optimized the processing device is further to:
determine that the inference serverless function is being invoked for a first time,
determine that a resource claim or limit of the inference serverless function has been modified since a prior invocation of the inference serverless function, or
determine that a container volume associated with the inference serverless function has been modified since a prior invocation of the inference serverless function.

### Claim 14 (depends on 8)

The system of claim 8, the processing device further to generate the init container for the inference serverless function in response to determining that the inference serverless function can be optimized.

### Claim 15

A non-transitory computer-readable storage medium including instructions that, when executed by a processing device, cause the processing device to:
generate an optimized version of an inference serverless function using a graph compiler; and
replace, by the processing device, a storage volume in an init container of the inference serverless function with a new storage volume comprising the optimized version of the inference serverless function.

### Claim 16 (depends on 15)

The non-transitory computer-readable storage medium of claim 15, the processing device further to:
invoke a graph compiler image of the graph compiler;
provide a location of a deep learning model of the inference serverless function to the graph compiler;
provide a resource claim and limit of the inference serverless function to the graph compiler;
generate the new storage volume of the init container; and
store the optimized version of the inference serverless function in the new storage volume of the init container.

### Claim 17 (depends on 15)

The non-transitory computer-readable storage medium of claim 15, the processing device further to:
determine that a resource claim or limit of the inference serverless function has been modified since a prior invocation of the inference serverless function; and
provide an updated resource claim or limit to the graph compiler for generating the optimized version of the inference serverless function.

### Claim 18 (depends on 15)

The non-transitory computer-readable storage medium of claim 15, the processing device further to provide a pre-configured cost model associated with the inference serverless function to the graph compiler for generating the optimized version of the inference serverless function.

### Claim 19 (depends on 15)

The non-transitory computer-readable storage medium of claim 15, wherein the processing device further to determine that the inference serverless function can be optimized.

### Claim 20 (depends on 19)

The non-transitory computer-readable storage medium of claim 19, the processing device further to generate the init container for the inference serverless function in response to determining that the inference serverless function can be optimized.
