# US 2021/0279100 A1 · Systems and methods for rate limiting of processing systems

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2021/0279100 A1
- Document ID: US20210279100A1
- Status: Published application
- Kind: A1
- Application number: 16/812,991
- Filed: 2020-03-09
- Published: 2021-09-09
- Earliest claimed: 2020-03-09
- Assignees: Walmart Apollo, LLC
- Applicants: Walmart Apollo, LLC
- Inventors: Menkae Jeng
- Web page: https://patentgrep.com/patent/US20210279100A1

## CPC

- G06F 9/4887: {involving deadlines, e.g. rate based, periodic}
- G06F 9/505: {considering the load}
- G06F 9/5083: {Techniques for rebalancing the load in a distributed system}
- G06F 2209/508: Monitor
- G06F 2209/548: Queue
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)

## Abstract

Systems and methods including one or more processors and one or more non-transitory storage devices storing computing instructions configured to run on the one or more processors and perform acts of receiving one or more processing requests; assigning each respective processing request of the one or more processing requests to a respective queue of one or more queues; assigning each respective queue of the one or more queues to a respective processing node of one or more processing nodes; calculating a respective processing request backlog for each respective processing node of the one or more processing nodes; and limiting a processing rate of the respective processing node for processing requests of the one or more processing requests of the respective queue based on the respective processing request backlog for the respective processing node. Other embodiments are disclosed herein.

## Claims

### Claim 1

A system comprising:
one or more processors; and
one or more non-transitory computer-readable storage devices storing computing instructions configured to run on the one or more processors and perform acts of:
receiving one or more processing requests;
assigning each respective processing request of the one or more processing requests to a respective queue of one or more queues;
assigning each respective queue of the one or more queues to a respective processing node of one or more processing nodes;
calculating a respective processing request backlog for each respective processing node of the one or more processing nodes; and
limiting a processing rate of the respective processing node for processing requests of the one or more processing requests of the respective queue based on the respective processing request backlog for the respective processing node.

### Claim 2 (depends on 1)

The system of claim 1, wherein assigning each respective queue to the respective processing node comprises:
re-assigning the respective queue to a different one of the one or more processing nodes.

### Claim 3 (depends on 2)

The system of claim 2, wherein re-assigning the respective queue to the different one of the one or more processing nodes occurs only when a queue re-assignment event is detected.

### Claim 4 (depends on 3)

The system of claim 3, wherein the queue re-assignment event comprises (1) a processing node of the one or more processing nodes fails, (2) the processing node of the one or more processing nodes recovers, (3) the processing node of the one or more processing nodes is removed from the system by an administrator, or (4) the processing node of the one or more processing nodes is added to the system by the administrator.

### Claim 5 (depends on 3)

The system of claim 3, wherein the queue re-assignment event is detected by the respective processing node of the one or more processing nodes.

### Claim 6 (depends on 1)

The system of claim 1, wherein limiting the processing rate of the respective processing node is performed by the respective processing node of the one or more processing nodes.

### Claim 7 (depends on 6)

The system of claim 6, wherein limiting the processing rate of the respective processing node is performed by an automated background process running on the respective processing node of the one or more processing nodes.

### Claim 8 (depends on 6)

The system of claim 6, wherein each respective processing node of the one or more processing nodes limits its own processing rate in synchrony with each other.

### Claim 9 (depends on 1)

The system of claim 1, wherein calculating the respective processing request backlog for a particular processing node of the one or more processing nodes comprises summing a total number of processing requests of the one or more processing requests assigned to a particular queue of the one or more queues assigned to the particular processing node.

### Claim 10 (depends on 1)

The system of claim 1, wherein calculating the respective processing request backlog comprises using a commit offset and an end offset to calculate the respective processing request backlog for each respective processing node.

### Claim 11

A method implemented via execution of computing instructions configured to run at one or more processors and configured to be stored at non-transitory computer-readable media, the method comprising:
receiving one or more processing requests;
assigning each respective processing request of the one or more processing requests to a respective queue of one or more queues;
assigning each respective queue of the one or more queues to a respective processing node of one or more processing nodes;
calculating a respective processing request backlog for each respective processing node of the one or more processing nodes; and
limiting a processing rate of the respective processing node for processing requests of the one or more processing requests of the respective queue based on the respective processing request backlog for the respective processing node.

### Claim 12 (depends on 11)

The method of claim 11, wherein assigning each respective queue to the respective processing node comprises:
re-assigning the respective queue to a different one of the one or more processing nodes.

### Claim 13 (depends on 12)

The method of claim 12, wherein re-assigning the respective queue to the different one of the one or more processing nodes occurs only when a queue re-assignment event is detected.

### Claim 14 (depends on 13)

The method of claim 13, wherein the queue re-assignment event comprises (1) a processing node of the one or more processing nodes fails, (2) the processing node of the one or more processing nodes recovers, (3) the processing node of the one or more processing nodes is removed from the system by an administrator, or (4) the processing node of the one or more processing nodes is added to the system by the administrator.

### Claim 15 (depends on 13)

The method of claim 13, wherein the queue re-assignment event is detected by the respective processing node of the one or more processing nodes.

### Claim 16 (depends on 11)

The method of claim 11, wherein limiting the processing rate of the respective processing node is performed by the respective processing node of the one or more processing nodes.

### Claim 17 (depends on 16)

The method of claim 16, wherein limiting the processing rate of the respective processing node is performed by an automated background process running on the respective processing node of the one or more processing nodes.

### Claim 18 (depends on 16)

The method of claim 16, wherein each respective processing node of the one or more processing nodes limits its own processing rate in synchrony with each other.

### Claim 19 (depends on 11)

The method of claim 11, wherein calculating the respective processing request backlog for a particular processing node of the one or more processing nodes comprises summing a total number of processing requests of the one or more processing requests assigned to a particular queue of the one or more queues assigned to the particular processing node.

### Claim 20 (depends on 11)

The method of claim 11, wherein calculating the respective processing request backlog comprises using a commit offset and an end offset to calculate the respective processing request backlog for each respective processing node.
