# US 11,451,601 B2 · Systems and methods for dynamic allocation of computing resources for microservice architecture type applications

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 11,451,601 B2
- Document ID: US11451601B2
- Status: Granted
- Kind: B2
- Application number: 16/995,927
- Filed: 2020-08-18
- Published: 2022-02-24
- Granted: 2022-09-20
- Earliest claimed: 2020-08-18
- Assignees: Spotify AB
- Applicants: Spotify AB
- Inventors: Arnthór Johann Jónsson; Emil B. L. Pedersen
- Web page: https://patentgrep.com/patent/US11451601B2

## CPC

- H04L 65/61: for supporting one-way streaming services, e.g. Internet radio
- H04L 67/10: in which an application is distributed across nodes in the network (software deployment G06F8/60; multiprogramming arrangements G06F9/46)
- G06F 9/5027: {the resource being a machine, e.g. CPUs, Servers, Terminals}
- G06F 9/505: {considering the load}
- G06F 9/542: {Event management; Broadcasting; Multicasting; Notifications}
- G06F 9/547: {Remote procedure calls [RPC]; Web services}
- H04L 65/80: Responding to QoS
- H04L 67/06: specially adapted for file transfer, e.g. file transfer protocol [FTP]
- H04L 67/306: User profiles
- G06F 2209/5013: Request control
- G06F 2209/5019: Workload prediction
- G06F 2209/508: Monitor
- H04L: TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION (arrangements common to telegraphic and telephonic communication H04M)
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)

## Abstract

Systems and methods for auto-scaling computing resources for applications that rely on microservices, such as media streaming applications. In example embodiments, the systems and methods use a hybrid approach that takes into account proactive and reactive metrics to predict future processing needs. The proactive metrics use data obtained from the microservices pipeline. In example embodiments, computing units are deployed or removed from a cluster of computing units in anticipation of a future computational need and in consideration of quality of service requirements.

## Claims

### Claim 1

A method of scaling a number of computer processing units, comprising the steps of:
receiving processing requests, each processing request requiring a first processing service and a subsequent second processing service that receives output from the first service, processes the output and completes the processing request;
monitoring, for each request, at each of a plurality of times that precede a predefined time, a plurality of previous processing events in an event sequence of the first processing service that processes the request, each of the previous processing events having an associated action type that is different from an associated action type of at least one later action performed by the second processing service that further processes the output of the first processing service for at least one of the requests, to provide previous events data;
selecting a number of computer processing units to perform the at least one later action of the second processing service at the predefined time, the selecting being based on the previous events data; and
making available at the predefined time the selected number of computer processing units to perform the at least one later action of the second processing service to complete the at least one of the requests.

### Claim 2 (depends on 1)

The method of claim 1, wherein the previous events data includes the associated action type of each previous event and a number of instances of each of the previous events occurring at each of the times that precede the predefined time.

### Claim 3 (depends on 1)

The method of claim 1, wherein each of the previous events is associated with one of a plurality of microservices of the first processing service.

### Claim 4 (depends on 3)

The method of claim 3,
wherein the action types include a parsing type and file downloading type; and
wherein the previous events data include:
when a request has been parsed;
when a file download has begun; and
when a file download has finished.

### Claim 5 (depends on 3)

The method of claim 3, wherein each event sequence and the selecting are triggered by an availability of at least one new or modified media content item.

### Claim 6 (depends on 1)

The method of claim 1,
wherein the selecting the number of computer processing units is also based on one or more of:
(i) a current length of a queue of the processing requests;
(ii) a predetermined processing throughput of each of the computer processing units; and
(iii) a predefined maximum length of time to complete processing of the queue.

### Claim 7 (depends on 6)

The method of claim 6,
wherein the first processing service includes a request ingestion service;
wherein the second processing service includes a transcoding service that receives and processes output of the request ingestion service; and
wherein each of the event sequences is triggered by a request to publish at least one media content item or by a request to stream at least one media content item received by the request ingestion service.

### Claim 8 (depends on 7)

The method of claim 7,
wherein the computer processing units are made available for the transcoding service; and
wherein the at least one later action includes a transcoding of a media content item for streaming.

### Claim 9 (depends on 7)

The method of claim 7, wherein the previous events data include:
when a request has been parsed by the request ingestion service;
when a file download has been begun by the request ingestion service;
when a file download has been finished by the request ingestion service; and
when the request has been received by the transcoder service.

### Claim 10 (depends on 1)

The method of claim 1,
wherein the previous events data is weighted based on:
the associated time that precedes the predefined time; and
the associated action type.

### Claim 11 (depends on 1)

The method of claim 1, further comprising: performing, at the predefined time, the at least one later action with the made available computer processing units.

### Claim 12

A system for scaling a number of computer processing units, comprising:
one or more processors and one or more storage devices storing instructions that are operable, when executed by the one or more processors, to cause the one or more processors to:
receive processing requests, each processing request requiring a first processing service and a subsequent second processing service that receives output from the first service, processes the output and completes the processing request;
monitor, for each request, at each of a plurality of times that precede a predefined time, a plurality of previous processing events in an event sequence of the first processing service that processes the request, each of the previous processing events having an associated action type that is different from an associated action type of at least one later action performed by the second processing service that further processes the output of the first processing service for at least one of the requests, to provide previous events data;
select a number of computer processing units to perform the at least one later action of the second processing service at the predefined time, to select being based on the previous events data; and
make available at the predefined time the selected number of computer processing units to perform the at least one later action of the second processing service to complete the at least one of the requests.

### Claim 13 (depends on 12)

The system of claim 12, wherein the previous events data includes the associated action type of each previous event and a number of instances of each of the previous events occurring at each of the times that precede the predefined time.

### Claim 14 (depends on 12)

The system of claim 12, wherein each of the previous events is associated with one of a plurality of microservices of the first processing service.

### Claim 15 (depends on 14)

The system of claim 14,
wherein the action types include a parse type and file download type; and
wherein the previous events data include:
when a request has been parsed;
when a file download has begun; and
when a file download has finished.

### Claim 16 (depends on 14)

The system of claim 14, wherein each event sequence is triggered by an availability of at least one new or modified media content item.

### Claim 17 (depends on 14)

The system of claim 14,
wherein the select the number of computer processing units is also based on one or more of:
(i) a current length of a queue of the processing requests;
(ii) a predetermined processing throughput of each of the computer processing units; and
(iii) a predefined maximum length of time to complete processing of the queue.

### Claim 18 (depends on 17)

The system of claim 17,
wherein the instructions are operable, when executed by the one or more processors, to cause the one or more processors to provide a request ingestion service and a transcoding service;
wherein each event sequence is associated with the request ingestion service;
wherein the transcoding service receives the processing requests from the request ingestion service; and
wherein each of the event sequences is triggered by a request to publish at least one media content item or by a request to stream at least one media content item received by the request ingestion service.

### Claim 19 (depends on 18)

The system of claim 18,
wherein the computer processing units are made available for the transcoding service; and
wherein the at least one later action includes a transcoding of a media content item for streaming.

### Claim 20 (depends on 18)

The system of claim 18, wherein the previous events data include:
when a request has been parsed by the request ingestion service;
when a file download has been begun by the request ingestion service;
when a file download has been finished by the request ingestion service; and
when the request has been received by the transcoder service.

### Claim 21 (depends on 12)

The system of claim 12,
wherein the previous events data is weighted based on:
the associated time that precedes the predefined time; and
the associated action type.

### Claim 22 (depends on 12)

The system of claim 12, wherein the instructions, when executed by the one or more processors, further cause the one or more made available processing units to perform, at the predefined time, the at least one later action.

### Claim 23

A non-transitory computer-readable medium having stored thereon one or more sequence of instructions for causing one or more processors to perform:
receiving processing requests, each processing request requiring a first processing service and a subsequent second processing service that receives output from the first service, processes the output and completes the processing request;
monitoring, for each request, at each of a plurality of times that precede a predefined time, a plurality of previous processing events in an event sequence of the first processing service that processes the request, each of the previous processing events having an associated action type that is different from an associated action type of at least one later action performed by the second processing service that further processes the output of the first processing service for at least one of the requests, to provide previous events data;
selecting a number of computer processing units to perform the at least one later action of the second processing service at the predefined time, the selecting being based on the previous events data; and
making available at the predefined time the selected number of computer processing units to perform the at least one later action of the second processing service to complete the at least one of the requests.

### Claim 24 (depends on 23)

The non-transitory computer-readable medium of claim 23, wherein the previous events data includes the associated action type of each previous event and a number of instances of each of the previous events occurring at each of the times that precede the predefined time.

### Claim 25 (depends on 23)

The non-transitory computer-readable medium of claim 23, wherein each of the previous events is associated with one of a plurality of microservices of the first processing service.

### Claim 26 (depends on 25)

The non-transitory computer-readable medium of claim 25,
wherein the action types include a parsing type and file downloading type; and
wherein the previous events data include:
when a request has been parsed;
when a file download has begun; and
when a file download has finished.

### Claim 27 (depends on 25)

The non-transitory computer-readable medium of claim 25, wherein each event sequence is triggered by an availability of at least one new or modified media content item.

### Claim 28 (depends on 23)

The non-transitory computer-readable medium of claim 23,
wherein the selecting the number of computer processing units is also based on one or more of:
(i) a current length of a queue of the processing requests;
(ii) a predetermined processing throughput of each of the computer processing units; and
(iii) a predefined maximum length of time to complete processing of the queue.

### Claim 29 (depends on 28)

The non-transitory computer-readable medium of claim 28,
wherein each event sequence is associated with a request ingestion service;
wherein a transcoding service receives the processing requests from the request ingestion service; and
wherein each of the event sequences is triggered by a request to publish at least one media content item or by a request to stream at least one media content item received by the request ingestion service.

### Claim 30 (depends on 29)

The non-transitory computer-readable medium of claim 29,
wherein the computer processing units are made available for the transcoding service; and
wherein the at least one later action includes a transcoding of a media content item for streaming.

### Claim 31 (depends on 29)

The non-transitory computer-readable medium of claim 29, wherein the previous events data include:
when a request has been parsed by the request ingestion service;
when a file download has been begun by the request ingestion service;
when a file download has been finished by the request ingestion service; and
when the request has been received by the transcoder service.

### Claim 32 (depends on 23)

The non-transitory computer-readable medium of claim 23,
wherein the previous events data is weighted based on:
the associated time that precedes the predefined time; and
the associated action type.

### Claim 33 (depends on 23)

The non-transitory computer-readable medium of claim 23, wherein the one or more sequences of instructions cause the made available computer processing units to perform, at the predefined time, the at least one later action.