# US 2024/0205290 A1 · Apparatus, method, and system for dynamic pod autoscaling in a serverless computing environment > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 2024/0205290 A1 - Document ID: US20240205290A1 - Status: Published application - Kind: A1 - Application number: 18/515,505 - Filed: 2023-11-21 - Published: 2024-06-20 - Earliest claimed: 2022-12-19 - Assignees: Foundation of Soongsil University-Industry Cooperation - Applicants: Foundation of Soongsil University-Industry Cooperation - Inventors: Young Han KIM; Seung Hyeon KIM - Web page: https://patentgrep.com/patent/US20240205290A1 ## CPC - H04L 67/101: based on network conditions - H04L 67/1008: based on parameters of servers, e.g. available memory or workload (monitoring of computer activity G06F11/30) - H04L: TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION (arrangements common to telegraphic and telephonic communication H04M) ## Abstract An apparatus, method, and system for dynamic pod autoscaling in a serverless computing environment are disclosed. A dynamic pod autoscaling apparatus in a serverless computing environment comprises a processor; and a memory connected to the processor, wherein the memory stores program instructions for performing operations comprising registering, if there is a service request, a predefined first target value, collecting concurrency/rps (request per second) information of a first number of pods corresponding to the number of replica sets that is pod autoscaled according to the first target value, calculating the first target value as a second target value using the concurrency/rps information and average resource usage of the pod, dynamically changing the number of replica sets from the first number to a second number through the second target value. ## Claims ### Claim 1 A dynamic pod autoscaling apparatus in a serverless computing environment comprising: a processor; and a memory connected to the processor, wherein the memory stores program instructions for performing operations comprising, registering, if there is a service request, a predefined first target value, collecting concurrency/rps (request per second) information of a first number of pods corresponding to the number of replica sets that is pod autoscaled according to the first target value, calculating the first target value as a second target value using the concurrency/rps information and average resource usage of the pod, dynamically changing the number of replicasets from the first number to a second number through the second target value. ### Claim 2 (depends on 1) The apparatus of claim 1, wherein the service request is traffic data received by Knative Service (ksvc) through Knative Revion. ### Claim 3 (depends on 2) The apparatus of claim 2, wherein pod autoscaling according to the first and second target values is performed by KnativePodAutoscaler (KPA) that receives the traffic data from the ksvc. ### Claim 4 (depends on 1) The apparatus of claim 1, wherein the operations further comprise, calculating an average value of CPU and memory usage of the pod, and calculating the second target value so that the average value is not close to minimum and maximum values of resource usage, including CPU and memory usage of a pod defined in CRD (Custom Resource Defined). ### Claim 5 (depends on 4) The apparatus of claim 4, wherein the operations further comprise, calculating a variance by dividing the number of replica sets, which is obtained by dividing an amount of the service request by the first target value, by the first target value, calculating the second target value using the variance. ### Claim 6 (depends on 5) The apparatus of claim 5, wherein the operations further comprise, determining the second target value by adding the variance to the first target value when the average value is less than the minimum value. ### Claim 7 (depends on 5) The apparatus of claim 5, wherein the operations further comprise, determining the second target value by subtracting the variance from the first target value when the average value is greater than the maximum value. ### Claim 8 (depends on 1) The apparatus of claim 1, wherein the concurrency/rps information is collected from a queue proxy of the pod. ### Claim 9 (depends on 1) The apparatus of claim 1, wherein the average resource usage is calculated in a metric server connected to the pod. ### Claim 10 A dynamic pod autoscaling system in a serverless computing environment comprising: Knative Service for receiving traffic data according to a service request from Knative Revision; KnativePodAutoscaler (KPA) for calculating the number of replica sets by performing pod autoscaling according to a predefined first target value in the Knative Service; and a custom controller for collecting concurrency/rps (request per second) information of a first number of pods corresponding to the number of replica sets, and calculating the first target value as a second target value using the concurrency/rps information and average resource usage of the pod, wherein the KPA dynamically changes the number of replica sets from the first number to a second number through the second target value. ### Claim 11 (depends on 10) The system of claim 10, wherein the custom controller, calculates an average value of CPU and memory usage of the pod, calculates the second target value so that the average value is not close to minimum and maximum values of resource usage, including CPU and memory usage of a pod defined in CRD (Custom Resource Defined). ### Claim 12 (depends on 11) The system of claim 11, wherein the custom controller, calculates a variance by dividing the number of replica sets, which is obtained by dividing an amount of the service request by the first target value, by the first target value, calculates the second target value using the variance. ### Claim 13 (depends on 12) The system of claim 12, wherein the custom controller, determines the second target value by adding the variance to the first target value when the average value is less than the minimum value, determines the second target value by subtracting the variance from the first target value when the average value is greater than the maximum value. ### Claim 14 A dynamic pod autoscaling method in a serverless computing environment comprising: registering, if there is a service request, a predefined first target value; collecting concurrency/rps (request per second) information of a first number of pods corresponding to the number of replica sets that is pod autoscaled according to the first target value; calculating the first target value as a second target value using the concurrency/rps information and average resource usage of the pod; and dynamically changing the number of replicasets from the first number to a second number through the second target value. ### Claim 15 (depends on 14) A computer program stored in a computer-readable recording medium for performing the method of claim 14.