# US 2025/0165295 A1 · Machine learning model scheduler > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 2025/0165295 A1 - Document ID: US20250165295A1 - Status: Published application - Kind: A1 - Application number: 18/950,046 - Filed: 2024-11-16 - Published: 2025-05-22 - Earliest claimed: 2023-11-17 - Assignees: Nokia Technologies Oy - Applicants: Nokia Technologies Oy - Inventors: Chulhong MIN; Utku Gunay ACER; Si Young JANG; Hyunjong LEE - Web page: https://patentgrep.com/patent/US20250165295A1 ## CPC - G06F 9/4881: {Scheduling strategies for dispatcher, e.g. round robin, multi-level priority queues} - G06F 9/5027: {the resource being a machine, e.g. CPUs, Servers, Terminals} - G06F 9/4887: {involving deadlines, e.g. rate based, periodic} - G06N 3/045: Combinations of networks - G06N 3/0464: Convolutional networks [CNN, ConvNet] - G06N 3/063: using electronic means - G06N 3/08: Learning methods - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) - G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS ## Abstract Apparatus, comprising: means for obtaining a first machine learning model, the first machine learning model comprising at least a first layer; means for obtaining a second machine learning model, the second machine learning model comprising at least a second layer; means for determining an execution schedule for executing the first machine learning model and the second machine learning model, wherein the means for determining the execution schedule are configured to: vary a number of processing resources and a start time allocated to the first layer and the second layer. The apparatus further comprises means for executing the first machine learning model and the second machine learning model according to the execution schedule. ## Claims ### Claim at position 1 -25. (canceled) ### Claim 26 Apparatus comprising: at least one processor; and at least one memory storing instructions that, when executed by the at least one processor, cause the apparatus at least to: obtain a first machine learning model, the first machine learning model comprising at least a first layer; obtain a second machine learning model, the second machine learning model comprising at least a second layer; determine an execution schedule for executing the first machine learning model and the second machine learning model, wherein the determining the execution schedule further comprises: varying a number of processing resources and a start time allocated to the first layer and the second layer; and execute the first machine learning model and the second machine learning model according to the execution schedule. ### Claim 27 (depends on 26) The apparatus according to claim 26, wherein the execution schedule comprises: a first start time defining when execution of the first layer begins; a second start time defining when execution of the second layer begins; and a resource allocation for the first layer and the second layer comprising at least one of: a first processing resource or a second processing resource. ### Claim 28 (depends on 27) The apparatus according to claim 27, wherein the executing the first machine learning model and the second machine learning model according to the execution schedule further comprises: execute the first layer of the first machine learning model at the first start time on the first processing resource and the second processing resource; and execute the second layer of the second machine learning model at a second start time on the first processing resource. ### Claim 29 (depends on 26) The apparatus according to claim 26, wherein the determining the execution schedule further comprises: generate a first possible execution schedule for executing the first machine learning model and the second machine learning model; generate a second possible execution schedule for executing the first machine learning model and the second machine learning model, wherein the first possible execution schedule is different to the second possible execution schedule; and select the first possible execution schedule or the second possible execution schedule as the execution schedule. ### Claim 30 (depends on 29) The apparatus according to claim 29, wherein when the apparatus is caused to select the first possible execution schedule or the second possible execution schedule as the execution schedule, further comprises to: determine a latency of the first possible execution schedule; determine a latency of the second possible execution schedule; and select the first possible execution schedule as the execution schedule in response to determining that the first possible execution schedule satisfies a criteria. ### Claim 31 (depends on 30) The apparatus according to claim 30, wherein the criteria is based on at least one of: a minimising of a total execution time of the first machine learning model and second machine learning model; or a maximising of a total inference throughput of the first machine learning model and second machine learning model. ### Claim 32 (depends on 26) The apparatus according to claim 26, wherein when the apparatus caused to determine the execution schedule for executing the first machine learning model and the second machine learning model, further comprises to: allocate at least one processing resource at a first time to execute the first layer of the first machine learning model; determine if the second layer of the second machine learning model can be executed in parallel on at least one unallocated processing resource; and allocate the at least one unallocated processing resource at the first time to execute the second layer in response to the determining that the second layer can be executed in parallel. ### Claim 33 (depends on 32) The apparatus according to claim 32, wherein when the apparatus is caused to determine the execution schedule, to: allocate the at least one processing resource processing resource to execute the second layer at a second time in response to the determining that the second layer of the cannot be executed in parallel, wherein the second time corresponds to a time when the first layer has completed execution on the at least one processing resource. ### Claim 34 (depends on 26) The apparatus according to claim 26, wherein the apparatus is further caused, when determining the execution schedule for executing the first machine learning model and the second machine learning model, to: determine a latency to execute the first layer of the machine learning model on a first set of processing resources. ### Claim 35 (depends on 34) The apparatus according to claim 34, wherein the first layer of the first machine learning model is associated with a type of operation performed by the first layer; and wherein the apparatus is further caused, when determining the latency to execute the first layer of the machine learning model on the first set of processing resources, to: determine a number of operations performed by the first layer on an input channel; and calculate the latency to execute the first layer based on: the number of operations performed by the first layer on the input channel; and a latency associated with the type of operation performed by the first layer. ### Claim 36 (depends on 35) The apparatus according to claim 35, where the type of operation performed by the first layer comprises a first convolutional operation using a first filter. ### Claim 37 (depends on 36) The apparatus according to claim 36, wherein the first layer of the first machine learning model is associated with a number of input channels to the first layer; and wherein the apparatus is further caused, when determining the latency, to calculate the latency to execute the first layer based on: the number of input channels; a number of times different input channels are executed on a single processing resource in the first set of processing resources. ### Claim 38 (depends on 37) The apparatus according to claim 37, wherein the apparatus is further caused, when determining the latency to execute the first layer of the machine learning model on the first set of processing resources to: determine the number of times different input channels are executed on the single processing resource based on: the number of input channels divided by a number of processing resources in the first set of processing resources. ### Claim 39 (depends on 26) The apparatus according to claim 26, wherein the first processing resource comprises at least one convolutional processing unit. ### Claim 40 (depends on 26) The apparatus according to claim 26, wherein the first machine learning model is an artificial neural network. ### Claim 41 A method comprising: obtaining a first machine learning model, the first machine learning model comprising at least a first layer; obtaining a second machine learning model, the second machine learning model comprising at least a second layer; determining an execution schedule for executing the first machine learning model and the second machine learning model, wherein determining the execution schedule further comprises: varying a number of processing resources and a start time allocated to the first layer and the second layer; and executing the first machine learning model and the second machine learning model according to the execution schedule. ### Claim 42 (depends on 41) The method according to claim 41, wherein the execution schedule comprises: a first start time defining when execution of the first layer begins; a second start time defining when execution of the second layer begins; and a resource allocation for the first layer and the second layer comprising at least one of: a first processing resource or a second processing resource. ### Claim 43 (depends on 42) The method according to claim 42, wherein executing the first machine learning model and the second machine learning model according to the execution schedule further comprises: executing the first layer of the first machine learning model at the first start time on the first processing resource and the second processing resource; and executing the second layer of the second machine learning model at a second start time on the first processing resource. ### Claim 44 (depends on 41) The method according to claim 41, wherein determining the execution schedule comprises: generating a first possible execution schedule for executing the first machine learning model and the second machine learning model; generating a second possible execution schedule for executing the first machine learning model and the second machine learning model, wherein the first possible execution schedule is different to the second possible execution schedule; and selecting the first possible execution schedule or the second possible execution schedule as the execution schedule. ### Claim 45 A non-transitory computer readable medium comprising program instructions that, when executed by an apparatus, cause the apparatus to perform at least the following: obtaining a first machine learning model, the first machine learning model comprising at least a first layer; obtaining a second machine learning model, the second machine learning model comprising at least a second layer; determining an execution schedule for executing the first machine learning model and the second machine learning model, wherein determining the execution schedule further comprises: varying a number of processing resources and a start time allocated to the first layer and the second layer; and executing the first machine learning model and the second machine learning model according to the execution schedule.