# US 12,182,669 B2 · Machine learning model compiler > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 12,182,669 B2 - Document ID: US12182669B2 - Status: Granted - Kind: B2 - Application number: 17/094,772 - Filed: 2020-11-10 - Published: 2021-12-23 - Granted: 2024-12-31 - Earliest claimed: 2020-06-19 - Assignees: Apple Inc. - Applicants: Apple Inc. - Inventors: Thomas Fogal; Adam Nemet; Gerolf F. Hoflehner - Web page: https://patentgrep.com/patent/US12182669B2 ## CPC - G06F 8/443: {Optimisation} - G06N 20/00: Machine learning - G06F 8/4434: {Reducing the memory space required by the program code} - G06F 8/447: {Target code generation} - G06F 8/47: {Retargetable compilers} - G06F 16/245: Query processing - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) - G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS ## Abstract The subject technology provides a framework for executable machine learning models that are executable in a zero-runtime operating environment. This allows the machine learning models to be deployed in limited memory environments such as embedded domains. A machine learning compiler is provided to generate the executable machine learning models. ## Claims ### Claim 1 A method, comprising: providing a machine learning model to a machine learning compiler; generating, with the machine learning compiler, compilable source code for all operations of the machine learning model, the compilable source code configured to be compiled by a separate compiler; and generating, with the machine learning compiler using compiled machine code generated by the separate compiler from the compilable source code, an executable machine learning model that is executable in a zero-runtime operating environment. ### Claim 2 (depends on 1) The method of claim 1, further comprising generating the machine learning model, wherein generating the compilable source code comprises: creating a set of machine learning kernels; and optimizing, with a front end of the machine learning compiler, the generated compilable source code for one or more of the operations. ### Claim 3 (depends on 2) The method of claim 2, further comprising optimizing the machine learning kernels, wherein optimizing the generated compilable source code for one or more of the operations and optimizing the machine learning kernels comprises balancing a performance metric and a size metric for the executable machine learning model. ### Claim 4 (depends on 2) The method of claim 2, further comprising optimizing the machine learning kernels, wherein optimizing the machine learning kernels comprises implementing a callback-based solution to bound memory use to prevent the executable machine learning model from allocating memory during execution time. ### Claim 5 (depends on 2) The method of claim 2, further comprising optimizing the machine learning kernels, wherein optimizing the machine learning kernels comprises removing run-time checks as directed by the front end of the machine learning compiler. ### Claim 6 (depends on 2) The method of claim 2, wherein optimizing the generated compilable source code for the one or more of the operations comprises: separating, with the front end of the machine learning compiler, the one or more of the operations into loops; determining directions to optimize the loops; and providing the loops and the directions to optimize the loops to a loop interface of the machine learning compiler. ### Claim 7 (depends on 6) The method of claim 6, further comprising, with the loop interface and using the directions to optimize the loops, operating a loop optimizer to optimize one or more of the loops. ### Claim 8 (depends on 6) The method of claim 6, wherein the directions to optimize comprise scheduling directives. ### Claim 9 (depends on 2) The method of claim 2, wherein optimizing the generated compilable source code comprises performing memory layout and alignment operations. ### Claim 10 (depends on 1) The method of claim 1, wherein the executable machine learning model comprises platform specific executable code. ### Claim 11 (depends on 1) The method of claim 1, wherein a portion of the compilable source code comprises C code. ### Claim 12 (depends on 1) The method of claim 1, wherein a portion of the compilable source code comprises intermediate representation code for the separate compiler. ### Claim 13 (depends on 1) The method of claim 1, further comprising executing the executable machine learning model in the zero-runtime operating environment to generate a model output. ### Claim 14 (depends on 1) The method of claim 1, further comprising providing, with the machine learning compiler, profiling tools to aid users in tuning machine learning models. ### Claim 15 A non-transitory computer-readable medium storing instructions that, when executed by one or more processors, cause the one or more processors to perform operations comprising: providing a machine learning model to a machine learning compiler; generating, with the machine learning compiler, compilable source code for all operations of the machine learning model, the compilable source code configured to be compiled by a separate compiler, the compilable source code including code for only a subset of a runtime library that is used by the machine learning model; and generating, with the machine learning compiler using compiled machine code generated by the separate compiler from the compilable source code, an executable machine learning model. ### Claim 16 (depends on 15) The non-transitory computer-readable medium of claim 15, the operations further comprising generating the machine learning model, wherein generating the compilable source code comprises: creating a set of machine learning kernels; and optimizing, with a front end of the machine learning compiler, the generated compilable source code for one or more of the operations. ### Claim 17 (depends on 16) The non-transitory computer-readable medium of claim 16, further comprising optimizing the machine learning kernels, wherein optimizing the machine learning kernels comprises implementing a callback-based solution to bound memory use to prevent the executable machine learning model from allocating memory during execution time. ### Claim 18 (depends on 16) The non-transitory computer-readable medium of claim 16, the operations further comprising optimizing the machine learning kernels, wherein optimizing the machine learning kernels comprises removing run-time checks as directed by the front end of the machine learning compiler. ### Claim 19 A device, comprising: one or more processors; and a memory including instructions that are executable by the one or more processors for causing the one or more processor to: provide a machine learning model to a machine learning compiler; generate, with the machine learning compiler, compilable source code for all operations of the machine learning model, the compilable source code configured to be compiled by a separate compiler; and generate, with the machine learning compiler using compiled machine code generated by the separate compiler from the compilable source code, an executable machine learning model that is executable in a zero-runtime operating environment. ### Claim 20 (depends on 19) The device of claim 19, wherein the executable machine learning model comprises platform specific executable code.