# US 12,223,425 B2 · End-to-end optimization > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 12,223,425 B2 - Document ID: US12223425B2 - Status: Granted - Kind: B2 - Application number: 17/204,870 - Filed: 2021-03-17 - Published: 2021-09-23 - Granted: 2025-02-11 - Earliest claimed: 2020-03-17 - Assignees: OnSpecta, Inc. - Applicants: Onspecta, Inc. - Inventors: Victor Jakubiuk; Sebastian Kaczor - Web page: https://patentgrep.com/patent/US12223425B2 ## CPC - G06N 3/08: Learning methods - G06N 3/045: Combinations of networks - G06N 3/063: using electronic means - G06N 3/082: modifying the architecture, e.g. adding, deleting or silencing nodes or connections - G06N 3/0985: Hyperparameter optimisation; Meta-learning; Learning-to-learn - G06N 5/01: Dynamic search techniques; Heuristics; Dynamic trees; Branch-and-bound - G06N 3/096: Transfer learning - G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS ## Abstract Methods, systems, and apparatus, including computer programs encoded on computer storage media, includes various embodiments for receiving a plurality of characteristics of a target artificial intelligence (AI) network. The various embodiments apply the plurality of characteristics of the target AI network to at least one of a static cost model and a heuristic AI network model. The various embodiments further receive optimized target AI network configuration data from at least one of static cost model and the heuristic AI network model, the optimized target AI network configuration data representative of a subset of the characteristics of the target AI network that minimize a cost function of execution of the target AI network. ## Claims ### Claim 1 A computer-implemented method comprising: receiving a plurality of characteristics of a target artificial intelligence (AI) network; applying the plurality of characteristics of the target AI network to at least one of a static cost model and a heuristic AI network model; and receiving optimized target AI network configuration data from at least one of static cost model and the heuristic AI network model, the optimized target AI network configuration data representative of a subset of the characteristics of the target AI network that minimize a cost function of execution of the target AI network; wherein the plurality of characteristics of a target AI network comprises: a plurality of operations that correspond to respective nodes on a computational graph of the target AI network; at least one kernel for implementing one or more operations, the kernel comprised of at least one micro-kernel for formatting kernel-operation an input data stream according to a target hardware input data format compatible with underlying target hardware supporting execution of the operation; and at least one post-processor module embedded into each micro-kernel, wherein each post-processor module converts target hardware output data located in one or more target hardware registers from a first output data format native to the underlying target hardware to a second output data format that matches an input data format of a subsequent micro-kernel. ### Claim 2 (depends on 1) The computer-implemented method of claim 1, wherein applying the plurality of characteristics of the target AI network to the static cost model comprises: simulating execution of the target AI network by iterating execution of the target AI network according to a plurality of differing kernel and post-processor module combinations with respect to at least a subset of the plurality of characteristics target AI network; and obtaining a time cost approximation for each simulated target AI network execution, wherein each simulated target AI network execution corresponds to a distinct set of kernel combinations. ### Claim 3 (depends on 2) The computer-implemented method of claim 2, wherein receiving optimized target AI network configuration data from the static cost model comprises: selecting a time cost approximation that corresponds with a particular simulated target AI network execution; and receiving a select distinct set of kernel and post-processor module combinations utilized during the particular simulated target AI network execution associated with the selected time cost approximation. ### Claim 4 (depends on 1) The computer-implemented method of claim 1, wherein applying the plurality of characteristics of the target AI network to the heuristic AI network model comprises: inputting at least a subset of the plurality of characteristics target AI network into a heuristic AI network trained according to heuristic network training data; and wherein receiving optimized target AI network configuration data from heuristic AI network model comprises: receiving an approximation of run time of the target AI network according to a combination of one or more optimal data formats, one or more optimal operation-calculation algorithms per kernel, one or more optimal kernel and post-processor module combinations and one or more optimal underlying hardware specifications. ### Claim 5 (depends on 4) The computer-implemented method of claim 4, wherein the heuristic network training data comprises at least one of: one or more underlying hardware microchip types, one or more memory hierarchies, one or more AI network structures, one or more operation hyperparameters, one or more optimal operation-calculation algorithms, one or more kernels and one or more post-processor modules. ### Claim 6 (depends on 1) The computer-implemented method of claim 1, wherein receiving optimized target AI network configuration data comprises: removing, from the target AI network, at least one of: one or more operations, one or more kernels, one or more micro-kernels and one or more post-processor modules that are excluded from the received optimized target AI network configuration data. ### Claim 7 A system comprising: one or more processors; and a non-transitory computer readable medium storing a plurality of instructions, which when executed, cause the one or more processors to: receive a plurality of characteristics of a target artificial intelligence (AI) network; apply the plurality of characteristics of the target AI network to at least one of a static cost model and a heuristic AI network model to; and receive optimized target AI network configuration data from at least one of static cost model and the heuristic AI network model, the optimized target AI network configuration data representative of a subset of the characteristics of the target AI network that minimize a cost function of execution of the target AI network; wherein the plurality of characteristics of a target AI network comprises: a plurality of operations that correspond to respective nodes on a computational graph of the target AI network; at least one kernel for implementing one or more operations, the kernel comprised of at least one micro-kernel for formatting kernel-operation an input data stream according to a target hardware input data format compatible with underlying target hardware supporting execution of the operation; and at least one post-processor module embedded into each micro-kernel, wherein each post-processor module converts target hardware output data located in one or more target hardware registers from a first output data format native to the underlying target hardware to a second output data format that matches an input data format of a subsequent micro-kernel. ### Claim 8 (depends on 7) The system of claim 7, wherein instructions, which when executed, cause the one or more processors to apply the plurality of characteristics of the target AI network to the static cost model further comprise one or more instructions, which when executed, cause the one or more processors: simulate execution of the target AI network by iterating execution of the target AI network according to a plurality of differing kernel and post-processor module combinations with respect to at least a subset of the plurality of characteristics target AI network; and obtain time cost approximation for each simulated target AI network execution, wherein each simulated target AI network execution corresponds to a distinct set of kernel combinations. ### Claim 9 (depends on 8) The system of claim 8, wherein the instructions, which when executed, cause the one or more processors to receive optimized target AI network configuration data from the static cost model further comprise one or more instructions, which when executed, cause the one or more processors to: select a time cost approximation that corresponds with a particular simulated target AI network execution; and receive a select distinct set of kernel and post-processor module combinations utilized during the particular simulated target AI network execution associated with the selected time cost approximation. ### Claim 10 (depends on 7) The system of claim 7, wherein the instructions, which when executed, cause the one or more processors to apply the plurality of characteristics of the target AI network to the heuristic AI network model further comprise one or more instructions, which when executed, cause the one or more processors to: input at least a subset of the plurality of characteristics target AI network into a heuristic AI network trained according to heuristic network training data; and receive optimized target AI network configuration data from heuristic AI network model, wherein the receiving the optimized target AI network configuration data comprises: receiving an approximation of run time of the target AI network according to a combination of one or more optimal data formats, one or more optimal operation-calculation algorithms per kernel, one or more optimal kernel and post-processor module combinations and one or more optimal underlying hardware specifications. ### Claim 11 (depends on 10) The system of claim 10, wherein the heuristic network training data comprises at least one of: one or more underlying hardware microchip types, one or more memory hierarchies, one or more AI network structures, one or more operation hyperparameters, one or more optimal operation-calculation algorithms, one or more kernels and one or more post-processor modules. ### Claim 12 (depends on 7) The system of claim 7, wherein the instructions, which when executed, cause the one or more processors to receive optimized target AI network configuration data further comprise one or more instructions, which when executed, cause the one or more processors to: remove, from the target AI network, at least one of: one or more operations, one or more kernels, one or more micro-kernels and one or more post-processor modules that are excluded from the received optimized target AI network configuration data. ### Claim 13 A non-transitory computer-readable medium comprising instructions for inferring information about a product, the non-transitory computer-readable medium comprising instructions for: receiving a plurality of characteristics of a target artificial intelligence (AI) network; applying the plurality of characteristics of the target AI network to at least one of a static cost model and a heuristic AI network model to; and receiving optimized target AI network configuration data from at least one of static cost model and the heuristic AI network model, the optimized target AI network configuration data representative of a subset of the characteristics of the target AI network that minimize a cost function of execution of the target AI network; wherein the plurality of characteristics of a target AI network comprises: a plurality of operations that correspond to respective nodes on a computational graph of the target AI network; at least one kernel for implementing one or more operations, the kernel comprised of at least one micro-kernel for formatting kernel-operation an input data stream according to a target hardware input data format compatible with underlying target hardware supporting execution of the operation; and at least one post-processor module embedded into each micro-kernel, wherein each post-processor module converts target hardware output data located in one or more target hardware registers from a first output data format native to the underlying target hardware to a second output data format that matches an input data format of a subsequent micro-kernel. ### Claim 14 (depends on 13) The non-transitory computer-readable medium of claim 13, wherein applying the plurality of characteristics of the target AI network to the static cost model comprises: simulating execution of the target AI network by iterating execution of the target AI network according to a plurality of differing kernel and post-processor module combinations with respect to at least a subset of the plurality of characteristics target AI network; and obtaining a time cost approximation for each simulated target AI network execution, wherein each simulated target AI network execution corresponds to a distinct set of kernel combinations. ### Claim 15 (depends on 14) The non-transitory computer-readable medium of claim 14, wherein receiving optimized target AI network configuration data from the static cost model comprises: selecting a time cost approximation that corresponds with a particular simulated target AI network execution; and receiving a select distinct set of kernel and post-processor module combinations utilized during the particular simulated target AI network execution associated with the selected time cost approximation. ### Claim 16 (depends on 13) The non-transitory computer-readable medium of claim 13, wherein applying the plurality of characteristics of the target AI network to the heuristic AI network model comprises: inputting at least a subset of the plurality of characteristics target AI network into a heuristic AI network trained according to heuristic network training data; and wherein receiving optimized target AI network configuration data from heuristic AI network model comprises: receiving an approximation of run time of the target AI network according to a combination of one or more optimal data formats, one or more optimal operation-calculation algorithms per kernel, one or more optimal kernel and post-processor module combinations and one or more optimal underlying hardware specifications, wherein the heuristic network training data comprises at least one of: one or more underlying hardware microchip types, one or more memory hierarchies, one or more AI network structures, one or more operation hyperparameters, one or more optimal operation-calculation algorithms, one or more kernels and one or more post-processor modules. ### Claim 17 (depends on 13) The non-transitory computer-readable medium of claim 13, wherein receiving optimized target AI network configuration data comprises: removing, from the target AI network, at least one of: one or more operations, one or more kernels, one or more micro-kernels and one or more post-processor modules that are excluded from the received optimized target AI network configuration data.