# US 11,030,005 B2 · Configuration of application software on multi-core image processor > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 11,030,005 B2 - Document ID: US11030005B2 - Status: Granted - Kind: B2 - Application number: 16/657,656 - Filed: 2019-10-18 - Published: 2020-02-13 - Granted: 2021-06-08 - Earliest claimed: 2017-05-12 - Assignees: Google Inc. - Applicants: Google LLC - Inventors: Hyunchul Park; Albert Meixner - Web page: https://patentgrep.com/patent/US11030005B2 ## CPC - G06F 9/5005: {to service a request} - G06F 9/505: {considering the load} - G06T 1/20: Processor architectures; Processor configuration, e.g. pipelining - Y02D 10/00: Energy efficient computing, e.g. low power processors, power management or thermal management - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) - G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL - Y02D: CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE ## Abstract A method is described. The method includes calculating data transfer metrics for kernel-to-kernel connections of a program having a plurality of kernels that is to execute on an image processor. The image processor includes a plurality of processing cores and a network connecting the plurality of processing cores. Each of the kernel-to-kernel connections include a producing kernel that is to execute on one of the processing cores and a consuming kernel that is to execute on another one of the processing cores. The consuming kernel is to operate on data generated by the producing kernel. The method also includes assigning kernels of the plurality of kernels to respective ones of the processing cores based on the calculated data transfer metrics. ## Claims ### Claim 1 A method performed by one or more computers, the method comprising: receiving a request to compute kernel assignments for an image processing pipeline to be executed on a device having a plurality of stencil processors, wherein the image processing pipeline comprises a plurality of kernels; generating a plurality of candidate kernel assignments, each candidate kernel assignment assigning each kernel of the image processing pipeline to a respective stencil processor of the plurality of stencil processors; computing a respective total weight for each of the plurality of candidate kernel assignments, the total weight for each candidate kernel assignment being based on respective transfer sizes of data transferred between kernels according to the candidate kernel assignment; selecting a candidate kernel assignment according to the respective total weights computed for each of the plurality of candidate kernel assignments; and assigning kernels of the plurality of kernels to respective stencil processors according to the selected candidate kernel assignment. ### Claim 2 (depends on 1) The method of claim 1, wherein the device comprises a plurality of line buffer units, and further comprising assigning one or more line buffer units to be a respective source of one or more kernels. ### Claim 3 (depends on 2) The method of claim 2, further comprising assigning one or more line buffer units to be a respective sink of one or more kernels. ### Claim 4 (depends on 3) The method of claim 3, wherein assigning the one or more line buffer units to be a respective sink of one or more kernels comprises: generating, for a particular producing kernel assigned to a particular stencil processor, a list of line buffer units sorted by transfer distances from the particular stencil processor; and assigning, to the particular producing kernel, a closest line buffer unit having enough memory to buffer data generated by the particular producing kernel. ### Claim 5 (depends on 4) The method of claim 4, wherein the transfer distances are based on a respective number of nodal hops within the network between kernels. ### Claim 6 (depends on 4) The method of claim 4, wherein the transfer distances are based on distances along a network ring of the network. ### Claim 7 (depends on 1) The computing system of claim 1, wherein each stencil processor comprises an execution lane array and a two-dimensional shift-register array. ### Claim 8 One or more non-transitory computer storage media encoded with computer program instructions that when executed by one or more computers cause the one or more computers to perform operations comprising: receiving a request to compute kernel assignments for an image processing pipeline to be executed on a device having a plurality of stencil processors, wherein the image processing pipeline comprises a plurality of kernels; generating a plurality of candidate kernel assignments, each candidate kernel assignment assigning each kernel of the image processing pipeline to a respective stencil processor of the plurality of stencil processors; computing a total weight for each of the plurality of candidate kernel assignments, the total weight for each candidate kernel assignment being based on respective transfer sizes of data transferred between kernels according to the candidate kernel assignment; selecting a candidate kernel assignment according to the respective total weights computed for each of the plurality of candidate kernel assignments; and assigning kernels of the plurality of kernels to respective stencil processors according to the selected candidate kernel assignment. ### Claim 9 (depends on 8) The one or more non-transitory computer storage media of claim 8, wherein the device comprises a plurality of line buffer units, and wherein the operations further comprise assigning one or more line buffer units to be a respective source of one or more kernels. ### Claim 10 (depends on 9) The one or more non-transitory computer storage media of claim 9, wherein the operations further comprise assigning one or more line buffer units to be a respective sink of one or more kernels. ### Claim 11 (depends on 10) The one or more non-transitory computer storage media of claim 10, wherein assigning the one or more line buffer units to be a respective sink of one or more kernels comprises: generating, for a particular producing kernel assigned to a particular stencil processor, a list of line buffer units sorted by transfer distances from the particular stencil processor; and assigning, to the particular producing kernel, a closest line buffer unit having enough memory to buffer data generated by the particular producing kernel. ### Claim 12 (depends on 11) The one or more non-transitory computer storage media of claim 11, wherein the transfer distances are based on a respective number of nodal hops within the network between kernels. ### Claim 13 (depends on 11) The one or more non-transitory computer storage media of claim 11, wherein the transfer distances are based on distances along a network ring of the network. ### Claim 14 (depends on 8) The one or more non-transitory computer storage media of claim 8, wherein each stencil processor comprises an execution lane array and a two-dimensional shift-register array. ### Claim 15 A system, comprising: one or more computers and one or more storage devices storing instructions that are operable, when executed by the one or more computers, to cause the one or more computers to perform operations comprising: receiving a request to compute kernel assignments for an image processing pipeline to be executed on a device having a plurality of stencil processors, wherein the image processing pipeline comprises a plurality of kernels; generating a plurality of candidate kernel assignments, each candidate kernel assignment assigning each kernel of the image processing pipeline to a respective stencil processor of the plurality of stencil processors; computing a total weight for each of the plurality of candidate kernel assignments, the total weight for each candidate kernel assignment being based on respective transfer sizes of data transferred between kernels according to the candidate kernel assignment; selecting a candidate kernel assignment according to the respective total weights computed for each of the plurality of candidate kernel assignments; and assigning kernels of the plurality of kernels to respective stencil processors according to the selected candidate kernel assignment. ### Claim 16 (depends on 15) The system of claim 15, wherein the device comprises a plurality of line buffer units, and wherein the operations further comprise assigning one or more line buffer units to be a respective source of one or more kernels. ### Claim 17 (depends on 16) The system of claim 16, wherein the operations further comprise assigning one or more line buffer units to be a respective sink of one or more kernels. ### Claim 18 (depends on 17) The system of claim 17, wherein assigning the one or more line buffer units to be a respective sink of one or more kernels comprises: generating, for a particular producing kernel assigned to a particular stencil processor, a list of line buffer units sorted by transfer distances from the particular stencil processor; and assigning, to the particular producing kernel, a closest line buffer unit having enough memory to buffer data generated by the particular producing kernel. ### Claim 19 (depends on 18) The system of claim 18, wherein the transfer distances are based on a respective number of nodal hops within the network between kernels. ### Claim 20 (depends on 18) The system of claim 18, wherein the transfer distances are based on distances along a network ring of the network.