# US 2019/0378239 A1 · Architecture for high performance, power efficient, programmable image processing > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 2019/0378239 A1 - Document ID: US20190378239A1 - Status: Published application - Kind: A1 - Application number: 16/547,801 - Filed: 2019-08-22 - Published: 2019-12-12 - Earliest claimed: 2015-04-23 - Assignees: Google Inc. - Applicants: Google LLC - Inventors: Qiuling Zhu; Ofer Shacham; Albert Meixner; Jason Rupert Redgrave; Daniel Frederic Finchelstein; David Patterson; Neeti Desai; Donald Stark; Edward Chang; William R. Mark - Web page: https://patentgrep.com/patent/US20190378239A1 ## CPC - G06T 1/20: Processor architectures; Processor configuration, e.g. pipelining - G06T 1/60: Memory management - H04N 5/91: Television signal processing therefor - Y02D 10/00: Energy efficient computing, e.g. low power processors, power management or thermal management - G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL - H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION - Y02D: CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE ## Abstract An apparatus is described. The apparatus includes an image processing unit. The image processing unit includes a plurality of stencil processor circuits each comprising an array of execution unit lanes coupled to a two-dimensional shift register array structure to simultaneously process multiple overlapping stencils through execution of program code. The image processing unit includes a plurality of sheet generators respectively coupled between the plurality of stencil processors and the network. The sheet generators are to parse input line groups of image data into input sheets of image data for processing by the stencil processors, and, to form output line groups of image data from output sheets of image data received from the stencil processors. The image processing unit includes a plurality of line buffer units coupled to the network to pass line groups in a direction from producing stencil processors to consuming stencil processors to implement an overall program flow. ## Claims ### Claim 1 A processor comprising a two-dimensional shift-register array and a two-dimensional array of processing elements, wherein each shift register in the two-dimensional shift-register array is dedicated to a respective processing element in the two-dimensional array of processing elements, wherein each processing element is configured to execute instructions of a kernel program that causes the two-dimensional shift-register array to shift data over a two-dimensional pattern and causes each processing element to compute a respective output value using multiple respective input values read from a respective shift register dedicated to the processing element while shifting the data over the two-dimensional pattern. ### Claim 2 (depends on 1) The processor of claim 1, further comprising a sheet generator configured to receive a sheet of output values computed by the two-dimensional array of processing elements and to provide the sheet of output values to a line buffer. ### Claim 3 (depends on 2) The processor of claim 2, wherein the line buffer is configured to receive the sheet of output values and to provide the sheet of output values to one or more consumer processors. ### Claim 4 (depends on 2) The processor of claim 2, wherein the sheet generator is configured to receive a line group from a producer line buffer and to load a subsequent sheet of input values into the two-dimensional shift-register array of the processor. ### Claim 5 (depends on 4) The processor of claim 4, wherein the processor, the sheet generator, and the line buffer are configured to implement a portion of an image processing pipeline. ### Claim 6 (depends on 4) The processor of claim 4, wherein the sheet generator is configured to write sheets of output values to multiple line buffers. ### Claim 7 (depends on 1) The processor of claim 1, wherein each shift register of the two-dimensional shift-register array is dedicated to a respective processing element in the two-dimensional array of processing elements. ### Claim 8 (depends on 1) The processor of claim 1, wherein each shift of data from a first shift register to a second shift register in the two-dimensional shift-register array replaces a value previously stored in the second shift register. ### Claim 9 (depends on 1) The processor of claim 1, wherein the two-dimensional shift-register array maintains a single copy of the data while shifting data over the two-dimensional pattern. ### Claim 10 (depends on 1) The processor of claim 1, wherein while shifting data over the two-dimensional pattern each processing element reads the multiple respective input values from a same respective shift register dedicated to the processing element. ### Claim 11 A method performed by a processor comprising a two-dimensional shift-register array and a two-dimensional array of processing elements, wherein each shift register in the two-dimensional shift-register array is dedicated to a respective processing element in the two-dimensional array of processing elements, the method comprising: executing, by each processing element in the two-dimensional array of processing elements, instructions of a kernel program that causes the two-dimensional shift-register array to shift data over a two-dimensional pattern; and computing, by each processing element in the two-dimensional array of processing elements, a respective output value using multiple respective input values read from a respective shift register dedicated to the processing element while shifting the data over the two-dimensional pattern. ### Claim 12 (depends on 11) The method of claim 11, further comprising: receiving, by a sheet generator, a sheet of output values computed by the two-dimensional array of processing elements; and providing the sheet of output values to a line buffer. ### Claim 13 (depends on 12) The method of claim 12, further comprising: receiving, by the line buffer, the sheet of output values; and providing the sheet of output values to one or more consumer processors. ### Claim 14 (depends on 12) The method of claim 12, further comprising: receiving, by the sheet generator, a line group from a producer line buffer; and loading, by the sheet generator, a subsequent sheet of input values into the two-dimensional shift-register array of the processor. ### Claim 15 (depends on 14) The method of claim 14, wherein the processor, the sheet generator, and the line buffer implement a portion of an image processing pipeline. ### Claim 16 (depends on 14) The method of claim 14, further comprising writing, by the sheet generator, sheets of output values to multiple line buffers. ### Claim 17 (depends on 11) The method of claim 11, wherein each shift register of the two-dimensional shift-register array is dedicated to a respective processing element in the two-dimensional array of processing elements. ### Claim 18 (depends on 11) The method of claim 11, wherein shifting data from a first shift register to a second shift register in the two-dimensional shift-register array comprises replacing a value previously stored in the second shift register. ### Claim 19 (depends on 11) The method of claim 11, wherein further comprising maintaining, by the two-dimensional shift-register array, a single copy of the data while shifting data over the two-dimensional pattern. ### Claim 20 (depends on 11) The method of claim 11, further comprising: while shifting data over the two-dimensional pattern, reading, by each processing element, the multiple respective input values from a same respective shift register dedicated to the processing element.