# US 10,769,749 B2 · Processor, information processing apparatus, and operation method of processor > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 10,769,749 B2 - Document ID: US10769749B2 - Status: Granted - Kind: B2 - Application number: 16/288,185 - Filed: 2019-02-28 - Published: 2019-06-27 - Granted: 2020-09-08 - Earliest claimed: 2017-01-27 - Assignees: Fujitsu Limited - Applicants: Fujitsu Limited - Inventors: Katsuhiro Yoda; Mitsuru Tomono; Takahiro Notsu - Web page: https://patentgrep.com/patent/US10769749B2 ## CPC - G06T 1/20: Processor architectures; Processor configuration, e.g. pipelining - G06F 9/3001: {Arithmetic instructions} - G06F 15/80: comprising an array of processing units with common control, e.g. single instruction multiple data processors (G06F15/82 takes precedence {; for correlation function computation G06F17/15}) - G06F 17/153: {Multidimensional correlation or convolution} - G06F 17/16: Matrix or vector computation {, e.g. matrix-matrix or matrix-vector multiplication, matrix factorization (matrix transposition G06F7/78)} - G06N 3/045: Combinations of networks - G06N 3/0464: Convolutional networks [CNN, ConvNet] - G06N 3/063: using electronic means - G06N 3/08: Learning methods - G06N 3/09: Supervised learning - G06T 1/60: Memory management - G06T 3/4023: based on decimating pixels or lines of pixels; based on inserting pixels or lines of pixels - G06T 5/20: using local operators - G09G 5/04: using circuits for interfacing with colour displays - G06T 2207/20024: Filtering details - G06T 2207/20081: Training; Learning - G06T 2207/20084: Artificial neural networks [ANN] - G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) - G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS - G09G: ARRANGEMENTS OR CIRCUITS FOR CONTROL OF INDICATING DEVICES USING STATIC MEANS TO PRESENT VARIABLE INFORMATION (arrangements for transferring data between digital computers and displays G06F3/14; static indicating arrangements comprising an association of a number of separate sources or light control cells G09F9/00; static indicating arrangements comprising integral associations of a number of light sources H01J, H01K, H10H, H05B33/12; scanning, transmission or reproduction of documents or the like, e.g. facsimile transmission, details thereof H04N1/00) ## Abstract A processor includes: a first memory configured to store image data including pixel data of a plurality of pixels that are two-dimensionally arranged; a second memory configured to store neighborhood matrix image data including pixel data of a neighborhood matrix; and a format converter that includes (a) a readout circuit configured to read out the image data from the first memory, (b) a padding arithmetic unit configured to receive the read-out image data, select pixel data of the received read-out image data and padding data inserted at periphery of the plurality of pixels in accordance with mask values of a padding mask, and generate the neighborhood matrix image data including the pixel data and the padding data, and (c) a writing circuit configured to write the neighborhood matrix image data to the second memory. ## Claims ### Claim 1 A processor comprising: a first memory configured to store image data including pixel data of a plurality of pixels that are two-dimensionally arranged; a second memory configured to store neighborhood matrix image data including pixel data of a neighborhood matrix; and a format converter that includes a readout circuit configured to read out the image data from the first memory, a padding arithmetic unit configured to receive the read-out image data, select pixel data of the received read-out image data and padding data inserted at periphery of the plurality of pixels in accordance with mask values of a padding mask, and generate the neighborhood matrix image data including the pixel data and the padding data, and a writing circuit configured to write the neighborhood matrix image data to the second memory. ### Claim 2 (depends on 1) The processor according to claim 1, wherein the format converter further includes a compression arithmetic unit, the compression arithmetic unit being configured to: receive the read-out image data; select pixel data included in the neighborhood matrix image data from a plurality of pieces of pixel data of the received read-out image data to generate compressed image data in accordance with mask values of a compression mask, and the padding arithmetic unit is configured to receive the compressed image data as the read-out image data, and output the neighborhood matrix image data. ### Claim 3 (depends on 2) The processor according to claim 2, wherein the format converter further includes a compression mask register configured to store the compression mask, a padding mask register configured to store the padding mask, and a controller configured to set the compression mask in the compression mask register and set the padding mask in the padding mask register, and the controller is further configured to change the mask values of each of the compression mask and the padding mask in accordance with a position of a matrix of the image data of an attention-paid pixel of the neighborhood matrix. ### Claim 4 (depends on 2) The processor according to claim 2, wherein the format converter includes a plurality of sets, each of the plurality of sets including the compression arithmetic unit and the padding arithmetic unit, the each of the plurality of sets being configured to generate a plurality of sets of neighborhood matrix image data in parallel. ### Claim 5 (depends on 1) The processor according to claim 1, wherein the format converter further includes a connecting circuit configured to connect a plurality of sets of neighborhood matrix image data sequentially output by the padding arithmetic unit, and a transposition circuit configured to transpose the pixel data and the padding data included in each of the connected plurality of sets of neighborhood matrix image data, and generate neighborhood matrix image data for input including a data format corresponding to an input configuration of an arithmetic unit. ### Claim 6 (depends on 1) The processor according to claim 1, wherein the format converter further includes a connecting circuit, the connecting circuit being configured to dispose a plurality of sets of neighborhood matrix image data sequentially output by the padding arithmetic unit into a matrix-manner data format with a smaller number of columns than the number of pixels of the neighborhood matrix in a raster-scan direction without vacancy and generate neighborhood matrix image data for input, and the writing circuit is configured to write the neighborhood matrix image data for input to the second memory. ### Claim 7 (depends on 1) The processor according to claim 1, wherein the format converter further includes a padding mask register configured to store the padding mask, and a controller configured to set the padding mask in the padding mask register, and the controller is further configured to change the mask values of the padding mask in accordance with a position of a matrix of the image data of an attention-paid pixel of the neighborhood matrix. ### Claim 8 (depends on 1) The processor according to claim 1, wherein the format converter includes a plurality of sets of the padding arithmetic units, the plurality of sets of the padding arithmetic units being configured to generate a plurality of sets of neighborhood matrix image data in parallel. ### Claim 9 (depends on 1) The processor according to claim 1, further comprising: a sum-of-product arithmetic unit configured to receive the neighborhood matrix image data, multiply each pixel of a plurality of pieces of pixel data in the neighborhood matrix image data by a corresponding one of coefficients in a coefficient filter, and add a multiplication result of the each pixel in the neighborhood matrix image data. ### Claim 10 An information processing apparatus comprising: a processor; and a main memory configured to be accessed by the processor, wherein the processor includes a first memory configured to store image data including pixel data of a plurality of pixels that are two-dimensionally arranged, a second memory configured to store neighborhood matrix image data including pixel data of a neighborhood matrix, and a format converter that includes a readout circuit configured to read out the image data from the first memory, a padding arithmetic unit configured to receive the read-out image data, select pixel data of the received read-out image data and padding data inserted at periphery of the plurality of pixels in accordance with mask values of a padding mask, and generate the neighborhood matrix image data including the pixel data and the padding data, and a writing circuit configured to e the neighborhood matrix image data to the second memory. ### Claim 11 An operation method performed by a processor including a first memory, a second memory, and a format converter, the first memory being configured to store image data including pixel data of a plurality of pixels that are two-dimensionally arranged, the second memory being configured to store neighborhood matrix image data including pixel data of a neighborhood matrix, the format converter being configured to generate the neighborhood matrix image data, the operation method comprising: reading out the image data from the first memory, receiving the read-out image data; selecting pixel data of the received read-out image data and padding data inserted at periphery of the plurality of pixels in accordance with mask values of a padding mask; generating the neighborhood matrix image data including the pixel data and the padding data; and writing the neighborhood matrix image data to the second memory.