# US 2022/0191524 A1 · Caching and Clearing Mechanism for Deep Convolutional Neural Networks

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2022/0191524 A1
- Document ID: US20220191524A1
- Status: Published application
- Kind: A1
- Application number: 17/549,039
- Filed: 2021-12-13
- Published: 2022-06-16
- Earliest claimed: 2020-12-14
- Assignees: Nokia Technologies Oy
- Applicants: Nokia Technologies Oy
- Inventors: Honglei ZHANG; Francesco Cricri; Hamed Rezazadegan Tavakoli; Jani Lainema; Emre Aksu; Nannan Zou
- Web page: https://patentgrep.com/patent/US20220191524A1

## CPC

- H04N 19/423: characterised by memory arrangements (H04N19/433 takes precedence)
- G06F 12/0875: with dedicated cache, e.g. instruction or stack
- G06F 12/126: with special data handling, e.g. priority of data or instructions, handling errors or pinning
- G06N 3/0455: Auto-encoder networks; Encoder-decoder networks
- G06N 3/0464: Convolutional networks [CNN, ConvNet]
- G06N 3/063: using electronic means
- H04N 19/107: between spatial and temporal predictive coding, e.g. picture refresh
- H04N 19/119: Adaptive subdivision aspects, e.g. subdivision of a picture into rectangular or non-rectangular coding blocks
- H04N 19/176: the region being a block, e.g. a macroblock
- G06F 2212/1016: Performance improvement
- G06F 2212/1044: Space efficiency improvement
- G06F 2212/302: In image processor or graphics adapter
- G06F 2212/454: Vector or matrix data
- G06F 2212/455: Image or video data
- G06N 3/08: Learning methods
- H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS

## Abstract

An apparatus includes circuitry configured to: partition an input tensor into one or more block tensors; partition at least one of the block tensors into one or more continuation bands, the one or more continuation bands being associated with a caching counter having a value; store the one or more continuation bands in a cache managed using a cache manager; retrieve, prior to a convolution or pooling operation on a current block tensor, the one or more continuation bands of a previous block tensor from the cache that are adjacent to a current block tensor; concatenate the retrieved continuation bands with the current block tensor; apply the convolution or pooling operation on the current block tensor after the concatenation; decrease the respective caching counter value of the retrieved continuation bands; and clear the continuation bands from the cache when its respective caching counter reaches a value of zero.

## Claims

### Claim 1

An apparatus comprising:
at least one processor; and
at least one non-transitory memory including computer program code;
wherein the at least one memory and the computer program code are configured to, with the at least one processor, cause the apparatus at least to perform:
partition an input tensor into one or more block tensors;
partition at least one of the block tensors into one or more continuation bands, the one or more continuation bands being associated with a caching counter having a value;
store the one or more continuation bands in a cache managed using a cache manager;
retrieve, prior to a convolution or pooling operation on a current block tensor, the one or more continuation bands of a previous block tensor from the cache that are adjacent to a current block tensor;
concatenate the retrieved continuation bands with the current block tensor;
apply the convolution or pooling operation on the current block tensor after the concatenation;
decrease the respective caching counter value of the retrieved one or more continuation bands; and
clear the one or more continuation bands from the cache when its respective caching counter reaches a value of zero.

### Claim 2 (depends on 1)

The apparatus of claim 1, wherein the at least one memory and the computer program code are further configured to, with the at least one processor, cause the apparatus at least to perform:
determine an initial value of the one or more caching counters based on a type of continuation band associated with the caching counter and a position of the block tensor associated with the continuation band.

### Claim 3 (depends on 1)

The apparatus of claim 1, wherein the at least one memory and the computer program code are further configured to, with the at least one processor, cause the apparatus at least to perform:
determine a size of the block tensors using the memory.

### Claim 4 (depends on 1)

The apparatus of claim 1, wherein the at least one memory and the computer program code are further configured to, with the at least one processor, cause the apparatus at least to perform:
process the block tensors in raster scan order.

### Claim 5 (depends on 1)

The apparatus of claim 1, where the memory is a memory of a graphics processing unit or a tensor processing unit.

### Claim 6 (depends on 5)

The apparatus of claim 5, where storing the one or more continuation bands in the cache comprises transferring the one or more continuation bands from a main memory to the memory of the graphics processing unit or the tensor processing unit using the cache manager.

### Claim 7 (depends on 1)

The apparatus of claim 1, where a size of the one or more continuation bands is determined based at least on a position width and a position height, and the position width is either positive or negative, and the position height is either positive or negative.

### Claim 8 (depends on 7)

The apparatus of claim 7, where the position width is a function of a width of an associated block tensor and a stride value of a convolution kernel, and the position height is a function of a height of the associated block tensor and the stride value of the convolution kernel.

### Claim 9 (depends on 7)

The apparatus of claim 7, where when the size of a continuation band is negative, a corresponding part of the current block tensor is removed prior to applying the convolution operation, instead of concatenating the retrieved one or more continuation bands with the current block tensor.

### Claim 10 (depends on 1)

The apparatus of claim 1, where the one or more continuation bands are one of three types, where a first type comprises a right side region of a block tensor, a second type comprises a bottom side region of the block tensor, and a third type comprises an overlapping area of the first region and the second region.

### Claim 11 (depends on 1)

The apparatus of claim 1, wherein the at least one memory and the computer program code are further configured to, with the at least one processor, cause the apparatus at least to perform:
pad the input tensor prior to partitioning the input tensor into one or more block tensors.

### Claim 12 (depends on 1)

The apparatus of claim 1, wherein the at least one memory and the computer program code are further configured to, with the at least one processor, cause the apparatus at least to perform:
apply padding to a block tensor depending at least on a position of the block tensor in the input tensor when the block tensor is processed during the convolution operation.

### Claim 13 (depends on 1)

The apparatus of claim 1, where the convolution or pooling operation is a neural network operation.

### Claim 14

A method comprising:
partitioning an input tensor into one or more block tensors;
partitioning at least one of the block tensors into one or more continuation bands, the one or more continuation bands being associated with a caching counter having a value;
storing the one or more continuation bands in a cache managed using a cache manager;
retrieving, prior to a convolution or pooling operation on a current block tensor, the one or more continuation bands of a previous block tensor from the cache that are adjacent to a current block tensor;
concatenating the retrieved continuation bands with the current block tensor;
applying the convolution or pooling operation on the current block tensor after the concatenation;
decreasing the respective caching counter value of the retrieved one or more continuation bands; and
clearing the one or more continuation bands from the cache when its respective caching counter reaches a value of zero.

### Claim 15 (depends on 14)

The method of claim 14, further comprising:
determining an initial value of the one or more caching counters based on a type of continuation band associated with the caching counter and a position of the block tensor associated with the continuation band.

### Claim 16 (depends on 14)

The method of claim 14, further comprising:
determining a size of the block tensors using a memory.

### Claim 17 (depends on 14)

The method of claim 14, further comprising:
processing the block tensors in raster scan order.

### Claim 18

A non-transitory program storage device readable by a machine, tangibly embodying a program of instructions executable by the machine for performing operations, the operations comprising:
partitioning an input tensor into one or more block tensors;
partitioning at least one of the block tensors into one or more continuation bands, the one or more continuation bands being associated with a caching counter having a value;
storing the one or more continuation bands in a cache managed using a cache manager;
retrieving, prior to a convolution or pooling operation on a current block tensor, the one or more continuation bands of a previous block tensor from the cache that are adjacent to a current block tensor;
concatenating the retrieved continuation bands with the current block tensor;
applying the convolution or pooling operation on the current block tensor after the concatenation;
decreasing the respective caching counter value of the retrieved one or more continuation bands; and
clearing the one or more continuation bands from the cache when its respective caching counter reaches a value of zero.

### Claim 19 (depends on 18)

The non-transitory program storage device of claim 18, the operations further comprising:
determining an initial value of the one or more caching counters based on a type of continuation band associated with the caching counter and a position of the block tensor associated with the continuation band.

### Claim 20 (depends on 18)

The non-transitory program storage device of claim 18, the operations further comprising:
determining a size of the block tensors using a memory.
