# US 2021/0334531 A1 · Generating shift-invariant neural network feature maps and outputs

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2021/0334531 A1
- Document ID: US20210334531A1
- Status: Published application
- Kind: A1
- Application number: 17/327,088
- Filed: 2021-05-21
- Published: 2021-10-28
- Earliest claimed: 2019-01-28
- Assignees: Adobe Inc.
- Applicants: Adobe Inc.
- Inventors: Richard Zhang
- Web page: https://patentgrep.com/patent/US20210334531A1

## CPC

- G06N 3/084: Backpropagation, e.g. using gradient descent
- G06N 3/045: Combinations of networks
- G06N 3/0464: Convolutional networks [CNN, ConvNet]
- G06N 3/09: Supervised learning
- G06N 20/10: using kernel methods, e.g. support vector machines [SVM]
- G06T 5/70: Denoising; Smoothing
- G06V 10/454: {Integrating the filters into a hierarchical structure, e.g. convolutional neural networks [CNN]}
- G06V 10/82: using neural networks
- G06F 2218/02: Preprocessing
- G06F 2218/08: Feature extraction
- G06F 2218/12: Classification; Matching
- G06N 3/048: Activation functions
- G06N 3/088: Non-supervised learning, e.g. competitive learning
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)

## Abstract

The present disclosure relates to systems, methods, and non-transitory computer readable media for generating shift-resilient neural network outputs based on utilizing a dense pooling layer, a low-pass filter layer, and a downsampling layer of a neural network. For example, the disclosed systems can generate a pooled feature map utilizing a dense pooling layer to densely pool feature values extracted from an input. The disclosed systems can further apply a low-pass filter to the pooled feature map to generate a shift-adaptive feature map. In addition, the disclosed systems can downsample the shift-adaptive feature map utilizing a downsampling layer. Based on the downsampled, shift-adaptive feature map, the disclosed systems can generate shift-resilient neural network outputs such as digital image classifications.

## Claims

### Claim 1

A method comprising:
extracting a feature map utilizing a neural network;
generating a densely pooled feature map from the feature map utilizing a dense pooling layer of the neural network;
applying a low-pass filter to the densely pooled feature map to generate a shift-adaptive feature map from the densely pooled feature map; and
downsampling the shift-adaptive feature map from the low-pass filter to generate a downsampled shift-adaptive feature map.

### Claim 2 (depends on 1)

The method of claim 1, further comprising generating a shift-resilient classification from the downsampled, shift-adaptive feature map.

### Claim 3 (depends on 2)

The method of claim 2, further comprising:
extracting, utilizing the neural network, a shifted feature map comprising a shifted version of the feature map;
generating a second densely pooled feature map from the shifted feature map;
applying the low-pass filter to the second densely pooled feature map to generate a second shift-adaptive feature map;
downsampling the second shift-adaptive feature map to generate a second downsampled shift-adaptive feature map; and
generating a second shift-resilient classification from the second downsampled shift-adaptive feature map, wherein the second shift-resilient classification matches the shift-resilient classification.

### Claim 4 (depends on 1)

The method of claim 1, wherein extracting the feature map comprises utilizing one or more extraction layers of the neural network to extract feature values from a digital image.

### Claim 5 (depends on 1)

The method of claim 1, wherein downsampling the shift-adaptive feature map comprises applying a downsampling layer of the neural network to the shift-adaptive feature map to generate the downsampled shift-adaptive feature map.

### Claim 6 (depends on 5)

The method of claim 5, wherein applying the low-pass filter comprises utilizing a low-pass filter layer that is between the dense pooling layer and the downsampling layer within the neural network.

### Claim 7 (depends on 1)

The method of claim 1, wherein applying the low-pass filter removes high-frequency feature values from the densely pooled feature map before downsampling feature values of the feature map.

### Claim 8

A non-transitory computer readable medium comprising instructions that, when executed by at least one processor, cause a computing device to:
extract feature values from a digital image utilizing one or more extraction layers of a neural network;
generate a densely pooled feature map from the feature values utilizing a dense pooling layer of the neural network;
apply a low-pass filter to the densely pooled feature map to generate a shift-adaptive feature map from the densely pooled feature map;
downsample the shift-adaptive feature map to generate a downsampled shift-adaptive feature map; and
generate a shift-resilient classification of the digital image from the downsampled shift-adaptive feature map.

### Claim 9 (depends on 8)

The non-transitory computer readable medium of claim 8, further comprising instructions that, when executed by the at least one processor, cause the computing device to generate a second shift-resilient classification for a second digital image comprising a shifted version of the digital image, wherein the second shift-resilient classification matches the shift-resilient classification.

### Claim 10 (depends on 8)

The non-transitory computer readable medium of claim 8, further comprising instructions that, when executed by the at least one processor, cause the computing device to apply the low-pass filter to the densely pooled feature map by determining a weighted combination of one or more adjacent pooled feature values within the densely pooled feature map.

### Claim 11 (depends on 8)

The non-transitory computer readable medium of claim 8, further comprising instructions that, when executed by the at least one processor, cause the computing device to apply the low-pass filter by utilizing a low-pass filter layer of the neural network that is between the dense pooling layer and a downsampling layer.

### Claim 12 (depends on 8)

The non-transitory computer readable medium of claim 8, further comprising instructions that, when executed by the at least one processor, cause the computing device to generate the shift-resilient classification by determining a classification for the digital image that is resilient to shifted variations of input to the neural network.

### Claim 13 (depends on 1)

The non-transitory computer readable medium of claim 1, further comprising instructions that, when executed by the at least one processor, cause the computing device to downsample the shift-adaptive feature map by utilizing a downsampling layer of the neural network to generate the downsampled shift-adaptive feature map comprising fewer shift-adaptive feature values than the shift-adaptive feature map from the low-pass filter.

### Claim 14 (depends on 8)

The non-transitory computer readable medium of claim 8, further comprising instructions that, when executed by the at least one processor, cause the computing device to generate the densely pooled feature map by pooling the feature values such that an input spatial resolution matches an output spatial resolution associated with the dense pooling layer.

### Claim 15

A system comprising:
one or more memory devices comprising a neural network comprising a dense pooling layer, a downsampling layer, and a low-pass filter layer between the dense pooling layer and the downsampling layer;
one or more computing devices that are configured to cause the system to:
generate a densely pooled feature map utilizing the dense pooling layer of the neural network;
generate, utilizing the low-pass filter layer of the neural network, a shift-adaptive feature map from the densely pooled feature map; and
downsample, utilizing the downsampling layer of the neural network, a downsampled shift-adaptive feature map from the shift-adaptive feature map.

### Claim 16 (depends on 15)

The system of claim 15, wherein the one or more computing devices are further configured to cause the system to generate a shift-resilient classification from the downsampled, shift-adaptive feature map.

### Claim 17 (depends on 15)

The system of claim 15, wherein:
the neural network further comprises one or more extraction layers; and
the one or more computing devices are further configured to cause the system to extract a feature map utilizing the one or more extraction layers of the neural network.

### Claim 18 (depends on 17)

The system of claim 17, wherein the one or more computing devices are further configured to cause the system to generate the densely pooled feature map by utilizing the dense pooling layer to pool feature values of the feature map.

### Claim 19 (depends on 15)

The system of claim 15, wherein the one or more computing devices are further configured to cause the system to generate the shift-adaptive feature map by utilizing the low-pass filter layer of the neural network utilizing the dense pooling layer and before utilizing the downsampling layer.

### Claim 20 (depends on 15)

The system of claim 15, wherein the one or more computing devices are further configured to cause the system to generate the shift-adaptive feature map by modifying the densely pooled feature map utilizing the low-pass filter layer to adapt to shift variations of feature values within the densely pooled feature map.
