# US 2023/0410496 A1 · Omni-scale convolution for convolutional neural networks

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2023/0410496 A1
- Document ID: US20230410496A1
- Status: Published application
- Kind: A1
- Application number: 18/252,164
- Filed: 2020-12-23
- Published: 2023-12-21
- Earliest claimed: 2020-12-23
- Assignees: Intel Corporation
- Applicants: Intel Corporation
- Inventors: Anbang YAO; Bo LIU; Ming LU; Feng CHEN; Yurong CHEN
- Web page: https://patentgrep.com/patent/US20230410496A1

## CPC

- G06V 10/82: using neural networks
- G06V 10/52: Scale-space analysis, e.g. wavelet analysis (multi-scale boundary representations G06V10/42)
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING

## Abstract

Omni-scale convolution for convolutional neural networks is disclosed. An example of an apparatus includes one or more processors to process data, including processing for a convolutional neural network (CNN); and a memory to store data, including CNN data, wherein processing of input data by the CNN includes implementing omni-scale convolution in one or more convolutional layers of the CNN, implementation of the omni-scale convolution into a convolutional layer of the one or more convolutional layers including at least applying multiple dilation rates in a plurality of kernels of a kernel lattice of the convolutional layer, and applying a cyclic pattern for the multiple dilation rates in the plurality of kernels of the convolutional layer.

## Claims

### Claim 1

An apparatus comprising:
one or more processors to process data, including processing for a convolutional neural network (CNN); and
a memory to store data, including data for CNN processing;
wherein processing of input data by the CNN includes the one or more processors implementing omni-scale convolution in one or more convolutional layers of the CNN, implementation of the omni-scale convolution into a convolutional layer of the one or more convolutional layers including at least:
applying a plurality of dilation rates in a plurality of kernels of a kernel lattice of the convolutional layer, and
applying a cyclic pattern for the plurality of dilation rates in the plurality of kernels of the convolutional layer.

### Claim 2 (depends on 1)

The apparatus of claim 1, wherein the omni-scale convolution is to mix multi-scale information in two orthogonal dimensions of the kernel lattice for the convolutional layer.

### Claim 3 (depends on 2)

The apparatus of claim 2, wherein mixing multi-scale information in two orthogonal dimensions includes the dilation rates of the plurality of kernels alternating along both an input channel and an output channel in the cyclic pattern.

### Claim 4 (depends on 2)

The apparatus of claim 2, wherein the implementation of the omni-scale convolution provides a combination of:
a cyclic operation in which dilation rates for the plurality of kernels vary in a periodic manner along an axis of input channels; and
a shift operation in which dilation rates for the plurality of kernels are shifted along an axis of output channels.

### Claim 5 (depends on 1)

The apparatus of claim 1, wherein applying the plurality of dilation rates in the plurality of kernels includes implementing the dilation rates in group convolution.

### Claim 6 (depends on 1)

The apparatus of claim 1, wherein the one or more processors are further to generate an output based at least in part on the omni-scale convolution in one or more convolutional layers of the CNN.

### Claim 7 (depends on 1)

The apparatus of claim 1, wherein the omni-scale convolution is implemented in multiple convolutional layers of the CNN.

### Claim 8 (depends on 1)

The apparatus of claim 1, wherein the omni-scale convolution is incorporated in an existing CNN structure.

### Claim 9

At least one non-transitory machine readable storage medium comprising instructions that, when executed, cause at least one processor to perform operations including:
implementing a convolution operation in one or more convolutional layers of a convolutional neural network (CNN), including at least:
applying a plurality of dilation rates in a plurality of kernels of a kernel lattice of a convolutional layer of the one or more convolutional layers, and
applying a cyclic pattern for the plurality of dilation rates in the plurality of kernels of the convolutional layer;
receiving a set of input data for processing by the CNN; and
utilizing the CNN to generate an output, including applying the plurality of dilation rates according to the cyclic pattern in the one or more convolutional layers.

### Claim 10 (depends on 9)

The at least one non-transitory machine readable storage medium of claim 9, further comprising instructions to perform operations including:
mixing multi-scale information in two orthogonal dimensions of the kernel lattice for the convolutional layer.

### Claim 11 (depends on 10)

The at least one non-transitory machine readable storage medium of claim 10, wherein mixing multi-scale information in two orthogonal dimensions includes the dilation rates of the plurality of kernels alternating along both an input channel and an output channel in the cyclic pattern.

### Claim 12 (depends on 10)

The at least one non-transitory machine readable storage medium of claim 10, wherein the implementation of the convolution operation includes a combination of:
a cyclic operation in which dilation rates for the plurality of kernels vary in a periodic manner along an axis of input channels; and
a shift operation in which dilation rates for the plurality of kernels are shifted along an axis of output channels.

### Claim 13 (depends on 9)

The at least one non-transitory machine readable storage medium of claim 9, wherein applying the plurality of dilation rates in the plurality of kernels includes implementing the dilation rates in group convolution.

### Claim 14 (depends on 9)

The at least one non-transitory machine readable storage medium of claim 9, wherein the implementation of convolution is provided in multiple convolutional layers of the CNN.

### Claim 15

A system comprising:
one or more processors to process data, including computer vision processing utilizing a convolutional neural network (CNN), the CNN including one or more convolutional layers;
a memory to store data, including data for CNN processing; and
an omni-convolution tool to provide support for objection recognition by the CNN in varying scales of object sizes;
wherein application of the omni-convolution tool includes at least:
applying a plurality of dilation rates in a plurality of kernels of a kernel lattice of a convolution layer of the one or more convolutional layers of the CNN, and
applying a cyclic pattern for the plurality of dilation rates in the plurality of kernels of the convolutional layer.

### Claim 16 (depends on 15)

The system of claim 15, wherein application of the omni-scale convolution tool is to mix multi-scale information in two orthogonal dimensions of the kernel lattice for the convolutional layer.

### Claim 17 (depends on 16)

The system of claim 16, wherein mixing multi-scale information in two orthogonal dimensions includes the dilation rates of the plurality of kernels alternating along both an input channel and an output channel in the cyclic pattern.

### Claim 18 (depends on 16)

The system of claim 16, wherein application of the omni-scale convolution tool provides a combination of:
a cyclic operation in which dilation rates for the plurality of kernels vary in a periodic manner along an axis of input channels; and
a shift operation in which dilation rates for the plurality of kernels are shifted along an axis of output channels.

### Claim 19 (depends on 16)

The system of claim 16, wherein applying the plurality of dilation rates in the plurality of kernels includes applying the dilation rates in group convolution.

### Claim 20 (depends on 16)

The system of claim 16, wherein the system is further to generate an output based at least in part on the application of the omni-convolution tool in one or more convolutional layers of the CNN.
