# US 2020/0401847 A1 · Calculation method using pixel-channel shuffle convolutional neural network and operating system using the same

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2020/0401847 A1
- Document ID: US20200401847A1
- Status: Published application
- Kind: A1
- Application number: 16/790,979
- Filed: 2020-02-14
- Published: 2020-12-24
- Earliest claimed: 2019-06-24
- Assignees: Realtek Semiconductor Corp.
- Applicants: Realtek Semiconductor Corp.
- Inventors: CHUN-CHANG WU; SHIH-TSE CHEN
- Web page: https://patentgrep.com/patent/US20200401847A1

## CPC

- G06V 10/7715: {Feature extraction, e.g. by transforming the feature space, e.g. multi-dimensional scaling [MDS]; Mappings, e.g. subspace methods}
- G06F 18/2148: {characterised by the process organisation or structure, e.g. boosting cascade}
- G06N 3/04: Architecture, e.g. interconnection topology
- G06N 3/045: Combinations of networks
- G06N 3/0464: Convolutional networks [CNN, ConvNet]
- G06N 3/08: Learning methods
- G06V 10/20: Image preprocessing
- G06V 10/7747: {Organisation of the process, e.g. bagging or boosting}
- G06V 10/82: using neural networks
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS

## Abstract

A calculation method using pixel-channel shuffle convolutional neural network is provided. In the method, an operating system receives original input data. The original input data is pre-processed by a pixel shuffle process to be separated into multiple groups in order to minimize dimension of the data. The multiple groups of data are then processed by a channel shuffle process so as to form multiple groups of new input data selected for convolution operation. The unselected data are abandoned. Therefore, the dimension of the input data can be much effectively minimized. A multiplier-accumulator of the operating system is used to execute convolution operation using a convolution kernel and the multiple new groups of input data. Multiple output data are then produced.

## Claims

### Claim 1

A calculation method using pixel-channel shuffle convolutional neural network, comprising:
in an operating system, receiving an original input data that is a value with a length, a width and a depth;
by a processor of the operating system, performing a pixel shuffle operation on the original input data, in which the original input data is separated into multiple values for reducing a dimension of each of the values;
by the processor, performing a channel shuffle operation on the multiple values for selecting values used in a convolution operation from each of the multiple values so as to form multiple new input data;
setting up convolution kernels with respect to the multiple new input data, wherein a second amount of convolution kernels are included, and each of the convolution kernels implements a filter; and
by a multiplier-accumulator of the processor, performing the convolution operation with the second amount of convolution kernels and the multiple new input data, so as to form a second amount of multiple output data.

### Claim 2 (depends on 1)

The method of claim 1, wherein the original input data is an image data, and the operating system performs the convolution operation for extracting image features from the original input data so as to form a plurality of feature maps with a second amount of depth.

### Claim 3 (depends on 2)

The method of claim 2, wherein the second amount of output data are calculated by an inverse pixel shuffle operation so as to form an image feature map with the second amount of depth.

### Claim 4 (depends on 1)

The method of claim 1, wherein, after forming the new input data by the channel shuffle operation, unselected values of each of the multiple values that are not in the convolution operation are abandoned.

### Claim 5 (depends on 1)

The method of claim 1, wherein the original input data is a value with a first amount of depth, and the original input data is processed by the pixel shuffle and the channel shuffle operations so as to form the plurality of input data with the depth smaller than the first amount of depth.

### Claim 6 (depends on 5)

The method of claim 5, wherein the original input data is an image data, and the operating system performs the convolution operation for extracting image features from the original input data so as to form a plurality of feature maps with the second amount of depth.

### Claim 7 (depends on 6)

The method of claim 6, wherein the second amount of output data are calculated by an inverse pixel shuffle operation so as to form an image feature map with the second amount of depth.

### Claim 8 (depends on 7)

The method of claim 7, wherein the image feature map is used to recognize the original input data.

### Claim 9 (depends on 1)

The method of claim 1, wherein, a length, a width and a depth of each of the convolution kernels used in the convolution operation are positive integers.

### Claim 10

An operating system using a pixel-channel shuffle convolutional neural network, comprising:
a processor, and a communication circuit and a memory that are electrically connected with the processor;
wherein the processor performs a calculation method using the pixel-channel shuffle convolutional neural network, comprising:
receiving an original input data that is a value with a length, a width and a depth;
performing a pixel shuffle operation on the original input data, in which the original input data is separated into multiple values for reducing dimension of each of the values;
performing a channel shuffle operation on the multiple values for selecting values used in a convolution operation from each of the multiple values so as to form multiple new input data;
setting up convolution kernels with respect to the multiple new input data, wherein a second amount of convolution kernels are included, and each of the convolution kernels implements a filter; and
by a multiplier-accumulator of the processor, performing the convolution operation with the second amount of convolution kernels and the multiple new input data, so as to form a second amount of multiple output data.

### Claim 11 (depends on 10)

The system of claim 10, wherein the original input data is an image data, and the operating system performs the convolution operation for extracting image features from the original input data so as to form a plurality of feature maps with the second amount of depth.

### Claim 12 (depends on 11)

The system of claim 11, wherein the second amount of output data are calculated by an inverse pixel shuffle operation so as to form an image feature map with a second amount of depth.

### Claim 13 (depends on 10)

The system of claim 10, wherein, after the new input data is formed by the channel shuffle operation, the unselected values for the convolution operation are abandoned.

### Claim 14 (depends on 10)

The system of claim 10, wherein the original input data is a value with a first amount of depth, and the original input data is processed by the pixel shuffle and the channel shuffle operations so as to form the plurality of input data with depth smaller than the first amount of depth.

### Claim 15 (depends on 14)

The system of claim 14, wherein the original input data is an image data, and the operating system performs the convolution operation for extracting image features from the original input data so as to form a plurality of feature maps with a second amount of depth.

### Claim 16 (depends on 15)

The system of claim 15, wherein the second amount of output data are calculated by an inverse pixel shuffle operation so as to form an image feature map with the second amount of depth.

### Claim 17 (depends on 16)

The system of claim 16, wherein image feature map is used to recognize the original input data.

### Claim 18 (depends on 10)

The system of claim 10, wherein, a length, a width and a depth of each of the convolution kernels used in the convolution operation are positive integers.

### Claim 19 (depends on 18)

The system of claim 18, wherein the operating system acts as a cloud system that provides a service of image recognition through the calculation method using the pixel-channel shuffle convolutional neural network.

### Claim 20 (depends on 18)

The system of claim 18, wherein the operating system implements a circuit system that performs image recognition through the calculation method using the pixel-channel shuffle convolutional neural network.