# US 12,182,717 B2 · Feature extraction with a convolutional neural network

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 12,182,717 B2
- Document ID: US12182717B2
- Status: Granted
- Kind: B2
- Application number: 17/504,488
- Filed: 2021-10-18
- Published: 2024-07-18
- Granted: 2024-12-31
- Earliest claimed: 2019-09-11
- Assignees: Expedera, Inc.
- Applicants: Expedera, Inc.
- Inventors: Shang-Tse Chuang; Sharad Vasantrao Chole; Siyad Chih-Hua Ma
- Web page: https://patentgrep.com/patent/US12182717B2

## CPC

- G06N 3/084: Backpropagation, e.g. using gradient descent
- G06F 18/2115: by evaluating different subsets according to an optimisation criterion, e.g. class separability, forward selection or backward elimination
- G06F 18/2148: {characterised by the process organisation or structure, e.g. boosting cascade}
- G06N 3/04: Architecture, e.g. interconnection topology
- G06N 3/045: Combinations of networks
- G06N 3/0464: Convolutional networks [CNN, ConvNet]
- G06N 3/09: Supervised learning
- G06V 10/454: {Integrating the filters into a hierarchical structure, e.g. convolutional neural networks [CNN]}
- G06V 10/764: using classification, e.g. of video objects
- G06V 10/82: using neural networks
- G06V 20/00: Scenes; Scene-specific elements (control of digital cameras H04N23/60)
- G06N 7/01: Probabilistic graphical models, e.g. probabilistic networks
- G06N 20/20: Ensemble learning
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING

## Abstract

Artificial intelligence is an increasingly important sector of the computer industry. One of the most important applications for artificial intelligence is object recognition and classification from digital images. Convolutional neural networks have proven to be a very effective tool for object recognition and classification from digital images. However, convolutional neural networks are extremely computationally intensive thus requiring high-performance processors, significant computation time, and significant energy consumption. To reduce the computation time and energy consumption a “cone of dependency” and “cone of influence” processing techniques are disclosed. These two techniques arrange the computations required in a manner that minimizes memory accesses such that computations may be performed in local cache memory. These techniques significantly reduce the time to perform the computations and the energy consumed by the hardware implementing a convolutional neural network.

## Claims

### Claim 1

A method of extracting features from a digital image with a multi-step convolutional processing system in a memory efficient manner, said method comprising the stages of:
dividing said digital image into a set of digital image subareas,
for each digital image subarea in said set of digital image subareas, performing the substages of:
loading a subset of digital image subarea data for a feature extraction operation,
executing said feature extraction operation on said subset of digital image data and any saved intermediate results to generate a first set of results,
executing any additional later step operations of said multi-step convolutional processing system using said first set of results of convolutional stride calculation to generate a second set of results,
saving intermediate data in said first set of results and said second set of results still needed for calculating later step operations in said multi-step convolutional processing system as saved intermediate results and discarding all other data, and
repeating said substages of loading, executing, executing, and saving until a final output from said multi-step convolutional processing system is complete for said digital image subarea; and
combining said final outputs for each said digital image subareas into a final result for said digital image.

### Claim 2 (depends on 1)

The method of extracting features from a digital image with a multi-step convolutional processing system in a memory efficient manner as set forth in claim 1 wherein said multi-step convolutional processing system comprises a convolutional neural network.

### Claim 3 (depends on 1)

The method of extracting features from a digital image with a multi-step convolutional processing system in a memory efficient manner as set forth in claim 1 wherein a step in said multi-step convolutional processing system comprises a pooling operation.

### Claim 4 (depends on 1)

The method of extracting features from a digital image with a multi-step convolutional processing system in a memory efficient manner as set forth in claim 1 wherein said digital image subareas sub areas of said set of digital image subareas sub areas overlap each other.

### Claim 5 (depends on 1)

The method of extracting features from a digital image with a multi-step convolutional processing system in a memory efficient manner as set forth in claim 1, said method further comprising the stages of:
first dividing said multi-step convolutional processing system into a set of second substages, each of said second substages comprising a group of steps from said multi-step convolutional processing system; and
performing said substages of loading, executing, executing, saving, and repeating upon each of said second substages comprising said group of steps.

### Claim 6 (depends on 1)

The method of extracting features from a digital image with a multi-step convolutional processing system in a memory efficient manner as set forth in claim 1 wherein a first processing task operates said method of claim 1 on a first portion of said digital image and a second processing task operates said method of claim 1 on a second portion of said digital image.

### Claim 7

A method of extracting features from a digital image with a multi-step convolutional processing system in a memory efficient manner, said method comprising the stages of:
dividing said digital image into a set of digital image subareas,
for each digital image subarea in said set of digital image subareas, performing the substages of:
loading a subset of digital image subarea data for a feature extraction operation,
executing as many full and partial feature extraction operations as possible with said subset of digital image subarea data and any saved partial results,
saving all partial results and final convolutional processing system output results and discarding all other data, and
repeating said substages of loading, executing, and saving until a final output for said digital image subarea from said multi-step convolutional processing system is complete; and
combining said final outputs for all said digital image subareas into a final result for said digital image.

### Claim 8 (depends on 7)

The method of extracting features from a digital image with a multi-step convolutional processing system in a memory efficient manner as set forth in claim 7 wherein said multi-step convolutional processing system comprises a convolutional neural network.

### Claim 9 (depends on 7)

The method of extracting features from a digital image with a multi-step convolutional processing system in a memory efficient manner as set forth in claim 7 wherein a step in said multi-step convolutional processing system comprises a pooling operation.

### Claim 10 (depends on 7)

The method of extracting features from a digital image with a multi-step convolutional processing system in a memory efficient manner as set forth in claim 7 wherein said digital image subareas overlap each other.

### Claim 11 (depends on 7)

The method of extracting features from a digital image with a multi-step convolutional processing system in a memory efficient manner as set forth in claim 7, said method further comprising the stages of:
first dividing said multi-step convolutional processing system into a set of second substages, each of said second substages comprising a group of steps from said multi-step convolutional processing system; and
performing said second substages of loading, executing, saving, and repeating upon each of said substages comprising said group of steps.

### Claim 12 (depends on 7, 8)

The method of extracting features from a digital image with a multi-step convolutional processing system in a memory efficient manner as set forth in claim 7 wherein a first processing task operates said method of claim 8 on a first portion of said digital image and a second processing task operates said method of claim 8 on a second portion of said digital image.