# US 12,762,408 B2 · Neuromorphic apparatus and method with neural network

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 12,762,408 B2
- Document ID: US12762408B2
- Status: Granted
- Kind: B2
- Application number: 17/078,714
- Filed: 2020-10-23
- Published: 2021-12-30
- Granted: 2026-10-13
- Earliest claimed: 2020-06-24
- Assignees: Samsung Electronics Co., Ltd.
- Applicants: Samsung Electronics Co., Ltd.
- Inventors: Hyunsoo Kim
- Web page: https://patentgrep.com/patent/US12762408B2

## CPC

- G06N 3/084: Backpropagation, e.g. using gradient descent
- G06N 3/04: Architecture, e.g. interconnection topology
- G06N 3/088: Non-supervised learning, e.g. competitive learning
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS

## Abstract

A processor-implemented neural network implementation method includes: learning each of first layers included in a neural network according to a first method; learning at least one second layer included in the neural network according to a second method; and generating output data from input data by using the learned first layers and the learned at least one second layer.

## Claims

### Claim 1

A processor-implemented neural network implementation method executed by a processor and an on-chip memory included in a neuromorphic apparatus for performing a neuromorphic operation, the method comprising:
performing, by the processor and the on-chip memory included in the neuromorphic apparatus, a first training of each of first layers included in a convolution neural network based on unsupervised learning according to a self-organizing map without using back-propagation, including determining a similarity between partial input vectors and an updated weight of at least one output neuron using input feature map data stored in the on-chip memory, wherein the self-organizing map comprises an input layer having a number of neurons equal to dimensions of an input and an output layer having a number of neurons equal to clustering target classes, and wherein each neuron of the output layer of the self-organizing map has a weight vector having the same dimension as the input and the input of the self-organizing map is clustered by classifying the input into one of most similar neuron of the output layer based on a calculated similarity between the input and the weight vector of each neuron of the output layer;
performing, by the processor and the on-chip memory, a second training of at least one second layer included in the convolution neural network based on supervised learning according to back-propagation; and
performing, by the processor and the on-chip memory, an inference operation, including generating, by the processor and the on-chip memory, output data based on input data stored in the on-chip memory, by using the learned first layers and the learned at least one second layer,
wherein the first layers comprise convolutional layers of the convolution neural network to extract features from the input data and the at least one second layer comprises at least one fully-connected layer of the convolution neural network to perform classification and identification based on output feature map data extracted from the input data by the first layers,
wherein the convolution neural network is implemented as a single convolution neural network including the first layers and the at least one second layer,
wherein the performing the first training comprises generating output feature map data for the convolutional layers via addition operation and subtraction operation instead of a MAC (multiply and accumulate) operation based on the self-organizing map,
wherein the generated output feature map data of the first layers is used as an input feature map for the at least one second layer to perform the supervised learning according to the back-propagation,
wherein the generating the output feature map data comprises:
generating partial input vectors based on input feature map data of an initial layer of the first layers;
determining the similarity between the partial input vectors and the updated weights, to generate similarity vectors; and
generating the output feature map data of the initial layer based on the similarity vectors, and
wherein the similarity vectors are determined as partial output feature map data, and the partial output feature map data is matched with a coordinate of the input feature map data of which the partial input vectors are generated, to generate the output feature map data of the initial layer, and
wherein the first training, the second training and the inference operation are performed by the processor and the on-chip memory.

### Claim 2 (depends on 1)

The method of claim 1, wherein the performing the first training of each of the first layers comprises:
generating the partial input vectors based on the input feature map data of the initial layer of the first layers;
learning the initial layer, based on the partial input vectors using the self-organizing map corresponding to the initial layer; and
generating the output feature map data of the initial layer using the learned initial layer.

### Claim 3 (depends on 2)

The method of claim 2, wherein the learning of the initial layer comprises:
determining, using the self-organizing map, an output neuron, among output neurons, having a weight most similar to at least one of the partial input vectors;
updating, using the self-organizing map, a weight of at least one output neuron located in a determined range of the output neurons based on the determined output neuron; and
learning the initial layer based on the updated weight.

### Claim 4 (depends on 1)

The method of claim 1, further comprising learning a next layer of the first layers based on the output feature map data of the initial layer.

### Claim 5 (depends on 1)

The method of claim 1, wherein the generating of the output data comprises:
generating output feature map data by applying the input data to the learned first layers; and
generating the output data by applying the output feature map data to the learned at least one second layer.

### Claim 6 (depends on 1)

A non-transitory computer-readable storage medium storing instructions that, when executed by one or more processors, configure the one or more processors to perform the method of claim 1.

### Claim 7

A neuromorphic neural network implementation apparatus for performing a neuromorphic operation, the neuromorphic neural network implementation apparatus comprising:
a processor; and
an on-chip memory comprising a plurality of cores and storing one or more instructions that, when executed by the processor,
the processor configured to
perform a first training of each of first layers included in a convolution neural network based on unsupervised learning according to a self-organizing map without using back-propagation, including determining a similarity between partial input vectors and an updated weight of at least one output neuron using input feature map data stored in the on-chip memory, wherein the self-organizing map comprises an input layer having a number of neurons equal to dimensions of an input and an output layer having a number of neurons equal to clustering target classes, and wherein each neuron of the output layer of the self-organizing map has a weight vector having the same dimension as the input and the input of the self-organizing map is clustered by classifying the input into one of most similar neuron of the output layer based on a calculated similarity between the input and the weight vector of each neuron of the output layer,
perform a second training of at least one second layer included in the convolution neural network based on supervised learning according to back-propagation, and
perform an inference operation by driving the processor and the on-chip memory to implement the convolution neural network, including generating output data based on input data stored in the on-chip memory, by using the learned first layers and the learned at least one second layer,
wherein the first layers comprise convolutional layers of the convolution neural network to extract features from the input data and the at least one second layer comprises at least one fully-connected layer of the convolution neural network to perform classification and identification based on output feature map data extracted from the input data by the first layers,
wherein the convolution neural network is implemented as a single convolution neural network including the first layers and the at least one second layer,
wherein, for performing the first training, the processor is further configured to generate output feature map data for the convolutional layers via addition operation and subtraction operation instead of a MAC (multiply and accumulate) operation based on the self-organizing map,
wherein the generated output feature map data of the first layers is used as an input feature map for the at least one second layer to perform the supervised learning according to the back-propagation,
wherein, for generating the output feature map data, the processor is further configured to:
generate partial input vectors based on input feature map data of an initial layer of the first layers;
determine the similarity between the partial input vectors and the updated weights, to generate similarity vectors; and
generate the output feature map data of the initial layer based on the similarity vectors, and
wherein the similarity vectors are determined as partial output feature map data, and the partial output feature map data is matched with a coordinate of the input feature map data of which the partial input vectors are generated, to generate the output feature map data of the initial layer, and
wherein the first training, the second training and the inference operation are performed by the processor and the on-chip memory.

### Claim 8 (depends on 7)

The apparatus of claim 7, wherein, for the performing the first training of each of the first layers, the processor is further configured to
generate the partial input vectors based on the input feature map data of the initial layer of the first layers,
learn the initial layer based on the partial input vectors using the self-organizing map corresponding to the initial layer, and
generate output feature map data of the initial layer using the learned initial layer.

### Claim 9 (depends on 8)

The apparatus of claim 8, wherein, for the learning of the initial layer, the processor is further configured to
determine, using the self-organizing map, an output neuron, among output neurons, having a weight most similar to at least one of the partial input vectors,
update, using the self-organizing map, a weight of at least one output neuron located in a determined range of the output neurons based on the determined output neuron, and
learn the initial layer based on the updated weight.

### Claim 10 (depends on 7)

The apparatus of claim 7, wherein the processor is further configured to learn a next layer of the first layers based on the output feature map data of the initial layer.

### Claim 11 (depends on 10)

The apparatus of claim 10, wherein, for the generating of the output data, the processor is further configured to
generate output feature map data by applying the input data to the learned first layers, and
generate the output data by applying the output feature map data to the learned at least one second layer.

### Claim 12 (depends on 9)

The apparatus of claim 9, wherein the on-chip memory stores the one or more instructions that configure the processor to:
perform the first training of each of the first layers;
perform the second training of the at least one second layer; and
drive the neural network to perform the generating of the output data.