US 12,182,577 B2Granted

Neural-processing unit tile for shuffling queued nibbles for multiplication with non-zero weight nibbles

AssigneeSamsung Electronics Co., Ltd.

InventorsIlia Ovsiannikov, Ali Shafiee Ardestani, Hamzah Ahmed Ali Abdelaziz, Joseph H. Hassoun

Life of this patent

Estimating the term…

Loading patent…

Abstract

A processor. In some embodiments, the processor includes: a first tile, the first tile being configured: to feed a first nibble from a third queue, through a first shuffler, to a first multiplier, and to multiply, in the first multiplier, the first nibble from the third queue by a first nibble of a third weight; to feed a second nibble from the third queue, through the first shuffler, to a second multiplier, and to multiply, in the second multiplier, the second nibble from the third queue by a second nibble of the third weight; and to feed a first nibble from a fourth queue, through the first shuffler, to a third multiplier, and to multiply, in the third multiplier, the first nibble from the fourth queue by a first nibble of a fourth weight.

Claims19

Dependent claims

A processor, comprising:

a first tile,
a second tile,
a memory, and
a bus,
the bus being connected to:
the memory,
the first tile, and
the second tile,
the first tile comprising:
a first weight register storing a first weight, the first weight being zero;
a second weight register storing a third weight, the third weight being an eight-bit number neither nibble of which is zero;
a third weight register storing a second weight, the second weight being zero;
a fourth weight register storing a fourth weight, the fourth weight being an eight-bit number neither nibble of which is zero;
an activations buffer;
a first shuffler;
a first multiplier connected to the first weight register;
a second multiplier connected to the second weight register;
a third multiplier connected to the third weight register;
and
a fourth multiplier connected to the fourth weight register, the activations buffer being configured to include:
a first queue,
a second queue,
a third queue, and
a fourth queue,
the first tile being configured:
to feed a first nibble from the third queue, through the first shuffler, to the first multiplier, and to multiply, in the first multiplier, the first nibble from the third queue by a first nibble of the third weight;
to feed a second nibble from the third queue, through the first shuffler, to the second multiplier, and to multiply, in the second multiplier, the second nibble from the third queue by a second nibble of the third weight;
to feed a first nibble from the fourth queue, through the first shuffler, to the third multiplier, and to multiply, in the third multiplier, the first nibble from the fourth queue by a first nibble of the fourth weight;
and
to feed a second nibble from the fourth queue, through the first shuffler, to the fourth multiplier, and to multiply, in the fourth multiplier, the second nibble from the fourth queue by a second nibble of the fourth weight.

27 more elements

A method for calculating with a processing circuit, the processing circuit comprising:

a first tile,
a second tile,
a memory, and
a bus,
the bus being connected to:
the memory,
the first tile, and
the second tile,
the first tile comprising:
a first weight register storing a first weight, the first weight being zero;
a second weight register storing a third weight, the third weight being an eight-bit number neither nibble of which is zero;
a third weight register storing a second weight, the second weight being zero;
a fourth weight register storing a fourth weight, the fourth weight being an eight-bit number neither nibble of which is zero;
an activations buffer;
a first shuffler;
a first multiplier connected to the first weight register;
a second multiplier connected to the second weight register;
a third multiplier connected to the third weight register;
and
a fourth multiplier connected to the fourth weight register,
the activations buffer being configured to include:
a first queue,
a second queue,
a third queue, and
a fourth queue,
the method comprising:
feeding a first nibble from the third queue, through the first shuffler, to the first multiplier, and multiplying, in the first multiplier, the first nibble from the third queue by a first nibble of the third weight;
feeding a second nibble from the third queue, through the first shuffler, to the second multiplier, and multiplying, in the second multiplier, the second nibble from the third queue by a second nibble of the third weight;
feeding a first nibble from the fourth queue, through the first shuffler, to the third multiplier, and multiplying, in the third multiplier, the first nibble from the fourth queue by a first nibble of the fourth weight;
and
feeding a second nibble from the fourth queue, through the first shuffler, to the fourth multiplier, and multiplying, in the fourth multiplier, the second nibble from the fourth queue by a second nibble of the fourth weight.

28 more elements

A method for calculating with a means for processing, the means for processing comprising:

a first tile,
a second tile,
a memory, and
a bus,
the bus being connected to:
the memory,
the first tile, and
the second tile,
the first tile comprising:
a first weight register storing a first weight, the first weight being zero;
a second weight register storing a third weight, the third weight being an eight-bit number neither nibble of which is zero;
a third weight register storing a second weight, the second weight being zero;
a fourth weight register storing a fourth weight, the fourth weight being an eight-bit number neither nibble of which is zero;
an activations buffer;
a first shuffler;
a first multiplier connected to the first weight register;
a second multiplier connected to the second weight register;
a third multiplier connected to the third weight register;
and
a fourth multiplier connected to the fourth weight register,
the activations buffer being configured to include:
a first queue,
a second queue,
a third queue, and
a fourth queue,
the method comprising:
feeding a first nibble from the third queue, through the first shuffler, to the first multiplier, and multiplying, in the first multiplier, the first nibble from the third queue by a first nibble of the third weight;
feeding a second nibble from the third queue, through the first shuffler, to the second multiplier, and multiplying, in the second multiplier, the second nibble from the third queue by a second nibble of the third weight;
feeding a first nibble from the fourth queue, through the first shuffler, to the third multiplier, and multiplying, in the third multiplier, the first nibble from the fourth queue by a first nibble of the fourth weight;
and
feeding a second nibble from the fourth queue, through the first shuffler, to the fourth multiplier, and multiplying, in the fourth multiplier, the second nibble from the fourth queue by a second nibble of the fourth weight,
wherein the first shuffler is a butterfly shuffler.

29 more elements

Dependencies are inferred from the claim text.

Facts

Application: 16/847,504Filed 13 Apr 2020 · us provisional application
Priority: 1 May 2019
Published: US 2020/0349106 A15 Nov 2020
Granted: 31 Dec 2024
Term ends: Not estimatedThe source records don’t support an estimate.

Classification6 codes

G06F 9/3885
Concurrent instruction execution, e.g. pipeline or look ahead: using a plurality of independent parallel functional units
G06N 3/063
Physical realisation, i.e. hardware implementation of neural networks, neurons or parts of neurons: using electronic means
G06F 15/80
Architectures of general purpose stored program computers: comprising an array of processing units with common control, e.g. single instruction…
G06N 3/082
Learning methods: modifying the architecture, e.g. adding, deleting or silencing nodes or connections

Family

Application62/841,606Us provisional application · filed 1 May 2019
US 2020/0349106 A1Pre-grant publication · published 5 Nov 2020
US 12,182,577 B2This patent

From the US filing history. Foreign family members aren’t listed.

Timeline

Citations

Open elsewhere

Building an agent or pulling many patents? Use the API or MCP server instead of scraping pages. Get an API key