US 11,455,144 B1Granted

Softmax calculation and architecture using a modified coordinate rotation digital computer (CORDIC) approach

AssigneeXilinx, Inc.

InventorTomas Figliolia

Life of this patent

Estimating the term…

Loading patent…

Abstract

Apparatus and associated methods relate to providing a modified CORDIC approach and implementing the modified CORDIC approach in SoftMax calculation to reduce usage of hardware resources. In an illustrative example, a system may include (a) a first circuit configured to transform each element Vi of an input vector V into Vi=Vpi+ki·ln 2 to generate a second data set, (b) a second circuit configured to perform exponential calculations on the second data set to generate a third data set that has a first mantissa ∑ j = 1 n ⁢ ⁢ 2 k j · f exp ⁡ ( V pj ) 2 k total and an exponent Ktotal, (c) a third circuit configured to perform logarithm calculations to generate a third sub data set that has a second mantissa 2 ⁢ ⁢ f ln ( ∑ j = 1 n ⁢ ⁢ 2 k j · f exp ⁡ ( V pj ) 2 k total ) , and, (d) a fourth circuit configured to perform exponential calculations to generate an output data set that has 2 k i - k total · f exp ( V pi + 2 ⁢ ⁢ f ln ( ∑ j = 1 n ⁢ ⁢ 2 k j · f exp ⁡ ( V pj ) 2 k total ) , the output data set may be interpreted as a probability distribution of each element Vi of the input vector V.

Claims13

Dependent claims

A system comprising:

a first circuit configured to receive an input vector V representative of a first data set and transform each element Vi of the input vector V into Vi=Vpi+ki·ln 2 to generate a second data set, the second data set comprising a first sub data set that has Vpi and a second sub data set that has ki, wherein −1≤Vpi≤1, and each ki is an integer;
a second circuit configured to receive the second data set from the first circuit and perform first exponential calculations on the second data set to generate a third data set, the third data set comprising a third sub data set that has a first mantissa
V
total
⁢
⁢
1
=
∑
j
=
1
n
⁢
2
k
j
·
f
e
⁢
x
⁢
p
⁡
(
V
p
⁢
j
)
2
k
total
and a fourth sub data set having a first exponent ktotal, wherein −1≤Vtotal1≤1,
a third circuit configured to receive the third sub data set and perform logarithm calculations on the third sub data set to generate a fifth sub data set that has a second mantissa
V
total
⁢
⁢
2
=
2
⁢
f
l
⁢
n
(
∑
j
=
1
n
⁢
2
k
j
·
f
e
⁢
x
⁢
p
⁡
(
V
p
⁢
j
)
2
k
total
)
,
wherein −1≤Vtotal2≤1;
and
a fourth circuit configured to receive the fourth sub data set that has the first exponent ktotal, the fifth sub data set that has the second mantissa, and the second data set that has Vpi and ki, and perform second exponential calculations to generate an output data set that has
2
k
i
-
k
t
⁢
o
⁢
t
⁢
a
⁢
l
·
f
e
⁢
x
⁢
p
(
V
p
⁢
i
+
2
⁢
f
l
⁢
n
(
∑
j
=
1
n
⁢
2
k
j
·
f
e
⁢
x
⁢
p
⁡
(
V
p
⁢
j
)
2
k
total
)
,
wherein the output data set is representative of a probability distribution of each element Vi of the input vector V.

140 more elements

A method comprising:

(a) receiving, at a first circuit, an input vector V representative of a first data set and transforming each element Vi of the input vector V into Vi=Vpi+ki·ln 2 to generate a second data set, the second data set comprising a first sub data set that has Vpi and a second sub data set that has ki, wherein −1≤Vpi≤1, and ki is an integer;
(b) receiving, at a second circuit, the second data set and performing first exponential calculations on the second data set to generate a third data set, the third data set comprising a third sub data set that has a first mantissa
V
total
⁢
⁢
1
=
∑
j
=
1
n
⁢
2
k
j
·
f
exp
⁡
(
V
pj
)
2
k
total
and a fourth sub data set having a first exponent ktotal, wherein −1≤Vtotal≤1,
(c) receiving, at a third circuit, the third sub data set and performing logarithm calculations on third sub data set to generate a fifth sub data set that has a second mantissa
V
total
⁢
⁢
2
=
2
⁢
f
l
⁢
n
(
∑
j
=
1
n
⁢
2
k
j
·
f
e
⁢
x
⁢
p
⁡
(
V
p
⁢
j
)
2
k
total
)
;
and
(d) receiving, at a fourth circuit, the fourth sub data set that has the first exponent ktotal, the fifth sub data set that has the second mantissa, and the second data set that has Vpi and ki, and performing second exponential calculations to generate an output data set that has
2
k
i
-
k
t
⁢
o
⁢
t
⁢
a
⁢
l
·
f
e
⁢
x
⁢
p
(
V
p
⁢
i
+
2
⁢
f
l
⁢
n
(
∑
j
=
1
n
⁢
2
k
j
·
f
e
⁢
x
⁢
p
⁡
(
V
p
⁢
j
)
2
k
total
)
,
wherein the output data set is representative of a probability distribution of each element Vi of the input vector V.

133 more elements

Dependencies are inferred from the claim text.

Facts

Application: 16/691,379Filed 21 Nov 2019
Granted: 27 Sep 2022
Term ends: Not estimatedThe source records don’t support an estimate.

Classification8 codes

G06F 7/5446
For evaluating functions by calculation: using crossaddition algorithms, e.g. CORDIC
G06F 7/485
Computations with numbers represented by a non-linear combination of denominational numbers, e.g. rational numbers, logarithmic number system or…:…
G06F 7/556
For evaluating functions by calculation: logarithmic or exponential functions
G06F 17/18
Complex mathematical operations: for evaluating statistical data , e.g. average values, frequency distributions, probability functions, regression…