# US 2020/0394252 A1 · Integrator voltage shifting for improved performance in softmax operation

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2020/0394252 A1
- Document ID: US20200394252A1
- Status: Published application
- Kind: A1
- Application number: 16/439,246
- Filed: 2019-06-12
- Published: 2020-12-17
- Earliest claimed: 2019-06-12
- Assignees: International Business Machines Corporation
- Applicants: International Business Machines Corporation
- Inventors: Seyoung Kim; Tayfun Gokmen; Malte Rasch
- Web page: https://patentgrep.com/patent/US20200394252A1

## CPC

- G06F 17/16: Matrix or vector computation {, e.g. matrix-matrix or matrix-vector multiplication, matrix factorization (matrix transposition G06F7/78)}
- G06N 3/048: Activation functions
- G06N 3/065: Analogue means
- G06N 3/08: Learning methods
- H03F 3/70: Charge amplifiers
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- H03F: AMPLIFIERS

## Abstract

An apparatus and method are provided for saturation prevention of a current integrator in a Resistive Processing Unit-based (RPU-based) accelerator. The apparatus includes a set of hardware switches. The apparatus further includes a voltage generator, operatively coupled between an input terminal and an output terminal of the current integrator, reducing a magnitude of an output voltage at the output terminal of the current integrator during a current integration operation by selectively applying a non-zero initial voltage to the current integrator prior to the current integration operation, responsive to an operating state of the set of hardware switches.

## Claims

### Claim 1

An apparatus for saturation prevention of a current integrator in a Resistive Processing Unit-based (RPU-based) accelerator, comprising:
a set of hardware switches; and
a voltage generator, operatively coupled between an input terminal and an output terminal of the current integrator, reducing a magnitude of an output voltage at the output terminal of the current integrator during a current integration operation by selectively applying a non-zero initial voltage to the current integrator prior to the current integration operation, responsive to an operating state of the set of hardware switches.

### Claim 2 (depends on 1)

The apparatus of claim 1, wherein the set of hardware switches comprises a first switch and a second switch, each operatively coupled in parallel between the input terminal and the output terminal of the current integrator, the first switch being in series with the voltage generator.

### Claim 3 (depends on 2)

The apparatus of claim 2, wherein the first switch is closed and the second switch is open to selectively apply the non-zero initial voltage to the current integrator prior to the current integration, and wherein the first switch and the second switch are open during the current integration.

### Claim 4 (depends on 2)

The apparatus of claim 2, wherein another input terminal of the current integrator is grounded.

### Claim 5 (depends on 2)

The apparatus of claim 2, further comprising a capacitor connected between the input terminal and the output terminal, in parallel with the first switch, the voltage generator, and the second switch.

### Claim 6 (depends on 1)

The apparatus of claim 1, wherein the non-zero initial voltage is generated to have a polarity opposing an expected polarity of the output voltage at the output terminal of the current integrator to reduce the magnitude of the output voltage at the output terminal of the current integrator below a saturating value.

### Claim 7 (depends on 1)

The apparatus of claim 1, wherein the non-zero initial voltage is selected to double an effective range of the output signal at the output terminal of the current integrator.

### Claim 8 (depends on 1)

The apparatus of claim 1, wherein the input terminal of the current integrator is connected to a column of RPU devices of the RPU-based accelerator.

### Claim 9 (depends on 1)

The apparatus of claim 1, wherein the voltage generator selectively pulls the output voltage at the output terminal of the current integrator up or down by an amount corresponding to the initial non-zero voltage to avoid a saturation state, responsive to the operating state of the set of hardware switches.

### Claim 10 (depends on 1)

The apparatus of claim 1, wherein the current integration operation is part of a softmax operation.

### Claim 11

A method for saturation prevention of a current integrator in a Resistive Processing Unit-based (RPU-based) accelerator, comprising:
reducing, by a voltage generator operatively coupled between an input terminal and an output terminal of the current integrator, a magnitude of an output voltage at the output terminal of the current integrator during a current integration operation by selectively applying a non-zero initial voltage to the current integrator prior to the current integration operation, responsive to an operating state of the set of hardware switches.

### Claim 12 (depends on 11)

The method of claim 11, wherein the set of hardware switches comprises a first switch and a second switch, the first switch and the second switch both being operatively coupled in parallel between the input terminal and the output terminal of the current integrator, the first switch being in series with the voltage generator, and the method further comprises:
closing the first switch and opening the second switch to selectively apply the non-zero initial voltage to the current integrator prior to the current integration; and
opening the first switch and the second switch during the current integration.

### Claim 13 (depends on 11)

The method of claim 11, further comprising connecting a capacitor between the input terminal and the output terminal, in parallel with the first switch, the voltage generator, and the second switch.

### Claim 14 (depends on 11)

The method of claim 11, wherein the non-zero initial voltage is generated to have a polarity opposing an expected polarity of the output voltage at the output terminal of the current integrator to reduce the magnitude of the output voltage at the output terminal of the current integrator below a saturating value.

### Claim 15 (depends on 11)

The method of claim 11, further comprising selecting the non-zero initial voltage so as to double an effective range of the output signal at the output terminal of the current integrator.

### Claim 16 (depends on 11)

The method of claim 11, wherein the voltage generator selectively pulls the output voltage at the output terminal of the current integrator up or down by an amount corresponding to the initial non-zero voltage to avoid a saturation state, responsive to the operating state of the set of hardware switches.

### Claim 17

An apparatus for saturation prevention of a current integrator in a Resistive Processing Unit-based (RPU-based) accelerator, comprising:
a saturation detector for detecting an impending saturation condition of an output of the current integrator;
a current source trigger signal generator for generating a trigger signal responsive to a detection of the impending saturation condition;
a current source for reducing a magnitude of an existing current at the input terminal of the current integrator during a current integration operation by generating a current of opposing polarity to the existing current at the input terminal to prevent an occurrence of the saturation condition, responsive to the trigger signal.

### Claim 18 (depends on 17)

The apparatus of claim 17, wherein the current of opposing polarity is generated so as to double an effective range of an expected output signal of the current integrator.

### Claim 18 (depends on 17)

The apparatus of claim 17, wherein the input terminal of the current integrator is connected to a column of RPU devices of the RPU-based accelerator.

### Claim 20 (depends on 16)

The apparatus of claim 16, wherein the current source trigger signal generator comprises an OR gate.
