# US 2025/0370710 A1 · Apparatus for operating deep neural network for energy-efficient floating-point operation and method for floating-point operation using the same

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2025/0370710 A1
- Document ID: US20250370710A1
- Status: Published application
- Kind: A1
- Application number: 19/023,539
- Filed: 2025-01-16
- Published: 2025-12-04
- Earliest claimed: 2024-05-31
- Assignees: Korea Advanced Institute of Science and Technology
- Applicants: Korea Advanced Institute of Science and Technology
- Inventors: Hoi Jun YOO; Won hoon PARK
- Web page: https://patentgrep.com/patent/US20250370710A1

## CPC

- G06F 7/49915: {Mantissa overflow or underflow in handling floating-point numbers}
- G06F 7/483: Computations with numbers represented by a non-linear combination of denominational numbers, e.g. rational numbers, logarithmic number system or floating-point numbers {(G06F7/4806, G06F7/4824, G06F7/49, G06F7/491, G06F7/544 take precedence)}
- G06F 7/5443: {Sum of products (for applications thereof, see the relevant places, e.g. G06F17/10, H03H17/00)}
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)

## Abstract

An apparatus for a DNN operation includes a preprocessor configured to classify outlier data and inlier data from a predetermined number of pieces of grouped and input floating-point data and to perform presorting on the inlier data, a CIM operator configured to perform a fixed-point operation on the inlier data, an NPU operator configured to receive the outlier data and corresponding input channel information from the preprocessor and to perform a floating-point operation on the outlier data, and an aggregation core configured to sum and output an operation result of each of the CIM operator and the NPU operator, wherein the NPU operator reads a weight for each input channel for the floating-point operation on the outlier data through a separate transmission line implemented in the CIM operator, and causes the outlier data to be processed in parallel with an operation cycle of the inlier data.

## Claims

### Claim 1

An apparatus for a deep neural network (DNN) operation for an energy-efficient floating-point operation, the apparatus comprising:
a preprocessor configured to classify outlier data and inlier data from a predetermined number of pieces of grouped and input floating-point data and to perform presorting on the inlier data;
a computing-in-memory (CIM) operator configured to perform a fixed-point operation on the inlier data;
a neural processing unit (NPU) operator configured to receive the outlier data and corresponding input channel information from the preprocessor and to perform a floating-point operation on the outlier data; and
an aggregation core configured to sum and output an operation result of each of the CIM operator and the NPU operator,
wherein the NPU operator reads a weight for each input channel for the floating-point operation on the outlier data through a separate transmission line implemented in the CIM operator, and causes the outlier data to be processed in parallel with an operation cycle of the inlier data.

### Claim 2 (depends on 1)

The apparatus according to claim 1, wherein the preprocessor comprises:
an outlier searcher configured to find a maximum exponent value Emax among exponent values of each piece of the floating-point data, and then determine floating-point data, in which a difference between an exponent value and the maximum exponent value Emax exceeds a preset threshold Th, as outlier data; and
a mantissa preprocessor configured to presort mantissa values based on a difference value between the maximum exponent value Emax and the exponent value for each piece of remaining inlier data excluding the outlier data among the pieces of floating-point data.

### Claim 3 (depends on 2)

The apparatus according to claim 2, wherein the outlier searcher comprises:
a comparator configured to extract the maximum exponent value Emax by a comparison tree;
a bias operator configured to calculate a difference value between the maximum exponent value Emax and an exponent value of each piece of the floating-point data; and
a comparator configured to compare each difference value with the preset threshold Th to determine whether data is outlier data.

### Claim 4 (depends on 2)

The apparatus according to claim 2, wherein the mantissa preprocessor comprises:
a converter configured to convert a mantissa value of each piece of the inlier data to a 2's complement form including a corresponding sign; and
a shift operator configured to perform a shift operation on the mantissa value based on the difference value.

### Claim 5 (depends on 1)

The apparatus according to claim 1, wherein:
the CIM operator comprises a plurality of CIM cells storing a 1-bit weight for the DNN operation, and
each of the CIM cell comprises:
an SRAM cell configured to support an operation of reading/writing the weight through a read word line RWL and a read bit line pair RBL/RBLB and to transfer the weight to the NPU operator; and
a NOR operator configured to receive input of the inlier data through a compute work line CWL implemented separately from the read word line RWL and to perform a multiplication operation on the inlier data and the weight.

### Claim 6 (depends on 1)

The apparatus according to claim 1, wherein:
the NPU operator comprises at least one single instruction multiple data (SIMD) core matched with the CIM operator to perform the floating-point operation, and
the SIMD core comprises:
a plurality of SIMD lines configured to perform a floating-point operation on pieces of outlier data sequentially input from the preprocessor according to an input channel thereof;
an outlier cache configured to store a weight for each input channel read from the CIM operator in a previous floating-point operation; and
a cache controller configured to read a weight for each input channel of each piece of outlier data of a currently input floating-point data group from the outlier cache and to load the read weight into the SIMD line.

### Claim 7 (depends on 6)

The apparatus according to claim 6, wherein the cache controller further performs a process of requesting a weight from the CIM operator for an input channel whose corresponding weight is not stored in the outlier data among input channels of the outlier data and storing a received weight in the outlier cache in response thereto.

### Claim 8

A method for a floating-point operation using an apparatus for a DNN operation comprising a preprocessor configured to perform preprocessing on a predetermined number of pieces of grouped and input floating-point data for a floating-point operation, a CIM operator configured to perform a fixed-point operation on inlier data, an NPU operator configured to perform a floating-point operation on outlier data, and an aggregation core configured to sum and output an operation result of each of the CIM operator and the NPU operator, the method comprising:
a preprocessing step of classifying, by the preprocessor, outlier data and inlier data from a predetermined number of pieces of grouped and input floating-point data and performing presorting on the inlier data;
a CIM operation step of performing, by the CIM operator, a fixed-point operation on the inlier data;
an NPU operation step of receiving, by the NPU operator, the outlier data and corresponding input channel information from the preprocessor and performing a floating-point operation on the outlier data; and
an aggregation step of summing and outputting, by the aggregation core, an operation result of each of the CIM operation step and the NPU operation step,
wherein the NPU operation step comprises reading a weight for each input channel for the floating-point operation on the outlier data through a separate transmission line implemented in the CIM operator, and causing the outlier data to be processed in parallel with an operation cycle of the inlier data.

### Claim 9 (depends on 8)

The method according to claim 8, wherein the preprocessing step comprises:
an outlier search step of finding a maximum exponent value Emax among exponent values of each piece of the floating-point data, and then determining floating-point data, in which a difference between an exponent value and the maximum exponent value Emax exceeds a preset threshold Th, as outlier data; and
a mantissa presorting step of presorting mantissa values based on a difference value between the maximum exponent value Emax and the exponent value for each piece of remaining inlier data excluding the outlier data among the pieces of floating-point data.

### Claim 10 (depends on 9)

The method according to claim 9, wherein the outlier search step comprises:
a maximum exponent value Emax extraction step of extracting the maximum exponent value Emax by a comparison tree;
a bias operation step of calculating a difference value between the maximum exponent value Emax and an exponent value of each piece of the floating-point data; and
a comparison step of comparing each difference value with the preset threshold Th to determine whether data is outlier data.

### Claim 11 (depends on 10)

The method according to claim 10, wherein the mantissa presorting step comprises:
a conversion step of converting a mantissa value of each piece of the inlier data to a 2's complement form including a corresponding sign; and
a shift operation step of performing a shift operation on the mantissa value based on the difference value.

### Claim 12 (depends on 8)

The method according to claim 8, wherein the CIM operation step comprises:
a weight storage step of storing a 1-bit weight for the DNN operation in a plurality of CIM cells for processing a CIM operation;
a fixed-point operation step of receiving input of the inlier data by a signal of a compute work line CWL implemented separately from a read word line RWL of each of the CIM cell and performing a multiplication operation on the inlier data and the weight; and
a weight transfer step of transferring the weight to the NPU operator by a read word line RWL and read bit line pair RBL/RBLB signal applied to the CIM cell.

### Claim 13 (depends on 8)

The method according to claim 8, wherein:
the NPU operation step comprises:
a floating-point operation step of performing a floating-point operation on the outlier data using a weight for each input channel read from the CIM operator; and
a weight caching step of storing the weight used for the floating-point operation for each input channel in an outlier cache, and
the floating-point operation step comprises a weight loading step of loading a weight for each input channel prestored in the outlier cache for an operation of each piece of the outlier data.

### Claim 14 (depends on 13)

The method according to claim 13, wherein the NPU operation step comprises:
a weight request step of requesting a weight from the CIM operator for an input channel whose corresponding weight is not stored in the outlier data among input channels of the outlier data; and
a weight storage step of storing a weight received from the CIM operator in the outlier cache.
