# US 12,223,414 B1 · Programming method of an activation function and an activation function programming unit > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 12,223,414 B1 - Document ID: US12223414B1 - Status: Granted - Kind: B1 - Application number: 18/599,120 - Filed: 2024-03-07 - Granted: 2025-02-11 - Earliest claimed: 2023-11-08 - Assignees: Deepx Co., Ltd. - Applicants: Deepx Co., Ltd. - Inventors: Jong Hoon Shin; Lok Won Kim; Hyung Jin Chun; Ho Seung Kim - Web page: https://patentgrep.com/patent/US12223414B1 ## CPC - G06N 3/048: Activation functions - G06N 3/045: Combinations of networks - G06N 3/063: using electronic means - G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS ## Abstract An activation function conversion program unit and method may be configured to approximate a target activation function to a programmed activation function through machine-learning of an artificial neural network. The method may include setting up a target activation function; approximating the target activation function to a programmed activation function by machine-learning an artificial neural network; and converting the programmed activation function into a slope and offset and storing it in a lookup table. Accordingly, the computation speed and power consumption of the programmed activation function execution unit of an NPU may be optimized. ## Claims ### Claim 1 A computing device, comprising: one or more processors; and memory storing instructions thereon, the instructions, when executed by the one or more processors, cause the one or more processors to: receive a target activation function of a first artificial neural network for implementing in a neural processor circuit, execute a second artificial neural network to determine an approximation activation function that approximates the target activation function and is programmable into the neural processor circuit to implement the first artificial neural network, wherein the second artificial neural network is trained to determine one or more parameters of the approximation activation function to reduce an error with respect to the target activation function, and cause the approximation activation function to be programmed into the neural processor circuit to implement the first artificial neural network. ### Claim 2 (depends on 1) The computing device of claim 1, wherein the second artificial neural network includes a first neural network section comprising at least one neuron and a second neural network section comprising at least one neuron, wherein a rectified linear unit (ReLU) function is applied to outputs of the at least one neuron of the first neural network section, and wherein a value to which the ReLU function is applied is an input to the at least one neuron of the second neural network section. ### Claim 3 (depends on 1) The computing device of claim 1, wherein the second artificial neural network comprises a first neural network section including at least one neuron and a second neural network section including at least one neuron, wherein each of the at least one neuron of the first neural network section includes a weight and a bias, and wherein the at least one neuron in the second neural network section include only weights. ### Claim 4 (depends on 1) The computing device of claim 1, wherein the approximation activation function comprises a plurality of segments including a segment of a first-order function. ### Claim 5 (depends on 1) The computing device of claim 1, wherein the approximation activation function comprises a plurality of segments, wherein a number of the plurality of the segments corresponds to hardware configuration of a subcircuit in the neural processor circuit programmable to store the approximation activation function, and wherein the hardware configuration is related to a number of comparators in the subcircuit. ### Claim 6 (depends on 1) The computing device of claim 1, wherein the second artificial neural network includes at least one neuron, and wherein the approximation activation function includes a plurality of segments, each of the segments separated by a breaking point of an output of the at least one neuron. ### Claim 7 (depends on 1) The computing device of claim 1, wherein the second artificial neural network includes a plurality of neurons, and wherein at least one of outputs of the plurality of neurons is pruned according to hardware information of a subcircuit in the neural processor circuit programmed to store the approximation activation function. ### Claim 8 (depends on 1) The computing device of claim 1, wherein the second artificial neural network includes a plurality of neurons, and wherein a number of the plurality of neurons is less than or equal to a number of comparators in a subcircuit in the neural processor circuit programmed to store the approximation activation function. ### Claim 9 (depends on 1) The computing device of claim 1, wherein the approximation activation function is defined by one or more slopes and one or more offsets. ### Claim 10 (depends on 9) The computing device of claim 9, wherein the instructions further cause the one or more processors to program the one or more slopes and the one or more offsets into a lookup table in the neural processor circuit. ### Claim 11 An activation function programming method comprising: receiving a target activation function of a first artificial neural network for implementing in a neural processor circuit; executing a second artificial neural network to determine an approximation activation function that approximates the target activation function and is defined by one or more slopes and one or more offsets that are programmable in a lookup table of the neural processor circuit, wherein the second artificial neural network is trained to determine one or more parameters of the approximation activation function to reduce an error with respect to the target activation function; and causing the one or more slopes and the one or more offsets to be programmed into the lookup table of the neural processor circuit to implement the first artificial neural network. ### Claim 12 (depends on 11) The activation function programming method of claim 11, wherein the second artificial neural network includes a first neural network section comprising at least one neuron and a second neural network section comprising at least one neuron, wherein a rectified linear unit (ReLU) function is applied to outputs of the at least one neuron of the first neural network section, and wherein a value to which the ReLU function is applied is an input to the at least one neuron of the second neural network section. ### Claim 13 (depends on 11) The activation function programming method of claim 11, wherein the second artificial neural network comprises a first neural network section including at least one neuron and a second neural network section including at least one neuron, wherein each of the at least one neuron of the first neural network section includes a weight and a bias, and wherein the at least one neuron in the second neural network section include only weights. ### Claim 14 (depends on 11) The activation function programming method of claim 11, wherein the approximation activation function comprises a plurality of segments including a segment of a first-order function. ### Claim 15 (depends on 11) The activation function programming method of claim 11, wherein the approximation activation function comprises a plurality of segments, wherein a number of the plurality of the segments corresponds to hardware configuration of a subcircuit in the neural processor circuit programmable to store the approximation activation function, and wherein the hardware configuration is related to a number of comparators in the subcircuit. ### Claim 16 (depends on 11) The activation function programming method of claim 11, wherein the second artificial neural network includes at least one neuron, and wherein the approximation activation function includes a plurality of segments, each of the segments separated by a breaking point of an output of the at least one neuron. ### Claim 17 (depends on 11) The activation function programming method of claim 11, wherein the second artificial neural network includes a plurality of neurons, and wherein a number of the plurality of neurons is less than or equal to a number of comparators included in a subcircuit of the neural processor circuit. ### Claim 18 (depends on 11) The activation function programming method of claim 11, wherein the second artificial neural network includes a plurality of neurons, and wherein at least one of outputs of the plurality of neurons is pruned according to hardware information of a subcircuit in the neural processor circuit programmed to store the approximation activation function.