Neural network circuitry having approximate multiplier units

#The present disclosure generally relates to artificial intelligence technology.

#Among the numerous neural network (NN) models, recurrent neural networks (RNN), which are distinguished by the presence of feedback connections, have been shown to be much better suited than feed-forward NNs, such as convolutional neural networks (CNNs), for many sequence labeling tasks in the field of machine learning. RNNs are designed to capture the temporal dependencies within data sequences and have been shown to learn the long-term trends and patterns inherent in sequences. To alleviate the vanishing gradient problem in standard RNNs and to be able to learn the patterns over a larger number of time steps, more advanced RNN models, such as gated recurrent units (GRUs) and long short-term memory (LSTM) have been developed. The LSTM model has been shown to be highly robust and accurate for many applications involving time series data, including natural language processing and video analysis. The LSTM model is now used in virtual assistant user interfaces such as Apple Siri, Amazon Alexa, and Google Assistant. Such applications are typically launched on mobile devices, but due to their compute-intensive nature, they are executed on cloud servers. With the emergence of the Internet of Things and the further proliferation of mobile devices, this approach is not scalable, and hence there is a need to move some or all of the NN computations to energy-constrained, performance-limited mobile devices. This poses difficult challenges associated with simultaneously achieving high energy efficiency and high throughput. These challenges are due to the recursive structure of the LSTM model and the compute-intensive operations on very large dimensional data as well as the high memory-bandwidth requirement for computing on a large number of parameters. The present disclosure relates to achievement of high energy efficiency by employing low power and compact computation units and aggressively maximizing the overall throughput.

#Disclosed is neural network circuitry having a first plurality of logic cells in which the logic cells are interconnected to form neural network computation units that are configured to perform approximate computations. The neural network circuitry further includes a second plurality of logic cells in which the logic cells are interconnected to form a controller hierarchy that is interfaced with the neural network computation units to control pipelining of the approximate computations with all other computations, all performed by the neural network computational units. In some embodiments the neural network computation units include approximate multipliers that are configured to perform approximate multiplications that comprise the approximate computations. The approximate multipliers include preprocessing units that reduce latency while maintaining accuracy. In some exemplary embodiments, the neural network circuitry is long short-term memory circuitry.

#The accompanying drawing figures incorporated in and forming a part of this specification illustrate several aspects of the disclosure and, together with the description, serve to explain the principles of the disclosure.

#FIG. 1 is a diagram showing a structure of a related-art long short-term memory (LSTM) layer.

#FIG. 2 is a diagram showing the structure of a related-art 4-bit signed approximate multiplier (AM).

#The embodiments set forth below represent the necessary information to enable those skilled in the art to practice the embodiments and illustrate the best mode of practicing the embodiments. Upon reading the following description in light of the accompanying drawing figures, those skilled in the art will understand the concepts of the disclosure and will recognize applications of these concepts not particularly addressed herein. It should be understood that these concepts and applications fall within the scope of the disclosure and the accompanying claims.

#It will be understood that, although the terms first, second, etc. may be used herein to describe various elements, these elements should not be limited by these terms. These terms are only used to distinguish one element from another. For example, a first element could be termed a second element, and, similarly, a second element could be termed a first element, without departing from the scope of the present disclosure. As used herein, the term “and/or” includes any and all combinations of one or more of the associated listed items.

#It will be understood that when an element such as a layer, region, or substrate is referred to as being “on” or extending “onto” another element, it can be directly on or extend directly onto the other element or intervening elements may also be present. In contrast, when an element is referred to as being “directly on” or extending “directly onto” another element, there are no intervening elements present. Likewise, it will be understood that when an element such as a layer, region, or substrate is referred to as being “over” or extending “over” another element, it can be directly over or extend directly over the other element or intervening elements may also be present. In contrast, when an element is referred to as being “directly over” or extending “directly over” another element, there are no intervening elements present. It will also be understood that when an element is referred to as being “connected” or “coupled” to another element, it can be directly connected or coupled to the other element or intervening elements may be present. In contrast, when an element is referred to as being “directly connected” or “directly coupled” to another element, there are no intervening elements present.