3D hybrid bonding 3D memory devices with NPU/CPU for AI inference application

An artificial intelligence (AI) inference method for applying weight data from a high-bandwidth memory to neural network computations, the method comprising:

• retrieving, by an AI processor element of a plurality of AI processor elements arrayed on a logic die, a plurality of neural network weights stored in a corresponding 3D memory cell of a plurality of 3D memory cells arrayed on a memory die;
• wherein the AI processor element is directly connected to the corresponding 3D memory cell by a plurality of metal-to-metal hybrid bonds formed between metal contacts placed upon contacting surfaces of each of the logic die and the memory die;
• applying, using an accelerator core coupled to the AI processor element, the plurality of neural network weights to input data for one or more nodes of a plurality of nodes of a neural network to obtain an intermediate output;
• retrieving activation data from an activation memory directly connected with the AI processor element on the logic die;
• applying the retrieved activation data to the intermediate output to obtain a result for a neural network layer;
• and
• storing the result in at least one of the corresponding 3D memory cell, a field programmable gate array (FPGA) buffer, and the activation memory, thereby facilitating computations at additional neural network layers until a final result is reached.