US 12,430,547 B1Granted

AI accelerator integrated circuit chip with integrated cell-based fabric adapter

AssigneeRecogni Inc.

InventorsGary S. Goldman, Ramalingam K. Anand, Kalyana S. Venkataraman, Berend Ozceri, Pradeep R. Joginipally, Chung Y. Lau, Jigar K. Savla, Ashwin Radhakrishnan, Michael Davie, Shijun Li

Life of this patent

Estimating the term…

patentgrep

Loading patent…

Abstract

An integrated circuit formed on (i) a single semiconductor die or (ii) a plurality semiconductor dies that are integrated into a single package. The integrated circuit may include a communication interface including a serializer/deserializer (SerDes) interface; a fabric adapter communicatively coupled to the communication interface; a plurality of inference engine clusters, each inference engine cluster including a respective memory element and/or memory interface; and a data interconnect communicatively coupling each respective memory element and/or memory interfaces of the plurality of inference engine clusters to the fabric adapter. The fabric adapter may be configured to facilitate remote direct memory access (RDMA) read and write services and/or datagram communication over a cell-based switch fabric to and from the respective memory elements and/or memory interfaces of the plurality of inference engine clusters via the data interconnect.

Claims19

Dependent claims

An integrated circuit formed on (i) a single semiconductor die or (ii) a plurality of semiconductor dies that are integrated into a single package, the integrated circuit comprising:

a communication interface;
a first inference engine cluster including a first memory element;
a second inference engine cluster including a second memory element;
a first fabric adapter communicatively coupled to the communication interface and the first memory element;
and
a second fabric adapter communicatively coupled to the communication interface and the second memory element,
wherein the first fabric adapter is configured to facilitate remote direct memory access (RDMA) read and write services over a cell-based switch fabric to and from the first memory element of the first inference engine cluster,
the first fabric adapter comprising logic to:
(i) segment transactions concerning the first memory element into first packets, assign the first packets to a first plurality of virtual output queues (VOQs), cellify the first packets into first cells, attach a respective cell header to each of the first cells, and transmit the first cells to the cell-based switch fabric;
and
(ii) receive second packets with second cells from the cell-based switch fabric, enqueue the second packets on a first plurality of output queues (OQs), decellify respective second cells of the second packets to reconstitute first packet data, and write the first packet data to the first memory element based on respective headers of the second packets,
wherein the second fabric adapter is configured to facilitate RDMA read and write services over the cell-based switch fabric to and from the second memory element of the second inference engine cluster,
the second fabric adapter comprising logic to:
(i) segment transactions concerning the second memory element into third packets, assign the third packets to a second plurality of virtual output queues (VOQs), cellify the third packets into third cells, attach a respective cell header to each of the third cells, and transmit the third cells to the cell-based switch fabric;
and
(ii) receive fourth packets with fourth cells from the cell-based switch fabric, enqueue the fourth packets on a second plurality of output queues (OQs), decellify respective fourth cells of the fourth packets to reconstitute second packet data, and write the second packet data to the second memory element based on respective headers of the fourth packets.

13 more elements

An integrated circuit formed on (i) a single semiconductor die or (ii) a plurality semiconductor dies that are integrated into a single package, the integrated circuit comprising:

a communication interface;
a first inference engine cluster including a first memory interface;
a second inference engine cluster including a second memory interface;
a first fabric adapter communicatively coupled to the communication interface and the first memory interface;
and
a second fabric adapter communicatively coupled to the communication interface and the second memory interface,
wherein the first fabric adapter is configured to facilitate remote direct memory access (RDMA) read and write services over a cell-based switch fabric to and from the first memory interface of the first inference engine cluster, the first fabric adapter comprising logic to:
(i) segment transactions concerning the first memory interface into first packets, assign the first packets to a first plurality of virtual output queues (VOQs), cellify the first packets into first cells, attach a respective cell header to each of the first cells, and transmit the first cells to the cell-based switch fabric;
and
(ii) receive second packets with second cells from the cell-based switch fabric, enqueue the second packets on a first plurality of output queues (OQs), decellify respective second cells of the second packets to reconstitute first packet data, and transmit the first packet data to the first memory interface based on respective headers of the second packets,
wherein the second fabric adapter is configured to facilitate RDMA read and write services over the cell-based switch fabric to and from the second memory interface of the second inference engine cluster,
the second fabric adapter comprising logic to:
(i) segment transactions concerning the second memory interface into third packets, assign the third packets to a second plurality of virtual output queues (VOQs), cellify the third packets into third cells, attach a respective cell header to each of the third cells, and transmit the third cells to the cell-based switch fabric;
and
(ii) receive fourth packets with fourth cells from the cell-based switch fabric, enqueue the fourth packets on a second plurality of output queues (OQs), decellify respective fourth cells of the fourth packets to reconstitute second packet data, and transmit the second packet data to the second memory interface based on respective headers of the fourth packets.

12 more elements

Dependencies are inferred from the claim text.

Facts

Application: 18/989,508Filed 20 Dec 2024 · us provisional application
Priority: 13 Sep 2024
Granted: 30 Sep 2025
Term ends: Not estimatedThe source records don’t support an estimate.

Classification10 codes

G06N 3/063
Physical realisation, i.e. hardware implementation of neural networks, neurons or parts of neurons: using electronic means
G06F 12/1081
Address translation: for peripheral access to main memory, e.g. direct memory access [DMA]
G06F 13/28
For access to input/output bus: using burst mode transfer, e.g. direct memory access DMA, cycle steal
G06F 13/4022
Coupling between buses: using switching circuits, e.g. switching matrix, connection or expansion network