US 12,664,413 B2Granted

Methods for an AI accelerator integrated circuit chip with integrated cell-based fabric adapter

AssigneeTensordyne, Inc.

InventorsGary S. Goldman, Ramalingam K. Anand, Kalyana S. Venkataraman, Berend Ozceri, Pradeep R. Joginipally, Chung Y. Lau, Jigar K. Savla, Ashwin Radhakrishnan, Michael Davie, Shijun Li

Life of this patent

Estimating the term…

patentgrep

Loading patent…

Abstract

An integrated circuit formed on (i) a single semiconductor die or (ii) a plurality semiconductor dies that are integrated into a single package. The integrated circuit may include a communication interface including a serializer/deserializer (SerDes) interface; a fabric adapter communicatively coupled to the communication interface; a plurality of inference engine clusters, each inference engine cluster including a respective memory element and/or memory interface; and a data interconnect communicatively coupling each respective memory element and/or memory interfaces of the plurality of inference engine clusters to the fabric adapter. The fabric adapter may be configured to facilitate remote direct memory access (RDMA) read and write services and/or datagram communication over a cell-based switch fabric to and from the respective memory elements and/or memory interfaces of the plurality of inference engine clusters via the data interconnect.

Claims12

Dependent claims

A method for transmitting data from a first artificial intelligence (AI) chip to a second AI chip over a cell-based switch fabric, the first AI chip being a leaf node of a Clos network and serving as a source endpoint, the first AI chip comprising a virtual output queue (VOQ) subsystem with a plurality of VOQs,

wherein the first AI chip comprises:
a communication interface;
a fabric adapter communicatively coupled to the communication interface, wherein the fabric adapter includes the VOQ subsystem;
an inference engine cluster including a memory element or a memory interface;
and
a data interconnect communicatively coupling the memory element or the memory interface of the inference engine cluster to the fabric adapter,
the method comprising:
dividing, by the first AI chip, the data into packets;
and
for each of the packets:
determining, by the first AI chip, a respective one of the VOQs to assign to the packet;
attaching, by the first AI chip, a packet header to the packet;
enqueuing, by the first AI chip, the packet to the respective one of the VOQs assigned to the packet;
and
when selected for transport to the second AI chip, dequeuing, by the first AI chip, the packet from the respective one of the VOQs, cellifying, by the first AI chip, the packet, attaching, by the first AI chip, a cell header to each cell to indicate a destination OQ on the second AI chip corresponding to the assigned VOQ, and transmitting, by the first AI chip as the source endpoint, the cells to one or more cell-fabric switch chips.

12 more elements

A method for receiving data at a first artificial intelligence (AI) chip comprising an output queue (OQ) subsystem with a plurality of output queues (OQs), the first AI chip being a leaf node of a Clos network and serving as a destination endpoint, the OQs being associated with a VOQ subsystem on a second AI chip, wherein the first AI chip comprises:

a communication interface;
a fabric adapter communicatively coupled to the communication interface, wherein the fabric adapter includes the OQ subsystem;
an inference engine cluster including a memory element or a memory interface;
and
a data interconnect communicatively coupling the memory element or the memory interface of the inference engine cluster to the fabric adapter,
the method comprising:
receiving, by the first AI chip, a plurality of packets from the second AI chip via one or more cell-fabric switch chips;
enqueueing, by the first AI chip, the plurality of packets to the OQs;
dequeuing, by the first AI chip, one or more of the packets from the OQs according to a scheduling policy based on a class of service assigned to each of the OQs;
and
for each of the packets,
decellifying, by the first AI chip as the destination endpoint, respective cells of the packet to reconstitute packet data;
examining, by the first AI chip, a header of the packet to identify the packet data and determine its appropriate processing;
removing, by the first AI chip, the header of the packet;
and
storing, by the first AI chip, the packet data to the memory element or communicating the packet data to the memory interface.

13 more elements

Dependencies are inferred from the claim text.

Facts

Application: 18/989,525Filed 20 Dec 2024 · us provisional application
Priority: 13 Sep 2024
Published: US 2026/0079868 A119 Mar 2026
Granted: 23 Jun 2026
Term ends: Not estimatedThe source records don’t support an estimate.

Classification9 codes

G06N 3/063
Physical realisation, i.e. hardware implementation of neural networks, neurons or parts of neurons: using electronic means
G06F 12/1081
Address translation: for peripheral access to main memory, e.g. direct memory access [DMA]
G06F 13/28
For access to input/output bus: using burst mode transfer, e.g. direct memory access DMA, cycle steal
G06F 13/4068
Device-to-bus coupling: electrical coupling