US 10,705,985 B1Granted

Integrated circuit with rate limiting

AssigneeAmazon Technologies, Inc.

InventorsBenny Pollak, Dana Michelle Vantrease, Adi Habusha

Life of this patent

Estimating the term…

Loading patent…

Abstract

In various implementations, provided are systems and methods for an integrated circuit implementing a processor that can include a rate limiting circuit that attempts to fairly distribute processor memory bandwidth between transaction generators in the processor. The rate limiting circuit can maintain a count of tokens for each transaction generator, where a transaction generator can only transmit a transaction when the transaction generator has enough tokens to do so. Each transaction generator can send a request to the rate limiting circuit when the transaction generator wants to transmit a transaction. The rate limiting circuit can then check whether the transaction generator has sufficient tokens to transmit the transaction. When the transaction generator has enough tokens, the rate limiting circuit will allow the transaction to enter the interconnect. When the transaction generator does not have enough tokens, the rate limiting circuit will not allow the transaction to enter the interconnect.

Claims20

Dependent claims

An integrated circuit for a processor, comprising:

a channel to a processor memory;
transaction generators, wherein each transaction generator is an integrated circuit component that is operable to generate transactions to the processor memory;
an interconnect coupled to the channel and the transaction generators, wherein the interconnect enables transactions to be transferred from the transaction generators to the channel and responses from processor memory to be transferred from the channel to the transaction generators, and wherein the interconnect includes a cache memory operable to store data recently accessed by the transaction generators;
and
a rate limiting circuit configured to:
maintain a separate count of tokens for each transaction generator of the transaction generators, wherein tokens are added to the separate count of tokens for each transaction generator at a pre-determined interval;
upon receiving, from a transaction generator, a request to transmit a transaction, determine that a count of tokens associated with the transaction generator is greater than zero and thus sufficient to allow the transaction to enter the interconnect;
allow the transaction to enter the interconnect;
and
deduct a token from the separate count of tokens for the transaction generator,
wherein the rate limiting circuit is further configured to:
determine to add a token to a first count of tokens associated with a first transaction generator from a first set of transaction generators;
determine, before adding the token, that the first count of tokens associated with the first transaction generator is equal to a maximum value;
determine that a second count of tokens associated with a second transaction generator from the first set of transaction generators is less than the maximum value;
and
add the token to the second count of tokens associated with the second transaction generator.

13 more elements

An integrated circuit, comprising:

a channel to a processor memory;
transaction generators, wherein each transaction generator comprises a circuit that is operable to generate transactions to the processor memory;
an interconnect coupled to the channel and the transaction generators, wherein the interconnect enables data to be exchanged between the transaction generators and the channel, and wherein the interconnect includes a cache memory for the transaction generators;
and
a rate limiting circuit configured to:
upon receiving, from one of the transaction generators, a request to transmit a transaction, determine whether a count of tokens associated with the one of the transaction generators is greater than or equal to a token amount assigned to the transaction;
allow the transaction to enter the interconnect when the count of tokens associated with the one of the transaction generators is greater than or equal to the token amount assigned to the transaction;
and
deduct the token amount from the count of tokens associated with the one of the transaction generators when the transaction is allowed to enter the interconnect,
wherein the rate limiting circuit is further configured to:
determine to add a token to a first count of tokens associated with a first transaction generator from a first set of transaction generators;
determine, before adding the token, that the first count of tokens associated with the first transaction generator is equal to a maximum value;
determine that a second count of tokens associated with a second transaction generator from the first set of transaction generators is less than the maximum value;
and
add the token to the second count of tokens associated with the second transaction generator.

12 more elements

A method, comprising:

receiving, by a rate limiting circuit of an integrated circuit, a request to transmit a transaction to processor memory, wherein the transaction is received from a first transaction generator from a plurality of transaction generators of the integrated circuit;
determining that a count of tokens associated with the first transaction generator is greater than or equal to an amount assigned to each transaction;
allowing the transaction to enter an interconnect of the integrated circuit based on the count of tokens associated with the first transaction generator being greater than or equal to the amount, wherein the interconnect enables data to be transferred between the plurality of transaction generators and a channel of the integrated circuit, the channel providing an interface to processor memory, and wherein the interconnect includes a cache memory operable to store data recently accessed by the plurality of transaction generators;
and
deducting a quantity of tokens from the count of tokens associated with the first transaction generator, wherein the quantity of tokens is equal to the amount assigned to each transaction,
wherein the method further comprises:
determining, by the rate limiting circuit, to add a token to a first count of tokens associated with the first transaction generator;
determining, before adding the token, that the first count of tokens associated with the first transaction generator is equal to a maximum value;
determining that a second count of tokens associated with a second transaction generator from the plurality of transaction generators is less than the maximum value;
and
adding the token to the second count of tokens associated with the second transaction generator.

8 more elements

Dependencies are inferred from the claim text.