Data Processor with Efficient Scalable Queuing and Method Therefor

A data processor comprising:

• a single-token-record memory having a plurality of first storage locations;
• a sequencer circuit coupled to said single-token-record memory, wherein said sequencer circuit, responsive to a request to dequeue a token from a head-end of a queue, either reads said token associated with said data packet from one of said plurality of first storage locations if said single-token-record memory stores more than a predetermined number of tokens associated with said head-end of said queue, or reads said token along with at least one additional token from one of a plurality of second storage locations and stores said additional tokens in said single-token-record memory otherwise;
• and
• a memory controller coupled to said sequencer circuit, for reading said token and said at least one additional token from said plurality of second storage locations in a multi-token-record memory.

A method for controlling queuing in a data processor, the method comprising:

• receiving a token enqueue request;
• transmitting a first token associated with said token enqueue request to a queuing mechanism;
• determining a first number of tokens in a tail-end of a queue associated with said first token stored in a single-token-record memory;
• if said first number is less than or equal to a first predetermined number, storing said first token in said single-token-record memory;
• and
• if said first number is greater than said first predetermined number, storing said first token with at least one additional token and a pointer to a next storage location in a multi-token-record memory.