US 10,592,286 B2 · Highly available distributed queue using replicated messages

Abstract

Methods and systems for implementing a highly available distributed queue using replicated messages are disclosed. An enqueue request is received from a client at a particular queue host of a plurality of queue hosts. The enqueue request comprises a message and a replica count greater than one. One or more copies of a replication request are sent from the particular queue host to one or more additional queue hosts. The replication request comprises the message. The quantity of copies of the replication request is determined based at least in part on the replica count. An initial replica of the message is enqueued at the particular queue host. One or more additional replicas of the message are enqueued at the one or more additional queue hosts. A quantity of the one or more additional replicas is determined based at least in part on the replica count.

Claims20

Dependent claims

A system, comprising:

one or more computing devices having one or more respective hardware processors and associated memory, configured to implement a distributed queue system, wherein the distributed queue system comprises a plurality of queue hosts, the distributed queue system is configured to:
store a plurality of messages in a queue, wherein individual ones of the plurality of messages are stored in replicas on multiple ones of the plurality of queue hosts;
receive, at a particular queue host, a dequeue request from a client that specifies a message type of a message to be dequeued from the queue;
return, by the particular queue host, an acknowledgment of the dequeue request, wherein the acknowledgment includes (a) a dequeued message of the message type from the queue and (b) a list of host identifiers of all queue hosts that store a replica of the dequeued message;
and
make the dequeued message unavailable in the queue without removing the dequeued message from the queue.

3 more elements

A computer-implemented method, comprising:

performing, via a particular queue host of a plurality of queue hosts of a distributed queue system:
storing messages in a queue, wherein individual ones of the messages are stored in replicas on other ones of the plurality of queue hosts;
receiving a dequeue request from a client, wherein the dequeue request specifies a message type of a message to be dequeued from the queue;
returning an acknowledgment of the dequeue request, wherein the acknowledgment includes (a) a dequeued message of the message type from the queue and (b) a list of host identifiers for the plurality of queue hosts that store a replica of the dequeued message;
and
making the dequeued message unavailable in the queue without removing the dequeued message from the queue.

3 more elements

A non-transitory computer-readable storage medium storing program instructions that when executed on one or more processors of a particular queue host of a distributed queue system, cause the particular queue host to:

store a plurality of messages in a queue, wherein individual ones of the plurality of messages are stored in replicas on multiple ones of the plurality of queue hosts;
receive a dequeue request from a client, wherein the dequeue request specifies a message type of a message to be dequeued from the queue;
send an acknowledgment of the dequeue request, wherein the acknowledgment includes (a) a dequeued message of the message type from the queue and (b) a list of host identifiers for other queue hosts of a distributed queue system that store a replica of the dequeued message;
and
make the dequeued message unavailable in the queue without removing the dequeued message from the queue.

2 more elements

Dependencies are inferred from the claim text.

Highly available distributed queue using replicated messages

Life of this patent

Highly available distributed queue using replicated messages

Life of this patent

Abstract

Claims20