US 2026/0246829 A1 · LAYERED INGRESS SHARDING

Abstract

Layered ingress sharding for multi-tenant services outperforms current sharding techniques, to enhance the reliability and scalability of services. A sharding controller assigns clients and service instances to shards in each of multiple layers. Assignments differ among the layers, at least for clients and may also for service instances. This minimizes adverse effects on clients assigned to a shard with a noisy neighbor, because there are other layers (with a high probability) in which they are not sharing a shard with that noisy neighbor. The sharding controller monitors service instance health and available capacity, which indicates shard health and capacity. Client requests are routed to healthy shards, where retries will eventually find a healthy service instance or, in some examples, requests are routed directly to healthy service instances, eliminating the need for a retry.

Claims20

Dependent claims

A system comprising:

a processor;
and
a computer-readable medium storing instructions that are operative upon execution by the processor to:
assign, by a sharding controller, within each layer of a plurality of layers, two or more service instances of a plurality of service instances to each shard of a plurality of shards, wherein each layer of the plurality of layers has a separate plurality of shards;
assign, within each layer, each client of a plurality of clients to one or more shards of the plurality of shards within that layer, wherein assignments of clients to shards differs for each separate plurality of shards across the plurality of layers;
monitor a health of each service instance within each shard of the pluralities of shards;
based on at least the monitoring of the service instances, identify healthy shards of the pluralities of shards;
receive a first request from a first client of the plurality of clients;
and
route the first request to a first healthy shard assigned to the first client.

7 more elements

A computer-implemented method comprising:

assigning, by a sharding controller, within each layer of a plurality of layers, two or more service instances of a plurality of service instances to each shard of a plurality of shards, wherein each layer of the plurality of layers has a separate plurality of shards;
assigning, within each layer, each client of a plurality of clients to one or more shards of the plurality of shards within that layer, wherein assignments of clients to shards differs for each separate plurality of shards across the plurality of layers;
monitoring a health of each service instance within each shard of the pluralities of shards;
based on at least the monitoring of the service instances, identifying healthy shards of the pluralities of shards;
receiving a first request from a first client of the plurality of clients;
and
routing the first request to a first healthy shard assigned to the first client.

4 more elements

A computer storage device having computer-executable instructions stored thereon, which, on execution by a computer, cause the computer to perform operations comprising:

assigning, by a sharding controller, within each layer of a plurality of layers, two or more service instances of a plurality of service instances to each shard of a plurality of shards, wherein each layer of the plurality of layers has a separate plurality of shards;
assigning, within each layer, each client of a plurality of clients to one or more shards of the plurality of shards within that layer, wherein assignments of clients to shards differs for each separate plurality of shards across the plurality of layers;
monitoring a health of each service instance within each shard of the pluralities of shards;
based on at least the monitoring of the service instances, identifying healthy shards of the pluralities of shards;
receiving a first request from a first client of the plurality of clients;
and
routing the first request to a first healthy shard assigned to the first client.

4 more elements

Dependencies are inferred from the claim text.

Layered ingress sharding

Life of this patent

Layered ingress sharding

Life of this patent

Abstract

Claims20