US 2026/0246828 A1 · INGRESS SHARDING

Abstract

The disclosed ingress sharding for multi-tenant services outperforms current sharding techniques, to enhance the reliability and scalability of services. A sharding controller efficiently routes client requests across shards/partitions to improve fault isolation, reduce impact across clients, and distribute loads more evenly. Faults may be isolated within individual shards, and hotspots are reduced to enhance overall system performance. Some examples provide enhanced routing guidance for client requests to reduce reliance on client retry behavior. The sharding controller monitors service instance health and available capacity, which indicates shard health and capacity. Client requests are routed to healthy shards, where retries will eventually find a healthy service instance or, in some examples, requests are routed directly to healthy service instances, eliminating the need for a retry. The underlying sharding arrangement is leveraged to provide well-behaved clients a path to a healthy service instance, whereas the noisy client remains isolated in the affected shard(s).

Claims20

Dependent claims

A system comprising:

a processor;
and
a computer-readable medium storing instructions that are operative upon execution by the processor to:
monitor, by a sharding controller, a health of each service instance within each shard of a plurality of shards, wherein each shard comprises two service instances of a plurality of service instances;
based on at least the monitoring, identify healthy service instances of the plurality of service instances, wherein a service instance is healthy if the service instance meets responsiveness criteria, and wherein a service instance is not healthy if the service instance is not able to meet the responsiveness criteria;
based on at least the identification of healthy service instances, identify healthy shards of the plurality of shards, wherein a shard is healthy if the shard has at least one healthy service instance, and wherein a shard is not healthy if the shard does not have at least one healthy service instance;
receive a first request from a first client of a plurality of clients;
and
based on at least the identification of the healthy shards, route the first request to a first healthy shard assigned to the first client.

6 more elements

A computer-implemented method comprising:

monitoring, by a sharding controller, a health of each service instance within each shard of a plurality of shards, wherein each shard comprises two service instances of a plurality of service instances;
based on at least the monitoring, identifying healthy service instances of the plurality of service instances, wherein a service instance is healthy if the service instance meets responsiveness criteria, and wherein a service instance is not healthy if the service instance is not able to meet the responsiveness criteria;
based on at least the identification of healthy service instances, identifying healthy shards of the plurality of shards, wherein a shard is healthy if the shard has at least one healthy service instance, and wherein a shard is not healthy if the shard does not have at least one healthy service instance;
receiving a first request from a first client of a plurality of clients;
and
based on at least the identification of the healthy shards, routing the first request to a first healthy shard assigned to the first client.

3 more elements

A computer storage device having computer-executable instructions stored thereon, which, on execution by a computer, cause the computer to perform operations comprising:

monitoring, by a sharding controller, a health of each service instance within each shard of a plurality of shards, wherein each shard comprises two service instances of a plurality of service instances;
based on at least the monitoring, identifying healthy service instances of the plurality of service instances, wherein a service instance is healthy if the service instance meets responsiveness criteria, and wherein a service instance is not healthy if the service instance is not able to meet the responsiveness criteria;
based on at least the identification of healthy service instances, identifying healthy shards of the plurality of shards, wherein a shard is healthy if the shard has at least one healthy service instance, and wherein a shard is not healthy if the shard does not have at least one healthy service instance;
receiving a first request from a first client of a plurality of clients;
and
based on at least the identification of the healthy shards, routing the first request to a first healthy shard assigned to the first client.

3 more elements

Dependencies are inferred from the claim text.

Ingress sharding

Life of this patent

Ingress sharding

Life of this patent

Abstract

Claims20