# US 10,768,983 B2 · Mechanism for facilitating a quorum-based coordination of broker health for management of resources for application servers in an on-demand services environment

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 10,768,983 B2
- Document ID: US10768983B2
- Status: Granted
- Kind: B2
- Application number: 13/841,588
- Filed: 2013-03-15
- Published: 2014-03-13
- Granted: 2020-09-08
- Earliest claimed: 2012-09-12
- Assignees: salesforce.com, inc.
- Applicants: salesforce.com, inc.
- Inventors: Xiaodan Wang; John Buisson; Vijayanth Devadhar; Praveen Murugesan
- Web page: https://patentgrep.com/patent/US10768983B2

## CPC

- G06Q 30/08: Auctions
- G06F 9/50: Allocation of resources, e.g. of the central processing unit [CPU]
- G06F 9/5027: {the resource being a machine, e.g. CPUs, Servers, Terminals}
- G06F 21/40: by quorum, i.e. whereby two or more security principals are required
- H04L 41/30: {Decision processes by autonomous network management units using voting and bidding}
- H04L 47/50: Queue scheduling
- H04L 47/83: based on usage prediction
- G06F 2209/508: Monitor
- G06Q: INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- H04L: TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION (arrangements common to telegraphic and telephonic communication H04M)

## Abstract

In accordance with embodiments, there are provided mechanisms and methods for facilitating a quorum-based coordination of broker health for management of resources for application servers in an on-demand services environment. In one embodiment and by way of example, a method includes monitoring, via health checkers, health of a cluster of brokers in a distributed environment having application servers in communication over a network, receiving an indication from at least one health checker that a broker is failing, wherein the broker is associated with a cluster of worker nodes, collecting health status reports relating to the broker from the health checkers, examining the health status reports based on a quorum-based voting policy, and classifying the broker as unhealthy if, based on the quorum-based voting policy, a percentage of the health status reports regards the broker as failed, wherein the percentage is greater than a first threshold percentage.

## Claims

### Claim 1

A database system-implemented method comprising:
monitoring, by a health checker of the database system, resource distribution health of message queue brokers managing resource domains having worker nodes and thread resources for processing job requests, wherein managing includes routing of the job requests to job queues associated with application servers in a multi-tenant resource distribution environment and in communication over a network, wherein each message queue broker is associated with at least one resource domain having a set of working nodes and a set of thread resources;
generating, by the health checker of the database system, a status report associated with a first message queue broker based on quorum-based votes casted by other message queue brokers, wherein the status report to indicate fair or unfair management of a first resource domain by the first message queue broker, wherein each message queue broker participates in a quorum-based voting system to vote to rate one or more of the message queue brokers on their management of one or more of the resource domains; and
classifying, based on based votes, the first message queue broker as one of successful, failed and recoverable, and failed and unrecoverable,
wherein the first message queue broker when classified as successful, the first message queue broker remains active such that a set of job requests distributed to the first resource domain associated with the first message queue broker is routed to one or more of the job queues without interruption,
wherein the first message queue broker when classified as failed and recoverable, the first message queue broker is recovered from failure and turned active to resume routing of the set of job requests, and
wherein the first message queue broker when classified as failed and unrecoverable, the first message queue broker is retired, and the set of job requests is distributed to a second message queue broker managing a second resource domain.

### Claim 2 (depends on 1)

The method of claim 1, wherein the message queue broker is classified as failed and recoverable if, based on the quorum-based votes, a percentage of failed queues is less than a first threshold percentage and greater than a second threshold percentage, wherein the first message queue broker serves as a master broker, while the other message queue brokers are peer brokers.

### Claim 3 (depends on 2)

The method of claim 2, wherein the message queue broker is classified as successful if, based on the quorum-based votes, the percentage is lower than the second threshold percentage.

### Claim 4 (depends on 1)

The method of claim 1, wherein the job requests are held in a temporary cache or temporarily routed to another message queue broker, if the message queue broker is classified as failed and recoverable.

### Claim 5

A database system comprising:
a health-checker server device having a memory to store instructions, and a processing device to execute the instructions, the health-checker server device to perform operations comprising:
monitoring resource distribution health of message queue brokers managing resource domains having worker nodes and thread resources for processing job requests, wherein managing includes routing of the job requests to job queues associated with application servers in a multi-tenant resource distribution environment and in communication over a network, wherein each message queue broker is associated with at least one resource domain having a set of working nodes and a set of thread resources;
generating, by the health checker of the database system, a status report associated with a first message queue broker based on quorum-based votes casted by other message queue brokers, wherein the status report to indicate fair or unfair management of a first resource domain by the first message queue broker, wherein each message queue broker participates in a quorum-based voting system to vote to rate one or more of the message queue brokers on their management of one or more of the resource domains; and
classifying, based on the quorum-based votes, the first message queue broker as one of successful, failed and recoverable, and failed and unrecoverable,
wherein the first message queue broker when classified as successful, the first message queue broker remains active such that a set of job requests distributed to the first resource domain associated with the first message queue broker is routed to one or more of the job queues without interruption
wherein the first message queue broker when classified as failed and recoverable, the first message queue broker is recovered from failure and turned active to resume routing of the set of job requests, and
wherein the first message queue broker when classified as failed and unrecoverable, the first message queue broker is retired, and the set of job requests is distributed to a second message queue broker managing a second resource domain.

### Claim 6 (depends on 5)

The database system of claim 5, wherein the message queue broker is classified as failed and recoverable if, based on the quorum-based votes, a percentage of failed queues is less than a first threshold percentage and greater than a second threshold percentage, wherein the first message queue broker serves as a master broker, while the other message queue brokers are peer brokers.

### Claim 7 (depends on 6)

The database system of claim 6, wherein the message queue broker is classified as successful if, based on the quorum-based votes, the percentage is lower than the second threshold percentage.

### Claim 8 (depends on 5)

The database system of claim 5, wherein the job requests are held in a temporary cache or temporarily routed to another message queue broker, if the message queue broker is classified as failed and recoverable.

### Claim 9

At least one non-transitory machine-readable medium having stored thereon instructions which, when executed by a health-checker processing device, causes the health-checker processing device to perform operations comprising:
monitoring resource distribution health of message queue brokers managing resource domains having worker nodes and thread resources for processing job requests, wherein managing includes routing of the job requests to job queues associated with application servers in a multi-tenant resource distribution environment and in communication over a network, wherein each message queue broker is associated with at least one resource domain having a set of working nodes and a set of thread resources;
generating, by the health checker of the database system, a status report associated with a first message queue broker based on quorum-based votes casted by other message queue brokers, wherein the status report to indicate fair or unfair management of a first resource domain by the first message queue broker, wherein each message queue broker participates in a quorum-based voting system to vote to rate one or more of the message queue brokers on their management of one or more of the resource domains; and
classifying, based on the quorum-based votes, the first message queue broker as one of successful, failed and recoverable, and failed and unrecoverable,
wherein the first message queue broker when classified as successful, the first message queue broker remains active such that a set of job requests distributed to the first resource domain associated with the first message queue broker is routed to one or more of the job queues without interruption,
wherein the first message queue broker when classified as failed and recoverable, the first message queue broker is recovered from failure and turned active to resume routing of the set of job requests, and
wherein the first message queue broker when classified as failed and unrecoverable, the first message queue broker is retired, and the set of job requests is distributed to a second message queue broker managing a second resource domain.

### Claim 10 (depends on 9)

The non-transitory machine-readable medium of claim 9, wherein the message queue broker is classified as failed and recoverable if, based on the quorum-based votes, a percentage of failed queues is less than a first threshold percentage and greater than a second threshold percentage, wherein the first message queue broker serves as a master broker, while the other message queue brokers are peer brokers.

### Claim 11 (depends on 10)

The non-transitory machine-readable medium of claim 10, wherein the message queue broker is classified as successful if, based on the quorum-based votes, the percentage is lower than the second threshold percentage.

### Claim 12 (depends on 9)

The non-transitory machine-readable medium of claim 9, wherein the job requests are held in a temporary cache or temporarily routed to another message queue broker, if the message queue broker is classified as failed and recoverable.