# US 2023/0267395 A1 · Scaling a rest cluster using task processing logic

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2023/0267395 A1
- Document ID: US20230267395A1
- Status: Published application
- Kind: A1
- Application number: 18/114,013
- Filed: 2023-02-24
- Published: 2023-08-24
- Earliest claimed: 2022-02-24
- Assignees: Nuix Limited
- Applicants: Nuix Limited
- Inventors: Daniel Berry; Bruce Brown; Frank Marrone
- Web page: https://patentgrep.com/patent/US20230267395A1

## CPC

- G06F 9/4881: {Scheduling strategies for dispatcher, e.g. round robin, multi-level priority queues}
- G06F 9/5038: {considering the execution order of a plurality of tasks, e.g. taking priority or time dependency constraints into consideration (scheduling strategies G06F9/4881 and subgroups)}
- G06F 9/3891: {organised in groups of units sharing resources, e.g. clusters}
- G06Q 10/06316: {Sequencing of tasks or work}
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G06Q: INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR

## Abstract

Illustrative embodiments process large amounts of unstructured data by producing, from the unstructured data, a plurality of work items. Each work item includes an item of unstructured data, and corresponding work item metadata. Each work item may be claimed by a corresponding consumer node. In illustrative embodiments, a consumer node claims a work item only when the work item is compatible with the consumer node. The consumer node then processes the claimed work item by performing a specified task on the item of unstructured data associated with the work item.

## Claims

### Claim 1

A computer-implemented system for processing a plurality of data items, the system comprising:
a plurality of consumer nodes, each consumer nodes of the plurality of consumer nodes comprising corresponding node status data, wherein each consumer node of the plurality of consumer nodes is specialized to perform an operation selected from the plurality of operations, and is configured to:
access a work item queue, the work item queue storing a set of work items, each work item comprising a plurality of work items, each work item of the plurality of work items comprising (i) a data item from the plurality of data items and (ii) work item metadata corresponding to said data item;
identify, using a filter chain, a work item from the plurality of work items that is compatible with the consumer node; and
claim said work item corresponding to the consumer node.

### Claim 2 (depends on 1)

The computer-implemented system of claim 1, wherein:
the work item metadata of each work item includes an operation tag specifying an operation to be performed on the data item of the work item, and the node status data of each consumer node specified an operation for which the consumer node is specialized, and
each consumer node is configured to identify a work item that is compatible with the consumer node by determining that the operation to be performed on the data item of the work item, as specified by the work item operation tag, matches the operation for which the consumer node is specialized as specified in the node status data.

### Claim 3 (depends on 1)

The computer-implemented system of claim 1, wherein:
the work item metadata of each work item includes a status tag recording the status of whether the work item has been previously claimed by a consumer node from the plurality of consumer nodes; and
each consumer node is configured to assess the status tag of a work item and to refrain from claiming any work item having a status tag indicating that the work item has been previously claimed by a consumer node from the plurality of consumer nodes.

### Claim 4 (depends on 1)

The computer-implemented system of claim 1, wherein each consumer node is configured to determine that it is stressed, and consequently to refrain from claiming any work item.

### Claim 5 (depends on 1)

The computer-implemented system of claim 1, wherein each consumer node is configured to determine that the consumer node lacks access to a work item, and consequently to refrain from claiming the work item.

### Claim 6 (depends on 1)

The computer-implemented system of claim 1, wherein each consumer node of the plurality of consumer nodes is configured to assess work items from the work item queue to determine, for each such work item, whether said consumer node is specialized to perform the operation specified by the operation tag of the work item.

### Claim 7 (depends on 1)

The computer-implemented system of claim 1, further comprising:
a set of producer nodes, each such producer node configured to:
receive a plurality of data items, the plurality of data items comprising a set of data items, and a plurality of operation tags, the plurality of operation tags comprising a set of operation tags, each operation tag of the set of operation tags specifying an operation selected from a set of operations, each operation tag uniquely associated with a corresponding data item from the set of data items;
create the plurality of work items, each work item comprising a data item from the set of data items bound to the operation tag uniquely associated with said data item;
populate the work item queue with the set of work items; and
expose the work item queue to the plurality of consumer nodes, each consumer node of the plurality of consumer nodes configured to selectively claim one or more work items from the work item queue.

### Claim 8

A computer-implemented method of distributing a plurality of data items among a plurality of consumer nodes, each consumer node of the plurality of consumer nodes having corresponding node status data, the method comprising, for each consumer node of the plurality of consumer node:
accessing a work item queue, wherein the work item queue comprises a plurality of work items, each work item of the plurality of work items comprising (i) a data item from the plurality of data items and (ii) work item metadata corresponding to said data item, said work item metadata comprising at least an operation tag specifying an operation to be performed on said data item;
identifying from the plurality of work items in the work item queue a work item that is compatible with the consumer node;
claiming said work item; and
performing the operation on the data item of the work item.

### Claim 9 (depends on 8)

The computer-implemented method of claim 8, wherein identifying a work item that is compatible with the consumer node comprises identifying said work item by application of a filter chain.

### Claim 10 (depends on 9)

The computer-implemented method of claim 9, wherein the filter chain comprises a plurality of true/false conditions.

### Claim 11 (depends on 10)

The computer-implemented method of claim 10, wherein a consumer node declines to claim a work item of any item of the filter chain analysis returns FALSE.

### Claim 12 (depends on 9)

The computer-implemented method of claim 9, wherein the filter chain comprises:
a case exists filter that assess whether the consumer node has access to the work item.

### Claim 13 (depends on 9)

The computer-implemented method of claim 9, wherein the filter chain comprises:
a task status filter that assesses a status tag to determine whether a given task has already been claimed by another consumer node.

### Claim 14 (depends on 9)

The computer-implemented method of claim 9, wherein the filter chain comprises:
a node tag filter that assesses a consumer node tag and a work item's operation tag to determine whether the consumer node is specialized to perform the operation specified by the work item's operation tag.

### Claim 15 (depends on 9)

The computer-implemented method of claim 9, wherein the filter chain comprises:
a capacity filter that assesses whether the consumer node is not stressed.

### Claim 16

A non-transitory computer-readable medium storing computer-executable code thereon, the code comprising:
code for accessing a work item queue, wherein the work item queue comprises a plurality of work items, each work item of the plurality of work items comprising (i) a data item from the plurality of data items and (ii) work item metadata corresponding to said data item, said work item metadata comprising at least an operation tag specifying an operation to be performed on said data item;
code for identifying from the plurality of work items in the work item queue a work item that is compatible with the consumer node;
code for claiming said work item; and
code for performing the operation on the data item of the work item.

### Claim 17 (depends on 16)

The non-transitory computer-readable medium of claim 16, wherein code for identifying from the plurality of work items in the work item queue a work item that is compatible with the consumer node comprises code for identifying said work item by application of a filter chain.

### Claim 18 (depends on 17)

The non-transitory computer-readable medium of claim 17, wherein the filter chain comprises a plurality of true/false conditions.

### Claim 19 (depends on 18)

The non-transitory computer-readable medium of claim 18, wherein the filter chain comprises:
a case exists filter that assess whether the consumer node has access to the work item; and
a task status filter that assesses a status tag to determine whether a given task has already been claimed by another consumer node.

### Claim 20 (depends on 18)

The non-transitory computer-readable medium of claim 18, wherein the filter chain comprises:
a node tag filter that assesses a consumer node tag and a work item's operation tag to determine whether the consumer node is specialized to perform the operation specified by the work item's operation tag; and
a capacity filter that assesses whether the consumer node is not stressed.
