# US 2022/0045955 A1 · Local and global quality of service shaper on ingress in a distributed system

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2022/0045955 A1
- Document ID: US20220045955A1
- Status: Published application
- Kind: A1
- Application number: 16/988,212
- Filed: 2020-08-07
- Published: 2022-02-10
- Earliest claimed: 2020-08-07
- Assignees: Hyannis Port Research, Inc.
- Applicants: Hyannis Port Research, Inc.
- Inventors: Anthony D. Amicangioli; Allen Bast; B. Joshua Rosen
- Web page: https://patentgrep.com/patent/US20220045955A1

## CPC

- G06Q 40/04: Trading; Exchange, e.g. stocks, commodities, derivatives or currency exchange
- G06Q 30/08: Auctions
- H04L 47/10: Flow control; Congestion control
- H04L 47/122: by diverting traffic away from congested entities
- H04L 47/193: at the transport layer, e.g. TCP related
- H04L 47/22: Traffic shaping
- H04L 47/6225: {Fixed service order, e.g. Round Robin}
- H04L 47/215: using token-bucket
- H04L 47/27: Evaluation or update of window size, e.g. using information derived from acknowledged [ACK] packets
- H04L 47/6245: {Modifications to standard FIFO or LIFO}
- G06Q: INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- H04L: TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION (arrangements common to telegraphic and telephonic communication H04M)

## Abstract

A distributed computing system, such as may be used to implement an electronic trading system, controls inbound message flow rates. Limiting a per-client or per-connection inbound message rate also helps ensure fair provisioning of computing resources, so that a single client's excessive use of resources cannot overwhelm the system to such an extent that it prevents other clients from interacting with the distributed system. It is also desirable to have system-wide control of the overall inbound message rate across all client connections. Such system-wide control ensures that the distributed system as a whole can maintain the required levels of service, including offering a predictable level of access for all clients.

## Claims

### Claim 1

A method of operating a distributed data processing system to control inbound flow of messages from a plurality of gateway nodes to a plurality of compute nodes and to a system-level node, wherein the distributed data processing system is an electronic trading system, and the messages are electronic trading messages, the method comprising:
at each of the plurality of gateway nodes:
receiving the messages over one or more client connections;
controlling a sustained flow rate and/or a burst flow rate of the messages on a per-client or per-connection basis; and
forwarding the messages to the system-level node;
at the system-level node:
receiving the messages from each of the plurality of gateway nodes;
controlling a system-wide message flow rate, wherein controlling the system-wide message flow rate further comprises controlling a sustained flow rate and/or a burst flow rate on a per-gateway-node-basis for each of the plurality of gateway nodes; and
forwarding the messages to the compute nodes;
at each of the plurality of compute nodes:
receiving the messages from the system-level node; and
operating on the messages to perform an electronic trading function;
at a selected one of the compute nodes:
generating a response message; and
returning the response message to a selected one of the gateway nodes; and
at the selected gateway node:
returning the response message over at least one of the one or more client connections.

### Claim 2 (depends on 1)

The method of claim 1 wherein
the messages are application layer messages, in which multiple application layer messages are contained in a lower layer protocol packet; and
wherein at least one of the steps of controlling the sustained flow rate and/or burst flow rate further comprises providing feedback to the lower layer protocol.

### Claim 3 (depends on 2)

The method of claim 2 wherein the lower layer protocol is a transport layer protocol, and the feedback is provided by controlling a transport layer window size.

### Claim 4 (depends on 1)

The method of claim 1 wherein the system-level node is a sequencer node, and wherein the method further comprises:
at one or more of the gateway nodes:
forwarding the messages to the sequencer node;
at each of the compute nodes:
receiving the messages from the sequencer node;
and wherein operating on the messages to perform an electronic trading function further comprises operating on the messages received from the one or more gateway nodes and/or operating on the messages received from the sequencer node.

### Claim 5 (depends on 1)

The method of claim 1 wherein at least one of the steps of controlling sustained flow rate and/or burst flow rate further comprises:
queuing the messages with a plurality of queues;
feeding the messages from the queues to a plurality of token buckets; and
selecting the messages from token buckets.

### Claim 6 (depends on 5)

The method of claim 5 wherein the queues are FIFOs.

### Claim 7 (depends on 5)

The method of claim 5 wherein the selecting is on a round-robin basis.

### Claim 8 (depends on 1)

The method of claim 1 wherein the sustained flow rate and/or burst flow rate is further controlled on the per-client or per-connection basis in response to a client request.

### Claim 9 (depends on 1)

The method of claim 1 wherein the system-wide message flow rate is further controlled by providing feedback to one or more of the gateway nodes.

### Claim 10 (depends on 9)

The method of claim 9 where the step of providing feedback further comprises at least one of:
lowering a TCP window size on a per-connection basis for all connections from the system-level node to the one or more of the gateway nodes, or
lowering the sustained flow rate and/or the burst flow rate by adjusting at least one parameter of a per-connection token bucket for all connections from the system-level node to the one or more of the gateway nodes, or
sending a feedback message to the one or more of the gateway nodes.

### Claim 11 (depends on 1)

The method of claim 1 wherein the system-wide message flow rate is further controlled by pausing one or more of the gateway nodes.

### Claim 12 (depends on 11)

The method of claim 11 wherein the step of pausing one or more of the gateway nodes further comprises at least one of:
setting a TCP window size to zero for a least one client connection on the one or more of the gateway nodes,
not adding new messages to at least one per-connection FIFO queue for the one or more of the gateway nodes,
not servicing messages from at least one per-connection FIFO queue for the one or more of the gateway nodes, or
setting at least one of the sustained flow rate and/or burst flow rate for at least one connection on the one or more gateway nodes to zero.

### Claim 13 (depends on 1)

The method of claim 1 wherein controlling the sustained flow rate and/or burst flow rate on the per-client or per-connection basis further comprises:
controlling a flow from one of the gateway nodes to a selected client and/or connection.

### Claim 14 (depends on 1)

The method of claim 1 where the step of controlling the sustained flow rate and/or burst flow rate further comprises at least one of:
lowering a TCP window size on a per-client or per-connection basis; or
lowering the sustained flow rate and/or burst flow rate by adjusting at least one parameter of a per-client or per-connection token bucket.

### Claim 15 (depends on 1)

The method of claim 1 wherein the sustained flow rate and/or burst flow rate on the per-client or per-connection basis is further controlled on the per-client or per-connection basis by pausing at least one client or connection.

### Claim 16 (depends on 15)

The method of claim 15 wherein the step of pausing at least one client or connection further comprises at least one of:
setting a TCP window size to zero on a per-client or per-connection basis,
not adding new messages to a per-client or a per-connection FIFO for at least one of the gateway nodes,
not servicing messages from a per-client or a per-connection FIFO queue, or
setting at least one of the sustained flow rate or burst flow rate for at least one client or connection to zero.

### Claim 17 (depends on 1)

The method of claim 1 additionally comprising:
receiving the messages at the system-level node from the gateway nodes via a full mesh set of point to point direct connections.

### Claim 18 (depends on 1)

The method of claim 1 additionally comprising:
at the system-level node:
receiving the messages from the compute nodes; and
slowing a rate at which the messages are received from the gateway nodes, when the messages from the compute nodes are received at greater than a predetermined rate.

### Claim 19 (depends on 1)

The method of claim 1 additionally comprising:
at each of the compute nodes:
slowing a rate at which the messages are received from the gateway nodes, when the messages received from or to be sent from the compute nodes to the gateway nodes exceed a rate greater than a predetermined rate.

### Claim 20

An electronic trading system comprising:
a plurality of gateway nodes configured to:
receive messages over one or more client connections, wherein the messages are electronic trading messages;
control a sustained flow rate and/or a burst flow rate of the messages on a per-client or per-connection basis; and
forward the messages to a system-level node;
the system-level node configured to:
receive the messages from each of the gateway nodes;
control a system-wide message flow rate for the system, and further to control a sustained flow rate and/or a burst flow rate on a per-gateway-node-basis for each of the gateway nodes; and
forward the messages to one or more compute nodes;
the one or more of the compute nodes configured to:
receive the messages from the system-level node; and
operate on the messages to perform an electronic trading function;
a selected one of the compute nodes further configured to:
generate a response message; and
return the response message to a selected one of the gateway nodes; and
the selected one of the gateway nodes further configured to:
return the response message over at least one of the one or more client connections.

### Claim 21

(canceled)

### Claim 22

(canceled)

### Claim 23 (depends on 1)

The method of claim 1 further comprising:
wherein controlling the system-wide message flow rate further comprises sending flow control messages from the system level node to the plurality of gateway nodes.

### Claim 24 (depends on 1)

The method of claim 1 wherein
forwarding the messages to the system-level node further comprises forwarding the messages using a layered protocol; and
receiving the messages from each of the plurality of gateway nodes further comprises receiving the messages using a layered protocol.

### Claim 25 (depends on 1)

The method of claim 1 wherein
forwarding the messages to the compute nodes further comprises forwarding the messages using a layered protocol; and
receiving the messages from the system level node further comprises receiving the messages using a layered protocol.
