# US 2025/0094386 A1 · Distributed query execution and aggregation including historical statistical analysis

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2025/0094386 A1
- Document ID: US20250094386A1
- Status: Published application
- Kind: A1
- Application number: 18/969,042
- Filed: 2024-12-04
- Published: 2025-03-20
- Earliest claimed: 2022-04-24
- Assignees: Morgan Stanley Services Group Inc.
- Applicants: Morgan Stanley Services Group Inc.
- Inventors: Luke HIGGINS; Robert BRUNO
- Web page: https://patentgrep.com/patent/US20250094386A1

## CPC

- G06F 16/13: File access structures, e.g. distributed indices (arrangements of input from, or output to, record carriers G06F3/06)
- G06F 16/2272: {Management thereof}
- G06F 16/24544: {Join order optimisation}
- G06F 16/24556: {Aggregation; Duplicate elimination}
- G06F 16/2456: {Join operations}
- G06F 16/2462: {Approximate or statistical queries}
- G06F 16/2471: {Distributed queries}
- G06N 20/00: Machine learning
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS

## Abstract

Computer-implemented methods and systems are disclosed for receiving and indexing a plurality of files for later querying, for dynamically generating scripts to be executed during a query of a data store, and for horizontally distributing a query and aggregating results of the distributed query.

## Claims

### Claim 1

A system for dynamically generating scripts to be executed during a query of a data store, comprising:
a server comprising one or more processors; and
non-transitory memory comprising instructions that, when executed by the one or more processors of the server, cause the one or more processors to:
receive a query comprising a time window of files to be searched, key values to search on in two or more distinct datasets, one or more output fields to track over a period of time and to generate one or more summary statistics of the one or more output fields, and one or both of a filter selecting a subset of the files to be searched and an aggregation of data from all of the files or from all of the files that are filtered;
dynamically generate a script based on contents of the query, optimizing the script from a template to include only features necessary to satisfy the query;
distribute the generated script horizontally to a plurality of computing devices that will execute the query by calling the generated script on each of the files to be searched;
store output; and
in response to receiving a second instance of the query at a later time, re-provide updated output to the query and the one or more summary statistics.

### Claim 2 (depends on 1)

The system of claim 1, wherein the comparison includes data on a relationship between output for the one or more output fields in the second instance of the query and the one or more summary statistics.

### Claim 3 (depends on 2)

The system of claim 2, wherein the one or more summary statistics include a standard deviation and wherein the relationship is a z-score for each of the one or more output fields in the second instance of the query.

### Claim 4 (depends on 2)

The system of claim 2, wherein the one or more summary statistics include a set of quantiles and wherein the relationship is which quantile each of the one or more output fields in the second instance of the query falls into.

### Claim 5 (depends on 1)

The system of claim 1, wherein one or more expected entries in a data structure storing output for the period of time is absent or not yet calculated, and wherein the one or more expected entries are generated and backfilled with default values before the one or more summary statistics are calculated.

### Claim 6 (depends on 5)

The system of claim 5, wherein an additional one or more expected entries in a data structure storing output for the period of time is absent or not yet calculated, but backfilling does not occur because a predetermined criterion for backfilling values is not met.

### Claim 7 (depends on 1)

The system of claim 1, wherein, in response to receiving a third instance of the query, whether before or after the second instance of the query, and further in response to determining that a count of one or more entries in a data structure storing output for the period of time is less than a predetermined threshold for sample size, the one or more summary statistics are not calculated.

### Claim 8 (depends on 1)

The system of claim 1, wherein one or more entries in a data structure storing output for the period of time are older than a length of the period of time, and are deleted before the one or more summary statistics are calculated.

### Claim 9 (depends on 1)

The system of claim 1, wherein entries in a data structure storing output for the period of time are more numerous than a predetermined threshold, and oldest entries are deleted before the one or more summary statistics are calculated on only a set of entries as numerous as the predetermined threshold.

### Claim 10 (depends on 1)

The system of claim 1, wherein a machine learning classifier is trained on the output and used to classify output generated after the second instance of the query.

### Claim 11

A computer-implemented method for dynamically generating scripts to be executed during a query of a data store, comprising:
receiving a query comprising a time window of files to be searched, key values to search on in two or more distinct datasets, one or more output fields to track over a period of time and to generate one or more summary statistics of the one or more output fields, and one or both of a filter selecting a subset of the files to be searched and an aggregation of data from all of the files or from all of the files that are filtered;
dynamically generating a script based on contents of the query, optimizing the script from a template to include only features necessary to satisfy the query;
distributing the generated script horizontally to a plurality of computing devices that will execute the query by calling the generated script on each of the files to be searched;
storing output; and
in response to receiving a second instance of the query at a later time, re-provide updated output to the query and the one or more summary statistics.

### Claim 12 (depends on 11)

The method of claim 11, wherein the comparison includes data on a relationship between output for the one or more output fields in the second instance of the query and the one or more summary statistics.

### Claim 13 (depends on 12)

The method of claim 12, wherein the one or more summary statistics include a standard deviation and wherein the relationship is a z-score for each of the one or more output fields in the second instance of the query.

### Claim 14 (depends on 12)

The method of claim 12, wherein the one or more summary statistics include a set of quantiles and wherein the relationship is which quantile each of the one or more output fields in the second instance of the query falls into.

### Claim 15 (depends on 11)

The method of claim 11, wherein one or more expected entries in a data structure storing output for the period of time is absent or not yet calculated, and wherein the one or more expected entries are generated and backfilled with default values before the one or more summary statistics are calculated.

### Claim 16 (depends on 15)

The method of claim 15, wherein an additional one or more expected entries in a data structure storing output for the period of time is absent or not yet calculated, but backfilling does not occur because a predetermined criterion for backfilling values is not met.

### Claim 17 (depends on 11)

The method of claim 11, wherein, in response to receiving a third instance of the query, whether before or after the second instance of the query, and further in response to determining that a count of one or more entries in a data structure storing output for the period of time is less than a predetermined threshold for sample size, the one or more summary statistics are not calculated.

### Claim 18 (depends on 11)

The method of claim 11, wherein one or more entries in a data structure storing output for the period of time are older than a length of the period of time, and are deleted before the one or more summary statistics are calculated.

### Claim 19 (depends on 11)

The method of claim 11, wherein entries in a data structure storing output for the period of time are more numerous than a predetermined threshold, and oldest entries are deleted before the one or more summary statistics are calculated on only a set of entries as numerous as the predetermined threshold.

### Claim 20 (depends on 11)

The method of claim 11, wherein a machine learning classifier is trained on the output and used to classify output generated after the second instance of the query.
