# US 2026/0093699 A1 · System, Method, and Computer Program Product for Dynamic Optimization of Complex Queries in Data Lakehouse Query Engines

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2026/0093699 A1
- Document ID: US20260093699A1
- Status: Published application
- Kind: A1
- Application number: 18/901,231
- Filed: 2024-09-30
- Published: 2026-04-02
- Earliest claimed: 2024-09-30
- Assignees: Visa International Service Association
- Applicants: Visa International Service Association
- Inventors: Udit Kumar; Madhumita Mallick; Gunjan Singh; Mahadev Khapali
- Web page: https://patentgrep.com/patent/US20260093699A1

## CPC

- G06F 16/24545: {Selectivity estimation or determination}
- G06F 16/24526: {Internal representations for queries}
- G06F 16/2471: {Distributed queries}
- G06F 16/285: {Clustering or classification}
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)

## Abstract

Systems, methods, and computer program products for dynamic optimization of complex SQL queries in data lakehouse query engines are provided. An example system may include at least one processor configured to receive data associated with operations of computational resources of a computer cluster receive data associated with a query to be carried out by the computer cluster during a time interval, provide an input to a cluster load machine learning model to generate an output of the cluster load machine learning model, where the output comprises a prediction of load on the computer cluster during the time interval, determine to adjust at least one session parameter associated with the query, and adjust the at least one session parameter associated with the query based on determining to adjust the at least one session parameter associated with the query.

## Claims

### Claim 1

A system, comprising:
at least one processor configured to:
receive data associated with operations of computational resources of a computer cluster;
receive data associated with a query to be carried out by the computer cluster during a time interval;
provide an input to a cluster load machine learning model to generate an output of the cluster load machine learning model, wherein the output comprises a prediction of load on the computer cluster required to carry out operations during the time interval;
determine whether to adjust at least one session parameter associated with the query based on an output of the cluster load machine learning model, the data associated with computational resources of the computer cluster, and the data associated with a query to be carried out by the computer cluster;
adjust the at least one session parameter associated with the query based on determining to adjust the at least one session parameter associated with the query; and
forego adjusting the at least one session parameter associated with the query based on determining not to adjust the at least one session parameter associated with the query.

### Claim 2 (depends on 1)

The system of claim 1, wherein the at least one session parameter associated with the query comprises at least one of the following:
a parameter associated with task concurrency for the computer cluster;
a parameter associated with resource overcommit for the computer cluster;
a parameter associated with a maximum hash partition count;
a parameter associated with a join distribution type of the query;
a parameter associated with an execution policy for the query; or
any combination thereof.

### Claim 3 (depends on 1)

The system of claim 1, wherein the at least one processor is further configured to:
train the cluster load machine learning model based on historical time-series data associated with operations of computational resources of the computer cluster.

### Claim 4 (depends on 3)

The system of claim 3, wherein the historical time-series data associated with operations of computational resources of the computer cluster comprises a plurality of timestamps associated with a plurality of features, and wherein the plurality of features associated with each timestamp comprises at least one of the following:
a feature associated with a number of concurrent queries for the computer cluster;
a feature associated with an amount of memory available for the computer cluster;
a feature associated with an amount of central processing unit resource available for the computer cluster;
a feature associated with a number of threads for the computer cluster;
a feature associated with an amount of information transmitted in a network associated with the computer cluster; or
any combination thereof.

### Claim 5 (depends on 1)

The system of claim 1, wherein, when receiving the data associated with operations of computational resources of the computer cluster, the at least one processor is configured to:
receive real time data associated with operations of computational resources of the computer cluster; and
wherein, when providing the input to a cluster load machine learning model to generate the output of the cluster load machine learning model, the at least one processor is configured to:
provide the real time data associated with operations of computational resources of the computer cluster as the input to a cluster load machine learning model to generate the output of the cluster load machine learning model;
wherein the at least one processor is further configured to:
generate the output of the cluster load machine learning model, wherein the output comprises a prediction of real time load on the computer cluster required to carry out operations during the time interval for the query.

### Claim 6 (depends on 1)

The system of claim 1, wherein, when adjusting the at least one session parameter associated with the query, the at least one processor is configured to:
generate an adjusted query based on adjusting the at least one session parameter associated with the query; and
wherein the at least one processor is further configured to:
perform the adjusted query on a data lakehouse based on generating the adjusted query.

### Claim 7 (depends on 1)

The system of claim 1, wherein, when receiving the data associated with the query to be carried out by the computer cluster during the time interval, the at least one processor is configured to:
receive data associated with a Structured Query Language (SQL) query to be carried out by the computer cluster during the time interval; and
wherein, when determining whether to adjust the at least one session parameter associated with the query, the at least one processor is configured to:
determine to adjust at least one session parameter associated with the SQL query.

### Claim 8

A computer-implemented method, comprising:
receiving, with at least one processor, data associated with operations of computational resources of a computer cluster;
receiving, with at least one processor, data associated with a query to be carried out by the computer cluster during a time interval;
providing, with at least one processor, an input to a cluster load machine learning model to generate an output of the cluster load machine learning model, wherein the output comprises a prediction of load on the computer cluster required to carry out operations during the time interval;
determining, with at least one processor, to adjust at least one session parameter associated with the query based on an output of the cluster load machine learning model, the data associated with computational resources of the computer cluster, and the data associated with a query to be carried out by the computer cluster; and
adjusting, with at least one processor, the at least one session parameter associated with the query based on determining to adjust the at least one session parameter associated with the query.

### Claim 9 (depends on 8)

The computer-implemented method of claim 8, wherein the at least one session parameter associated with the query comprises at least one of the following:
a parameter associated with task concurrency for the computer cluster;
a parameter associated with resource overcommit for the computer cluster;
a parameter associated with a maximum hash partition count;
a parameter associated with a join distribution type of the query;
a parameter associated with an execution policy for the query; or
any combination thereof.

### Claim 10 (depends on 8)

The computer-implemented method of claim 8, further comprising:
training the cluster load machine learning model based on historical time-series data associated with operations of computational resources of the computer cluster.

### Claim 11 (depends on 10)

The computer-implemented method of claim 10, wherein the historical time-series data associated with operations of computational resources of the computer cluster comprises a plurality of timestamps associated with a plurality of features, and wherein the plurality of features associated with each timestamp comprises at least one of the following:
a feature associated with a number of concurrent queries for the computer cluster;
a feature associated with an amount of memory available for the computer cluster;
a feature associated with an amount of central processing unit resource available for the computer cluster;
a feature associated with a number of threads for the computer cluster;
a feature associated with an amount of information transmitted in a network associated with the computer cluster; or
any combination thereof.

### Claim 12 (depends on 8)

The computer-implemented method of claim 8, wherein receiving the data associated with operations of computational resources of the computer cluster comprises:
receiving real time data associated with operations of computational resources of the computer cluster; and
wherein providing the input to a cluster load machine learning model to generate the output of the cluster load machine learning model comprises:
providing the real time data associated with operations of computational resources of the computer cluster as the input to a cluster load machine learning model to generate the output of the cluster load machine learning model;
wherein the method further comprises:
generating the output of the cluster load machine learning model, wherein the output comprises a prediction of real time load on the computer cluster required to carry out operations during the time interval for the query.

### Claim 13 (depends on 8)

The computer-implemented method of claim 8, wherein adjusting the at least one session parameter associated with the query comprises:
generating an adjusted query based on adjusting the at least one session parameter associated with the query; and
wherein the method further comprising:
performing the adjusted query on a data lakehouse based on generating the adjusted query.

### Claim 14 (depends on 8)

The computer-implemented method of claim 8, wherein receiving the data associated with the query to be carried out by the computer cluster during the time interval comprises:
receiving data associated with a Structured Query Language (SQL) query to be carried out by the computer cluster during the time interval; and
wherein determining whether to adjust the at least one session parameter associated with the query comprises:
determining to adjust at least one session parameter associated with the SQL query.

### Claim 15

A computer program product, comprising at least one non-transitory computer-readable medium including program instructions that, when executed by at least one processor, cause the at least one processor to:
receive data associated with operations of computational resources of a computer cluster;
receive data associated with a query to be carried out by the computer cluster during a time interval;
provide an input to a cluster load machine learning model to generate an output of the cluster load machine learning model, wherein the output comprises a prediction of load on the computer cluster required to carry out operations during the time interval;
determine whether to adjust at least one session parameter associated with the query based on an output of the cluster load machine learning model, the data associated with computational resources of the computer cluster, and the data associated with a query to be carried out by the computer cluster;
adjust the at least one session parameter associated with the query based on determining to adjust the at least one session parameter associated with the query; and
forego adjusting the at least one session parameter associated with the query based on determining not to adjust the at least one session parameter associated with the query.

### Claim 16 (depends on 15)

The computer program product of claim 15, wherein the at least one session parameter associated with the query comprises at least one of the following:
a parameter associated with task concurrency for the computer cluster;
a parameter associated with resource overcommit for the computer cluster;
a parameter associated with a maximum hash partition count;
a parameter associated with a join distribution type of the query;
a parameter associated with an execution policy for the query; or
any combination thereof.

### Claim 17 (depends on 15)

The computer program product of claim 15, wherein the program instructions further cause the at least one processor to:
train the cluster load machine learning model based on historical time-series data associated with operations of computational resources of the computer cluster.

### Claim 18 (depends on 17)

The computer program product of claim 17, wherein the historical time-series data associated with operations of computational resources of the computer cluster comprises a plurality of timestamps associated with a plurality of features, and wherein the plurality of features associated with each timestamp comprises at least one of the following:
a feature associated with a number of concurrent queries for the computer cluster;
a feature associated with an amount of memory available for the computer cluster;
a feature associated with an amount of central processing unit resource available for the computer cluster;
a feature associated with a number of threads for the computer cluster;
a feature associated with an amount of information transmitted in a network associated with the computer cluster; or
any combination thereof.

### Claim 19 (depends on 15)

The computer program product of claim 15, wherein, the program instructions that cause the at least one processor to receive the data associated with operations of computational resources of the computer cluster, cause the at least one processor to:
receive real time data associated with operations of computational resources of the computer cluster; and
wherein, the program instructions that cause the at least one processor to provide the input to a cluster load machine learning model to generate the output of the cluster load machine learning model, cause the at least one processor to:
provide the real time data associated with operations of computational resources of the computer cluster as the input to a cluster load machine learning model to generate the output of the cluster load machine learning model;
wherein the program instructions further cause the at least one processor to:
generate the output of the cluster load machine learning model, wherein the output comprises a prediction of real time load on the computer cluster required to carry out operations during the time interval for the query.

### Claim 20 (depends on 15)

The computer program product of claim 15, wherein, the program instructions that cause the at least one processor to receive the data associated with the query to be carried out by the computer cluster during the time interval, cause the at least one processor to:
receive data associated with a Structured Query Language (SQL) query to be carried out by the computer cluster during the time interval; and
wherein, the program instructions that cause the at least one processor to determine whether to adjust the at least one session parameter associated with the query, cause the at least one processor to:
determine to adjust at least one session parameter associated with the SQL query;
wherein, the program instructions that cause the at least one processor to adjust the at least one session parameter associated with the query, cause the at least one processor to:
generate an adjusted SQL query based on adjusting the at least one session parameter associated with the SQL query; and
wherein the program instructions further cause the at least one processor to:
perform the adjusted SQL query on a data lakehouse based on generating the adjusted SQL query.
