# US 2020/0004749 A1 · Systems and methods for intelligent capture and fast transformations of granulated data summaries in database engines

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2020/0004749 A1
- Document ID: US20200004749A1
- Status: Published application
- Kind: A1
- Application number: 16/459,274
- Filed: 2019-07-01
- Published: 2020-01-02
- Earliest claimed: 2018-06-29
- Assignees: Security On-Demand, Inc.
- Applicants: Security On-Demand, Inc.
- Inventors: Dominik Slezak; Richard Glick; Pawel Betlinkski; Piotr Synak; Jakub Wroblewski; Agnieszka Chadzynska-Krasowska; Janusz Borkowski; Arkadiusz Wojna; Joel Alan Holland
- Web page: https://patentgrep.com/patent/US20200004749A1

## CPC

- G06F 16/24539: {using cached or materialised query results}
- G06F 16/2282: {Tablespace storage structures; Management thereof}
- G06F 16/2462: {Approximate or statistical queries}
- G06N 5/01: Dynamic search techniques; Heuristics; Dynamic trees; Branch-and-bound
- G06N 7/01: Probabilistic graphical models, e.g. probabilistic networks
- G06N 20/00: Machine learning
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS

## Abstract

Embodiments may provide methods and systems for intelligent capture and fast transformation of granulated data summaries. An engine may be used to transform input data summaries into result sets representing query outcomes. The data summaries contain enough knowledge about the original data to accurately perform operations on the summaries without needing to access the original data. In an embodiment, the contents of data summaries are accessible via an SQL approximate engine which retrieves summaries stored on disk and utilizes them for its operations. Alternatively, the contents of data summaries are accessible via virtual tables which give users direct access to the summary contents and allow for the creation and implementation of algorithms to work with the data summaries independently from the SQL approximate engine.

## Claims

### Claim 1

A method for capture and transformation of granulated data summaries from a data set stored in a server implemented in a computer system comprising a processor, a memory adapted to store program instructions and data, and program instructions executable by the processor to perform:
receiving, at the computer system, a query for at least one chunk of data stored in the server;
generating, at the computer system, a response to the query including a result generated from at least one data summary of the at least one chunk of data without accessing the data itself; and
transmitting, at the computer system, the generated response to the query.

### Claim 2 (depends on 1)

The method of claim 1, wherein the at least one summary of the at least one chunk of data takes the form of a relational data table with columns and rows.

### Claim 3 (depends on 1)

The method of claim 1, wherein generating the response to the query comprises generating, at the computer system, a response summary from the at least one summary of the at least one chunk of data and generating the response to the query from the response summary.

### Claim 4 (depends on 1)

The method of claim 1, wherein generating at least one data summary for the at least one chunk of data comprises:
dividing, at the computer system, the data stored in the server into a plurality of chunks;
generating, at the computer system, statistical information for at least some columns in some of the chunks of the data;
generating, at the computer system, statistical information about relationships between at least some pairs of columns of the chunks of data; and
generating, at the computer system, at least one summary based on the generated statistical information for the at least some columns and the generated statistical information about relationships between at least some pairs of columns.

### Claim 5 (depends on 1)

The method of claim 1, wherein generating at least one data summary for at least one chunk of the data comprises:
receiving, at the computer system, data including a plurality of rows of the data at the computer system;
generating, at the computer system, at least a chunk of the data including a predetermined number of received rows of the data;
generating, at the computer system, statistical information for the generated chunks of the data;
generating, at the computer system, statistical information about relationships between at least some pairs of columns of the chunks of data; and
generating, at the computer system, at least one summary based on the generated statistical information for the generated chunks of data and on the generated statistical information about relationships between the at least some pairs of columns of the chunks of data.

### Claim 6 (depends on 5)

The method of claim 5, wherein the statistical information for the generated chunks of data and the statistical information about relationships are updated using the at least one data summary that is generated as data is received by the computer system.

### Claim 7 (depends on 1)

The method of claim 1, wherein generating a response to the query comprises utilizing, at the computer system, a mechanism of tree-based belief propagation adapted to populate changes in the at least one data summary.

### Claim 8 (depends on 1)

The method of claim 1, wherein the generated response to the query is a standard SQL result.

### Claim 9 (depends on 2)

The method of claim 2, wherein the at least one summary comprises at least one histogram reflecting intensities of occurrences of values of at least one column, and at least one co-occurrence ratio reflecting intensities of co-occurrences of values of a plurality of columns.

### Claim 10 (depends on 9)

The method of claim 9, wherein the at least one histogram reflects intensities of occurrences within a quantized domain of values of the at least one column within the at least one data chunk, and wherein the at least one co-occurrence ratio reflects intensities of co-occurrences of quantized values of the plurality of columns.

### Claim 11 (depends on 10)

The method of claim 10, wherein the domain of values of the at least one column is quantized by basing on heuristic derivation of the estimated expected accuracy of the generated response to the query subject to different choices of quantization settings.

### Claim 12 (depends on 10)

The method of claim 10, wherein the at least one co-occurrence ratio does not represent information about all possible combinations of values of the plurality of columns and the co-occurrence ratios related to unrepresented combinations of values are estimated based on information about the at least one co-occurrence ratio related to combinations of the quantized values.

### Claim 13 (depends on 10)

The method of claim 10, wherein the at least one co-occurrence ratio to be represented is selected from a set of all combinations of values of the plurality of columns by basing on heuristic derivation of the estimated expected accuracy of the generated response to the query.

### Claim 14

A system for capture and transformation of granulated data summaries from a data set stored in a server, the system comprising a processor, a memory adapted to store program instructions and data, and program instructions executable by the processor to perform:
receiving a query for at least one chunk of data stored in the server;
generating a response to the query including a result generated from at least one data summary of the at least one chunk of data without accessing the data itself; and
transmitting the generated response to the query.

### Claim 15 (depends on 14)

The system of claim 14, wherein the at least one summary of the at least one chunk of data takes the form of a relational data table with columns and rows.

### Claim 16 (depends on 14)

The system of claim 14, wherein generating the response to the query comprises generating a response summary from the at least one summary of the at least one chunk of data and generating the response to the query from the response summary.

### Claim 17 (depends on 14)

The system of claim 14, wherein generating at least one data summary for the at least one chunk of data comprises:
dividing the data stored in the server into a plurality of chunks;
generating statistical information for at least some columns in some of the chunks of the data;
generating statistical information about relationships between at least some pairs of columns of the chunks of data; and
generating at least one summary based on the generated statistical information for the at least some columns and the generated statistical information about relationships between at least some pairs of columns.

### Claim 18 (depends on 14)

The system of claim 14, wherein generating at least one data summary for at least one chunk of the data comprises:
receiving data including a plurality of rows of the data at the computer system;
generating at least a chunk of the data including a predetermined number of received rows of the data;
generating statistical information for the generated chunks of the data;
generating statistical information about relationships between at least some pairs of columns of the chunks of data; and
generating at least one summary based on the generated statistical information for the generated chunks of data and on the generated statistical information about relationships between the at least some pairs of columns of the chunks of data.

### Claim 19 (depends on 18)

The system of claim 18, wherein the statistical information for the generated chunks of data and the statistical information about relationships are updated using the at least one data summary that is generated as data is received by the computer system.

### Claim 20 (depends on 14)

The system of claim 14, wherein generating a response to the query comprises utilizing a mechanism of tree-based belief propagation adapted to populate changes in the at least one data summary.

### Claim 21 (depends on 14)

The system of claim 14, wherein the generated response to the query is a standard SQL result.

### Claim 22 (depends on 15)

The system of claim 15, wherein the at least one summary comprises at least one histogram reflecting intensities of occurrences of values of at least one column, and at least one co-occurrence ratio reflecting intensities of co-occurrences of values of a plurality of columns.

### Claim 23 (depends on 22)

The system of claim 22, wherein the at least one histogram reflects intensities of occurrences within a quantized domain of values of the at least one column within the at least one data chunk, and wherein the at least one co-occurrence ratio reflects intensities of co-occurrences of quantized values of the plurality of columns.

### Claim 24 (depends on 23)

The system of claim 23, wherein the domain of values of the at least one column is quantized by basing on heuristic derivation of the estimated expected accuracy of the generated response to the query subject to different choices of quantization settings.

### Claim 25 (depends on 23)

The system of claim 23, wherein the at least one co-occurrence ratio does not represent information about all possible combinations of values of the plurality of columns and the co-occurrence ratios related to unrepresented combinations of values are estimated based on information about the at least one co-occurrence ratio related to combinations of the quantized values.

### Claim 26 (depends on 23)

The system of claim 23, wherein the at least one co-occurrence ratio to be represented is selected from a set of all combinations of values of the plurality of columns by basing on heuristic derivation of the estimated expected accuracy of the generated response to the query.

### Claim 27

A computer program product for capture and transformation of granulated data summaries from a data set stored in a server, the computer program product comprising a non-transitory computer readable storage medium having program instructions embodied therewith, the program instructions executable by a computer, to cause the computer to perform a method comprising:
receiving, at the computer system, a query for at least one chunk of data stored in the server;
generating, at the computer system, a response to the query including a result generated from at least one data summary of the at least one chunk of data without accessing the data itself; and
transmitting, at the computer system, the generated response to the query.

### Claim 28 (depends on 27)

The computer program product of claim 27, wherein the at least one summary of the at least one chunk of data takes the form of a relational data table with columns and rows.

### Claim 29 (depends on 27)

The computer program product of claim 27, wherein generating the response to the query comprises generating, at the computer system, a response summary from the at least one summary of the at least one chunk of data and generating the response to the query from the response summary.

### Claim 30 (depends on 27)

The computer program product of claim 27, wherein generating at least one data summary for the at least one chunk of data comprises:
dividing, at the computer system, the data stored in the server into a plurality of chunks;
generating, at the computer system, statistical information for at least some columns in some of the chunks of the data;
generating, at the computer system, statistical information about relationships between at least some pairs of columns of the chunks of data; and
generating, at the computer system, at least one summary based on the generated statistical information for the at least some columns and the generated statistical information about relationships between at least some pairs of columns.

### Claim 31 (depends on 27)

The computer program product of claim 27, wherein generating at least one data summary for at least one chunk of the data comprises:
receiving, at the computer system, data including a plurality of rows of the data at the computer system;
generating, at the computer system, at least a chunk of the data including a predetermined number of received rows of the data;
generating, at the computer system, statistical information for the generated chunks of the data;
generating, at the computer system, statistical information about relationships between at least some pairs of columns of the chunks of data; and
generating, at the computer system, at least one summary based on the generated statistical information for the generated chunks of data and on the generated statistical information about relationships between the at least some pairs of columns of the chunks of data.

### Claim 32 (depends on 31)

The computer program product of claim 31, wherein the statistical information for the generated chunks of data and the statistical information about relationships are updated using the at least one data summary that is generated as data is received by the computer system.

### Claim 33 (depends on 27)

The computer program product of claim 27, wherein generating a response to the query comprises utilizing, at the computer system, a mechanism of tree-based belief propagation adapted to populate changes in the at least one data summary.

### Claim 34 (depends on 27)

The computer program product of claim 27, wherein the generated response to the query is a standard SQL result.

### Claim 35 (depends on 28)

The computer program product of claim 28, wherein the at least one summary comprises at least one histogram reflecting intensities of occurrences of values of at least one column, and at least one co-occurrence ratio reflecting intensities of co-occurrences of values of a plurality of columns.

### Claim 36 (depends on 35)

The computer program product of claim 35, wherein the at least one histogram reflects intensities of occurrences within a quantized domain of values of the at least one column within the at least one data chunk, and wherein the at least one co-occurrence ratio reflects intensities of co-occurrences of quantized values of the plurality of columns.

### Claim 37 (depends on 36)

The computer program product of claim 36, wherein the domain of values of the at least one column is quantized by basing on heuristic derivation of the estimated expected accuracy of the generated response to the query subject to different choices of quantization settings.

### Claim 38 (depends on 36)

The computer program product of claim 36, wherein the at least one co-occurrence ratio does not represent information about all possible combinations of values of the plurality of columns and the co-occurrence ratios related to unrepresented combinations of values are estimated based on information about the at least one co-occurrence ratio related to combinations of the quantized values.

### Claim 39 (depends on 36)

The computer program product of claim 36, wherein the at least one co-occurrence ratio to be represented is selected from a set of all combinations of values of the plurality of columns by basing on heuristic derivation of the estimated expected accuracy of the generated response to the query.

### Claim 40

A system for capture and transformation of granulated data summaries from a data set stored in a server comprising:
a computer system comprising a processor, a memory adapted to store program instructions and data, and program instructions executable by the processor to receive a query for data stored in the computer system; and
an approximate query database engine comprising:
a knowledge capture layer comprising at least one server system; and
a knowledge transformation layer comprising at least one approximate database engine; and
at least one client system comprising a database client, wherein the at least one server system generates at least one data summary from the data and the approximate database engine utilizes the at least one data summary for its operations and generates an approximate result based on the at least one data summary in response to a query from the at least one client system.

### Claim 41 (depends on 40)

The system of claim 40, wherein the at least one data summary is accessible via virtual tables from the at least one client system.

### Claim 42 (depends on 41)

The system of claim 41, wherein the at least one client system implements algorithms on the at least one data summary independently from the approximate database engine.

### Claim 43

A system for capture and transformation of granulated data summaries from a data set stored in a server comprising:
a computer system comprising a processor, a memory adapted to store program instructions and data, and program instructions executable by the processor to receive a query for data stored in the computer system;
at least one client system;
an approximate database engine; and
an exact database engine, wherein in response to a query from the at least one client system, the exact database engine generates an exact response from the data set, the approximate database engine generates at least one data summary from the data set and utilizes the at least one data summary for its operations and generates an approximate result based on the at least one data summary in response to the query, and a confidence interval is utilized to switch between providing the approximate result and the exact result to the at least one client system.

### Claim 44

A computer program product for capture and transformation of granulated data summaries from a data set stored in a server comprising a non-transitory computer readable storage having program instructions embodied therewith, the program instructions executable by a computer to cause the computer to perform a method comprising:
receiving a query for at least one chunk of data stored in the server;
generating at least one data summary from the data set;
generating an approximate result based on the at least one data summary in response to the query;
applying a confidence interval to the approximate result; and
switching between providing the approximate result and an exact result based on the confidence interval.

### Claim 45

A method for capture and transformation of granulated data summaries from a data set stored in a server implemented in a computer system comprising a processor, a memory adapted to store program instructions and data, and program instructions executable by the processor to perform:
receiving a query for at least one chunk of data stored in the server;
generating at least one data summary from the data set;
generating an approximate result based on the at least one data summary in response to the query;
applying a confidence interval to the approximate result; and
switching between providing the approximate result and an exact result based on the confidence interval.
