# US 12,602,384 B2 · Methods for enhancing rapid data analysis > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 12,602,384 B2 - Document ID: US12602384B2 - Status: Granted - Kind: B2 - Application number: 18/642,424 - Filed: 2024-04-22 - Published: 2024-08-15 - Granted: 2026-04-14 - Earliest claimed: 2015-02-12 - Assignees: Scuba Analytics, Inc. - Applicants: Scuba Analytics, Inc. - Inventors: Robert Johnson; Oleksandr Barykin; Alex Suhan; Lior Abraham; Don Fossgreen - Web page: https://patentgrep.com/patent/US12602384B2 ## CPC - G06F 16/24554: {Unary operations; Data partitioning operations} - G06F 16/278: {Data partitioning, e.g. horizontal or vertical partitioning} - H04L 63/1425: {Traffic logging, e.g. anomaly detection} - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) - H04L: TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION (arrangements common to telegraphic and telephonic communication H04M) ## Abstract A method for enhancing rapid data analysis includes receiving a set of data; storing the set of data in a first set of data shards sharded by a first field; and identifying anomalous data from the set of data by monitoring a range of shard indices associated with a first shard of the first set of data shards, detecting that the range of shard indices is smaller than an expected range by a threshold value, and identifying data of the first shard as anomalous data. ## Claims ### Claim 1 A system, comprising: a database, wherein data in the database is partitioned into a plurality of data shards; and a processing subsystem configured to: receive a query; identify a set of relevant data shards from the plurality of data shards containing data relevant to the query; determine a data sample, by: selecting a subset of the set of relevant data shards according to a set of sampling parameters; and collecting the data sample from the subset of the set of relevant data shards; and calculate a result based on an analysis of the data sample. ### Claim 2 (depends on 1) The system of claim 1, wherein the subset of the set of relevant data shards comprises a representative sample of a broader population. ### Claim 3 (depends on 1) The system of claim 1, wherein each data shard in the plurality of data shards comprises data from an intersection of a subset of rows in the data in the database and a subset of columns of the data in the database. ### Claim 4 (depends on 1) The system of claim 1, further comprising a distributed computing system comprising a plurality of data nodes storing the plurality of data shards. ### Claim 5 (depends on 4) The system of claim 4, wherein the plurality of data shards are distributed across the plurality of data nodes such that each data node stores data representative of a broader population. ### Claim 6 (depends on 1) The system of claim 1, wherein the processing subsystem is further configured to determine a confidence parameter for an accuracy of the result, wherein the set of sampling parameters are determined based on a target confidence parameter. ### Claim 7 (depends on 1) The system of claim 1, wherein the result comprises an intermediate calculation for a first pass, wherein the processing subsystem is configured to perform a multipass method comprising at least the first pass and a final pass, the final pass comprising: determining a final data sample based on the intermediate calculation and a final set of sampling parameters; and determining a final result based on the final data sample, wherein the processing system is configured to return the final result. ### Claim 8 (depends on 7) The system of claim 7, wherein the final set of sampling parameters is different from the set of sampling parameters. ### Claim 9 (depends on 1) The system of claim 1, wherein the processing subsystem is further configured to: identify anomalous data in the database, wherein the analysis of the data sample is based on weights assigned to the anomalous data and non-anomalous data, wherein the weights are different for the anomalous data and the non-anomalous data. ### Claim 10 (depends on 1) The system of claim 1, wherein the processing subsystem is further configured to: identify data shards in the plurality of data shards containing anomalous data based on an analysis of a sharding structure of the plurality of data shards; and flag the anomalous data with metadata, wherein the data sample is determined based on the metadata. ### Claim 11 A system, comprising: a processing subsystem configured to: partition a dataset into a plurality of data shards, wherein each data shard of the plurality of data shards stores a representative sample of a population; compress each data shard of the plurality of data shards; receive a query; collect a data sample from a subset of the plurality of data shards; and calculate a result based on an analysis of the data sample. ### Claim 12 (depends on 11) The system of claim 11, wherein the processing subsystem is configured to partition the dataset into the plurality of data shards using vertical partitioning and horizontal partitioning. ### Claim 13 (depends on 12) The system of claim 12, wherein vertical partitioning comprises partitioning the dataset into vertical partitions, wherein each vertical partition comprises data for a subset of vertical fields of the dataset, wherein horizontal partitioning comprises partitioning each vertical partition into a predetermined number of horizontal partitions. ### Claim 14 (depends on 13) The system of claim 13, wherein each subset of vertical fields comprises a time field. ### Claim 15 (depends on 11) The system of claim 11, wherein the processing subsystem is further configured to: identify data shards of the plurality of data shards containing anomalous data based on an analysis of a sharding structure of the plurality of data shards; and flag the anomalous data with metadata, wherein the data sample is collected based on the metadata. ### Claim 16 (depends on 15) The system of claim 15, wherein analyzing the sharding structure of the plurality of data shards comprises identifying shard keys that are statistical outliers based on a number of entries for each shard key. ### Claim 17 (depends on 11) The system of claim 11, wherein the processing subsystem is further configured to: identify anomalous data in the dataset base, wherein the analysis of the data sample is based on weights assigned to the anomalous data and non-anomalous data, wherein the weights are different for the anomalous data and the non-anomalous data. ### Claim 18 (depends on 11) The system of claim 11, further comprising a distributed computing system comprising a plurality of nodes, wherein the processing subsystem is further configured to encode the plurality of data shards, and store the encoded plurality of data shards across the plurality of nodes. ### Claim 19 (depends on 11) The system of claim 11, wherein the query is determined by a user using a graphical user interface. ### Claim 20 (depends on 11) The system of claim 11, wherein the processing subsystem is further configured to return the result of the query.