Multi-dimensional range-index searching using search box approximation and splitting

A system comprising:

• a distributed data storage network comprising a plurality of data storage nodes, wherein a multi-dimensional dataset comprising a plurality of data objects is spread out over more than one of the plurality of data storage nodes;
• a processor operatively connected to the distributed data storage network, the processor configured to:
• receive, from a user, a user query for data contained within a multi-dimensional dataset, wherein the query for data comprises a user query for overlaps of data objects within the multi-dimensional dataset and a user-selected value wherein the user-selected value is a range of values;
• generate a multi-dimensional search box for use in searching the multi-dimensional dataset according to the user query, wherein the size of the multi-dimensional search box is approximated based upon a percentage of the scale of the dataset and upon the standard deviation of the data object distribution of the dataset, wherein the multi-dimensional search box is generated in part by modeling the range of values as an arbitrary start point within the multi-dimensional data set with extents that are Gaussian distributed with a standard deviation less than the scale of the dataset;
• search data from the multi-dimensional dataset using the multi-dimensional search box, wherein the processor is further configured to iteratively splitting the multi-dimensional search box along a longest identified bit boundary so long as PFANobjects≥Sobject≥nsplitsSData, wherein PFA is a probability of a false alarm, Nobjects is the number of data objects in the multi-dimensional dataset, Sobject is the average size of a data object, nsplits is the current number of splits of the multi-dimensional search box, and SData is the average size of a data dump given the structure of the multi dimensional dataset;
• and
• return, to the user, data from the multi-dimensional dataset contained within the multi-dimensional search box.