System and method for statistics-based pattern searching of compressed data and encrypted data

A method, comprising:

• receiving, at a processor, a search request including a search string;
• generating, via the processor, a search string hash value based on the search string;
• detecting, via the processor and based on the search string hash value, a hash table position of a hash table;
• when a position of a bit of an Nth output data file, the bit having a value corresponding to the hash table position, is true:
• flagging the Nth output data file as relevant to the search request, and
• transmitting a signal representing the Nth output data file to a compute device of a requestor associated with the search request;
• determining whether at least one additional output data file exists;
• and
• when at least one additional output data file exists, inspecting the at least one additional output data file to determine whether the at least one additional output data file is relevant to the search request.

A system, comprising:

• a processor;
• and
• a processor-readable memory storing instructions that, when executed by the processor, cause the processor to:
• receive a search request including a search string;
• compute a search string hash value based on the search string;
• when a position stored at a location within a Boolean filter, the location having an address equal to the search string hash value, is true:
• flag that at least a portion of a data file is relevant to the search request.

A method, comprising:

• receiving, at a processor, an input data file;
• identifying, via the processor, a plurality of patterns of the input data file, each pattern from the plurality of patterns including a predetermined number of bits;
• hashing, via the processor, each pattern from the plurality of patterns into a hash value, to produce a plurality of hash values;
• storing the plurality of hash values in a hash table, each record of a plurality of records of the hash table including a hash value from the plurality of hash values and an associated position value, to produce a output data file associated with the input data file;
• and
• generating, via the processor, a Boolean filter based on the hash table, each bit from a plurality of bits of the Boolean filter associated with a different hash value from the plurality of hash values.

A system, comprising:

• a processor;
• and
• a processor-readable memory storing instructions that, when executed by the processor, cause the processor to:
• receive an input data file;
• identify a plurality of patterns of the input data file, each pattern from the plurality of patterns including a predetermined number of bits;
• hash each pattern from the plurality of patterns into a hash value, to produce a plurality of hash values;
• store the plurality of hash values in a hash table, each record of a plurality of records of the hash table including a hash value from the plurality of hash values and an associated position value, to produce an output data file associated with the input data file;
• and
• generate a Boolean filter based on the hash table, each bit from a plurality of bits of the Boolean filter associated with a different hash value from the plurality of hash values.