Generating probabilistic data structures for lookup tables in computer memory for multi-token searching

A system comprising:

• one or more non-transitory computer readable media comprising a digital data repository of digital documents;
• and
• processing hardware configured to cause the system to:
• determine a probabilistic data structure comprising a bit vector with sets of bit values mapped to a plurality of items in a lookup list, the plurality of items including one or more multi-token items;
• determine a maximum number of tokens of the one or more multi-token items in the lookup list;
• determine, for a selected token from text content in a digital document, a set of sequential tokens including the selected token based on the maximum number of tokens;
• generate classifications for the text content in a digital document of the digital data repository by iteratively:
• comparing the set of sequential tokens to the sets of bit values mapped to the plurality of items in the lookup list;
• and
• reducing a number of tokens in the set of sequential tokens for a subsequent comparison;
• and
• cause a computing device to modify the digital document or a digital data repository operation of the digital data repository based on the classifications of the text content in the digital document.

A non-transitory computer readable medium comprising instructions that, when executed by processing hardware, cause the processing hardware to perform operations comprising:

• determining a probabilistic data structure comprising a bit vector with sets of bit values mapped to a plurality of items in a lookup list, the plurality of items including one or more multi-token items;
• determining a maximum number of tokens of the one or more multi-token items in the lookup list;
• determining, for a selected token from text content in a digital document, a set of sequential tokens including the selected token based on the maximum number of tokens;
• generating, for the selected token in the digital document, a plurality of classifications for the text content in the digital document by comparing the set of sequential tokens to the sets of bit values of the probabilistic data structure and iteratively reducing a number of tokens in the set of sequential tokens for comparing to the sets of bit values;
• and
• providing, for display within a graphical user interface of a client device, indications of the plurality of classifications of the text content in the digital document relative to the plurality of items in the lookup list.