Probabilistic bitmaps for efficient data comparison

A non-transitory computer-readable medium storing instructions that, when executed by at least one processor, cause the at least one processor to:

• access a dictionary table shared across a plurality of systems, the dictionary table mapping each of a plurality of transaction attribute names to a unique bit position, the dictionary table defining a determined order of the plurality of transaction attribute names that is consistent across the plurality of systems;
• generate a probabilistic bitmap for transaction data processed by a system of the plurality of systems by:
• applying a hash function to an attribute value of each transaction attribute to produce a hashed output that is deterministic for a given attribute value;
• applying a Mod function to the hashed output for each transaction attribute to generate a bit value mapped to a bit position specified by the dictionary table for that transaction attribute;
• and
• organizing bit values in a series in the determined order defined by the dictionary table;
• and
• transmit the probabilistic bitmap for comparison with one or more other probabilistic bitmaps generated by one or more other systems of the plurality of systems processing related transaction data, the comparison detecting data inconsistencies between the system and the one or more other systems without direct transmission of the transaction data.

A networked system for transaction data comparison, the system comprising:

• a network interface communicatively coupled to a first component system and a second component system, each of the first component system and the second component system configured to process transaction data;
• one or more processors;
• and
• one or more non-transitory computer-readable media storing instructions that, when executed by the one or more processors, cause the networked system to:
• receive, via the network interface, transaction attributes of transaction data from the first component system;
• receive, via the network interface, transaction attributes of related transaction data from the second component system;
• generate a first probabilistic structure for the first component system based on the transaction attributes received from the first component system, wherein the first probabilistic structure represents the transaction attributes in a determined order, wherein generating the first probabilistic structure comprises:
• hashing attribute values of the transaction attributes received from the first component system;
• performing a Mod function on results of the hashing of the attribute values to generate bits representative of the attribute values;
• and
• organizing the bits in a series based on the determined order of the transaction attributes;
• generate a second probabilistic structure for the second component system based on the transaction attributes received from the second component system, wherein generating the second probabilistic structure comprises:
• hashing attribute values of the transaction attributes received from the second component system;
• performing a Mod function on results of the hashing of the attribute values to generate bits representative of the attribute values;
• and
• organizing the bits in a series based on the determined order of the transaction attributes;
• compare the first probabilistic structure with the second probabilistic structure;
• and
• detect inconsistencies between the first component system and the second component system based on the comparison.