Data perturbation and anonymization using one way hash

A method for anonymizing data from multiple data sources, the data sources comprising personal identifiers identifying entities associated with the data, the data being collected by a central data aggregation module connected to the data sources, said method comprising:

• replacing a personal identifier with an anonymous identifier using an anonymization engine, wherein said replacing of the personal identifier with the anonymous identifier comprises generating the first anonymous identifier with a salt and a key;
• searching a key-hash bucket for the first anonymous identifier to determine whether the anonymization engine has anonymized the personal identifier before, the key-hash bucket including:
• anonymous identifiers generated using the key, and
• the salt used with the key;
• sending the first anonymous identifier to the first data source when it is determined that the anonymization engine has anonymized the personal identifier before;
• generating new anonymous identifiers with the anonymization engine when it is determined that the anonymization engine has not anonymized the personal identifier before, the new anonymous identifiers being generated using key-salt combinations used previously by the anonymization engine;
• creating an equivalent hashes table for the personal identifier, the equivalent hashes table including the first anonymous identifier and the new anonymous identifiers;
• receiving the record identifier from a second data source, the record identifier being received by the anonymization engine;
• generating a second anonymous identifier to replace the record identifier with the anonymization engine, the second anonymous identifier being generated using a second salt and a second key, wherein both the first anonymous identifier and the second anonymous identifier are linked to the record identifier;
• searching a second key-hash bucket for the second anonymous identifier, the second key-hash bucket including:
• anonymous identifiers generated using the second key, and
• the second salt used with the second key;
• sending the second anonymous identifier to the second data source when the second anonymous identifier is found in the second key-hash bucket;
• generating second new anonymous identifiers with the anonymization engine when the second anonymous identifier is not found in the second key-hash bucket, the second new anonymous identifiers being generated using key-salt combinations used previously by the anonymization engine;
• creating a second equivalent hashes table for the record identifier, the second equivalent hashes table including the second anonymous identifier and the second new anonymous identifiers;
• and
• sending the second equivalent hashes table, the second anonymous identifier and second data associated with the second anonymous identifier to the data aggregation module.

A system for anonymizing data from multiple data sources, the data sources comprising personal identifiers identifying entities associated with the data, said system comprising:

• a processor;
• a central data aggregation module connected to the data sources, said central data aggregation module collects the data;
• an anonymization engine, said anonymization engine:
• generates a first anonymous identifier to replace the record identifier, the first anonymous identifier being generated using a salt and a key,
• searches a key-hash bucket for the first anonymous identifier to determine whether said anonymization engine has anonymized the record identifier before, the key-hash bucket including:
• anonymous identifiers generated using the key, and
• the salt used with the key,
• sends the first anonymous identifier to the first data source when it is determined that said anonymization engine has anonymized the record identifier before,
• generates new anonymous identifiers when it is determined that said anonymization engine has not anonymized the record identifier before, the new anonymous identifiers being generated using key-salt combinations used previously by said anonymization engine, and
• creates an equivalent hashes table for the record identifier, the equivalent hashes table including the first anonymous identifier and the new anonymous identifiers;
• a mapping module connected to said anonymization engine and the data aggregation module, said mapping module provides the equivalent hashes table, the first anonymous identifier and first data associated with the first anonymous identifier to the data aggregation module;
• said anonymization engine:
• receives the record identifier from a second data source;
• generates a second anonymous identifier to replace the record identifier with the anonymization engine, the second anonymous identifier being generated using a second salt and a second key, wherein both the first anonymous identifier and the second anonymous identifier are linked to the record identifier;
• searches a second key-hash bucket for the second anonymous identifier, the second key-hash bucket including:
• anonymous identifiers generated using the second key, and
• the second salt used with the second key;
• sends the second anonymous identifier to the second data source when the second anonymous identifier is found in the second key-hash bucket;
• generating second new anonymous identifiers with the anonymization engine when the second anonymous identifier is not found in the second key-hash bucket, the second new anonymous identifiers being generated using key-salt combinations used previously by the anonymization engine;
• creates a second equivalent hashes table for the record identifier, the second equivalent hashes table including the second anonymous identifier and the second new anonymous identifiers;
• and
• sends the second equivalent hashes table, the second anonymous identifier and second data associated with the second anonymous identifier to the data aggregation module.

A computer program product for anonymizing data from multiple data sources, the data sources comprising personal identifiers identifying entities associated with the data, the data being collected by a central data aggregation module connected to the data sources, said computer program product comprising:

• a non-transitory computer readable storage medium;
• first program instructions to replace a personal identifier with an anonymous identifier said first program instructions generate the first anonymous identifier with a salt and a key;
• second program instructions to search a key-hash bucket for the first anonymous identifier to determine whether an anonymization engine has anonymized the personal identifier before, the key-hash bucket including:
• anonymous identifiers generated using the key, and
• the salt used with the key;
• third program instructions to send the first anonymous identifier to the first data source when it is determined that the anonymization engine has anonymized the personal identifier before;
• fourth program instructions to generate new anonymous identifiers when it is determined that the anonymization engine has not anonymized the personal identifier before, the new anonymous identifiers being generated using key-salt combinations used previously by the anonymization engine;
• fifth program instructions to create an equivalent hashes table for the personal identifier, the equivalent hashes table including the first anonymous identifier and the new anonymous identifiers;
• sixth program instructions to receive the record identifier from a second data source, the record identifier being received by the anonymization engine;
• seventh program instructions to generate a second anonymous identifier to replace the record identifier with the anonymization engine, the second anonymous identifier being generated using a second salt and a second key, wherein both the first anonymous identifier and the second anonymous identifier are linked to the record identifier;
• eighth program instructions to search a second key-hash bucket for the second anonymous identifier, the second key-hash bucket including:
• anonymous identifiers generated using the second key, and
• the second salt used with the second key;
• ninth program instructions send the second anonymous identifier to the second data source when the second anonymous identifier is found in the second key-hash bucket;
• tenth program instructions generate second new anonymous identifiers with the anonymization engine when the second anonymous identifier is not found in the second key-hash bucket, the second new anonymous identifiers being generated using key-salt combinations used previously by the anonymization engine;
• eleventh program instructions create a second equivalent hashes table for the record identifier, the second equivalent hashes table including the second anonymous identifier and the second new anonymous identifiers;
• and
• twelfth program instructions send the second equivalent hashes table, the second anonymous identifier and second data associated with the second anonymous identifier to the data aggregation module,
• said first program instructions, said second program instructions, said third program instructions, and said fourth program instructions, and said fifth program instructions, and said sixth program instructions, and said seventh program instructions, and said eighth program instructions, and said ninth program instructions, and said tenth program instructions, and said eleventh program instructions, and said twelfth program instructions are stored on said computer readable storage medium.