System for simulating a de-identified healthcare data set and creating simulated personal data while retaining profile of authentic data
AssigneeDatavant, Inc.
InventorsShahir Kassam-Adams, Jason A. LaBonte, Paul J. Bayless, Joseph Austin
Life of this patent
Estimating the term…
Abstract
A method and apparatus for the creation of simulated records from a small sample data set with configurable levels of variability, the creation of simulated data from an encrypted token that uniquely identifies an individual, and the creation of simulated values using as the basis retained data (birth years, 3-digit zip areas, gender, etc.) from the de-identification process.
Claims41
A method for creating and configuring simulated data corresponding to de-identified healthcare data sets while retaining profiles of source data, the method comprising:
- registering and authenticating, using a register module of a computing device comprising a processor and memory, one or more user devices of a client configured to communicate data to and from the computing device over a telecommunications network;
- receiving, from the one or more user devices, an input data set comprising a sample data set or a complete data set, a record set request, and a configuration comprising a requested set of fields;
- aggregating and ingesting, using a data aggregation module, data records of the input data set comprising sensitive information from a plurality of data sources and/or one or more data stores comprising previously de-identified healthcare data sets with encrypted tokens;
- generating a projected data set comprising simrecords by uniquely associating the data records with identifiable individuals using a merging module, and augmenting the data records, using a simrecord module, by replicating sample data records of the sample data set until reaching a number set by the record set request, wherein the simrecord module creates variation in the sample data records as specified in the configuration;
- creating a de-identified and tokenized data set, using a de-identification module, comprising removing the sensitive information from the projected data set to create a de-identified data set comprising data fields to be simulated that are blank or nulled, and adding one or more unique encrypted tokens, each comprising a randomized character string, to each de-identified record of the de-identified data set;
- simulating personal data from the de-identified and tokenized data set, using a simPII module, by creating simulated values from characters found in each de-identified and tokenized data record by converting the characters or replacing the characters with stored characters selected from a plurality of look-up lists, then populating each of the data fields that are blank or nulled with the simulated values and replacing any existing values or modified values specified by the configuration with simulated values, making each de-identified and tokenized record unique to a particular individual but without the sensitive information and sensitive data values used to create each de-identified and tokenized record;
- applying validation rules to convert any invalid data to the simulated personal data that is valid, or to shift passed through information to match the simulated personal data;
- and
- transmitting, to the one or more user devices, one or more simulated data sets comprising the simulated personal data in simulated data records comprising the simulated personal data values, preventing disclosure of sensitive information associated with the particular individual while preserving a demographic profile of the sensitive information that is simulated and a connection with nonpersonal data for the particular individual based on matching unique encrypted tokens, wherein the simPII module consistently creates the same simulated personal data values for the particular individual wherever and whenever executed, and storing the simulated personal data in a data store segregated from sensitive information;
- wherein the simulating personal data from the de-identified and tokenized data set using the simPII module by creating simulated values from characters further comprises:
- selecting, using the simPII module, specific token characters by position for each of the one or more unique encrypted tokens, wherein the selected specific token characters are then used to access a simulated value from a look-up list, with a number of simulated values contained in the look-up list determining a quantity of specific token characters used for selecting simulated values that bear no relation to the sensitive data values originally input, and simulating personal data further comprises one or more of the group consisting of simulating general text terms, simulating record-consistent text values, simulating numbers, simulating address values, and simulating birthdate values, and
- wherein the simulating personal data further comprises simulating numbers comprising:
- selecting characters of the one or more unique encrypted tokens;
- converting to decimal numeric values for each of the selected characters;
- concatenating, truncating to required length, and converting to correct format the decimal numeric values of the selected characters to create a numeric string;
- and
- adding the created numeric string to record to replace nulled value.
A system creating and configuring simulated data corresponding to de-identified healthcare data sets while retaining profiles of source data, the system comprising:
- one or more user devices configured to communicate data to and from one or more computing devices over one or more telecommunications networks, the one or more computing devices comprising:
- one or more processors for executing instruction;
- memory for storing data and instructions for the one or more processors;
- one or more interfaces;
- one or more input-output devices;
- a register module configured to register and authenticate one or more user devices of a client;
- one or more databases or data stores comprising previously de-identified healthcare data sets with unique encrypted tokens and comprising lookup lists stored therein, and configured to receive, from the one or more user devices, an input data set comprising a sample data set or a complete data set, a record set request, and a configuration comprising a requested set of fields;
- a data aggregation module configured to aggregate and ingest data records of the input data set comprising sensitive information from a plurality of data sources and/or the one or more databases or data stores comprising previously de-identified healthcare data sets with encrypted tokens;
- a merging module configured to generate a projected data set comprising simrecords by uniquely associating the data records with identifiable individuals, and a simrecord module configured to augment the data records by replicating sample data records of the sample data set until reaching a number set by the record set request, wherein the simrecord module creates variation in the sample data records as specified in the configuration;
- a de-identification module configured to create a de-identified and tokenized data set by removing the sensitive information from the projected data set to create a de-identified data set comprising data fields to be simulated that are blank or nulled, and adding one or more unique encrypted tokens, each comprising a randomized character string, to each de-identified record of the de-identified data set;
- a simPII module configured to simulate personal data from the de-identified and tokenized data set by creating simulated values from characters found in each de-identified and tokenized data record by converting the characters or replacing the characters with stored characters selected from a plurality of look-up lists, then populating each of the data fields that are blank or nulled with the simulated values and replacing any existing values or modified values specified by the configuration with simulated values, making each de-identified and tokenized record unique to a particular individual but without the sensitive information and sensitive data values used to create each de-identified and tokenized record, and further configured to apply validation rules to convert any invalid data to simulated data that is valid, or to shift passed through information to be consistent with the simulated data;
- and
- wherein the one or more computing devices are further configured to transmit to the one or more user devices, using the one or more input-output devices and the one or more telecommunications networks, one or more simulated data sets comprising simulated personal data in simulated data records comprising simulated personal data values, preventing disclosure of sensitive information associated with the particular individual while preserving a demographic profile of the sensitive information that is simulated and a connection with nonpersonal data for the particular individual based on matching unique encrypted tokens, wherein the simPII module consistently creates the same simulated personal data values for the particular individual wherever and whenever executed, and wherein the one or more computing devices store the simulated personal data in the one or more databases or data stores segregated from sensitive information;
- wherein the simulation of personal data from the de-identified and tokenized data set using the simPII module by creating simulated values from characters further comprises:
- selecting, using the simPII module, specific token characters by position for each of the one or more unique encrypted tokens, wherein the selected specific token characters are then used to access a simulated value from a look-up list, with a number of simulated values contained in the look-up list determining a quantity of specific token characters used for selecting simulated values that bear no relation to the sensitive data values originally input, and simulating personal data further comprises one or more of the group consisting of simulating general text terms, simulating record-consistent text values, simulating numbers, simulating address values, and simulating birthdate values, and
- wherein the simulation of personal data further comprises simulating numbers comprising:
- selecting characters of the one or more unique encrypted tokens;
- converting to decimal numeric values for each of the selected characters;
- concatenating, truncating to required length, and converting to correct format the decimal numeric values of the selected characters to create a numeric string;
- and
- adding the created numeric string to record to replace nulled value.
Dependencies are inferred from the claim text.