Self-contained system for de-identifying unstructured data in healthcare records

A system for de-identifying unstructured data within data sets, comprising:

• memory and a processor configured for accessing the data sets from data sources and parsing data within the data sets and identifying elements of unstructured data within the data sets and de-identifying unstructured data using a de-identification engine initializing a blacklist dictionary having a structure that facilitates identification of personally identifiable information (PII) and protected health information (PHI) and that facilitates removal and replacement of PII and PHI with a case-type tag, and a whitelist dictionary comprising terms selected to remain in the data sets despite being included in the blacklist dictionary, and comprising:
• a register module configured to modify the blacklist dictionary by removing terms included within the whitelist dictionary to generate an adjusted blacklist dictionary, and augmenting the adjusted blacklist dictionary with a record-specific blacklist for each individual record within the data sets to create a record-specific adjusted blacklist dictionary;
• a de-identification module configured to determine which unstructured elements are to be removed from each individual record of a data set by scrubbing PII and PHI from each individual record utilizing the record-specific adjusted blacklist dictionary, the scrubbing comprising:
• removing all elements within each individual record determined to be PII or PHI according to terms in the record-specific adjusted blacklist dictionary;
• and
• replacing removed elements with a case-type tag identifying a type of information being removed according to the record-specific adjusted blacklist dictionary;
• wherein the de-identification module repeats the scrubbing PII and PHI for each individual record within the data sets;
• and
• one or more user devices configured to receive input from one or more users by an input-output interface and communicate with the processor and the de-identification engine over a telecommunication network, providing functionality for the register module, de-identification module, and merging module that share a secured network connection.