# US 2019/0377901 A1 · Obfuscating information related to personally identifiable information (pii) > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 2019/0377901 A1 - Document ID: US20190377901A1 - Status: Published application - Kind: A1 - Application number: 16/004,128 - Filed: 2018-06-08 - Published: 2019-12-12 - Earliest claimed: 2018-06-08 - Assignees: Microsoft Technology Licensing, LLC - Applicants: Microsoft Technology Licensing, LLC - Inventors: Andreas Balzer; David Mowatt; Muiris Woulfe - Web page: https://patentgrep.com/patent/US20190377901A1 ## CPC - G06F 21/6254: {by anonymising data, e.g. decorrelating personal data from the owner's identification} - H04L 63/0421: {Anonymous communication, i.e. the party's identifiers are hidden from the other party or parties, e.g. using an anonymizer} - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) - H04L: TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION (arrangements common to telegraphic and telephonic communication H04M) ## Abstract A system for protecting personally identifiable information (PII) associated with audio, image and video. The system includes an output device and a processor. The processor receives a document including an audio, an image, or a video containing both non-personally identifiable information and personally identifiable information, scans the document for a voice, a face, a graphically rendered text, or a personal attribute, match the voice, face, graphically rendered text, or personal attribute with records in a database to determine whether the voice, face, graphically rendered text, or personal attribute in the document is associated with personally identifiable information. The processor also determines a start time and an end time associated with the presence of the voice or video in response to determining that the voice, or video is associated with PII, generates an obfuscated audio or a video between the start time and the end time, and causes the output device to output the obfuscated audio, graphically rendered text or video. ## Claims ### Claim 1 A computer system for protecting personally identifiable information (PII) associated with textual, audio, image, or video data, the computer system comprising: an output device; an electronic processor electrically connected to the output device and configured to receive a document containing both non-personally identifiable information and personally identifiable information; scan the document for a voice or a first personal attribute; match the voice or the first personal attribute with records in a database to determine whether the voice or the first personal attribute in the document is associated with personally identifiable information; determine a start time and an end time associated with the presence of the voice or the first personal attribute in an audio in the document; generate obfuscated version of the document, wherein the voice or the first personal attribute in the audio is obfuscated between the start time and the end time; and generate a signal causing the output device to output the obfuscated version of the document. ### Claim 2 (depends on 1) The computer system of claim 1, wherein the document includes personally identifiable information included in an item selected from the group consisting of graphically rendered text, audio, image, and video data. ### Claim 3 (depends on 1) The computer system of claim 1, wherein obfuscated audio includes an item selected from the group consisting of a tone, a beep, a second audio and a period of silence to replace the voice or the first personal attribute. ### Claim 4 (depends on 1) The computer system of claim 1, wherein determining that the first personal attribute is associated with personally identifiable information includes converting the audio into textual information and parsing the textual information for personally identifiable information. ### Claim 5 (depends on 4) The computer system of claim 4, wherein the textual information for personally identifiable information includes an item selected from a group consisting of name, date of birth, place of birth, email address, phone number, fax number, particular content, social networking credential, biometric information, financial account number, organization issued identification, and government issued identification. ### Claim 6 (depends on 5) The computer system of claim 5, wherein the electronic processor is further configured to in response to the personally identifiable information matching none of the records in the database, update the database with a new record associated with the personally identifiable information. ### Claim 7 (depends on 1) The computer system of claim 1, wherein the electronic processor is further configured to scan the document for a facial image or a second personal attribute; match the facial image or the second personal attribute with records in the database to determine whether the facial image or the second personal attribute in the document is associated with personally identifiable information; in response to determining that the facial image or the second personal attribute is associated with personally identifiable information, obfuscate the portion of the image including the facial image or the second personal attribute; generate an obfuscated image including an obfuscated facial image or an obfuscated second personal attribute; and display the obfuscated image. ### Claim 8 (depends on 1) The computer system of claim 1, wherein the database is accessible based on an authorization level associated with a user. ### Claim 9 (depends on 1) The computer system of claim 1, wherein the electronic processor is further configured to delete all documents containing personally identifiable information associated with an individual. ### Claim 10 A method for protecting personally identifiable information of entities, the method comprising: receiving, with an electronic processor, a document including an item selected from the group consisting of graphically rendered text, an audio, an image, and a video containing both non-personally identifiable information and personally identifiable information; scanning, with the electronic processor, the document for a voice or a first personal attribute; matching, with the electronic processor, the voice or personal attribute with records in a database to determine whether the voice or the first personal attribute in the document is associated with personally identifiable information; determining a start time and an end time associated with the presence of the voice or the first personal attribute in the audio; generating obfuscated version of the document, wherein the voice or the first personal attribute in the audio is obfuscated between the start time and the end time; and generating a signal causing the output device to output the obfuscated version of the document. ### Claim 11 (depends on 10) The method of claim 10, further comprising: retrieving all documents containing personally identifiable information associated with an individual; and deleting all documents containing personally identifiable information associated with the individual. ### Claim 12 (depends on 10) The method of claim 10, wherein receiving the document containing both non-personally identifiable data and personally identifiable information of one or more entities includes: receiving the document including personally identifiable information included in an item selected from the group consisting of graphically rendered text, audio, image, and video data. ### Claim 13 (depends on 10) The method of claim 10, wherein determining that the personal attribute is associated with personally identifiable information includes converting the audio into textual information and parsing the graphically rendered text for personally identifiable information. ### Claim 14 (depends on 13) The method of claim 13, wherein parsing the graphically rendered text for personally identifiable information includes parsing the graphically rendered text information for an item selected from a group consisting of name, date of birth, place of birth, email address, phone number, fax number, particular content, social networking credential, biometric information, financial account number, organization issued identification, and government issued identification. ### Claim 15 (depends on 10) The method of claim 10, further comprising: updating the database with a new record associated with the personally identifiable information, in response to the personally identifiable information matching none of the records in the database. ### Claim 16 (depends on 10) The method of claim 10, further comprising: scanning the document for a facial image or a second personal attribute; matching the facial image or the second personal attribute with records in the database to determine whether the facial image or the second personal attribute in the document is associated with personally identifiable information; in response to determining that the facial image or the second personal attribute is associated with personally identifiable information, obfuscating the portion of the image including the facial image or the second personal attribute; generating an obfuscated image including an obfuscated facial image or obfuscated second personal attribute; and displaying the obfuscated image. ### Claim 17 (depends on 10) The method of claim 10, further comprising: accessing the database based on an authorization level associated with the user. ### Claim 18 A tangible machine-readable medium containing computer-readable instructions that when executed by one or more processors cause the one or more processors to perform a method, the method comprising: receiving a document including an item selected from the group consisting of an audio. an image, and a video containing both non-personally identifiable information and personally identifiable information; scanning the document for a voice or a first personal attribute; matching the voice or personal attribute with records in a database to determine whether the voice or the first personal attribute in the document is associated with personally identifiable information; determining a start time and an end time associated with the presence of the voice or the first personal attribute in the audio; generating obfuscated version of the document, wherein the voice or the first personal attribute in the audio is obfuscated between the start time and the end time; and generating a signal causing the output device to output the obfuscated version of the document. ### Claim 19 (depends on 18) The tangible machine-readable medium of claim 18, further comprising: scanning the document for a facial image or a second personal attribute; matching the facial image or the second personal attribute with records in the database to determine whether the facial image or the second personal attribute in the document is associated with personally identifiable information; in response to determining that the facial image or the second personal attribute is associated with personally identifiable information, obfuscating the portion of the image including the facial image or the second personal attribute; generating the obfuscated document including an obfuscated facial image or an obfuscated second personal attribute; and displaying the obfuscated document including the obfuscated facial image or the obfuscated second personal attribute. ### Claim 20 (depends on 19) The tangible machine-readable medium of claim 19, wherein determining that the first personal attribute is associated with personally identifiable information includes converting the audio into textual information and parsing the textual information for personally identifiable information.