# US 2018/0322310 A1 · Method/system for the online identification and blocking of privacy vulnerabilities in data streams

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2018/0322310 A1
- Document ID: US20180322310A1
- Status: Published application
- Kind: A1
- Application number: 16/037,356
- Filed: 2018-07-17
- Published: 2018-11-08
- Earliest claimed: 2016-07-22
- Assignees: International Business Machines Corporation
- Applicants: International Business Machines Corporation
- Inventors: Spyridon Antonatos; Stefano Braghin; Aris Gkoulalas-Divanis; Olivier Verscheure
- Web page: https://patentgrep.com/patent/US20180322310A1

## CPC

- G06F 21/6254: {by anonymising data, e.g. decorrelating personal data from the owner's identification}
- G06F 11/3006: {where the computing system is distributed, e.g. networked systems, clusters, multiprocessor systems (multiprogramming arrangements G06F9/46; allocation of resources G06F9/50)}
- G06F 16/2228: {Indexing structures}
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)

## Abstract

A method, system and computer program product for providing privacy protection to data streams in a distributed computing environment. The method includes concurrently processing, by a plurality of computer machines, data streams of attributes containing data values received by each of the plurality of local computer machines; indexing the data values for each attribute 1of the plurality of data streams received by each of the plurality of local computer machines; providing the indexed data values to a main computer machine; integrating, by the main computer machine, the local computer machine indexed data values into a global index data structure for the plurality of data streams; and identifying privacy vulnerabilities of the attributes that are direct identifiers and quasi-identifiers based on the global index.

## Claims

### Claim 1

A method for providing privacy protection to data streams in a distributed computing environment comprising;
concurrently processing, by a plurality of computer machines, data streams of attributes containing data values received by each of the plurality of local computer machines;
indexing the data values for each attribute of the plurality of data streams received by each of the plurality of local computer machines;
providing the indexed data values to a main computer machine;
integrating, by the main computer machine, the local computer machine indexed data values into a global index data structure for the plurality of data streams; and
identifying privacy vulnerabilities of the attributes that are direct identifiers and quasi-identifiers based on the global index.

### Claim 2 (depends on 1)

The method of claim 1, wherein identifying privacy vulnerabilities comprises:
identifying attributes that are direct identifiers based on the attribute data values stored in the global index data structure; and
identifying combinations of attributes that are quasi-identifiers based on the low frequency of certain combinations of attribute data value pairs by computing the frequency based on the single attribute data values stored in the global index data structure; and
providing privacy protection to the data streams by applying data transformations on the discovered direct identifiers and the quasi-identifiers.

### Claim 3 (depends on 1)

The method of claim 1, further including classifying the data attributes of the plurality of data streams into known data types, the classification being performed either before identifying the privacy vulnerabilities in the data, or after identifying the privacy vulnerabilities in the data and wherein the classification includes a set of classifiers, each classifier recognizing a specific data type to which a data value may belong and selecting a preference order for testing an incoming data value against different known specific data types, based on the specific data type that was selected previously for a data value of the same schema and attribute.

### Claim 4 (depends on 2)

The method of claim 2, wherein the privacy protection for the direct identifiers in the data stream is provided by applying relevant masking providers to the data based on the data type to which they were assigned in the data type classification.

### Claim 5 (depends on 1)

The method of claim 1, wherein the privacy protection for the quasi-identifiers in the data stream is provided by data generalization and data suppression operations applied on batches of data records held by different local computing machines.

### Claim 6 (depends on 1)

The method of claim 1, wherein each local computer machine receives incoming data from a data stream on a per-record basis and the data streams are split into attribute-value pairs.

### Claim 7 (depends on 1)

The method of claim 1, further including updating the global index data structure by one of: (a) each local computer machine issuing an update request every time it reads one or more data values, (b) each local computer machine issuing an update request every time it reads one or more data records, (c) each local computer machine issuing an update request for one or more read data values that are associated with counters that have values below a certain threshold k, and (d) at a time a data export has been requested, the main computer machine requesting the local indexes from all local computer machines to update the global index data structure before initiating the data export.

### Claim 8 (depends on 1)

The method of claim 1, further including assigning each of the local computer machines a set of attributes belonging to a schema, providing to each local computer machine access to the part of the global index data structure corresponding to the assigned attributes, and computing by each local computer machine the direct identifiers and quasi-identifiers that are related to the corresponding attributes.

### Claim 9 (depends on 1)

The method of claim 1, wherein the plurality of local computer machines are computer machines, the main computer machine is a computer machine and the plurality local computer machines and main computer machine are operating in a cloud environment.

### Claim 10

A system comprising:
at least one data processor connected to at least one memory that stores software instructions, where execution of the software instructions by the at least one data processor causes the system to provide privacy protection to data streams in a distributed computing environment by:
concurrently processing, by a plurality of computer machines, data streams of attributes containing data values received by each of the plurality of local computer machines;
indexing the data values for each attribute of the plurality of data streams received by each of the plurality of local computer machines;
providing the indexed data values to a main computer machine;
integrating, by the main computer machine, the local computer machine indexed data values into a global index data structure for the plurality of data streams; and
identifying privacy vulnerabilities of the attributes that are direct identifiers and quasi-identifiers based on the global index.

### Claim 11 (depends on 10)

The system of claim 10, further including classifying the data attributes of the plurality of data streams into known data types, the classification being performed either before identifying the privacy vulnerabilities in the data, or after identifying the privacy vulnerabilities in the data and the classification includes a set of classifiers, each classifier recognizing a specific data type to which a data value may belong and selecting a preference order for testing an incoming data value against different known specific data types, based on the specific data type that was selected previously for a data value of the same schema and attribute.

### Claim 12 (depends on 10)

The system of claim 10, wherein each local computer machine receives incoming data from a data stream on a per-record basis and the data streams are split into attribute-value pairs and further including examining each pair of attributes that are not direct identifiers in the data for rare combinations of their respective values, progressively extending the pairs of attributes that are examined to include an additional attribute from the data that is not a direct identifier until all possible attribute combinations have been examined and identifying the pairs of attributes that contain rare combinations of attribute-value pairs as quasi-identifiers.

### Claim 13 (depends on 10)

The system of claim 10, wherein the privacy protection for the direct identifiers in the data stream is provided by applying relevant masking providers to the data based on the data type to which they were assigned in the data type classification, and wherein the privacy protection for the quasi-identifiers in the data stream is provided by data generalization and data suppression operations applied on batches of data records held by different local computer machines performed in a principled way following a formal privacy model.

### Claim 14 (depends on 10)

The system of claim 10, further including updating the global index data structure by one of: (a) each local computer machine issuing an update request every time it reads one or more data values, (b) each local computer machine issuing an update request every time it reads one or more data records, (c) each local computer machine issuing an update request for one or more read data values that are associated with counters that have values below a certain threshold k, and (d) at a time a data export has been requested, the main computer machine requesting the local indexes from all local computer machines to update the global index data structure before initiating the data export and further including assigning each of the local computer machines a set of attributes belonging to a schema, providing to each local computer machine access to the part of the global index data structure corresponding to the assigned attributes, and computing by each local computer machine the direct identifiers and quasi-identifiers that are related to the corresponding attributes.

### Claim 15 (depends on 10)

The system of claim 10, wherein the plurality of local computer machines are computer machines, the main computer machine is a computer machine and the plurality local computer machines and main computer machine are operating in a cloud environment.

### Claim 16

A computer program product comprising:
software instructions on a non-transitory computer-readable medium, where execution of the software instructions using a computer causes the computer to provide privacy protection to data streams in a distributed computing environment by:
concurrently processing, by a plurality of computer machines, data streams of attributes containing data values received by each of the plurality of local computer machines;
indexing the data values for each attribute of the plurality of data streams received by each of the plurality of local computer machines;
providing the indexed data values to a main computer machine;
integrating, by the main computer machine, the local computer machine indexed data values into a global index data structure for the plurality of data streams; and
identifying privacy vulnerabilities of the attributes that are direct identifiers and quasi-identifiers based on the global index.

### Claim 17 (depends on 16)

The computer program product of claim 16, wherein each local computer machine receives incoming data from a data stream on a per-record basis and the data streams are split into attribute-value pairs and further including examining each pair of attributes that are not direct identifiers in the data for rare combinations of their respective values, progressively extending the pairs of attributes that are examined to include an additional attribute from the data that is not a direct identifier until all possible attribute combinations have been examined and identifying the pairs of attributes that contain rare combinations of attribute-value pairs as quasi-identifiers.

### Claim 18 (depends on 16)

The computer program product of claim 16, wherein the privacy protection for the direct identifiers in the data stream is provided by applying relevant masking providers to the data based on the data type to which they were assigned in the data type classification, and wherein the privacy protection for the quasi-identifiers in the data stream is provided by data generalization and data suppression operations applied on batches of data records held by different local computer machines performed in a principled way following a formal privacy model.

### Claim 19 (depends on 16)

The computer program product of claim 16, further including updating the global index data structure by one of: (a) each local computer machine issuing an update request every time it reads one or more data values, (b) each local computer machine issuing an update request every time it reads one or more data records, (c) each local computer machine issuing an update request for one or more read data values that are associated with counters that have values below a certain threshold k, and (d) at a time a data export has been requested, the main computer machine requesting the local indexes from all local computer machines to update the global index data structure before initiating the data export.

### Claim 20 (depends on 16)

The computer program product of claim 16, further including classifying the data attributes of the plurality of data streams into known data types, the classification being performed either before identifying the privacy vulnerabilities in the data, or after identifying the privacy vulnerabilities in the data and wherein the classification includes a set of classifiers, each classifier recognizing a specific data type to which a data value may belong and selecting a preference order for testing an incoming data value against different known specific data types, based on the specific data type that was selected previously for a data value of the same schema and attribute, and further including assigning each of the local computer machines a set of attributes belonging to a schema, providing to each local computer machine access to the part of the global index data structure corresponding to the assigned attributes, and computing by each local computer machine the direct identifiers and quasi-identifiers that are related to the corresponding attributes.
