# US 2022/0286438 A1 · Machine learning techniques for mitigating aggregate exposure of identifying information

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2022/0286438 A1
- Document ID: US20220286438A1
- Status: Published application
- Kind: A1
- Application number: 17/195,349
- Filed: 2021-03-08
- Published: 2022-09-08
- Earliest claimed: 2021-03-08
- Assignees: Adobe Inc.
- Applicants: Adobe Inc.
- Inventors: Robert W. Burke, JR.; Ronald Oribio
- Web page: https://patentgrep.com/patent/US20220286438A1

## CPC

- H04L 63/0428: {wherein the data content is protected, e.g. by encrypting or encapsulating the payload}
- G06N 3/04: Architecture, e.g. interconnection topology
- G06N 3/0442: characterised by memory or gating, e.g. long short-term memory [LSTM] or gated recurrent units [GRU]
- G06N 3/045: Combinations of networks
- G06N 3/0464: Convolutional networks [CNN, ConvNet]
- G06N 3/08: Learning methods
- G06N 3/09: Supervised learning
- H04L 63/0407: {wherein the identity of one or more communicating identities is hidden (cryptographic mechanisms or cryptographic arrangements for anonymous credentials or for identity based cryptographic systems H04L9/00)}
- H04L 63/1416: {Event detection, e.g. attack signature detection}
- H04L: TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION (arrangements common to telegraphic and telephonic communication H04M)
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS

## Abstract

Systems and methods mitigate aggregate exposure of identifying information using machine learning. A privacy monitoring system identifies entities and corresponding entity types by applying a set of domain-specific neural networks, each trained to recognize a particular entity type, to media data extracted from two or more content items associated with a user. The privacy monitoring system computes a privacy score indicating a cumulative privacy risk for potential exposure of identifying information associated with the user from the two or more content items by identifying connections between the identified entities. The connections between the entities are weighted according to the entity types and contribute to the privacy score. A reporting subsystem outputs an indication of a recommended action for mitigating the cumulative privacy risk.

## Claims

### Claim 1

A computer-implemented method comprising:
identifying, with an entity identification subsystem, a plurality of entities and an entity type for each entity of the plurality of entities by at least applying a set of domain-specific neural networks, each trained to recognize a particular entity type, to media data extracted from two or more content items associated with a user;
computing, by a scoring subsystem, a privacy score indicating a cumulative privacy risk for potential exposure of identifying information associated with the user from the two or more content items, the privacy score computed by identifying connections between at least a subset of the identified plurality of entities, the connections between the entities weighted according to the entity types and contributing to the privacy score; and
outputting, by a reporting subsystem, an indication of a recommended action for mitigating the cumulative privacy risk.

### Claim 2 (depends on 1)

The method of claim 1, wherein the privacy score is further computed by:
identifying a user-specific data set comprising identifying information associated with the user;
extracting data from the user-specific data set, wherein the extracted data is used to identify additional connections contributing to the privacy score; and
updating the user-specific data set to include at least a subset of the identified entities.

### Claim 3 (depends on 1)

The method of claim 1, further comprising:
identifying, by a PII risk signature subsystem, a privacy breach event associated with the user; and
applying a deep learning model to information associated with the privacy breach event to generate a risk signature indicating entity types that pose a heightened risk for the user,
wherein the connections between the entities are further weighted based on the generated risk signature.

### Claim 4 (depends on 1)

The method of claim 1, wherein:
a content item of the two or more content items is a video; and
the method further comprises extracting the media data from the content item by at least:
extracting an image from the video; and
segmenting the extracted image to isolate one or more of the plurality of entities.

### Claim 5 (depends on 4)

The method of claim 4, wherein the media data comprises audio signals, the method further comprising:
extracting audio data from the video;
analyzing the extracted audio data for transitions;
extracting the audio signals from the analyzed audio data; and
segmenting the extracted audio signals to isolate one or more of the plurality of entities.

### Claim 6 (depends on 1)

The method of claim 1, further comprising:
training each of the domain-specific neural networks to identify the entities by:
collecting additional content items from a public source;
extracting image or audio data from the collected additional content items;
associating the extracted image or audio data with labels for a set of entity types to generate domain-specific training data; and
training each domain-specific neural network to identify entity types against the domain-specific training data.

### Claim 7 (depends on 1)

The method of claim 1, further comprising:
detecting user interaction with an upload element of a graphical interface; and
retrieving a content item of the two or more content items in response to detecting the user interaction with the upload element.

### Claim 8 (depends on 1)

The method of claim 1, further comprising:
responsive to outputting the indication of the recommended action for mitigating the cumulative privacy risk, applying a modification to a content item, of the two or more content items.

### Claim 9 (depends on 1)

The method of claim 1, further comprising:
receiving user identification of a set of data sources,
wherein the two or more content items are collected from the identified set of data sources.

### Claim 10

A computing system comprising:
a content retrieval subsystem configured to retrieve two or more content items associated with a user;
a content segmentation subsystem configured to extract data from the two or more content items;
an entity identification subsystem configured to identify a plurality of entities and an entity type for each entity of the plurality of entities by at least applying a set of domain-specific neural networks, each trained to recognize a particular entity type, to the extracted data;
a scoring subsystem configured to compute a privacy score indicating a cumulative privacy risk for potential exposure of identifying information associated with the user from the two or more content items, the privacy score computed by identifying connections between at least a subset of the identified plurality of entities, the connections between the entities contributing to the privacy score based on weights assigned to links between the entities in a graph model according to identified entity types for the respective entities; and
a reporting subsystem configured to output an indication of a recommended action for mitigating the cumulative privacy risk.

### Claim 11 (depends on 10)

The computing system of claim 10, wherein the privacy score is further computed by:
identifying a user-specific data set comprising identifying information associated with the user;
extracting data from the user-specific data set, wherein the extracted data is used to identify additional connections contributing to the privacy score; and
updating the user-specific data set to include at least a subset of the identified entities.

### Claim 12 (depends on 10)

The computing system of claim 10, wherein:
an item of the two or more content items is a video; and
extracting the data from the item comprises:
extracting an image from the video; and
segmenting the extracted image to isolate one or more of the plurality of entities.

### Claim 13 (depends on 12)

The computing system of claim 12, wherein:
extracting the data from the item further comprises:
extracting audio data from the video;
analyzing the extracted audio data for transitions;
extracting audio signals from the analyzed audio data; and
segmenting the extracted audio signals to isolate one or more of the plurality of entities.

### Claim 14 (depends on 10)

The computing system of claim 10, further comprising:
a training subsystem configured to train each of the domain-specific neural networks to identify the entities by:
collecting additional content items from a public source;
extracting image or audio data from the collected additional content items;
associating the extracted image or audio data with labels for a set of entity types to generate domain-specific training data; and
training each domain-specific neural network to identify entity types against the domain-specific training data.

### Claim 15 (depends on 10)

The computing system of claim 10, wherein the content retrieval subsystem is further configured to:
detect user interaction with an upload element of a graphical interface, wherein retrieving a content item of the two or more content items is in response to detecting the user interaction with the upload element.

### Claim 16 (depends on 10)

The computing system of claim 10, further comprising a PII risk signature subsystem configured to:
identify a privacy breach event associated with the user; and
apply a deep learning model to information associated with the privacy breach event to generate a risk signature indicating entity types that pose a heightened risk for the user,
wherein the connections between the entities are further weighted based on the generated risk signature.

### Claim 17

A non-transitory computer-readable medium having instructions stored thereon, the instructions executable by a processing device to perform operations comprising:
retrieving two or more content items associated with a user;
a step for computing a privacy score indicating a cumulative privacy risk for potential exposure of identifying information associated with the user from the two or more content items using a set of domain-specific neural networks trained to identify entities of specific types within the content items; and
outputting an indication of a recommended action for mitigating the cumulative privacy risk.

### Claim 18 (depends on 17)

The non-transitory computer-readable medium of claim 17, wherein the privacy score is further computed by:
identifying a user-specific data set comprising identifying information associated with the user;
extracting data from the user-specific data set, wherein the extracted data is used to identify additional connections contributing to the privacy score; and
updating the user-specific data set to include at least a subset of the identified entities.

### Claim 19 (depends on 17)

The non-transitory computer-readable medium of claim 17, wherein a content item of the two or more content items is a video;
the operations further comprising:
extracting an image from the video; and
segmenting the extracted image to isolate an entity that is used in computing the privacy score.

### Claim 20 (depends on 19)

The non-transitory computer-readable medium of claim 19, the operations further comprising:
extracting audio data from the video;
analyzing the extracted audio data for transitions;
extracting audio signals from the analyzed audio data; and
segmenting the extracted audio signals to isolate a second entity that is used in computing the privacy score.
