# US 12,223,767 B2 · Systems and methods of image processing for privacy management

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 12,223,767 B2
- Document ID: US12223767B2
- Status: Granted
- Kind: B2
- Application number: 17/725,160
- Filed: 2022-04-20
- Published: 2023-10-26
- Granted: 2025-02-11
- Earliest claimed: 2022-04-20
- Assignees: QUALCOMM Incorporated
- Applicants: QUALCOMM Incorporated
- Inventors: Binil Francis Joseph; Bapineedu Chowdary Gummadi
- Web page: https://patentgrep.com/patent/US12223767B2

## CPC

- G06V 40/16: Human faces, e.g. facial parts, sketches or expressions
- G06N 20/00: Machine learning
- G06T 11/60: Creating or editing images; Combining images with text
- G06V 10/82: using neural networks
- G06V 20/20: in augmented reality scenes
- G06V 40/166: {using acquisition arrangements}
- G06V 40/172: {Classification, e.g. identification}
- G06F 21/6245: {Protecting personal data, e.g. for financial or medical purposes}
- G06T 2200/24: involving graphical user interfaces [GUIs]
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)

## Abstract

Systems and techniques are described for processing one or more images. For example, an imaging system may receive an image of an environment and detect a face of a person in the image. The imaging system may identify profile data associated with the person and retrieve, based on the profile data, a synthesized face. The imaging system may generate a modified image of the environment at least in part by modifying the image to use the synthesized face for the person in place of the face of the person. The imaging system may further output the modified image of the environment. In some examples, the imaging system uses trained machine learning model(s) to generate the modified image.

## Claims

### Claim 1

An apparatus for processing one or more images, the apparatus comprising:
at least one memory; and
one or more processors coupled to the at least one memory, the one or more processors configured to:
receive image data depicting an environment, the image data including one or more images;
detect, in the image data, a first face of a first person;
identify first profile data associated with the first person;
detect, in the image data, a second face of a second person;
identify second profile data associated with the second person;
generate, based on the first profile data and the second profile data, a modified image of the environment by replacing an area depicting the first face in an image of the image data with a synthesized face that is distinct from the first face, wherein the modified image depicts the second face; and
output the modified image of the environment.

### Claim 2 (depends on 1)

The apparatus of claim 1, wherein to generate the modified image, the one or more processors are configured to use the synthesized face and at least a portion of the image of the image data as inputs to a trained machine learning model.

### Claim 3 (depends on 1)

The apparatus of claim 1, wherein the first profile data does not match a stored profile of a person in the at least one memory.

### Claim 4 (depends on 1)

The apparatus of claim 1, wherein the first profile data indicates one or more privacy settings for the first person.

### Claim 5 (depends on 1)

The apparatus of claim 1, wherein to identify the first profile data associated with the first person, the one or more processors are configured to determine an identity of the first person based on the image data.

### Claim 6 (depends on 1)

The apparatus of claim 1, wherein to identify the first profile data associated with the first person, the one or more processors are configured to identify the first profile data associated with the first person based on one or more communications from a user device associated with the first person.

### Claim 7 (depends on 1)

The apparatus of claim 1, wherein the one or more processors are configured to:
determine, based on the second profile data, that the second face of the second person is not to be modified in the modified image.

### Claim 8 (depends on 7)

The apparatus of claim 7, wherein the second profile data does not match a stored profile of a person in the at least one memory.

### Claim 9 (depends on 7)

The apparatus of claim 7, wherein the second profile data indicates one or more privacy settings for the second person.

### Claim 10 (depends on 1)

The apparatus of claim 1, wherein the one or more processors are configured to:
retrieve the synthesized face based on the first profile data; and
generate the synthesized face before retrieving the synthesized face, wherein to generate the synthesized face, the one or more processors are configured to use a trained machine learning model that combines attributes from a plurality of different images of a plurality of different faces into the synthesized face.

### Claim 11 (depends on 1)

The apparatus of claim 1, wherein the one or more processors are configured to re-generate the synthesized face, wherein the synthesized face changes in appearance upon being re-generated.

### Claim 12 (depends on 1)

The apparatus of claim 1, wherein the synthesized face has a skin tone that matches a skin tone of the first person in the modified image.

### Claim 13 (depends on 12)

The apparatus of claim 12, wherein to generate the modified image, the one or more processors are configured to modify a first skin tone of a portion of the first person in the image data to correspond to a second skin tone of the synthesized face.

### Claim 14 (depends on 1)

The apparatus of claim 1, wherein the one or more processors are configured to:
identify a head pose of a head of the first person in the image data, wherein to generate the modified image, the one or more processors are configured to orient the synthesized face according to the head pose.

### Claim 15 (depends on 1)

The apparatus of claim 1, wherein the one or more processors are configured to:
identify, from the first profile data, at least one privacy setting associated with the first person, wherein the at least one privacy setting indicates that output of the face of the first person is to be avoided, wherein the one or more processors are configured to generate the modified face in response to identifying the at least one privacy setting.

### Claim 16 (depends on 1)

The apparatus of claim 1, wherein the one or more processors are configured to:
identify an expression on the face of the first person in the image data; and
modify the synthesized face before generating the modified image based on the expression.

### Claim 17 (depends on 1)

The apparatus of claim 1, further comprising:
a display, wherein to output the modified image, the one or more processors are configured to display the modified image using at least the display.

### Claim 18 (depends on 1)

The apparatus of claim 1, further comprising:
a communication interface, wherein to output the modified image, the one or more processors are configured to send the modified image to at least a recipient device using at least the communication interface.

### Claim 19 (depends on 1)

The apparatus of claim 1, wherein the one or more processors are configured to receive the image data from an image sensor of the apparatus, wherein the one or more processors are configured to output the modified image using an output device of the apparatus, and wherein the apparatus includes at least one of a head-mounted display (HMD), a mobile handset, or a wireless communication device.

### Claim 20 (depends on 1)

The apparatus of claim 1, wherein the one or more processors are configured to receive the image data from a user device over a network, wherein the one or more processors are configured to send the modified image to the user device over the network, and wherein the apparatus includes one or more servers that are communicatively coupled to the user device over the network.

### Claim 21 (depends on 1)

The apparatus of claim 1, wherein the one or more processors are configured to replace the area depicting the first face in the image of the image data with the synthesized face using an inpainting process.

### Claim 22

A method of processing one or more images, the method comprising:
receiving image data depicting an environment, the image data including one or more images;
detecting, in the image data, a first face of a first person;
identifying first profile data associated with the first person;
detecting, in the image data, a second face of a second person;
identifying second profile data associated with the second person;
generating, based on the first profile data and the second profile data, a modified image of the environment by replacing an area depicting the first face in an image of the image data with a synthesized face that is distinct from the first face, wherein the modified image depicts the second face; and
outputting the modified image of the environment.

### Claim 23 (depends on 22)

The method of claim 22, wherein generating the modified image includes using the synthesized face and at least a portion of the image of the image data as inputs to a trained machine learning model.

### Claim 24 (depends on 22)

The method of claim 22, wherein the first profile data does not match a stored profile of a person in at least one memory.

### Claim 25 (depends on 22)

The method of claim 22, wherein the first profile data indicates one or more privacy settings for the first person.

### Claim 26 (depends on 22)

The method of claim 22, wherein identifying the first profile data associated with the first person includes determining an identity of the first person in based on the image data.

### Claim 27 (depends on 22)

The method of claim 22, wherein identifying the first profile data associated with the first person includes identifying the first profile data associated with the first person based on one or more communications from a user device associated with the first person.

### Claim 28 (depends on 22)

The method of claim 22, further comprising:
determining, based on the second profile data, that the second face of the second person is not to be modified in the modified image.

### Claim 29 (depends on 22)

The method of claim 22, further comprising:
identifying, from the first profile data, at least one privacy setting associated with the first person, wherein the at least one privacy setting indicates that output of the face of the first person is to be avoided; and
generating the modified face in response to identifying the at least one privacy setting.

### Claim 30 (depends on 22)

The method of claim 22, wherein generating the modified image includes replacing the area depicting the first face in the image of the image data with the synthesized face using an inpainting process.