# US 2025/0217918 A1 · Semi-fragile neural watermarks for media authentication and countering deepfakes

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2025/0217918 A1
- Document ID: US20250217918A1
- Status: Published application
- Kind: A1
- Application number: 18/850,041
- Filed: 2023-03-23
- Published: 2025-07-03
- Earliest claimed: 2022-03-24
- Assignees: The Regents of the University of California
- Applicants: The Regents of the University of California
- Inventors: Shehzeen Samarah Hussain; Paarth Neekhara; Farinaz Koushanfar; Xinqiao Zhang; Julian McAuley
- Web page: https://patentgrep.com/patent/US20250217918A1

## CPC

- G06N 3/0455: Auto-encoder networks; Encoder-decoder networks
- G06T 1/0042: {Fragile watermarking, e.g. so as to detect tampering}
- G06F 21/16: Program or content traceability, e.g. by watermarking
- G06N 3/094: Adversarial learning
- H04N 21/8358: involving watermark {(protecting executable software by watermarking G06F21/16; image watermarking in general G06T1/0021; watermarks inserted in still images for transmission purposes H04N1/32144; inserting watermarks during video coding H04N19/467)}
- H04N 19/467: characterised by the embedded information being invisible, e.g. watermarking
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION

## Abstract

In some implementations, there is provided a method comprising: receiving an image and a digital watermark; outputting a watermarked image generated based on the image and the digital watermark; selecting, from a set of benign transforms, a benign transform; selecting, from a set of malicious transforms, a malicious transform; performing the benign transform on the watermarked image to generate a benign image; performing the malicious transform on the watermarked image to generate a malicious image; decoding, by a decoder, the benign image to a first predicted value of the digital watermark; decoding, by the decoder, the malicious image to a second predicted value of the digital watermark; and adjusting at least one weight of the decoder during a learning phase of the decoder.

## Claims

### Claim 1

A computer-implemented method, comprising:
receiving, at an encoder, an image and a digital watermark;
outputting, by the encoder, a watermarked image generated based on the image and the digital watermark;
selecting, from a set of benign transforms, a benign transform;
selecting, from a set of malicious transforms, a malicious transform;
performing the benign transform on the watermarked image to generate a benign image;
performing the malicious transform on the watermarked image to generate a malicious image;
decoding, by a decoder, the benign image to a first predicted value of the digital watermark;
decoding, by the decoder, the malicious image to a second predicted value of the digital watermark; and
adjusting at least one weight of the decoder during a learning phase of the decoder by at least learning a minimum amount of error between the first predicted value, which corresponds to the benign image, and the digital watermark and learning a maximum amount of error between the second predicted value, which corresponds to the malicious image, and the digital watermark.

### Claim 2 (depends on 1)

The computer-implemented method of claim 1 further comprising:
training, the encoder, to minimize an error between the image and the watermarked image, wherein the error comprises an image reconstruction loss and/or an adversarial loss.

### Claim 3 (depends on 2)

The computer-implemented method of claim 2, wherein the training to minimize the error between the image and the watermarked image further comprises using a discriminator to determine the adversarial loss indicative of whether the watermarked image is the image.

### Claim 4 (depends on 1)

The computer-implemented method of claim 1, wherein the encoder comprises a convolutional neural network and/or a U-NET, and wherein the decoder comprises a convolutional neural network and/or a U-NET.

### Claim 5 (depends on 1)

The computer-implemented method of claim 1, wherein the benign transform is selected from a set of benign transforms comprising an image compression of the watermarked image, a color adjustment of the watermarked image, a lighting adjustment of the watermarked image, a contrast adjustment of the watermarked image, a downsizing of the watermarked image, an upsizing of the watermarked image transformation, a horizontal and/or vertical translation of the watermarked image, and/or a rotation of the watermarked image.

### Claim 6 (depends on 1)

The computer-implemented method of claim 1, wherein the malicious transform replaces an image portion of a subject of the watermarked image with another image portion, and/or wherein the malicious transform replaces at least a portion of face image of the subject of the watermarked image with another face portion.

### Claim 7 (depends on 1)

The computer-implemented method of claim 1, wherein the malicious transform uses a mask that replaces at least a portion of the watermarked image with another image portion.

### Claim 8 (depends on 1)

The computer-implemented method of claim 1, further comprising:
comparing the first predicted value of the digital watermark and/or the second predicted to the digital watermark to determine whether the watermarked image has been maliciously transformed.

### Claim 9 (depends on 1)

The computer-implemented method of claim 1, wherein the digital watermark comprises an encrypted message, wherein the encrypted message is generated using a key and a message.

### Claim 10 (depends on 1)

The computer-implemented method of claim 1, wherein the encoder receives a plurality of images to enable the learning phase of the decoder.

### Claim 11 (depends on 1)

The computer-implemented method of claim 1, wherein the adjusting further comprises adjusting at least one weight of an encoder during the learning phase.

### Claim 12

A system, comprising:
at least one data processor; and
at least one memory storing instructions which, when executed by the at least one data processor, cause operations comprising:
receiving, at an encoder, an image and a digital watermark;
outputting, by the encoder, a watermarked image generated based on the image and the digital watermark;
selecting, from a set of benign transforms, a benign transform;
selecting, from a set of malicious transforms, a malicious transform;
performing the benign transform on the watermarked image to generate a benign image;
performing the malicious transform on the watermarked image to generate a malicious image;
decoding, by a decoder, the benign image to a first predicted value of the digital watermark;
decoding, by the decoder, the malicious image to a second predicted value of the digital watermark; and
adjusting at least one weight of the decoder during a learning phase of the decoder by at least learning a minimum amount of error between the first predicted value, which corresponds to the benign image, and the digital watermark and learning a maximum amount of error between the second predicted value, which corresponds to the malicious image, and the digital watermark.

### Claim 13 (depends on 12)

The system of claim 12 further comprising:
training, the encoder, to minimize an error between the image and the watermarked image, wherein the error comprises an image reconstruction loss and/or an adversarial loss.

### Claim 14 (depends on 13)

The system of claim 13, wherein the training to minimize the error between the image and the watermarked image further comprises using a discriminator to determine the adversarial loss indicative of whether the watermarked image is the image.

### Claim 15 (depends on 12)

The system of claim 12, wherein the encoder comprises a convolutional neural network and/or a U-NET, and wherein the decoder comprises a convolutional neural network and/or a U-NET.

### Claim 16 (depends on 12)

The system of claim 12, wherein the benign transform is selected from a set of benign transforms comprising an image compression of the watermarked image, a color adjustment of the watermarked image, a lighting adjustment of the watermarked image, a contrast adjustment of the watermarked image, a downsizing of the watermarked image, an upsizing of the watermarked image transformation, a horizontal and/or vertical translation of the watermarked image, and/or a rotation of the watermarked image.

### Claim 17 (depends on 12)

The system of claim 12, wherein the malicious transform replaces an image portion of a subject of the watermarked image with another image portion, and/or wherein the malicious transform replaces at least a portion of face image of the subject of the watermarked image with another face portion.

### Claim 18 (depends on 12)

The system of claim 12, wherein the malicious transform uses a mask that replaces at least a portion of the watermarked image with another image portion.

### Claim 19 (depends on 12)

The system of claim 12, further comprising:
comparing the first predicted value of the digital watermark and/or the second predicted to the digital watermark to determine whether the watermarked image has been maliciously transformed, wherein the digital watermark comprises an encrypted message, wherein the encrypted message is generated using a key and a message, wherein the encoder receives a plurality of images to enable the learning phase of the decoder, wherein the adjusting further comprises adjusting at least one weight of an encoder during the learning phase.

### Claim 20

A non-transitory computer-readable medium including instructions which, when executed by at least one data processor, cause operations comprising:
receiving, at an encoder, an image and a digital watermark;
outputting, by the encoder, a watermarked image generated based on the image and the digital watermark;
selecting, from a set of benign transforms, a benign transform;
selecting, from a set of malicious transforms, a malicious transform;
performing the benign transform on the watermarked image to generate a benign image;
performing the malicious transform on the watermarked image to generate a malicious image;
decoding, by a decoder, the benign image to a first predicted value of the digital watermark;
decoding, by the decoder, the malicious image to a second predicted value of the digital watermark; and
adjusting at least one weight of the decoder during a learning phase of the decoder by at least learning a minimum amount of error between the first predicted value, which corresponds to the benign image, and the digital watermark and learning a maximum amount of error between the second predicted value, which corresponds to the malicious image, and the digital watermark.
