# US 2024/0005441 A1 · Systems and methods for intelligent steganographic protection

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2024/0005441 A1
- Document ID: US20240005441A1
- Status: Published application
- Kind: A1
- Application number: 18/369,929
- Filed: 2023-09-19
- Published: 2024-01-04
- Earliest claimed: 2021-06-16
- Assignees: Bank of America Corporation
- Applicants: Bank of America Corporation
- Inventors: Madhusudhanan Krishnamoorthy; Paramita De; Pawan Kumar Jha
- Web page: https://patentgrep.com/patent/US20240005441A1

## CPC

- G06T 1/0028: {Adaptive watermarking, e.g. Human Visual System [HVS]-based watermarking}
- G06T 1/0085: {Time domain based watermarking, e.g. watermarks spread over several images}
- G06F 21/6254: {by anonymising data, e.g. decorrelating personal data from the owner's identification}
- G06F 21/84: output devices, e.g. displays or monitors
- G06N 3/045: Combinations of networks
- G06N 3/0455: Auto-encoder networks; Encoder-decoder networks
- G06N 3/0464: Convolutional networks [CNN, ConvNet]
- H04W 4/029: Location-based management or tracking services
- G06T 2201/0061: Embedding of the watermark in each block of the image, e.g. segmented watermarking
- H04W 4/33: for indoor environments, e.g. buildings
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- H04W: WIRELESS COMMUNICATION NETWORKS (broadcast communication H04H; communication systems using wireless links for non-selective communication, e.g. wireless extensions H04M1/72)

## Abstract

The present invention generally relates to the field of automated and flexible information extraction and protection for graphical data. In particular, the invention provides a unique platform for analyzing, classifying, extracting, and processing information from images using deep learning image detection models. Embodiments of the inventions are configured to provide an end to end automated solution for intelligently hiding or obscuring private data from graphical displays via the use of embedded steganographic image data techniques.

## Claims

### Claim 1

A system for intelligent steganographic graphical data protection for use in screen protection of information displayed on a graphical interface of a user device, the system comprising:
at least one memory device with computer-readable program code stored thereon;
at least one communication device;
at least one processing device operatively coupled to the at least one memory device and the at least one communication device, wherein executing the computer-readable code is configured to cause the at least one processing device to:
receive an original image for analysis from a user device, wherein the original image comprises an image to be displayed on a graphical user interface of the user device;
encode the original image using multiple convolutional neural network layers;
embed, simultaneously, a steganographic layer containing randomized portions of a secret image into the encoded image such that the randomized portions of the secret image obscure portions of the original image;
encrypt and store pooling indices for the randomized portions of the secret image in encoding feature variance layers of the encoded image to thereby create an obscured image; access user-specific location or time preference data, wherein the user-specific location or time preference data comprise specific locations or times wherein the user prefers to activate screen protection;
compare the user-specific location or time preference data to current geolocation of the user device or a time of day at the current geolocation of the user device; and
in response to the user-specific location or time preference data matching either the geolocation of the user device or the time of day at the current geolocation of the user device, send the obscured image to the user device for display on the graphical user interface.

### Claim 2 (depends on 1)

The system of claim 1, wherein the system is further configured to:
apply a stegoanalyser to the encoded image to remove the steganographic layer resulting in a processed encoded image; and
decode the processed encoded image using additional multiple convolutional neural network layers to identify and extract feature data.

### Claim 3 (depends on 1)

The system of claim 1, wherein the multiple convolutional neural network layers further comprise batch normalization to normalize a scale of the encoded image and reduce internal covariance shift between the multiple convolutional neural network layers.

### Claim 4 (depends on 1)

The system of claim 1, wherein the multiple convolutional neural network layers further comprise using one or more linear-activation functions.

### Claim 5 (depends on 1)

The system of claim 1, wherein embedding the steganographic layer containing randomized portions of the secret image further comprises applying multiple hidden convolutional layers during encoding and wherein the hidden layers are applied between at least two separate convolutional neural network layers.

### Claim 6 (depends on 1)

The system of claim 1, wherein the system is further configured to:
receive geolocation data associated with the user device;
based on the geolocation data, determine that the user device is in a public location; and
determine portions of the original image to obscure based on the user device being in a public location.

### Claim 7 (depends on 1)

The system of claim 1, wherein the system is further configured to:
store the user-specific location or time preferences in the at least one memory device; and
continuously monitor geolocation data or time of day data at the geolocation of the user device.

### Claim 8

A computer program product for intelligent steganographic graphical data protection for use in screen protection of information displayed on a graphical interface of a user device with at least one non-transitory computer-readable medium having computer-readable program code portions embodied therein, the computer-readable program code portions comprising:
an executable portion configured to receive an original image for analysis from a user device, wherein the original image comprises an image to be displayed on a graphical user interface of the user device;
an executable portion configured to encode the original image using multiple convolutional neural network layers;
an executable portion configured to embed, simultaneously, a steganographic layer containing randomized portions of a secret image into the encoded image such that the randomized portions of the secret image obscure portions of the original image;
an executable portion configured to encrypt and store pooling indices for the randomized portions of the secret image in encoding feature variance layers of the encoded image to thereby create an obscured image;
an executable portion configured to access user-specific location or time preference data, wherein the user-specific location or time preference data comprise specific locations or times wherein the user prefers to activate screen protection;
an executable portion configured to compare the user-specific location or time preference data to current geolocation of the user device or a time of day at the current geolocation of the user device; and
in response to the user-specific location or time preference data matching either the geolocation of the user device or the time of day at the current geolocation of the user device, an executable portion configured to send the obscured image to the user device for display on the graphical user interface.

### Claim 9 (depends on 8)

The computer program product of claim 8, further comprising:
an executable portion configured to apply a stegoanalyser to the encoded image to remove the steganographic layer resulting in a processed encoded image; and
an executable portion configured to decode the processed encoded image using additional multiple convolutional neural network layers to identify and extract feature data.

### Claim 10 (depends on 8)

The computer program product of claim 8, wherein the multiple convolutional neural network layers further comprise batch normalization to normalize a scale of the encoded image and reduce internal covariance shift between the multiple convolutional neural network layers.

### Claim 11 (depends on 8)

The computer program product of claim 8, wherein the multiple convolutional neural network layers further comprise using one or more linear-activation functions.

### Claim 12 (depends on 8)

The computer program product of claim 8, wherein embedding the steganographic layer containing randomized portions of the secret image further comprises applying multiple hidden convolutional layers during encoding and wherein the hidden layers are applied between at least two separate convolutional neural network layers.

### Claim 13 (depends on 8)

The computer program product of claim 8, further comprising:
an executable portion configured to receive geolocation data associated with the user device;
an executable portion configured to, based on the geolocation data, determine that the user device is in a public location; and
an executable portion configured to determine portions of the original image to obscure based on the user device being in a public location.

### Claim 14 (depends on 8)

The computer program product of claim 8, further comprising:
an executable portion configured to store the user-specific location or time preferences in the at least one memory device; and
an executable portion configured to continuously monitor geolocation data or time of day data at the geolocation of the user device.

### Claim 15

A computer-implemented method for intelligent steganographic graphical data protection for use in screen protection of information displayed on a graphical interface of a user device, the method comprising:
receiving an original image for analysis from a user device, wherein the original image comprises an image to be displayed on a graphical user interface of the user device;
encoding the original image using multiple convolutional neural network layers;
embedding, simultaneously, a steganographic layer containing randomized portions of a secret image into the encoded image such that the randomized portions of the secret image obscure portions of the original image;
encrypting and store pooling indices for the randomized portions of the secret image in encoding feature variance layers of the encoded image to thereby create an obscured image;
accessing user-specific location or time preference data, wherein the user-specific location or time preference data comprise specific locations or times wherein the user prefers to activate screen protection;
comparing the user-specific location or time preference data to current geolocation of the user device or a time of day at the current geolocation of the user device; and
in response to the user-specific location or time preference data matching either the geolocation of the user device or the time of day at the current geolocation of the user device, sending the obscured image to the user device for display on the graphical user interface.

### Claim 16 (depends on 15)

The computer-implemented method of claim 15, the method further comprising:
applying a stegoanalyser to the encoded image to remove the steganographic layer resulting in a processed encoded image; and
decoding the processed encoded image using additional multiple convolutional neural network layers to identify and extract feature data.

### Claim 17 (depends on 15)

The computer-implemented method of claim 15, wherein the multiple convolutional neural network layers further comprise batch normalization to normalize a scale of the encoded image and reduce internal covariance shift between the multiple convolutional neural network layers.

### Claim 18 (depends on 15)

The computer-implemented method of claim 15, wherein the multiple convolutional neural network layers further comprise using one or more linear-activation functions.

### Claim 19 (depends on 15)

The computer-implemented method of claim 15, the method further comprising:
receiving geolocation data associated with the user device;
based on the geolocation data, determining that the user device is in a public location; and
determining portions of the original image to obscure based on the user device being in a public location.

### Claim 20 (depends on 15)

The computer-implemented method of claim 15, the method further comprising:
storing the user-specific location or time preferences in the at least one memory device; and
continuously monitoring geolocation data or time of day data at the geolocation of the user device.
