# US 2024/0153146 A1 · Encoding data matrices into color channels of images using neural networks and deep learning

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2024/0153146 A1
- Document ID: US20240153146A1
- Status: Published application
- Kind: A1
- Application number: 17/981,729
- Filed: 2022-11-07
- Published: 2024-05-09
- Earliest claimed: 2022-11-07
- Assignees: Rezolve Ltd.
- Applicants: Rezolve Ltd.
- Inventors: Salman AHMAD; Daniel Maurice WAGNER
- Web page: https://patentgrep.com/patent/US20240153146A1

## CPC

- G06T 9/00: Image coding (bandwidth or redundancy reduction for static pictures H04N1/41; coding or decoding of static colour picture signals H04N1/64; methods or arrangements for coding, decoding, compressing or decompressing digital video signals H04N19/00)
- H04N 19/186: the unit being a colour or a chrominance component
- G06F 9/30036: {Instructions to perform operations on packed data, e.g. vector, tile or matrix operations}
- G06F 11/00: Error detection; Error correction; Monitoring (error detection, correction or monitoring in information storage based on relative movement between record carrier and transducer G11B20/18; monitoring, i.e. supervising the progress of recording or reproducing G11B27/36; in static stores G11C29/00)
- G06F 11/08: Error detection or correction by redundancy in data representation, e.g. by using checking codes
- G06F 17/16: Matrix or vector computation {, e.g. matrix-matrix or matrix-vector multiplication, matrix factorization (matrix transposition G06F7/78)}
- G06K 7/1473: {error correction}
- G06N 3/044: Recurrent networks, e.g. Hopfield networks
- G06N 3/0455: Auto-encoder networks; Encoder-decoder networks
- G06N 3/0464: Convolutional networks [CNN, ConvNet]
- G06N 3/08: Learning methods
- G06N 20/00: Machine learning
- G06T 1/0021: {Image watermarking}
- G06T 5/60: using machine learning, e.g. neural networks
- G06T 7/11: Region-based segmentation
- G06T 7/90: Determination of colour characteristics
- G06V 10/70: using pattern recognition or machine learning (optical pattern recognition or electronic computations therefor G06V10/88)
- G06V 10/761: {Proximity, similarity or dissimilarity measures}
- G06V 10/82: using neural networks
- H03M 13/00: Coding, decoding or code conversion, for error detection or error correction; Coding theory basic assumptions; Coding bounds; Error probability evaluation methods; Channel models; Simulation or testing of codes (error detection or error correction for analogue/digital, digital/analogue or code conversion H03M1/00 – H03M11/00; specially adapted for digital computers G06F11/08; for information storage based on relative movement between record carrier and transducer G11B, e.g. G11B20/18; for static stores G11C)
- H03M 13/1105: {Decoding}
- H03M 13/37: Decoding methods or techniques, not specific to the particular type of coding provided for in groups H03M13/03 - H03M13/35
- H03M 13/61: {Aspects and characteristics of methods and arrangements for error correction or error detection, not provided for otherwise}
- H03M 13/616: {Matrix operations, especially for generator matrices or check matrices, e.g. column or row permutations}
- H04N 19/467: characterised by the embedded information being invisible, e.g. watermarking
- G06N 3/048: Activation functions
- G06T 2207/10024: Color image
- H04L 2209/34: Encoding or coding, e.g. Huffman coding or error correction
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G06K: GRAPHICAL DATA READING (image or video recognition or understanding G06V); PRESENTATION OF DATA; RECORD CARRIERS; HANDLING RECORD CARRIERS
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- H03M: CODING; DECODING; CODE CONVERSION IN GENERAL (using fluidic means F15C4/00; optical analogue/digital converters G02F7/00; coding, decoding or code conversion, specially adapted for particular applications, see the relevant subclasses, e.g. G01D, G01R, G06F, G06T, G09G, G10L, G11B, G11C, H04B, H04L, H04M, H04N; ciphering or deciphering for cryptography or other purposes involving the need for secrecy G09C)
- H04L: TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION (arrangements common to telegraphic and telephonic communication H04M)

## Abstract

Provided herein are systems and methods of encoding messages into images. At least one server can identify a first image having a first plurality of pixels in a color space having a set of channels. The at least one server can generate, using a message to encode in the first image, a data matrix identifying a plurality of values. The at least one server can apply a machine learning (ML) model comprising a plurality of convolutional layers to the first image and to the data matrix to generate a second image having a second plurality of pixels in the color space. The second image can correspond to the first image encoded with the data matrix across the set of channels in the color space.

## Claims

### Claim 1

A system to encode messages into images, comprising:
at least one server having one or more processors coupled with memory, to:
identify a first image having a first plurality of pixels in a color space having a set of channels;
generate, using a message to encode in the first image, a data matrix identifying a plurality of values; and
apply a machine learning (ML) model comprising a plurality of convolutional layers to the first image and to the data matrix to generate a second image having a second plurality of pixels in the color space, the second image corresponding to the first image encoded with the data matrix across the set of channels in the color space.

### Claim 2 (depends on 1)

The system of claim 1, comprising:
the at least one server to generate, from a third image, a plurality of image segments including at least the first image, each of the plurality of image segments to be encoded with the message.

### Claim 3 (depends on 1)

The system of claim 1, comprising:
the at least one server to generate, from application of the ML model on a first plurality of image segments and the data matrix, a second plurality of image segments to form the second image, each of the second plurality of image segments encoded with the data matrix across the set of channels in the color space.

### Claim 4 (depends on 1)

The system of claim 1, comprising:
the at least one server to concatenate the plurality of values identified by data matrix to the first plurality of pixels of the image as a channel separate from the set of channels of the color space.

### Claim 5 (depends on 1)

The system of claim 1, comprising:
the at least one server to add, to the message to generate the data matrix, an error correction code determined based at least on a portion of the message.

### Claim 6 (depends on 1)

The system of claim 1, comprising:
the at least one server to receive the message including a resource identifier corresponding to an information resource.

### Claim 7 (depends on 1)

The system of claim 1, comprising:
the at least one server to provide the second image to be acquired by a sensor of a client device to decode the message, the second plurality of pixels of the second image having a degree of deviation from the first plurality of pixels of the first image within a threshold.

### Claim 8

A system to decode messages from images, comprising:
at least one server having one or more processors coupled with memory, to:
receive, from a client device, an image having a plurality of pixels in a color space having a set of channels across which a data matrix is encoded, responsive to the client device determining that the image is encoded;
apply a machine learning (ML) model comprising a plurality of convolutional layers to the image to identify the data matrix decoded from the plurality of pixels of the color space; and
generate a message using a plurality of values of the data matrix decoded from the image.

### Claim 9 (depends on 8)

The system of claim 8, comprising:
the at least one server to identify, from a second image acquired by a sensor of the client device, a plurality of image segments including at least the image, at least one of the plurality of image segments encoded with the message.

### Claim 10 (depends on 8)

The system of claim 8, comprising:
the at least one server to determine, based on a visual characteristic of the image, that the image is encoded, the plurality of pixels of the image having a degree of deviation from a second plurality of pixels of a second image within a threshold, the second image generated using a third image and the message.

### Claim 11 (depends on 8)

The system of claim 8, comprising:
the at least one server to recover the message in accordance with an error correction code in the plurality of values of the data matrix.

### Claim 12 (depends on 8)

The system of claim 8, comprising:
the at least one server to provide, to the client device, the message decoded from image.

### Claim 13 (depends on 8)

The system of claim 8, comprising:
the at least one server to identify a resource identifier corresponding to an information resource included in the message to provide to the client device.

### Claim 14 (depends on 8)

The system of claim 8, comprising:
the at least one server to store an association between the image and the message decoded from the image.

### Claim 15

A system to train models to encode messages into images, comprising:
at least one server having one or more processors coupled with memory, to:
identify a training dataset including: (i) a first image having a first plurality of pixels in a color space having a set of channels, (ii) a data matrix corresponding to a message to be encoded, and (iii) a second image having a second plurality of pixels in the color space corresponding to the first image encoded with the data matrix;
apply a machine learning (ML) model comprising a plurality of convolutional layers to the first image and to the data matrix to generate a third image having a third plurality of pixels in the color space, the third image corresponding to the first image encoded with the data matrix across the set of channels in the color space;
compare the third image generated from applying the ML model with the second image from the training dataset; and
update at least one of the plurality of convolutional layers in the ML model in accordance with the comparison.

### Claim 16 (depends on 15)

The system of claim 15, comprising the at least one server to:
identify a second training dataset including: (i) a fourth image having a fourth plurality of pixels in a color space having a set of channels across which a second data matrix is encoded, (ii) the second data matrix to be recovered from the fourth image;
apply a second ML model comprising a second plurality of convolutional layers to the fourth image to identify a third data matrix decoded from the fourth plurality of pixels of the color space;
compare the third data matrix identified from the second ML model and the second data matrix of the second training dataset; and
update at least one of the second plurality of convolutional layers in the second ML model in accordance with the comparison between third data matrix and the second data matrix.

### Claim 17 (depends on 16)

The system of claim 16, comprising:
the at least one server to update at least one of the second plurality of convolutional layers in accordance with a loss metric determined based on a comparison between a second message of the second data matrix and a third message of the

### Claim 18 (depends on 16)

The system of claim 16, comprising:
the second ML comprising at least one spatial pooler to combine a plurality of feature maps generated from the second plurality of convolutional layers to output the third data matrix.

### Claim 19 (depends on 15)

The system of claim 15, comprising:
the at least one server to update at least one of the first plurality of convolutional layers in accordance with a loss metric determined based on the comparison of the third image with the second image.

### Claim 20 (depends on 15)

The system of claim 15, comprising:
the ML model comprising at least one integrator to combine a plurality of feature maps generated from the plurality of convolutional layers to output the third image.
