# US 2023/0410471 A1 · Method and network to embed image data and meta data

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2023/0410471 A1
- Document ID: US20230410471A1
- Status: Published application
- Kind: A1
- Application number: 18/031,850
- Filed: 2021-11-12
- Published: 2023-12-21
- Earliest claimed: 2020-11-12
- Assignees: Volpara Health Technologies Limited
- Applicants: Volpara Health Technologies Limited
- Inventors: Ralph HIGHNAM
- Web page: https://patentgrep.com/patent/US20230410471A1

## CPC

- G06V 10/454: {Integrating the filters into a hierarchical structure, e.g. convolutional neural networks [CNN]}
- G06V 10/764: using classification, e.g. of video objects
- G06V 10/7788: {the supervisor being a human, e.g. interactive learning with a human teacher}
- G06V 10/82: using neural networks
- G06V 2201/03: Recognition of patterns in medical or anatomical images
- G06V 2201/10: Recognition assisted with metadata
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING

## Abstract

The present invention relates to a system and method to embed meta data from an imaging and communications system whereby the meta data is combined with image data as an input to a deep learning network. An image classification learning network is disclosed which comprises: a means to input image data and meta data; and an embedding layer comprising learnable embedding weights to encode the meta data to provide a learned object, and a softmax layer to classify a combination of the image data and the learned object.

## Claims

### Claim 1

An image classification learning network comprising:
a means to input image data and meta data; and an embedding layer comprising learnable embedding weights to encode the meta data to provide a learned object, and
a softmax layer to classify a combination of the image data and the learned object.

### Claim 2 (depends on 1)

The image classification learning network according to claim 1, wherein the embedding layer comprises a fully connected layer of neurons to regress an embedding matrix comprising the learnable embedding weights when encoded with the meta data.

### Claim 3 (depends on 2)

The image classification learning network according to claim 2, wherein the embedding matrix is connected to the fully connected layer of neurons.

### Claim 4 (depends on 1)

The image classification learning network according to claim 1 wherein the embedding layer comprises a multiplication of a meta data vector comprising the meta data multiplying a weights matrix comprising the learnable embedding weights to create the embedding matrix.

### Claim 5 (depends on 4)

The image classification learning network according to claim 4, wherein the weights matrix comprises one row of the learnable embedding weights to weight all elements of the meta data equally.

### Claim 6 (depends on 4)

The image classification learning network according to claim 4, wherein the weights matrix comprises a number of rows equal to a number of elements of the meta data to weight the elements of the meta data individually.

### Claim 7 (depends on 1)

The image classification learning network according to claim 1, wherein the number of learnable embedding weights are selectable by a user to influence accuracy of classification.

### Claim 8 (depends on 1)

The image classification learning network according to claim 1, comprising a convolution layer configured to convolve the learned object provided by the embedding layer when concatenated with the image data.

### Claim 9

The image classification network according to claim comprising a convolution layer which follows a concatenation layer which combines the image data with the learned object.

### Claim 10 (depends on 1)

The image classification learning network according to claim 1, comprising a convolution layer configured to provide convolved image data which are further concatenated with the learned object from the embedding layer.

### Claim 11 (depends on 1)

The image classification learning network according to claim 1, comprising a convolution layer which leads a concatenation layer which combines the image data convolved by the convolution layer with the learned object.

### Claim 12 (depends on 1)

The image classification learning network according to claim 1, where the input means is configured to extract the image data and/or the meta data from a DICOM® object defined by an international standard for the communication, exchange, transmission and management of medical imaging information across various imaging modalities.

### Claim 13 (depends on 1)

The image classification learning network according to claim 1, wherein the softmax layer is configured to provide a probability that a classification category characterizes an image or feature from the image data and meta data.

### Claim 14

An image classification learning method including:
inputting image data and meta data from a standardized transmission or storage;
transferring the meta data to an embedding layer wherein learnable embedding weights encode the meta data to provide a learned object, and
using a softmax layer to classify a combination of the image data and the learned object.
