# US 12,223,700 B2 · Automatic image analysis method for automatically recognizing at least one rare characteristic

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 12,223,700 B2
- Document ID: US12223700B2
- Status: Granted
- Kind: B2
- Application number: 17/610,098
- Filed: 2020-05-07
- Published: 2022-07-28
- Granted: 2025-02-11
- Earliest claimed: 2019-05-10
- Assignees: Universite Brest Bretagne Occidentale; Institut National De La Sante Et De La Recherche Medicale (Inserm)
- Applicants: Universite De Brest; Inserm
- Inventors: Gwenolé Quellec
- Web page: https://patentgrep.com/patent/US12223700B2

## CPC

- G06V 10/82: using neural networks
- G06F 18/2135: based on approximation criteria, e.g. principal component analysis
- G06F 18/2431: Multiple classes
- G06N 3/045: Combinations of networks
- G06N 3/0464: Convolutional networks [CNN, ConvNet]
- G06N 3/09: Supervised learning
- G06T 7/74: {involving reference images or patches}
- G06V 10/761: {Proximity, similarity or dissimilarity measures}
- G06V 10/7715: {Feature extraction, e.g. by transforming the feature space, e.g. multi-dimensional scaling [MDS]; Mappings, e.g. subspace methods}
- G06T 2207/20081: Training; Learning
- G06T 2207/20084: Artificial neural networks [ANN]
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL

## Abstract

Disclosed is an automatic image analysis method that can be used to automatically recognise at least one rare characteristic in an image to be analysed. The method comprises a learning phase during which at least one convolutional neural network is trained to recognise characteristics, a parameter space of dimension n, in which n≥2, is constructed from at least one intermediate layer of the network, a presence probability function is determined for each characteristic in the parameter space from a projection of reference images in the parameter space. During a phase of analysing the image to be analysed, the method comprises a step of recognising the at least one rare characteristic in the image to be analysed on the basis of the presence probability function determined for the at least one rare characteristic.

## Claims

### Claim 1

An automatic image analysis method, said method comprising a learning phase and an analysis phase, said method comprising:
during said learning phase:
a step of supervised deep learning of at least one convolutional neural network on the basis of a collection of images associated with annotations for generating at least one trained convolutional neural network, arranged in layers and forming a learning model configured to recognize at least one frequent characteristic in an image to be analyzed, said collection of images comprising images having said at least one frequent characteristic and images having at least one rare characteristic, a frequent characteristic corresponding to a characteristic presenting an appearance frequency on the collection of images which is equal or greater than a frequency threshold; a rare characteristic corresponding to a characteristic presenting an appearance frequency on the collection of images which is lower than a frequency threshold;
an annotation indicating whether a rare or frequent characteristic is present in an image of the collection of images;
a step of constructing a parameter space of dimension n, with n ≥2, each parameter originating from at least one intermediate layer of said at least one trained convolutional neural network and each parameter representing a dimension of said parameter space;
a step of determining a presence probability function in said parameter space for at least one rare characteristic on the basis of a projection, in said parameter space, of said images of the collection of images and annotations relating to said at least one rare characteristic;
during said analysis phase:
for an image to be analyzed, a step of recognizing said at least one rare characteristic in said image to be analyzed on the basis of the presence probability function determined for said at least one rare characteristic.

### Claim 2 (depends on 1)

The analysis method as claimed in claim 1, further comprising, during the learning phase, a step of determining a presence probability function in said parameter space for said at least one frequent characteristic on the basis of a projection, in said parameter space, of said images of the collection of images and annotations relating to said at least one frequent characteristic.

### Claim 3 (depends on 2)

The analysis method as claimed in claim 2, wherein the supervised deep learning step of said at least one convolutional neural network is supervised by said presence probability function of said at least one frequent characteristic.

### Claim 4 (depends on 1)

The analysis method as claimed in claim 1, wherein, during the supervised deep learning step, a plurality of convolutional neural networks is trained separately or jointly, with each of said convolutional neural networks being trained to recognize said at least one frequent characteristic.

### Claim 5 (depends on 1)

The analysis method as claimed in claim 1, wherein said parameter space is constructed on the basis of values of output parameters originating from at least one intermediate layer of said at least one convolutional neural network, said at least one intermediate layer being selected from among the penultimate layers of said at least one convolutional neural network.

### Claim 6 (depends on 1)

The analysis method as claimed in claim 1, comprising a step of determining an absence probability function in said parameter space for said at least one rare characteristic on the basis of a projection, in said parameter space, of said images of the collection of images and annotations relating to said at least one rare characteristic,
said at least one intermediate layer being selected based on the maximization, for at least one considered rare characteristic, of the Patrick-Fischer distance between the presence probability function of said at least one considered rare characteristic and the absence probability function of said at least one considered rare characteristic.

### Claim 7 (depends on 1)

The analysis method as claimed in claim 1, wherein the parameter space is a reduced parameter space and the step of constructing the reduced parameter space comprises a step of reducing the dimension of an initial parameter space at the output of said at least one intermediate layer of said at least one trained convolutional neural network.

### Claim 8 (depends on 7)

The analysis method as claimed in claim 7, wherein said step of reducing the dimension of the initial parameter space is based on a principal component analysis algorithm and/or on a t-SNE algorithm.

### Claim 9 (depends on 1)

The analysis method as claimed in claim 1, further comprising, for at least one rare characteristic, a step of determining, in the image to be analyzed, pixels responsible for recognizing said considered rare characteristic.

### Claim 10 (depends on 8)

The analysis method as claimed in claim 8, the recognition step comprising:
constructing a second reduced parameter space by reducing the parameter space at the output of said at least one intermediate layer of said at least one trained convolutional neural network;
projecting, in the second parameter space, the image to be analyzed in order to obtain a projected image;
obtaining reference projected images in the second parameter space by projecting reference images of the collection of images; and
estimating a probability that the image to be analyzed contains the rare characteristic, said probability being computed by regression on the basis of presence probabilities of the rare characteristic determined for the reference images for which the reference projected images are the nearest neighbors of the projected image.

### Claim 11 (depends on 10)

The analysis method as claimed in claim 10, wherein the second reduced parameter space is constructed by means of a principal component analysis applied to the parameter space at the output of said at least one intermediate layer and wherein the projection of an image in the second reduced parameter space is obtained by applying said at least one trained convolutional network to the considered image in order to obtain an output, then applying a projection function originating from the principal component analysis to this output.

### Claim 12 (depends on 10)

The analysis method as claimed in claim 10, wherein the presence probability of a rare characteristic in a reference image is obtained by projecting the reference image in the reduced parameter space constructed for the rare characteristic so as to obtain a reference projected image, then applying the presence probability function defined in the reduced parameter space to the reference projected image.

### Claim 13 (depends on 1)

An analysis device configured to implement the steps of the analysis method as claimed in claim 1.