# US 2020/0401851 A1 · Image Classification and Labeling

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2020/0401851 A1
- Document ID: US20200401851A1
- Status: Published application
- Kind: A1
- Application number: 16/074,399
- Filed: 2017-02-01
- Published: 2020-12-24
- Earliest claimed: 2016-02-01
- Assignees: See-Out Pty Ltd.
- Applicants: See-Out Pty Ltd.
- Inventors: SANDRA MAU; SABESAN SIVAPALAN
- Web page: https://patentgrep.com/patent/US20200401851A1

## CPC

- G06N 3/08: Learning methods
- G06F 16/55: Clustering; Classification
- G06F 16/583: using metadata automatically derived from the content
- G06F 18/2155: {characterised by the incorporation of unlabelled data, e.g. multiple instance learning [MIL], semi-supervised techniques using expectation-maximisation [EM] or naïve labelling}
- G06F 18/2415: based on parametric or probabilistic models, e.g. based on likelihood ratio or false acceptance rate versus a false rejection rate
- G06F 18/254: {of classification results, e.g. of results related to same input data}
- G06N 3/045: Combinations of networks
- G06N 3/0464: Convolutional networks [CNN, ConvNet]
- G06N 3/09: Supervised learning
- G06V 10/454: {Integrating the filters into a hierarchical structure, e.g. convolutional neural networks [CNN]}
- G06V 10/764: using classification, e.g. of video objects
- G06V 10/774: Generating sets of training patterns; Bootstrap methods, e.g. bagging or boosting
- G06V 10/776: Validation; Performance evaluation
- G06V 10/82: using neural networks
- G06V 20/10: Terrestrial scenes (scenes under surveillance with static cameras G06V20/52; scenes perceived from the exterior of a vehicle G06V20/56; scenes perceived from the interior of a vehicle G06V20/59)
- G06V 20/70: Labelling scene content, e.g. deriving syntactic or semantic representations
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING

## Abstract

A method of training an image classification model includes obtaining training images associated with labels, where two or more labels of the labels are associated with each of the training images and where each label of the two or more labels corresponds to an image classification class. The method further includes classifying training images into one or more classes using a deep convolutional neural network, and comparing the classification of the training images against labels associated with the training images. The method also includes updating parameters of the deep convolutional neural network based on the comparison of the classification of the training images against the labels associated with the training images.

## Claims

### Claim 1

A computer implemented method of classifying images using one or more image classification models, the method comprising:
obtaining training images associated with labels, wherein two or more labels of the labels are associated with each of the training images and wherein each label of the two or more labels corresponds to an image classification class;
training a deep convolutional neural network using the training images and the labels associated with the training images; and
classifying an input image into two or more classes based on the trained deep convolutional neural network.

### Claim 2 (depends on 1)

The method of claim 1, wherein a classification layer of the deep convolutional neural network is based on soft-sigmoid activation.

### Claim 3 (depends on 1)

The method of claim 1, wherein the training images and the input images include graphically designed images.

### Claim 4 (depends on 1)

The method of claim 1, wherein the labels have a hierarchical structure and wherein the labels are non-mutually exclusive labels.

### Claim 5 (depends on 4)

The method of claim 4, wherein a separate deep convolutional neural network is trained for each hierarchy level of the labels.

### Claim 6 (depends on 1)

The method of claim 1, wherein the labels are codes used by a trademark registration organization.

### Claim 7 (depends on 1)

The method of claim 1, wherein the labels are codes used to classify design patent images or industrial design images.

### Claim 8 (depends on 1)

The method of claim 1, wherein the labels are available as metadata of the training images associated with the labels.

### Claim 9 (depends on 1)

The method of claim 1, wherein classifying the input image into two or more classes based on the trained deep convolutional neural network includes tagging or labelling the input image with two or more labels corresponding to the two or more classes.

### Claim 10 (depends on 1)

The method of claim 1, further comprising pre-processing the training images, wherein training the deep convolutional neural network using the training images and the labels associated with the training images comprises training the deep convolutional neural network using the pre-processed training images and the labels associated with the training images.

### Claim 11

A computer implemented method of training an image classification model, the method comprising:
obtaining training images associated with labels, wherein two or more labels of the labels are associated with each of the training images and wherein each label of the two or more labels corresponds to an image classification class;
classifying training images into one or more classes using a deep convolutional neural network;
comparing the classification of the training images against labels associated with the training images; and
updating parameters of the deep convolutional neural network based on the comparison of the classification of the training images against the labels associated with the training images.

### Claim 12 (depends on 11)

The method of claim 11, wherein the training images include graphically designed images.

### Claim 13 (depends on 11)

The method of claim 11, wherein the labels have a hierarchical structure.

### Claim 14 (depends on 13)

The method of claim 13, wherein a separate deep convolutional neural network is trained for each hierarchy level of the labels.

### Claim 15 (depends on 11)

The method of claim 11, wherein the labels are codes used by a trademark registration organization.

### Claim 16 (depends on 11)

The method of claim 11, further comprising pre-processing the training images before classifying the training images comprises classifying the pre-processed training images.

### Claim 17 (depends on 11)

The method of claim 11, wherein a classification layer of the deep convolutional neural network is based on soft-sigmoid activation.

### Claim 18

A system for classifying images using one or more image classification models, the system comprising:
a training image source comprising training images associated with labels, wherein two or more labels of the labels are associated with each of the training images;
an image processing device communicably coupled to the training image source, wherein the image processing device is configured to obtain the training images from the training image source and to train a deep convolutional neural network using the training images and the labels associated with the training images; and
an input image source comprising input images to be classified into classes, wherein classifying an input image into multiple classes associates the input image with multiple classes corresponding to the multiple classes.

### Claim 19 (depends on 18)

The system of claim 18, wherein a classification layer of the deep convolutional neural network is based on soft-sigmoid activation.

### Claim 20 (depends on 18)

The system of claim 18, wherein the labels are codes used by a trademark registration organization or an industrial design registration organization.