# US 2017/0286809 A1 · Visual object recognition

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2017/0286809 A1
- Document ID: US20170286809A1
- Status: Published application
- Kind: A1
- Application number: 15/089,707
- Filed: 2016-04-04
- Published: 2017-10-05
- Earliest claimed: 2016-04-04
- Assignees: International Business Machines Corporation
- Applicants: International Business Machines Corporation
- Inventors: Sharathchandra U. Pankanti; Xi Peng; Nalini K. Ratha
- Web page: https://patentgrep.com/patent/US20170286809A1

## CPC

- G06V 10/82: using neural networks
- G06F 18/2414: {Smoothing the distance, e.g. radial basis function networks [RBFN]}
- G06N 3/045: Combinations of networks
- G06N 3/0464: Convolutional networks [CNN, ConvNet]
- G06N 3/08: Learning methods
- G06N 3/084: Backpropagation, e.g. using gradient descent
- G06N 3/09: Supervised learning
- G06T 7/30: Determination of transform parameters for the alignment of images, i.e. image registration
- G06T 7/70: Determining position or orientation of objects or cameras (camera calibration G06T7/80)
- G06V 10/454: {Integrating the filters into a hierarchical structure, e.g. convolutional neural networks [CNN]}
- G06V 20/647: {by matching two-dimensional images to three-dimensional objects}
- G06V 30/19173: {Classification techniques}
- G06V 40/161: {Detection; Localisation; Normalisation}
- G06V 40/172: {Classification, e.g. identification}
- G06N 3/048: Activation functions
- G06T 2207/20081: Training; Learning
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL

## Abstract

Technical solutions are described for training an object-recognition neural network that identifies an object in a computer-readable image. An example method includes assigning a first neural network for determining a visual alignment model of the images for determining a normalized alignment of the object. The method further includes assigning a second neural network for determining a visual representation model of the images for recognizing the object. The method further includes determining the visual alignment model by training the first neural network and determining the visual representation model by training the second neural network independent of the first. The method further includes determining a combined object recognition model by training a combination of the first neural network and the second neural network. The method further includes recognizing the object in the image based on the combined object recognition model by passing the image through each of the neural networks.

## Claims

### Claim 1

A computer-implemented method for training an object-recognition neural network to identify an object in a computer-readable image, the method comprising:
assigning, using a processor system, a first neural network for determining a visual alignment model of the images, wherein the visual alignment model is used to determine a normalized alignment of an object in input images;
assigning, using the processor system, a second neural network for determining a visual representation model of the images, wherein the visual representation model is used to recognize the object in the input images;
determining the visual alignment model by training the first neural network;
determining the visual representation model by training the second neural network;
determining a combined object recognition model by training a combination of the first neural network and the second neural network; and
recognizing the object in the computer-readable image based on the combined object recognition model by passing the computer-readable image through each of the combined neural networks.

### Claim 2 (depends on 1)

The computer-implemented method of claim 1, further comprising:
splitting an object-recognition neural network into a plurality of independent networks including the first neural network and the second neural network, wherein a first subset of the independent networks operates on content-based attributes of the images and a second subset of the independent networks operates on content-unaware attributes of the images.

### Claim 3 (depends on 1)

The computer-implemented method of claim 1, wherein the visual alignment model is determined by training the first neural network using a first set of images and the visual representation model is determined by training the second neural network using a second set of images.

### Claim 4 (depends on 3)

The computer-implemented method of claim 3, wherein the combined object recognition model is determined by training a combination of the first neural network and the second neural network using a third set of images, distinct from the first set of images and the second set of images.

### Claim 5 (depends on 1)

The computer-implemented method of claim 1, wherein the first neural network is trained in parallel to the second neural network.

### Claim 6 (depends on 5)

The computer-implemented method of claim 5, wherein the first neural network is trained by a first computer system and the second neural network is trained by a second computer system, which is distinct from the first computer system.

### Claim 7 (depends on 1)

The computer-implemented method of claim 1, wherein the first neural network is trained by a first processor in parallel to the second neural network being trained by a second processor.

### Claim 8 (depends on 1)

The computer-implemented method of claim 1, further comprising:
assigning a third neural network for determining a contrast normalization model of the images, wherein the contrast normalization model is used to determine a normalized contrast value of the input images; and
determining the combined object recognition model by further training a combination of the first neural network, the second neural network, and the third neural network.

### Claim 9 (depends on 8)

The computer-implemented method of claim 8, wherein recognizing the object in the computer-readable image based on the combined object recognition model comprises:
passing the computer-readable image the first neural network and the third neural network in parallel; and
passing outputs of the first neural network and the third neural network to the second neural network, wherein the second neural network recognizes the object in the computer-readable image based on the respective outputs.

### Claim 10 (depends on 1)

The computer-implemented method of claim 1, wherein determining the normalized alignment by training the first neural network comprises determining transformations in 3-dimensional space to align the object in the computer-readable image in a normalized position.

### Claim 11

A system for training an object-recognition neural network to identify an object in a computer-readable image, the system comprising:
a memory; and
a processor communicatively coupled to the memory, wherein the processor is configured to:
assign a first neural network for determining a visual alignment model of the images, wherein the visual alignment model is used to determine a normalized alignment of an object in input images;
assign a second neural network for determining a visual representation model of the images, wherein the visual representation model is used to recognize the object in the input images;
determine the visual alignment model by training the first neural network;
determine the visual representation model by training the second neural network;
determine a combined object recognition model by training a combination of the first neural network and the second neural network; and
recognize the object in the computer-readable image based on the combined object recognition model by passing the computer-readable image through each of the combined neural networks.

### Claim 12 (depends on 11)

The system of claim 11, wherein the processor is further configured to split an object-recognition neural network into a plurality of independent networks including the first neural network and the second neural network, wherein a first subset of the independent networks operates on content-based attributes of the images and a second subset of the independent networks operates on content-unaware attributes of the images.

### Claim 13 (depends on 11)

The system of claim 11, wherein the visual alignment model is determined by training the first neural network using a first set of images and the visual representation model is determined by training the second neural network using a second set of images.

### Claim 14 (depends on 13)

The system of claim 13, wherein the combined object recognition model is determined by training a combination of the first neural network and the second neural network using a third set of images, distinct from the first set of images and the second set of images.

### Claim 15 (depends on 11)

The system of claim 11, wherein the first neural network is trained in parallel to the second neural network.

### Claim 16 (depends on 11)

The system of claim 11, wherein the first neural network is trained by a first computer system and the second neural network is trained by a second computer system, which is distinct from the first computer system.

### Claim 17

A computer program product for training an object-recognition neural network to identify an object in a computer-readable image, the computer program product comprising a non-transitory computer readable storage medium, the computer readable storage medium comprising computer executable instructions, wherein the computer readable storage medium comprises instructions to:
assign a first neural network for determining a visual alignment model of the images, wherein the visual alignment model is used to determine a normalized alignment of an object in input images;
assign a second neural network for determining a visual representation model of the images, wherein the visual representation model is used to recognize the object in the input images;
determine the visual alignment model by training the first neural network;
determine the visual representation model by training the second neural network;
determine a combined object recognition model by training a combination of the first neural network and the second neural network; and
recognize the object in the computer-readable image based on the combined object recognition model by passing the computer-readable image through each of the combined neural networks.

### Claim 18 (depends on 17)

The computer program product of claim 17, wherein the visual alignment model is determined by training the first neural network using a first set of images and the visual representation model is determined by training the second neural network using a second set of images.

### Claim 19 (depends on 18)

The computer program product of claim 18, wherein the combined object recognition model is determined by training a combination of the first neural network and the second neural network using a third set of images, distinct from the first set of images and the second set of images.

### Claim 20 (depends on 17)

The computer program product of claim 17, wherein the first neural network is trained in parallel to the second neural network, and wherein the first neural network is trained by a first computer system and the second neural network is trained by a second computer system, which is distinct from the first computer system.