# US 10,872,258 B2 · Adaptive image cropping for face recognition

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 10,872,258 B2
- Document ID: US10872258B2
- Status: Granted
- Kind: B2
- Application number: 16/355,665
- Filed: 2019-03-15
- Published: 2020-09-17
- Granted: 2020-12-22
- Earliest claimed: 2019-03-15
- Assignees: Huawei Technologies Co., Ltd.
- Applicants: Ali Ghobadzadeh; Juwei Lu; Richard Patrick Wildes; Wei Li
- Inventors: Ali Ghobadzadeh; Juwei Lu; Richard Patrick Wildes; Wei Li
- Web page: https://patentgrep.com/patent/US10872258B2

## CPC

- G06N 3/084: Backpropagation, e.g. using gradient descent
- G06F 18/21: Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
- G06N 3/04: Architecture, e.g. interconnection topology
- G06N 3/045: Combinations of networks
- G06N 3/0464: Convolutional networks [CNN, ConvNet]
- G06N 3/09: Supervised learning
- G06V 10/25: Determination of region of interest [ROI] or a volume of interest [VOI]
- G06V 10/449: {Biologically inspired filters, e.g. difference of Gaussians [DoG] or Gabor filters}
- G06V 10/764: using classification, e.g. of video objects
- G06V 10/82: using neural networks
- G06V 40/161: {Detection; Localisation; Normalisation}
- G06V 40/168: {Feature extraction; Face representation}
- G06N 7/01: Probabilistic graphical models, e.g. probabilistic networks
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING

## Abstract

By adding a side network to a face recognition network, output of early convolution blocks may be used to determine relative bounding box values. The relative bounding box values may be used to refine existing boundary box value with an eye on improving the generation, by the face recognition network, of embedding vectors.

## Claims

### Claim 1

A method of operating a face recognition system, the face recognition system including a side network and a face recognition network arranged to receive a cropped image of a face and produce an embedding vector representative of features of the face, the face recognition network implemented as a convolutional neural network including a series connection of a plurality of face recognition convolutional blocks terminating at a face recognition fully connected network, the cropped image of the face having a cropped bounding box relative to an original bounding box of an original image of the face, the method comprising:
receiving, at the side neural network, output from selected face recognition convolutional blocks among the plurality of face recognition convolutional blocks;
processing, at the side neural network, the output to produce relative bounding box values; and
employing the relative bounding box values to define a new bounding box for the image.

### Claim 2 (depends on 1)

The method of claim 1, wherein the side network comprises a plurality of branches that implement one or more of a max-pooling block, a convolution block and a resizing block.

### Claim 3 (depends on 2)

The method of claim 2, further comprising:
concatenating, at the side neural network, output from each branch among the plurality of branches to, thereby, produce a combined output vector; and
processing the combined output vector to, thereby, generate the relative bounding box values.

### Claim 4 (depends on 3)

The method of claim 3 wherein the processing the combined output vector comprises applying a 1×1 convolution block to the combined output vector to, thereby generate 1×1 convolution block output.

### Claim 5 (depends on 4)

The method of claim 4 further comprising flattening the 1×1 convolution block output.

### Claim 6 (depends on 1)

The method of claim 1, wherein the side network comprises a branch that implements a max-pooling block and a side network convolutional block.

### Claim 7 (depends on 6)

The method of claim 6 wherein the max-pooling block comprises a 2×2 max-pooling block connected to output from a first face recognition convolutional block of the plurality of face recognition convolutional blocks.

### Claim 8 (depends on 7)

The method of claim 7 wherein the max-pooling block has stride 2.

### Claim 9 (depends on 4)

The method of claim 4 wherein the side network convolutional block has 256 filters of size 3×3.

### Claim 10 (depends on 4)

The method of claim 4 wherein the side network convolutional block is connected to output from the max-pooling block.

### Claim 11 (depends on 1)

The method of claim 1 wherein the side network comprises a branch that implements a side network convolutional block with 256 filters of 4×4.

### Claim 12 (depends on 11)

The method of claim 11 wherein the side network convolutional block is connected to a third face recognition convolutional block of the plurality of face recognition convolutional blocks.

### Claim 13 (depends on 1)

The method of claim 1 wherein the side network comprises a branch that implements a side network convolutional block with 256 filters of size 5×5.

### Claim 14 (depends on 13)

The method of claim 13 wherein the side network convolutional block is connected to a fifth face recognition convolutional block of the plurality of face recognition convolutional blocks.

### Claim 15

A face recognition apparatus comprising:
a memory storing instructions; and
a processor configured, by executing the instructions, to:
implement a face recognition network, the face recognition network arranged to:
receive a cropped image of a face; and
produce an embedding vector representative of features of the face;
where the face recognition network is implemented as a convolutional neural network including a series connection of a plurality of face recognition convolutional blocks terminating at a face recognition fully connected network, the cropped image of the face having a cropped bounding box relative to an original bounding box of an original image of the face;
implement a side neural network to:
receive output from selected face recognition convolutional blocks among the plurality of face recognition convolutional blocks;
process the output to produce relative bounding box values; and
implement an image processing unit to:
receive the relative bounding box values; and
employ the relative bounding box values to define a new bounding box for the image.

### Claim 16 (depends on 15)

The face recognition apparatus of claim 15, wherein the side network comprises a plurality of branches that implement one or more of a max-pooling block, a convolution block and a resizing block.

### Claim 17 (depends on 16)

The face recognition apparatus of claim 16, wherein the processor is further configured to implement the side neural network to:
concatenate output from each branch among the plurality of branches to, thereby, produce a combined output vector; and
process the combined output vector to, thereby, generate the relative bounding box values.

### Claim 18

A non-transitory computer-readable medium having instructions stored thereon, the instructions executable by a processor of a face recognition apparatus, the instructions causing the processor to:
implement a face recognition network, the face recognition network arranged to:
receive a cropped image of a face; and
produce an embedding vector representative of features of the face;
where the face recognition network is implemented as a convolutional neural network including a series connection of a plurality of face recognition convolutional blocks terminating at a face recognition fully connected network, the cropped image of the face having a cropped bounding box relative to an original bounding box of an original image of the face;
implement a side neural network to:
receive output from selected face recognition convolutional blocks among the plurality of face recognition convolutional blocks;
process the output to produce relative bounding box values; and
implement an image processing unit to:
receive the relative bounding box values; and
employ the relative bounding box values to define a new bounding box for the image.

### Claim 19 (depends on 18)

The computer-readable medium of claim 18, wherein the side network comprises a plurality of branches that implement one or more of a max-pooling block, a convolution block and a resizing block.

### Claim 20 (depends on 19)

The computer-readable medium of claim 19, wherein the instructions further cause the processor to:
concatenate output from each branch among the plurality of branches to, thereby, produce a combined output vector; and
process the combined output vector to, thereby, generate the relative bounding box values.