# US 10,311,332 B2 · Orientation-based subject-matching in images

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 10,311,332 B2
- Document ID: US10311332B2
- Status: Granted
- Kind: B2
- Application number: 15/006,558
- Filed: 2016-01-26
- Published: 2017-07-27
- Granted: 2019-06-04
- Earliest claimed: 2016-01-26
- Assignees: Huawei Technologies Co., Ltd.
- Applicants: Huawei Technologies Co., Ltd
- Inventors: Shane Anthony Bergsma
- Web page: https://patentgrep.com/patent/US10311332B2

## CPC

- G06T 7/74: {involving reference images or patches}
- G06F 18/22: Matching criteria, e.g. proximity measures
- G06V 10/761: {Proximity, similarity or dissimilarity measures}
- G06T 2207/10016: Video; Image sequence
- G06T 2207/10024: Color image
- G06T 2207/20081: Training; Learning
- G06V 10/462: {Salient features, e.g. scale invariant feature transforms [SIFT]}
- G06V 30/194: References adjustable by an adaptive method, e.g. learning
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING

## Abstract

A system and method for subject-matching in images comprises detecting a first orientation of a first subject in a target image; comparing the first orientation with a second orientation of a second subject in a source image to obtain at least one orientation parameter; extracting at least one features from the target image; computing a visual similarity score between the target image and the source image using the features, the visual similarity score being computed as a function of the at least one feature and the at least one orientation parameter; and determining a match between the first subject and the second subject in accordance with the visual similarity score.

## Claims

### Claim 1

A computer-implemented method for subject-matching in images, the method comprising:
detecting, by a processor, a first orientation of a first subject in a target image using at least two angles, the at least two angles comprising a first angle α, in a horizontal plane, between a facing direction of the first subject and a viewing axis of an image acquisition device having acquired the target image, and a second angle β, in a vertical plane, between the viewing axis of the image acquisition device and a horizontal axis, the first orientation indicative of the facing direction of the first subject relative to the image acquisition device;
comparing, by the processor, the first orientation of the first subject with a second orientation of a second subject in a source image to obtain an orientation parameter indicative of a relative orientation of the first subject in the target image to the second subject in the source image;
extracting, by the processor, one or more features from the target image;
computing, by the processor, a visual similarity score between the target image and the source image as a function of the one or more features extracted from the target image and weights assigned to the one or more features extracted from the target image as a function of the orientation parameter; and
determining, by the processor, a match between the first subject and the second subject in accordance with the visual similarity score.

### Claim 2 (depends on 1)

The method of claim 1, further comprising assigning a non-zero weight to feature types that are associated with the orientation parameter.

### Claim 3 (depends on 1)

The method of claim 1, wherein extracting one or more features from the target image comprises extracting the one or more features from the target image based on a feature type that is associated with the orientation parameter.

### Claim 4 (depends on 1)

The method of claim 1, wherein the orientation parameter corresponds to an orientation pair representative of the first orientation and the second orientation.

### Claim 5 (depends on 4)

The method of claim 4, wherein the orientation pair is an expected orientation pair that is associated with a higher weight than an unexpected orientation pair.

### Claim 6 (depends on 1)

The method of claim 1, further comprising assigning a first set of weights when the first orientation and the second orientation are the same, and assigning a second set of weights when the first orientation and the second orientation are different.

### Claim 7 (depends on 6)

The method of claim 6, wherein assigning a second set of weights comprises selecting the second set of weights from a group of weights as a function of a difference between the first orientation and the second orientation.

### Claim 8 (depends on 1)

The method of claim 1, wherein each weight is assigned by feature type.

### Claim 9

A system comprising:
a memory having stored therein a program comprising instructions for subject-matching in images; and
at least one processor coupled to the memory for executing the instructions for:
detecting a first orientation of a first subject in a target image using at least two angles, the at least two angles comprising a first angle α, in a horizontal plane, between a facing direction of the first subject and a viewing axis of an image acquisition device having acquired the target image, and a second angle β, in a vertical plane, between the viewing axis of the image acquisition device and a horizontal axis, the first orientation indicative of the facing direction of the first subject relative to the image acquisition device;
comparing the first orientation of the first subject with a second orientation of a second subject in a source image to obtain an orientation parameter indicative of a relative orientation of the first subject in the target image to the second subject in the source image;
extracting one or more features from the target image;
computing a visual similarity score between the target image and the source image as a function of the one or more features extracted from the target image and weights assigned to the one or more features extracted from the target image as a function of the orientation parameter;
determining a match between the first subject and the second subject in accordance with the visual similarity score.

### Claim 10 (depends on 9)

The system of claim 9, wherein assigning weights comprises applying a non-zero weight to feature types that are associated with the at least one orientation parameter.

### Claim 11 (depends on 9)

The system of claim 9, wherein extracting one or more features from the target image comprises extracting the one features based on a feature type that is associated with the orientation parameter.

### Claim 12 (depends on 9)

The system of claim 9, wherein the orientation parameter corresponds to an orientation pair representative of the first orientation and the second orientation.

### Claim 13 (depends on 12)

The system of claim 12, wherein the orientation pair is an expected orientation pair is associated with a higher weight than an unexpected orientation pair.

### Claim 14 (depends on 9)

The system of claim 9, further comprising assigning a first set of weights when the first orientation and the second orientation are the same, and assigning a second set of weights when the first orientation and the second orientation are different.

### Claim 15 (depends on 14)

The system of claim 14, wherein assigning a second set of weights comprises selecting the second set of weights from a group of weights as a function of a difference between the first orientation and the second orientation.

### Claim 16 (depends on 9)

The system of claim 9, wherein the weights are assigned by feature type.

### Claim 17

A non-transitory computer readable medium or media having stored thereon computer readable instructions which when executed by at least one processor, cause the at least one processor to:
detect a first orientation of a first subject in a target image using at least two angles, the at least two angles comprising a first angle α, in a horizontal plane, between a facing direction of the first subject and a viewing axis of an image acquisition device having acquired the target image, and a second angle β, in a vertical plane, between the viewing axis of the image acquisition device and a horizontal axis, the first orientation indicative of the facing direction of the first subject relative to the image acquisition device;
compare the first orientation of the first subject with a second orientation of a second subject in a source image to obtain an orientation parameter indicative of a relative orientation of the first subject in the target image to the second subject in the source image;
extract one or more features from the target image;
compute a visual similarity score between the target image and the source image as a function of the one or more features and weights assigned to the one or more features extracted from the target image as a function of the orientation parameter; and
determine a match between the first subject and the second subject in accordance with the visual similarity score.

### Claim 18 (depends on 1)

The method of claim 1, further comprising:
extracting, by the processor, one or more features from the source image; and
wherein computing further comprises computing, by the processor, the visual similarity score between the target image and the source image as a function of the one or more features extracted from the source image and weights assigned to the one or more features extracted from the source image as a function of the orientation parameter.

### Claim 19 (depends on 9)

The system of claim 9, wherein the at least one processor is coupled to the memory for executing the instructions for:
extracting one or more features from the source image; and
wherein computing further comprises computing the visual similarity score between the target image and the source image as a function of the one or more features extracted from the source image and weights assigned to the one or more features extracted from the source image as a function of the orientation parameter.