# US 2020/0401857 A1 · Image processing method and apparatus, electronic device, and storage medium

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2020/0401857 A1
- Document ID: US20200401857A1
- Status: Published application
- Kind: A1
- Application number: 17/015,189
- Filed: 2020-09-09
- Published: 2020-12-24
- Earliest claimed: 2018-12-21
- Assignees: Shanghai Sensetime Intelligent Technology Co., Ltd.
- Applicants: Shanghai Sensetime Intelligent Technology Co., Ltd.
- Inventors: Yi LU; Li CAO; Chunlei HONG
- Web page: https://patentgrep.com/patent/US20200401857A1

## CPC

- G06V 40/173: {face re-identification, e.g. recognising unknown faces across different face tracks}
- G06F 18/22: Matching criteria, e.g. proximity measures
- G06F 18/25: Fusion techniques
- G06F 18/254: {of classification results, e.g. of results related to same input data}
- G06V 10/763: {Non-hierarchical techniques, e.g. based on statistics of modelling distributions}
- G06V 10/809: {of classification results, e.g. where the classifiers operate on the same input data}
- G06V 20/52: Surveillance or monitoring of activities, e.g. for recognising suspicious objects (recognising microscopic objects G06V20/69)
- G06V 40/16: Human faces, e.g. facial parts, sketches or expressions
- G06V 40/50: Maintenance of biometric data or enrolment thereof
- H04N 23/90: Arrangement of cameras or camera modules, e.g. multiple cameras in TV studios or sports stadiums
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION

## Abstract

An image processing method and apparatus, and a storage medium are provided. The method includes: obtaining a first image and a second image of a to-be-authenticated object, where the first image is captured by a first camera module, and the second image is captured by at least one second camera module; comparing the first image with image data in a target library for identity authentication, to obtain a first authentication result; and in response to that the first authentication result is authentication failure, performing joint authentication on the first image and the second image, and determining the identity of the to-be-authenticated object according to a second authentication result of the joint authentication.

## Claims

### Claim 1

An image processing method, comprising:
obtaining a first image and a second image of a to-be-authenticated object, wherein the first image is captured by a first camera module, and the second image is captured by at least one second camera module;
comparing the first image with image data in a target library for identity authentication, to obtain a first authentication result; and
in response to that the first authentication result is authentication failure, performing joint authentication on the first image and the second image, and determining the identity of the to-be-authenticated object according to a second authentication result of the joint authentication.

### Claim 2 (depends on 1)

The method according to claim 1, wherein the target library comprises a whitelist and blacklist library;
comparing the first image with the image data in the target library for identity authentication, to obtain the first authentication result comprises:
comparing first feature data of the first image with feature data of each image in the whitelist and blacklist library; and
in the case that feature data matching the first feature data exists in the whitelist and blacklist library, determining that the first authentication result is authentication success, and determining a to-be-authenticated object corresponding to the first image as a blacklist or whitelist object.

### Claim 3 (depends on 1)

The method according to claim 1, wherein the target library comprises an annotated stranger library;
comparing the first image with the image data in the target library for identity authentication, to obtain the first authentication result comprises:
comparing first feature data of the obtained first image with feature data of an image in the annotated stranger library; and
in the case that feature data matching the first feature data exists in the annotated stranger library, determining that the first authentication result is authentication success, and determining the to-be-authenticated object corresponding to the first image as an annotated stranger.

### Claim 4 (depends on 3)

The method according to claim 3, wherein the method further comprises: in the case that the feature data matching the first feature data exists in the annotated stranger library,
collecting statistics about a number of times that the to-be-authenticated object corresponding to the first image is annotated as a stranger.

### Claim 5 (depends on 1)

The method according to claim 1, further comprising:
in the case that the first authentication result is authentication success, adding the first image and associated information of the first image to a matching record corresponding to the matched feature data, wherein the associated information of the first image comprises at least one of: information of time when the first camera module captures the first image, identification information of the first camera module, or position information of the first camera module.

### Claim 6 (depends on 1)

The method according to claim 1, wherein the method further comprises: before performing joint authentication on the first image and the second image,
performing deduplication processing on at least one of the first image or the second image of which authentication fails within a first time range, to obtain at least one of a first image or a second image that satisfies a first preset condition for each to-be-authenticated object within the first time range.

### Claim 7 (depends on 1)

The method according to claim 1, wherein in response to that the first authentication result is authentication failure, performing joint authentication on the first image and the second image, and determining the identity of the to-be-authenticated object according to the second authentication result of the joint authentication comprises:
performing clustering processing on a first image whose first authentication result is authentication failure and a second image whose first authentication result is authentication failure within a second time range, to obtain an image set for each to-be-authenticated object;
determining similarities between each of a plurality of images in the image set and other images in the image set;
determining whether the image set satisfies a second preset condition based on a similarity corresponding to each image in the image set; and
in the case that the image set satisfies the second preset condition, determining that the to-be-authenticated object corresponding to the image set is a stranger.

### Claim 8 (depends on 7)

The method according to claim 7, wherein determining the similarities between each of the plurality of images in the image set and other images in the image set comprises:
obtaining a sum value of dot products of feature data of each of the plurality of images in each image set and feature data of all images; and
determining the similarities between each of the plurality of images and remaining images except the image based on the sum value and a number of feature data in the image set.

### Claim 9 (depends on 7)

The method according to claim 7, wherein performing clustering processing on the first image whose first authentication result is authentication failure and the second image whose first authentication result is authentication failure within the second time range, to obtain the image set for each to-be-authenticated object comprises:
obtaining first feature data and second feature data respectively corresponding to the first image and the second image of which authentication fails within the second time range;
comparing the first feature data and the second feature data for matching to determine whether each piece of first feature data and each piece of second feature data correspond to a same to-be-authenticated object; and
performing clustering on the first feature data and the second feature data of the same to-be-authenticated object to form an image set of the same to-be-authenticated object.

### Claim 10 (depends on 7)

The method according to claim 7, wherein determining whether the image set satisfies the second preset condition based on the similarity corresponding to each image in the image set comprises at least one of the following manners:
a maximum similarity among the similarities corresponding to the plurality of images in the image set is greater than a first similarity threshold;
an amount of feature data with a similarity among the similarities corresponding to the plurality of images in the image set greater than a second similarity threshold exceeds a preset ratio; or
a minimum similarity among the similarities corresponding to the plurality of images in the image set is greater than a third similarity threshold.

### Claim 11 (depends on 7)

The method according to claim 7, wherein in response to that the first authentication result is authentication failure, performing joint authentication on the first image and the second image, and determining the identity of the to-be-authenticated object according to the second authentication result of the joint authentication further comprises:
in the case that the similarities between the plurality of images in the image set does not satisfy a preset condition, deleting all images corresponding to the image set.

### Claim 12 (depends on 7)

The method according to claim 7, wherein in the case that the image set satisfies the second preset condition, determining that the to-be-authenticated object corresponding to the image set is the stranger comprises:
in the case that the images corresponding to feature data in the image set are images captured by different camera modules in different time ranges, determining the to-be-authenticated object corresponding to the feature data set to be the stranger.

### Claim 13 (depends on 1)

The method according to claim 1, wherein obtaining the first image and the second image of the to-be-authenticated object comprises:
respectively obtaining a first video captured by the first camera module and a second video captured by at least one second camera module, and preprocessing the first video to obtain a plurality of third images and preprocessing the second video to obtain a plurality of fourth images, or receiving the plurality of third images and the plurality of fourth images; and
determining an image of the plurality of third images that satisfies quality requirements as the first image, and determining an image of the plurality of fourth images that satisfies the quality requirements as the second image.

### Claim 14 (depends on 13)

The method according to claim 13, wherein the method further comprises: after obtaining the first image and the second image of the to-be-authenticated object, and before obtaining first feature data of the first image, and comparing the first feature data with feature data in the target library for identity authentication, to obtain the first authentication result,
detecting whether at least one of the first image or the second image comprises a predetermined feature; and
in response to that at least one of the first image or the second image comprises the predetermined feature, annotating at least one of the first image or the second image comprising the predetermined feature, wherein the predetermined feature comprises at least one of a mask, a hat, or sunglasses.

### Claim 15 (depends on 1)

The method according to claim 1, further comprising:
outputting and prompting the first authentication result or the second authentication result.

### Claim 16 (depends on 15)

The method according to claim 15, wherein outputting and prompting the first authentication result or the second authentication result comprises:
in response to that the first authentication result is authentication success, outputting the identity of the to-be-authenticated object and associated information of the to-be-authenticated object in a preset manner, and in response to determining that the to-be-authenticated object is an annotated stranger, outputting the number of times that the to-be-authenticated object is annotated as a stranger; or
outputting the second authentication result.

### Claim 17 (depends on 1)

The method according to claim 1, further comprising:
in response to the second authentication result indicating that the to-be-authenticated object is the stranger, storing the first image, the second image, and the associated information that correspond to the to-be-authenticated object in the target library; and
displaying an authentication result that the to-be-authenticated object is determined as the stranger, statistical information, and prompt information through a user interaction interface.

### Claim 18

An image processing apparatus, comprising:
a processor; and
a memory configured to store instructions executable by the processor;
wherein the processor is configured to:
obtain a first image and a second image of a to-be-authenticated object, wherein the first image is captured by a first camera module, and the second image is captured by at least one second camera module;
compare the first image with image data in a target library for identity authentication, to obtain a first authentication result; and
perform, in response to that the first authentication result is authentication failure, joint authentication on the first image and the second image, and determine the identity of the to-be-authenticated object according to a second authentication result of the joint authentication.

### Claim 19 (depends on 18)

The apparatus according to claim 18, wherein the target library comprises a whitelist and blacklist library; and the processor is configured to:
compare first feature data of the first image with feature data of each image in the whitelist and blacklist library; and
in the case that feature data matching the first feature data exists in the whitelist and blacklist library, determine that the first authentication result is authentication success, and determine a to-be-authenticated object corresponding to the first image as a blacklist or whitelist object.

### Claim 20

A non-transitory computer-readable storage medium, having computer program instructions stored thereon, wherein the computer program instructions, when being executed by a processor, enable the processor to implement the following operations:
obtaining a first image and a second image of a to-be-authenticated object, wherein the first image is captured by a first camera module, and the second image is captured by at least one second camera module;
comparing the first image with image data in a target library for identity authentication, to obtain a first authentication result; and
in response to that the first authentication result is authentication failure, performing joint authentication on the first image and the second image, and determining the identity of the to-be-authenticated object according to a second authentication result of the joint authentication.