US 12,613,906 B2 · High-speed search matching method for large-scale heterogeneous databases

Abstract

The high-speed search matching method with a large-scale heterogeneous database includes the process of extracting and storing the user's registered image features, forming a database, and being a reference when matching and searching; The process of initializing the search engine, in which two objects in the search engine are initialized, including a matching object, calculating the distance, and a data table object to determine the target search object for each input image sample; The process of matching and searching for objects, including the process of calculating the distance between the features and determining the target array, and finally comparing and returning the final result. The proposed method has demonstrated the ability to calculate accurately as well as very high performance, especially in large-scale data processing cases.

A high-speed search matching method with a large-scale heterogeneous database, the method includes three processes performed in turn, including:

a registration image processing process:
used to extract the features of the object including a face, or a human shape, a vehicle and update, store into the large-scale heterogeneous database according to a fragmented structure of rows and columns;
accordingly, an image used to initialize the database of users is registered through the channel of an operating partner using cloud computing services to deploy solutions for end users, used for the object recognition problem;
a search engine initialization process:
perform a database reading process, including loading feature data and accompanying information, and packaging the feature data and accompanying information into two objects, the first object is a matrix matching object and the second object is a data table object;
these two objects constitute a search engine, and perform the tasks of identifying a target data array as well as determining a distance between image features to be searched and a feature matrix of the available database;
this process includes:
reading and extracting information from a binary file, loading the data which is the deep learning features and accompanying information from the pre-specified database;
creating a matching and searching object:
the matching and searching object is created by a process of stringing together deep learning features of all the samples in the database from top to bottom into a unified array;
this feature array is loaded into a numpy object for parallel processing in blocks on a CPU and into a cupy object for parallel processing in blocks on a graphics microprocessors (GPU);
creating an object-label data table object:
created after extracting the accompanying information with a “key:
value” method of the dictionary, the object label is stringified into a table object, with an order number synchronized with a position of the deep learning feature array;
create an accompanying information data table object:
separated and synchronized with the label data table after separating the accompanying information with the “key:
value” method;
a search process search engine operators cluster:
includes a process of extracting features from the image that needs to be matched for search and performing the query, comparing with the available database, finding the most matching data samples with a smallest distance, this process includes:
extracting information from the images that need to be identified:
the information is analyzed into two data streams:
deep learning features of the object image the model synchronizes with the deep learning model when creating the database and determining a search identification code group, including an operating partner code, user code and database group;
determine a search target of each image sample in the group to be identified:
a search identification code information of the image group to be identified is compared with the accompanying information data table data of the database to determine a search array of each image sample in the above image group;
specifically, the search identification code of each sample is compared with the integer code of the database, to find the data sequence numbers that need to be matched;
let M be the scale of the database, the integer code of the database is the matrix R with size (M×3), besides, with X image samples to be searched simultaneously, the corresponding integer code is T with size (X×3), then the marking matrix Ψ has size (M×X), with values of only 0 or 1;
in which, at coordinates (m,x), a value of 0 indicates that the database's denominator m is not within the search range of the xth sample of the search group, and vice versa;
method of determining the array:
Ψ
⁡
(
m
,
x
)
=
{
0
⁢
if
⁢
M
(
m
,
:
)
≠
X
(
x
,
:
)
1
⁢
others
,
At this time, the marking matrix with values 0 and 1 will determine the search range of each image sample to be recognized, representing the search range of each image sample is a vector with length l×M, the position with value 1 is the sequence number of the data to be compared;
calculate a distance between the feature of image group to be recognized and the matching and searching object;
determine the group of effective distance values for each image sample to be recognized, specifically:
the group of features from the image group to be recognized is put into the matching and searching object to calculate the distance by using a cosine formula to calculate a distance between two vectors, the larger the value of cosine distance, the higher the similarity between the two vectors;
in the process of calculating the distance, use operators to process on the data array block processing, with high calculation speed, especially when used on GPU;
specifically, the distance matrix distance(m,x) is defined:
distance(m,x)=dot(f(M,:N),g(X,N)), in there, f(M,:N) vàg(X,N) are the feature matrix objects of the database and the image samples to be recognized, respectively, with N being the dimension of the feature vector;
from the distance calculation results and the marker matrix, filter the matching results and present the final recognition result;
finally, based on the effective distance value, sort and get the corresponding label position and return the final result, specifically:
first, remove the incorrect matching results, use the marker matrix to determine the usable distance values for each sample to be recognized, the unusable distance matrix values are set to 0 by the multiplication between the two distance matrices and the marker matrix, specifically:
distance′(m,x)=matmul(distance(m,x),Ψ(m,x));
according to each column of distance′(m,x), the higher the value of the element, the more it matches the corresponding sample to be recognized, according to the order number;
to get the K most matching data samples with K set in advance, arrange the values from large to small, will return the order number indices (k=1→K) and the corresponding distance value corr(k=1→K) of those K most matching samples:
indices
(
k
)
=
argsort
⁡
(
distance
′
)
[
:
K
]
corr
⁡
(
k
)
=
sort
(
distance
′
)
[
:
K
]
and the labels of these matching data sample label(k) are determined by:
label
(
k
)
=
L
⁡
(
indices
)
with L being the object storing the data label string determined from the initialization of the search engine;
finally, at the end of the search process, returning the results.

111 more elements

High-speed search matching method for large-scale heterogeneous databases

Life of this patent

High-speed search matching method for large-scale heterogeneous databases

Life of this patent

Abstract

Claims8