# US 2025/0156881 A1 · Merchant authenticity verification system and method based on street view image recognition

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2025/0156881 A1
- Document ID: US20250156881A1
- Status: Published application
- Kind: A1
- Application number: 18/838,663
- Filed: 2022-08-19
- Published: 2025-05-15
- Earliest claimed: 2022-02-17
- Assignees: China Unionpay Co., Ltd.
- Applicants: China Unionpay Co., Ltd.
- Inventors: Zhijun Fei; Yanming Yang; Xuetao Qiu; Pengfei Gao
- Web page: https://patentgrep.com/patent/US20250156881A1

## CPC

- G06Q 30/018: Certifying business or products
- G06F 18/00: Pattern recognition
- G06V 10/462: {Salient features, e.g. scale invariant feature transforms [SIFT]}
- G06Q: INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING

## Abstract

The present disclosure relates to a merchant authenticity verification system and method. The system comprises: a street view roaming module that calculates a translation scale for translating coordinates of an image acquisition point, calculates the coordinates of the image acquisition point based on the translation scale, and interacts with the street view image system to obtain a corresponding street view image based on the coordinates of the image acquisition point; an image comparison module that matches the merchant storefront image obtained with the street view image obtained from the street view roaming module to calculate merchant image similarity; a character recognition module for recognizing a merchant name from the street view image obtained from the street view roaming module; and a text comparison module for matching the merchant name recognized with the merchant name obtained from the merchant information platform to calculate merchant name similarity, and determining whether the merchant name similarity reaches a specified threshold, where in the case that the merchant name similarity reaches the specified threshold, merchant authenticity verification is successful. According to the present disclosure, the accuracy of merchant verification can be improved.

## Claims

### Claim 1

A merchant authenticity verification method, wherein the method verifies authenticity of a merchant based on merchant location, merchant name, merchant storefront image from a merchant information platform, and street view images from a street view image system, the method comprising following steps:
calculating a translation scale based on a given reference image, where the translation scale is used to translate coordinates of an image acquisition point;
obtaining, based on merchant location information obtained from the merchant information platform, a first street view image corresponding to the merchant location information from the street view image system, and conducting a feature point matching between the merchant storefront image obtained from the merchant information platform and the first street view image, and translating the first street view image along a road direction based on the translation scale to obtain coordinates of the image acquisition point;
obtaining, based on the coordinates of the image acquisition point, a second street view image corresponding to the coordinates of the image acquisition point from the street view image system;
cropping a portion of image from the second street view image to obtain a third street view image;
conducting a feature point matching between the merchant storefront image obtained from the merchant information platform and the third street view image to calculate merchant image similarity, and determining whether the merchant image similarity reaches a specified threshold, where in the case that it is determined that the merchant image similarity reaches the specified threshold, proceed with the following steps; and
recognizing a merchant name from the third street view image and matching it with the merchant name obtained from the merchant information platform to calculate merchant name similarity, and determining whether the merchant name similarity reaches a specified threshold, where in the case that the merchant name similarity reaches the specified threshold, merchant authenticity verification is successful.

### Claim 2 (depends on 1)

The merchant authenticity verification method according to claim 1, wherein calculating a translation scale based on a given reference image comprises:
providing two reference images I1 and I2, where reference images I1 and I2 contain a calibration point A; and
in the case that shooting coordinates of reference image I1 are (x1, y1), shooting coordinates of reference image I2 are (x2, y2) (x2>x1), and horizontal coordinates of the calibration point A in the reference images I1 and I2 are p1 and p2 (p2-p1), respectively, the translation scale is:
k
=
(
x
1
-
x
2
)
2
+
(
y
1
-
y
2
)
2
/
(
p
1
-
p
2
)
.

### Claim 3 (depends on 2)

The merchant authenticity verification method according to claim 2, wherein prior to calculating a translation scale based on a given reference image, the following step is further included:
obtaining, based on the merchant location information obtained from the merchant information platform, a search range of the image acquisition point corresponding to the merchant location information.

### Claim 4 (depends on 3)

The merchant authenticity verification method according to claim 3, wherein prior to translating the first street view image along the road direction to obtain the coordinates of the image acquisition point, further determining whether the coordinates of the image acquisition point are within the search range of the image acquisition point, where in the case that it is determined to be within the search range of the image acquisition point, proceed with subsequent steps; otherwise, terminate the subsequent steps.

### Claim 5 (depends on 4)

The merchant authenticity verification method according to claim 4, wherein obtaining the search range of the image acquisition point corresponding to the merchant location information comprises:
obtaining, based on the merchant location information obtained from the merchant information platform, latitude and longitude range and merchant list C {(cxi, cyi)} corresponding to the merchant location information obtained from the merchant information platform,
where the search range is represented as: (min (cxi), min (cyi)) to (max (cxi), max (cyi)).

### Claim 6 (depends on 5)

The merchant authenticity verification method according to claim 5,
wherein conducting a feature point matching between the merchant storefront image obtained from the merchant information platform and the first street view image and translating the first street view image along the road direction based on the translation scale to obtain the coordinates of the image acquisition point, comprises:
setting shooting position of the first street view image to be (sx, sy);
obtaining, using a specified feature matching algorithm, a matching feature point set P={p1, p2, . . . , pn} from the merchant storefront image and the first street view image, where pi is the horizontal coordinate of the matching feature point;
calculating a mean value L of respective differences between a center position f of the merchant storefront image and the matching feature point set P in the case that the number of feature points n in the matching feature point set P is greater than a threshold m, and setting L as the minimum movement distance I in the case that n is less than the threshold m; and
setting coordinates of the image acquisition point as N, where N is (sx+Lkcos (t), sy+kLsin (t)),
wherein, k is the translation scale, and t is an included angle between street direction and longitude line,
where the included angle t is represented by the following equation:
t
=
arg
⁢
tan
⁡
(
(
mean
(
cxi
)
-
min
⁡
(
cxi
)
)
/
mean
(
cyi
)
-
min
⁡
(
cyi
)
)
)
.

### Claim 7 (depends on 6)

The merchant authenticity verification method according to claim 6, wherein the specified feature matching algorithm includes any of the following:
SIFT algorithm, SURF algorithm, SIFT algorithm, ORB algorithm, FAST algorithm, and Harri algorithm.

### Claim 8 (depends on 6)

The merchant authenticity verification method according to claim 6, wherein cropping a portion of image from the second street view image to obtain a third street view image, comprises:
obtaining, using a specified feature matching algorithm, a matching feature point set Q={(qx1, qy1), (qx2, qy2), . . . , (qxn, qyn)} from the merchant storefront image and the second street view image, where (qxi, qyi) are horizontal and vertical coordinates of the matching feature point; and
cropping a portion of image from the second street view image to obtain the third street view image in the way so that upper left and lower right coordinates of the third street view image are: {(min (qxi)-a, min (qyi)-a), (max (qxi)+a, max (qyi)+a)}, where a is a pixel distance, with a value range of 10-100.

### Claim 9 (depends on 8)

The merchant authenticity verification method according to claim 8, wherein the specified feature matching algorithm includes any of the following:
SIFT algorithm, SURF algorithm, SIFT algorithm, ORB algorithm, FAST algorithm, and Harri algorithm.

### Claim 10 (depends on 1)

The merchant authenticity verification method according to claim 1, wherein calculating merchant image similarity employs any of the following matching methods:
image grayscale-based matching method, feature-based matching method, model-based matching method, and transform domain-based matching method.

### Claim 11 (depends on 1)

The merchant authenticity verification method according to claim 1, wherein calculating merchant name similarity employs any of the following comparison methods:
string-based text similarity calculation method, corpus-based text similarity calculation method, and world knowledge-based text similarity calculation method.

### Claim 12

A merchant authenticity verification system, wherein the system verifies authenticity of a merchant based on merchant location, merchant name and merchant storefront image from a merchant information platform, and street view images from a street view image system, the system comprising:
a processing system; and
a memory storing instructions that, when executed by the processing system, cause the system to:
obtain merchant location, merchant name, and merchant storefront image from the merchant information platform, calculate a translation scale for translating coordinates of an image acquisition point, and calculate the coordinates of the image acquisition point obtained by moving the street view image based on the translation scale, and interact with the street view image system to obtain a corresponding street view image based on the coordinates of the image acquisition point;
match the merchant storefront image obtained from the merchant information platform with the street view image obtained from the street view roaming module to calculate merchant image similarity, and determine whether the merchant image similarity reaches a specified threshold;
recognize a merchant name from the street view image obtained from the street view roaming module; and
match the merchant name recognized by the character recognition module with the merchant name obtained from the merchant information platform to calculate merchant name similarity, and determine whether the merchant name similarity reaches a specified threshold, where in the case that the merchant name similarity reaches the specified threshold, merchant authenticity verification is successful.

### Claim 13 (depends on 12)

The merchant authenticity verification system according to claim 12, wherein execution of the instructions further causes the system to obtain merchant location, merchant name, and merchant storefront image from the merchant information platform, calculate the translation scale based on a given reference image, obtain the first street view image corresponding to the merchant location information from the street view image system based on the merchant location information obtained from the merchant information platform, translate the first street view image along the road direction to obtain the coordinates of the image acquisition point based on feature point matching between the merchant storefront image obtained from the merchant information platform and the first street view image as well as the translation scale, obtain the second street view image corresponding to the coordinates of the image acquisition point from the street view image system based on the coordinates of the image acquisition point, and crop a portion of image from the second street view image to obtain the third street view image;
match the merchant storefront image obtained from the merchant information platform with the third street view image obtained from the street view roaming module to calculate merchant image similarity; and
recognize the merchant name from the third street view image.

### Claim 14 (depends on 13)

The merchant authenticity verification system according to claim 13, wherein calculating a translation scale based on a given reference image comprises:
providing two reference images I1 and I2, where reference images I1 and I2 contain a calibration point A;
in the case that shooting coordinates of reference image I1 are (x1, y1), shooting coordinates of reference image I2 are (x2, y2) (x2>x1), and horizontal coordinates of the calibration point A in the reference images I1 and I2 are p1 and p2 (p2-p1), respectively, the translation scale is:
k
=
(
x
1
-
x
2
)
2
+
(
y
1
-
y
2
)
2
/
(
p
1
-
p
2
)
.

### Claim 15 (depends on 14)

The merchant authenticity verification system according to claim 14, wherein execution of the instructions further causes the system to obtain a search range of the image acquisition point corresponding to the merchant location information based on the merchant location information obtained from the merchant information platform.

### Claim 16 (depends on 15)

The merchant authenticity verification system according to claim 15, wherein execution of the instructions further causes the system to prior to translating the first street view image along the road direction to obtain the coordinates of the image acquisition point, further determine whether the coordinates of the image acquisition point are within the search range of the image acquisition point, where in the case that it is determined to be within the search range of the image acquisition point, proceed with subsequent steps; otherwise, terminate the subsequent steps.

### Claim 17 (depends on 16)

The merchant authenticity verification system according to claim 16, wherein obtaining the search range of the image acquisition point corresponding to the merchant location information comprises:
obtaining, based on the merchant location information obtained from the merchant information platform, latitude and longitude range and merchant list C {(cxi, cyi)} corresponding to the merchant location information from the merchant information platform,
where the search range is represented as: (min (cxi), min (cyi)) to (max (cxi), max (cyi)).

### Claim 18 (depends on 17)

The merchant authenticity verification system according to claim 17,
wherein translating the first street view image along the road direction to obtain the coordinates of the image acquisition point based on the feature point matching between the merchant storefront image obtained from the merchant information platform and the first street view image as well as and the translation scale, comprises:
setting the shooting position of the first street view image to be (sx, sy);
obtaining, using a specified feature matching algorithm, a matching feature point set P={p1, p2, . . . , pn} from the merchant storefront image and the first street view image, where pi is the horizontal coordinate of the matching feature point;
calculating a mean value L of respective differences between the center position f of the merchant storefront image and the matching feature point set P in the case that the number of feature points n in the matching feature point set P is greater than a threshold m, and setting L as the minimum movement distance I in the case that n is less than the threshold m; and
setting coordinates of the image acquisition point as N, where N is (sx+Lkcos (t), sy+kLsin (t)),
wherein, k is the translation scale, and t is an included angle between the street direction and longitude line,
where the included angle t is represented by the following equation:
T
=
arg
⁢
tan
⁡
(
(
mean
(
cxi
)
-
min
⁡
(
cxi
)
)
/
mean
(
cyi
)
-
min
⁡
(
cyi
)
)
)

### Claim 19 (depends on 18)

The merchant authenticity verification system according to claim 18,
wherein the specified feature matching algorithm includes any of the following:
SIFT algorithm, SURF algorithm, SIFT algorithm, ORB algorithm, FAST algorithm, and Harri algorithm.

### Claim 20

(canceled)

### Claim 21

(canceled)

### Claim 22

(canceled)

### Claim 23

(canceled)

### Claim 24 (depends on 1)

A computer-readable medium storing a computer program, wherein, the computer program, when executed by a processor, implements the merchant authenticity verification method according to any of claim 1.

### Claim 25

(canceled)
