# US 2021/0063159 A1 · Positioning Method, Positioning Server and Positioning System

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2021/0063159 A1
- Document ID: US20210063159A1
- Status: Published application
- Kind: A1
- Application number: 16/961,719
- Filed: 2018-11-12
- Published: 2021-03-04
- Earliest claimed: 2018-04-09
- Assignees: Boe Technology Group Co., Ltd.
- Applicants: Boe Technology Group Co., Ltd.
- Inventors: Yue Li; Jingtao Xu; Bo Xu; Zhiguo Zhang
- Web page: https://patentgrep.com/patent/US20210063159A1

## CPC

- G01C 21/005: {with correlation of navigation data from several sources, e.g. map or contour matching (G01C21/30 takes precedence)}
- B62B 3/14: characterised by provisions for nesting or stacking, e.g. shopping trolleys {(collapsible shopping trolleys B62B3/027)}
- B62B 5/00: Accessories or details specially adapted for hand carts ({B62B3/1404} , B62B9/00 take precedence; wheels, axles, or axle bearings for vehicles B60B; castors for vehicles, castors in general B60B33/00)
- G01C 21/206: {specially adapted for indoor navigation}
- G01S 5/16: using electromagnetic waves other than radio waves
- G06F 16/29: Geographical information databases
- G06F 16/587: using geographical or spatial information, e.g. location
- G06Q 30/0639: {Locating goods or services, e.g. based on physical position of the goods or services within a shopping facility}
- G06T 7/74: {involving reference images or patches}
- G06T 2207/20021: Dividing image into blocks, subimages or windows
- G01C: MEASURING DISTANCES, LEVELS OR BEARINGS; SURVEYING; NAVIGATION; GYROSCOPIC INSTRUMENTS; PHOTOGRAMMETRY OR VIDEOGRAMMETRY (measuring liquid level G01F; radio navigation, determining distance or velocity by use of propagation effects, e.g. Doppler effects, propagation time, of radio waves, analogous arrangements using other waves G01S)
- B62B: HAND-PROPELLED VEHICLES, e.g. HAND CARTS OR PERAMBULATORS; SLEDGES (characterised by animal propulsion B62C; propulsion of sledges by driver or engine B62M)
- G01S: RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G06Q: INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL

## Abstract

The present disclosure relates to a positioning method, a positioning server, and a positioning system. The positioning method includes: receiving a wireless positioning coordinate of a moving object; determining a visual sub-map to be matched in a visual map database based on the wireless positioning coordinate; and obtaining a visual positioning coordinates corresponding to a current image of the moving object captured at time of wireless positioning as a positioning result, based on the determined visual sub-map to be matched.

## Claims

### Claim 1

A positioning method, comprising:
receiving a wireless positioning coordinate of a moving object;
determining a visual sub-map to be matched in a visual map database based on the wireless positioning coordinate; and
obtaining a visual positioning coordinate corresponding to a current image of the moving object captured at time of wireless positioning as a positioning result, based on the determined visual sub-map to be matched.

### Claim 2 (depends on 1)

The positioning method according to claim 1, wherein determining the visual sub-map to be matched comprises:
selecting at least one of a visual sub-map where the wireless positioning coordinate is located and an adjacent visual sub-map, as the visual sub-map to be matched.

### Claim 3 (depends on 1)

The positioning method according to claim 1, wherein determining the visual sub-map to be matched comprises:
drawing a geometric pattern with the wireless positioning coordinate as a reference point based on a system error of the wireless positioning coordinate, and determining a plurality of visual sub-maps pre-stored in the visual map database which are entirely or partly covered by the geometric pattern, as the visual sub-map to be matched.

### Claim 4 (depends on 1)

The positioning method according to claim 1, wherein obtaining the visual positioning coordinate comprises:
determining a matching key frame in the visual sub-map to be matched that matches the current image; and
performing an iterative pose estimation on the current image and the matching key frame to obtain the visual positioning coordinate corresponding to the current image.

### Claim 5 (depends on 4)

The positioning method according to claim 4, wherein determining a matching key frame comprises:
performing a feature extraction on the current image and obtaining corresponding visual words;
searching a candidate matching frame in key frames of the visual sub-map to be matched, wherein a matching score between at least one visual word of the candidate matching frame and visual words of the current image is greater than a first threshold;
counting an amount of visual words of each candidate matching frame, wherein a matching score between the visual words and the visual words of the current image is greater than a second threshold, and selecting a candidate matching frame with an amount greater than a third threshold as a common-view frame;
making each common-view frame and a preset number of key frames before or after the common-view frame into a group, and calculating an accumulative score of matching degree between visual words of all key frames in each group and the visual words of the current image; and
referring a key frame with the highest matching score in a group with the accumulative score of matching degree greater than a fourth threshold as the matching key frame.

### Claim 6 (depends on 1)

The positioning method according to claim 1, further comprising a step of constructing a visual map database, which comprises:
registering coordinate systems of a wireless positioning process and a visual positioning process;
dividing a positioning coverage area into a plurality of visual sub-maps, and collecting sparse three-dimensional map feature points and visual words of a plurality of key frames within a range of each visual sub-map; and
saving the plurality of key frames corresponding to respective visual sub-maps, the sparse three-dimensional map feature points and the visual words to the visual map database.

### Claim 7 (depends on 1)

The positioning method according to claim 1, further comprising:
predicting a visual sub-map to be entered by the moving object based on the visual positioning coordinate and a direction of the moving object when it is determined that the moving object is in a boundary area of a visual sub-map at a current moment based on the visual positioning coordinate; and
determining the predicted visual sub-map as the visual sub-map to be matched, and performing an operation of obtaining a visual positioning coordinate corresponding to a next image.

### Claim 8 (depends on 7)

The positioning method according to claim 7, wherein predicting a visual sub-map comprises:
determining a straight line passing through the visual positioning coordinate and having the same direction as the moving object based on the visual positioning coordinate and a direction of the moving object; and
when there is an intersection between the straight line and a boundary of an adjacent visual sub-map of a visual sub-map at the current moment, and a distance between the visual positioning coordinate and the boundary is shorter than a distance between the visual positioning coordinate and other boundaries of the visual sub-map at the current moment, predicting that the adjacent visual sub-map is the visual sub-map to be entered by the moving object.

### Claim 9 (depends on 1)

The positioning method according to claim 1, further comprising:
determining a projection point of the wireless positioning coordinate on a preset path as a positioning result, based on a planar environment map of a positioning coverage area.

### Claim 10 (depends on 9)

The positioning method according to claim 9, wherein determining a projection point comprises:
determining whether a navigation path has been generated within the planar environment map;
if so, projecting the wireless positioning coordinate onto the navigation path, and if not, projecting the wireless positioning coordinate onto an accessible path in the planar environment map.

### Claim 11 (depends on 10)

The positioning method according to claim 10, further comprising:
when the wireless positioning coordinate is projected onto the navigation path, determining whether the shortest distance of projecting the wireless positioning coordinate onto the navigation path is greater than an absolute value of a system error of the wireless positioning coordinate;
if so, projecting the wireless positioning coordinate onto the accessible path, and if not, using a projection point corresponding to the shortest distance as a positioning result.

### Claim 12 (depends on 10)

The positioning method according to claim 10, wherein an operation of projecting the wireless positioning coordinate onto the accessible path comprises:
determining an accessible path in the planar environment map with the smallest included angle relative to a direction of the moving object and at the closest distance to the wireless positioning coordinate; and
taking a projection point of projecting the wireless positioning coordinate onto the accessible path which has been determined as a positioning result.

### Claim 13 (depends on 1)

A positioning server, comprising:
a memory; and
a processor coupled to the memory, wherein the processor is configured to perform the positioning method according to claim 1 based on instructions stored in the memory.

### Claim 14 (depends on 13)

A positioning system, comprising:
the positioning server according to claim 13; and
moving object moving in a positioning coverage area, and configured to perform wireless communication with the positioning server.

### Claim 15 (depends on 14)

The positioning system according to claim 14, further comprising:
a user interaction unit configured to send and receive positioning messages, which communicates with the positioning server.

### Claim 16 (depends on 15)

The positioning system according to claim 15, wherein the moving object comprising a shopping cart, and the user interaction unit comprises a mobile phone of user.
