# US 2025/0148792 A1 · Repeatability predictions of interest points

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2025/0148792 A1
- Document ID: US20250148792A1
- Status: Published application
- Kind: A1
- Application number: 19/015,616
- Filed: 2025-01-09
- Published: 2025-05-08
- Earliest claimed: 2021-04-30
- Assignees: Niantic, Inc.
- Applicants: Niantic, Inc.
- Inventors: Dung Anh Doan; Daniyar Turmukhambetov; Soohyun Bae
- Web page: https://patentgrep.com/patent/US20250148792A1

## CPC

- G06V 20/20: in augmented reality scenes
- G06V 20/50: Context or environment of the image
- G06T 7/73: using feature-based methods
- G06T 7/74: {involving reference images or patches}
- G06V 10/768: {using context analysis, e.g. recognition aided by known co-occurring patterns}
- G06V 10/774: Generating sets of training patterns; Bootstrap methods, e.g. bagging or boosting
- G06V 20/90: Identifying an image sensor based on its output data
- G06T 2207/10016: Video; Image sequence
- G06T 2207/20081: Training; Learning
- G06T 2207/30232: Surveillance
- G06T 2207/30244: Camera pose
- G06T 2207/30252: Vehicle exterior; Vicinity of vehicle
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL

## Abstract

The present disclosure describes approaches for evaluating interest points for localization uses based on a repeatability of the detection of the interest point in images capturing a scene that includes the interest point. The repeatability of interest points is determined by using a trained repeatability model. The repeatability model is trained by analyzing a time series of images of a scene and determining repeatability functions for each interest point in the scene. The repeatability function is determined by identifying which images in the time series of images allowed for the detection of the interest point by an interest point detection model.

## Claims

### Claim 1

A computer-implemented method, comprising:
receiving first identification of a first location of a client device;
generating a summary map of a vicinity of the location of the client device based on a plurality of interest points located in the vicinity of the first location of the client device;
receiving a second identification of a second location of the client device;
determining a repeatability score of each interest point in the plurality of interest points based on sensor data collected by the client device at the second location;
updating the summary map by removing a subset of interest points from the summary map based on each interest point of the subset having a repeatability score below a threshold value; and
sending the updated summary map to the client device.

### Claim 2 (depends on 1)

The computer-implemented method of claim 1, wherein determining a repeatability score of each interest point in the plurality of interest points comprises:
applying a repeatability model based on the interest point, the repeatability model trained based on a time series of images of a scene across a set time period.

### Claim 3 (depends on 2)

The computer-implemented method of claim 2, wherein the second identification includes a current time and the repeatability score of the interest point is further based on the current time.

### Claim 4 (depends on 3)

The computer-implemented method of claim 3, wherein the current time is determined based on at least one of a received time from the client device and an internal time of a server.

### Claim 5 (depends on 2)

The computer-implemented method of claim 2, wherein the repeatability model is trained by:
receiving the time series of images of the scene across the set time period;
identifying, using an interest point detection model, a set of training interest points in the received time series of images;
for each training interest point in the set of training interest points, determining a repeatability function by identifying images in the time series of images in which the training interest point is detected by the interest point detection model; and
training the repeatability model using information associated with one or more training interest points from the set of training interest points and corresponding repeatability functions of the one or more training interest points.

### Claim 6 (depends on 1)

The computer-implemented method of claim 1, wherein the repeatability score is indicative of a likelihood that a trained interest point detection model will detect the interest point based on images captured by the client device.

### Claim 7 (depends on 1)

The computer-implemented method of claim 1, further comprising:
updating the summary map by adding interest points not in the summary map based on each additional interest point having a repeatability score above a threshold value.

### Claim 8

A computer-implemented method, comprising:
receiving first identification of a first location of a client device;
generating a summary map of a vicinity of the location of the client device based on a plurality of interest points located in the vicinity of the location of the client device;
receiving a second identification of a second location of the client device;
determining a repeatability score of each interest point in the plurality of interest points based on sensor data collected by the client device at the second location;
updating the summary map by adding interest points not in the summary map based on each additional interest point having a repeatability score above a threshold value; and
sending the updated summary map to the client device.

### Claim 9 (depends on 8)

The computer-implemented method of claim 8, wherein determining a repeatability score of each interest point in the plurality of interest points comprises:
applying a repeatability model based on the interest point, the repeatability model trained based on a time series of images of a scene across a set time period.

### Claim 10 (depends on 9)

The computer-implemented method of claim 9, wherein the second identification includes a current time and the repeatability score of the interest point is further based on the current time.

### Claim 11 (depends on 10)

The computer-implemented method of claim 10, wherein the current time is determined based on at least one of a received time from the client device and an internal time of a server.

### Claim 12 (depends on 9)

The computer-implemented method of claim 9, wherein the repeatability model is trained by:
receiving the time series of images of the scene across the set time period;
identifying, using an interest point detection model, a set of training interest points in the received time series of images;
for each training interest point in the set of training interest points, determining a repeatability function by identifying images in the time series of images in which the training interest point is detected by the interest point detection model; and
training the repeatability model using information associated with one or more training interest points from the set of training interest points and corresponding repeatability functions of the one or more training interest points.

### Claim 13 (depends on 8)

The computer-implemented method of claim 8, wherein the repeatability score is indicative of a likelihood that a trained interest point detection model will detect the interest point based on images captured by the client device.

### Claim 14 (depends on 8)

The computer-implemented method of claim 8, further comprising:
updating the summary map by removing a subset of interest points from the summary map based on each interest point of the subset having a repeatability score below a threshold value.

### Claim 15

A non-transitory computer readable storage medium configured to store instructions, the instructions that, when executed by a processor, cause the processor to:
receive first identification of a first location of a client device;
generate a summary map of a vicinity of the location of the client device based on a plurality of interest points located in the vicinity of the first location of the client device;
receive a second identification of a second location of the client device;
determine a repeatability score of each interest point in the plurality of interest points based on sensor data collected by the client device at the second location;
update the summary map by adding or removing a subset of interest points from the summary map based a repeatability score of each respective interest point; and
send the updated summary map to the client device.

### Claim 16 (depends on 15)

The non-transitory computer readable storage medium of claim 15, wherein the instruction that causes the processor to determine a repeatability score of each interest point in the plurality of interest points further causes the processor to:
apply a repeatability model based on the interest point, the repeatability model trained based on a time series of images of a scene across a set time period.

### Claim 17 (depends on 16)

The non-transitory computer readable storage medium of claim 16, wherein the second identification includes a current time and the repeatability score of the interest point is further based on the current time.

### Claim 18 (depends on 17)

The non-transitory computer readable storage medium of claim 17, wherein the current time is determined based on at least one of a received time from the client device and an internal time of a server.

### Claim 19 (depends on 16)

The non-transitory computer readable storage medium of claim 16, wherein the repeatability model is trained by:
receiving the time series of images of the scene across the set time period;
identifying, using an interest point detection model, a set of training interest points in the received time series of images;
for each training interest point in the set of training interest points, determining a repeatability function by identifying images in the time series of images in which the training interest point is detected by the interest point detection model; and
training the repeatability model using information associated with one or more training interest points from the set of training interest points and corresponding repeatability functions of the one or more training interest points.

### Claim 20 (depends on 15)

The non-transitory computer readable storage medium of claim 15, wherein the repeatability score is indicative of a likelihood that a trained interest point detection model will detect the interest point based on images captured by the client device.
