# US 2026/0065501 A1 · Method and system for motion prior based visual odometry for ground robot navigation

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2026/0065501 A1
- Document ID: US20260065501A1
- Status: Published application
- Kind: A1
- Application number: 19/308,442
- Filed: 2025-08-25
- Published: 2026-03-05
- Earliest claimed: 2024-08-30
- Assignees: Tata Consultancy Services Limited
- Applicants: Tata Consultancy Services Limited
- Inventors: Sayan PAUL; Ruddra Dev ROYCHOUDHURY; Brojeshwar BHOWMICK
- Web page: https://patentgrep.com/patent/US20260065501A1

## CPC

- G05D 1/2437: {Extracting relative motion information}
- G06T 7/73: using feature-based methods
- G05D 1/2435: {Extracting 3D information}
- G06V 10/766: using regression, e.g. by projecting features on hyperplanes
- G06V 10/82: using neural networks
- G05D 2101/15: using machine learning, e.g. neural networks
- G05D 2111/10: Optical signals
- G05D 2111/65: {taken successively, e.g. visual odometry or optical flow}
- G06T 2207/10024: Color image
- G06T 2207/10028: Range image; Depth image; 3D point clouds
- G06T 2207/20084: Artificial neural networks [ANN]
- G05D: SYSTEMS FOR CONTROLLING OR REGULATING NON-ELECTRIC VARIABLES
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING

## Abstract

Traditional Visual Odometry (VO) systems face challenges in wide-baseline scenarios leading to drift and catastrophic failures in robot navigation and suffer from sample inefficiency during training. Embodiments of present disclosure provide a method and system for motion prior based VO for ground robot navigation. A robot acquires a previous observation at a previous time step, an action prior, and a current observation at a current time step. Using this data, VO module estimates relative pose between previous and current observations. The VO module comprises a Geometric Coarse Pose Estimator module (GCPE) to estimate a coarse relative pose, and Neural Fine Pose Regression model (NFPR) to regress fine pose using the coarse pose and the acquired inputs. Then, using the relative pose, the current goal coordinates are updated. Finally, a navigation policy determines next action to take to reach the goal using updated goal coordinates and the current observation.

## Claims

### Claim 1

A processor implemented method comprising:
obtaining, via one or more hardware processors, a plurality of inputs comprising: i) a previous observation comprising an RGB image and a depth image captured at a previous time step from a robot, ii) an action prior defining an action performed by the robot at the previous time step, and iii) a current observation an RGB image and a depth image captured at a current time step obtained after performing the action by the robot;
determining, via the one or more hardware processors, a relative pose of the robot between the previous observation and the current observation based on the plurality of inputs by using a Visual Odometry (VO) module, wherein the VO module comprises i) a Geometric Coarse Pose Estimator (GCPE) module, and ii) a Neural Fine Pose Regression (NFPR) module, wherein the GCPE module estimates a coarse relative pose of the robot using the plurality of inputs, and wherein the NFPR module determines the relative pose of the robot using the plurality of inputs and the coarse relative pose as motion prior;
updating, via the one or more hardware processors, a plurality of goal coordinates of the robot at the previous timestep based on the relative pose of the robot; and
determining, via the one or more hardware processors, a next action to be performed by the robot at the current timestep based on the updated plurality of goal coordinates using a navigation policy for ground navigation by the robot.

### Claim 2 (depends on 1)

The processor implemented method of claim 1, wherein the GCPE module estimates the coarse relative pose of the robot using the plurality of inputs by:
extracting a first set of visual key points from the RGB image comprised in the previous observation and a second set of visual key points from the RGB image comprised in the current observation using a key point extractor;
matching a plurality of visual key points between the first set of visual key points and the second set of visual key points using a key point matcher to obtain a plurality of matched key point pairs, wherein the plurality of matched key point pairs are 2-Dimensional (2D);
filtering a set of matched key point pairs from the plurality of matched key point pairs based on a ratio test;
back-projecting the filtered set of matched key point pairs to obtain a set of 3-Dimensional (3D) matched key point pairs by using the depth image comprised in the previous observation, the depth image comprised in the current observation and a plurality of parameters of a camera of the robot used to capture the current and the previous observations;
performing a plurality of steps for a plurality of iterations until percentage increase of a best pose score at an iteration is higher than a predefined threshold value, wherein the plurality of steps comprise:
generating a plurality of candidate relative poses using a relative pose sampler function;
computing a plurality of pose scores associated with the plurality of candidate poses, wherein a pose score for a candidate pose among the plurality of candidate poses is computed by:
determining a plurality of weights corresponding to the set of 3D matched key point pairs for the candidate pose using a Motion Prior based Correspondence Weighing (MPCW) function; and
summing up the plurality of weights to obtain the pose score; and
selecting a best pose among the plurality of candidate relative poses having a best pose score, wherein the best pose score is a highest pose score among the plurality of pose scores, and wherein the best pose selected at the end of the plurality of iterations is considered as the coarse relative pose of the robot.

### Claim 3 (depends on 2)

The processor implemented method of claim 2, wherein generating a plurality of candidate relative poses using a relative pose sampler function comprises alternatively sampling a rotation and a translation from a normal distribution of poses, wherein the normal distribution is generated in a first iteration among the plurality of iterations using the action prior as mean and a predefined standard deviation, and wherein the normal distribution is generated in a current iteration among the plurality of iterations using a coarse relative pose estimated in a previous iteration as mean and half of standard deviation used in the previous iteration.

### Claim 4 (depends on 2)

The processor implemented method of claim 2, wherein the MPCW function determines weight as
W
C
ab
i
j
=
W
C
ab
i
j
-
1
⁢
(
best
)
(
T
sampled
(
p
a
i
)
-
p
b
i
)
2
+
(
T
sampled
-
1
(
p
b
i
)
-
p
a
i
)
2
,
and wherein
W
C
ab
i
j
is a weight associated with an ith matched key point pair
C
a
⁢
b
i
from among the set of 3D matched key point pairs at a current iteration j,
W
C
a
⁢
b
i
j
-
1
⁢
(
best
)
is plurality of weights of the set of 3D matched key point pairs for a best pose obtained at a previous iteration j−1, Tsampled represents the plurality of candidate poses,
p
a
i
is ith key point from among the first set of key points and
p
b
i
is ith key point from among the second set of key points.

### Claim 5 (depends on 1)

The processor implemented method of claim 1, wherein the NFPR module determines the relative pose of the robot using the plurality of inputs and the coarse relative pose as motion prior by:
generating a current observation overlap mask and a previous observation overlap mask for an overlapping region between the previous observation and the current observation;
masking the current observation and the previous observation using the current observation overlap mask, and the previous observation overlap mask to obtain a current masked observation and a previous masked observation;
stacking the current masked observation and the previous masked observation together channel-wise to obtain a stacked feature; and
regressing a fine pose estimate using a deep neural network based on the stacked feature, the action prior and the motion prior, wherein the regressed fine pose estimate is considered as the relative pose of the robot.

### Claim 6 (depends on 5)

The processor implemented method of claim 5, wherein the previous observation overlap mask is generated by warping RGB image comprised in the current observation onto the previous observation using the depth image comprised in the current observation and an inverse of the action prior, and wherein the current observation overlap mask is generated by warping RGB image comprised in the previous observation onto the current observation using the depth the depth image comprised in the previous observation and the action prior.

### Claim 7

A system comprising:
a memory storing instructions;
one or more Input/Output (I/O) interfaces; and
one or more hardware processors coupled to the memory via the one or more communication interfaces, wherein the one or more hardware processors are configured by the instructions to:
obtain a plurality of inputs comprising i) a previous observation comprising an RGB image and a depth image captured at a previous time step from a robot, ii) an action prior defining an action performed by the robot at the previous time step, and iii) a current observation an RGB image and a depth image captured at a current time step obtained after performing the action by the robot;
determine a relative pose of the robot between the previous observation and the current observation based on the plurality of inputs by using a Visual Odometry (VO) module, wherein the VO module comprises i) a Geometric Coarse Pose Estimator (GCPE) module, and ii) a Neural Fine Pose Regression (NFPR) module, wherein the GCPE module estimates a coarse relative pose of the robot using the plurality of inputs, and wherein the NFPR module determines the relative pose of the robot using the plurality of inputs and the coarse relative pose as motion prior;
update a plurality of goal coordinates of the robot at the previous timestep based on the relative pose of the robot; and
determine a next action to be performed by the robot at the current timestep based on the updated plurality of goal coordinates using a navigation policy for ground navigation by the robot.

### Claim 8 (depends on 7)

The system of claim 7, wherein the GCPE module estimates the coarse relative pose of the robot using the plurality of inputs by:
extracting a first set of visual key points from the RGB image comprised in the previous observation and a second set of visual key points from the RGB image comprised in the current observation using a key point extractor;
matching a plurality of visual key points between the first set of visual key points and the second set of visual key points using a key point matcher to obtain a plurality of matched key point pairs, wherein the plurality of matched key point pairs are 2-Dimensional (2D);
filtering a set of matched key point pairs from the plurality of matched key point pairs based on a ratio test;
back-projecting the filtered set of matched key point pairs to obtain a set of 3-Dimensional (3D) matched key point pairs by using the depth image comprised in the previous observation, the depth image comprised in the current observation and a plurality of parameters of a camera of the robot used to capture the current and the previous observations;
performing a plurality of steps for a plurality of iterations until percentage increase of a best pose score at an iteration is higher than a predefined threshold value, wherein the plurality of steps comprise:
generating a plurality of candidate relative poses using a relative pose sampler function;
computing a plurality of pose scores associated with the plurality of candidate poses, wherein a pose score for a candidate pose among the plurality of candidate poses is computed by:
determining a plurality of weights corresponding to the set of 3D matched key point pairs for the candidate pose using a Motion Prior based Correspondence Weighing (MPCW) function; and
summing up the plurality of weights to obtain the pose score; and
selecting a best pose among the plurality of candidate relative poses having a best pose score, wherein the best pose score is a highest pose score among the plurality of pose scores, and wherein the best pose selected at the end of the plurality of iterations is considered as the coarse relative pose of the robot.

### Claim 9 (depends on 8)

The system of claim 8, wherein generating a plurality of candidate relative poses using a relative pose sampler function comprises alternatively sampling a rotation and a translation from a normal distribution of poses, wherein the normal distribution is generated in a first iteration among the plurality of iterations using the action prior as mean and a predefined standard deviation, and wherein the normal distribution is generated in a current iteration among the plurality of iterations using a coarse relative pose estimated in a previous iteration as mean and half of standard deviation used in the previous iteration.

### Claim 10 (depends on 8)

The system of claim 8, wherein the MPCW function determines weight as
W
C
a
⁢
b
i
j
=
W
C
a
⁢
b
i
j
-
1
⁢
(
best
)
(
T
sampled
(
p
a
i
)
-
p
b
i
)
2
+
(
T
sampled
-
1
(
p
b
i
)
-
p
a
i
)
2
,
and wherein
W
C
a
⁢
b
i
j
is a weight associated with an ith matched key point pair
C
a
⁢
b
i
from among the set of 3D matched key point pairs at a current iteration j,
W
C
a
⁢
b
i
j
-
1
⁢
(
best
)
is plurality of weights of the set of 3D matched key point pairs for a best pose obtained at a previous iteration j−1, Tsampled represents the plurality of candidate poses,
p
a
i
is an ith key point from among the first set of key points and
p
b
i
is ith key point from among the second set of key points.

### Claim 11 (depends on 7)

The system of claim 7, wherein the NFPR module determines the relative pose of the robot using the plurality of inputs and the coarse relative pose as motion prior by:
generating a current observation overlap mask and a previous observation overlap mask for an overlapping region between the previous observation and the current observation;
masking the current observation and the previous observation using the current observation overlap mask, and the previous observation overlap mask to obtain a current masked observation and a previous masked observation;
stacking the current masked observation and the previous masked observation together channel-wise to obtain a stacked feature; and
regressing a fine pose estimate using a deep neural network based on the stacked feature, the action prior and the motion prior, wherein the regressed fine pose estimate is considered as the relative pose of the robot.

### Claim 12 (depends on 11)

The system of claim 11, wherein the previous observation overlap mask is generated by warping RGB image comprised in the current observation onto the previous observation using the depth image comprised in the current observation and an inverse of the action prior, and wherein the current observation overlap mask is generated by warping RGB image comprised in the previous observation onto the current observation using the depth the depth image comprised in the previous observation and the action prior.

### Claim 13

One or more non-transitory machine-readable information storage mediums comprising one or more instructions which when executed by one or more hardware processors cause:
obtaining a plurality of inputs comprising: i) a previous observation comprising an RGB image and a depth image captured at a previous time step from a robot, ii) an action prior defining an action performed by the robot at the previous time step, and iii) a current observation an RGB image and a depth image captured at a current time step obtained after performing the action by the robot;
determining a relative pose of the robot between the previous observation and the current observation based on the plurality of inputs by using a Visual Odometry (VO) module, wherein the VO module comprises i) a Geometric Coarse Pose Estimator (GCPE) module, and ii) a Neural Fine Pose Regression (NFPR) module, wherein the GCPE module estimates a coarse relative pose of the robot using the plurality of inputs, and wherein the NFPR module determines the relative pose of the robot using the plurality of inputs and the coarse relative pose as motion prior;
updating a plurality of goal coordinates of the robot at the previous timestep based on the relative pose of the robot; and
determining a next action to be performed by the robot at the current timestep based on the updated plurality of goal coordinates using a navigation policy for ground navigation by the robot.

### Claim 14 (depends on 13)

The one or more non-transitory machine-readable information storage mediums of claim 13, wherein the GCPE module estimates the coarse relative pose of the robot using the plurality of inputs by:
extracting a first set of visual key points from the RGB image comprised in the previous observation and a second set of visual key points from the RGB image comprised in the current observation using a key point extractor;
matching a plurality of visual key points between the first set of visual key points and the second set of visual key points using a key point matcher to obtain a plurality of matched key point pairs, wherein the plurality of matched key point pairs are 2-Dimensional (2D);
filtering a set of matched key point pairs from the plurality of matched key point pairs based on a ratio test;
back-projecting the filtered set of matched key point pairs to obtain a set of 3-Dimensional (3D) matched key point pairs by using the depth image comprised in the previous observation, the depth image comprised in the current observation and a plurality of parameters of a camera of the robot used to capture the current and the previous observations;
performing a plurality of steps for a plurality of iterations until percentage increase of a best pose score at an iteration is higher than a predefined threshold value, wherein the plurality of steps comprise:
generating a plurality of candidate relative poses using a relative pose sampler function;
computing a plurality of pose scores associated with the plurality of candidate poses, wherein a pose score for a candidate pose among the plurality of candidate poses is computed by:
determining a plurality of weights corresponding to the set of 3D matched key point pairs for the candidate pose using a Motion Prior based Correspondence Weighing (MPCW) function; and
summing up the plurality of weights to obtain the pose score; and
selecting a best pose among the plurality of candidate relative poses having a best pose score, wherein the best pose score is a highest pose score among the plurality of pose scores, and wherein the best pose selected at the end of the plurality of iterations is considered as the coarse relative pose of the robot.

### Claim 15 (depends on 14)

The one or more non-transitory machine-readable information storage mediums of claim 14, wherein generating a plurality of candidate relative poses using a relative pose sampler function comprises alternatively sampling a rotation and a translation from a normal distribution of poses, wherein the normal distribution is generated in a first iteration among the plurality of iterations using the action prior as mean and a predefined standard deviation, and wherein the normal distribution is generated in a current iteration among the plurality of iterations using a coarse relative pose estimated in a previous iteration as mean and half of standard deviation used in the previous iteration.

### Claim 16 (depends on 14)

The one or more non-transitory machine-readable information storage mediums of claim 14, wherein the MPCW function determines weight as
W
C
a
⁢
b
i
j
=
W
C
a
⁢
b
i
j
-
1
⁢
(
best
)
(
T
sampled
(
p
a
i
)
-
p
b
i
)
2
+
(
T
sampled
-
1
(
p
b
i
)
-
p
a
i
)
2
,
and wherein
W
C
a
⁢
b
i
j
is a weight associated with an ith matched key point pair
C
a
⁢
b
i
from among the set of 3D matched key point pairs at a current iteration j,
W
C
a
⁢
b
i
j
-
1
⁢
(
best
)
is plurality of weights of the set of 3D matched key point pairs for a best pose obtained at a previous iteration j−1, Tsampled represents the plurality of candidate poses,
p
a
i
is an ith key point num among the first set of key points and
p
b
i
is ith key point from among the second set of key points.

### Claim 17 (depends on 13)

The one or more non-transitory machine-readable information storage mediums of claim 13, wherein the NFPR module determines the relative pose of the robot using the plurality of inputs and the coarse relative pose as motion prior by:
generating a current observation overlap mask and a previous observation overlap mask for an overlapping region between the previous observation and the current observation;
masking the current observation and the previous observation using the current observation overlap mask, and the previous observation overlap mask to obtain a current masked observation and a previous masked observation;
stacking the current masked observation and the previous masked observation together channel-wise to obtain a stacked feature; and
regressing a fine pose estimate using a deep neural network based on the stacked feature, the action prior and the motion prior, wherein the regressed fine pose estimate is considered as the relative pose of the robot.

### Claim 18 (depends on 17)

The one or more non-transitory machine-readable information storage mediums of claim 17, wherein the previous observation overlap mask is generated by warping RGB image comprised in the current observation onto the previous observation using the depth image comprised in the current observation and an inverse of the action prior, and wherein the current observation overlap mask is generated by warping RGB image comprised in the previous observation onto the current observation using the depth the depth image comprised in the previous observation and the action prior.
