# US 2024/0004388 A1 · Method and system for navigation of robot from one area to another area

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2024/0004388 A1
- Document ID: US20240004388A1
- Status: Published application
- Kind: A1
- Application number: 18/310,283
- Filed: 2023-05-01
- Published: 2024-01-04
- Earliest claimed: 2022-06-30
- Assignees: Tata Consultancy Services Limited
- Applicants: Tata Consultancy Services Limited
- Inventors: SNEHASIS BANERJEE; SAYAN PAUL; RUDDRA DEV ROYCHOUDHURY; ABHIJAN BHATTACHARYYA
- Web page: https://patentgrep.com/patent/US20240004388A1

## CPC

- G05D 1/0212: {with means for defining a desired trajectory (involving a plurality of land vehicles G05D1/0287)}
- G01C 21/005: {with correlation of navigation data from several sources, e.g. map or contour matching (G01C21/30 takes precedence)}
- G01C 21/206: {specially adapted for indoor navigation}
- G01C 21/383: {Indoor data}
- G05D 1/0238: {using obstacle or wall sensors (G05D1/0246 and G05D1/0289 take precedence; lidar systems designed for anti-collision purposes G01S17/93)}
- G05D 1/0246: {using a video camera in combination with image processing means}
- G05D 1/0248: {in combination with a laser (lasers per se H01S)}
- G05D 1/0272: {comprising means for registering the travel distance, e.g. revolutions of wheels (measuring distance traversed on the ground by vehicles, e.g. using odometers G01C22/00)}
- G05D 1/0274: {using mapping information stored in a memory device (navigation using map-matching G01C21/30)}
- G05D 1/2462: {using feature-based mapping}
- G05D 1/249: from positioning sensors located off-board the vehicle, e.g. from cameras
- G05D 1/622: Obstacle avoidance (predicting or avoiding probable or impending collision of road vehicles B60W30/08)
- G05D 1/646: Following a predefined trajectory, e.g. a line marked on the floor or a flight path
- G06T 7/73: using feature-based methods
- G05D 2111/54: {for measuring the travel distances, e.g. by counting the revolutions of wheels}
- G06T 2207/10024: Color image
- G06T 2207/10028: Range image; Depth image; 3D point clouds
- G05D: SYSTEMS FOR CONTROLLING OR REGULATING NON-ELECTRIC VARIABLES
- G01C: MEASURING DISTANCES, LEVELS OR BEARINGS; SURVEYING; NAVIGATION; GYROSCOPIC INSTRUMENTS; PHOTOGRAMMETRY OR VIDEOGRAMMETRY (measuring liquid level G01F; radio navigation, determining distance or velocity by use of propagation effects, e.g. Doppler effects, propagation time, of radio waves, analogous arrangements using other waves G01S)
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL

## Abstract

A system and method for navigation of a robot from a first area to a second area in a facility is provided. The present disclosure is providing robot navigation using the ‘Areagoal’ Navigation technique. ‘Areagoal’ class of problem is divided into two subtasks: identifying the area; and navigation from one area to another. The robot starts in first location and goes out of the current area if it is not in the target area. If there are multiple openings from the first area, it needs to select the most statistically close one to the target area and go there. If the target area is not reached, it backtracks to an earlier viable branch position to continue the target area search. The system takes input from RGB-D camera and odometer, while the output is action space (left, right, forward) with goal of moving to target area.

## Claims

### Claim 1

A processor implemented method for navigation of a robot from a first area to a second area in a facility, the method comprising:
providing, via a camera, a stream of egocentric images as the robot moves, wherein the camera is located on the robot;
recording, via an odometer, coordinates of a current location of the robot in the first area;
providing, via a user interface, a target area as input, wherein the target area is the second area, wherein the first area and the second area comprises a plurality of objects;
generating, via one or more hardware processors, a growing semantic map of the facility using the stream of egocentric images and the recording coordinates from the odometer, for storing a plurality of landmarks present in the facility, wherein the plurality of landmarks is stored at a predefined time interval;
creating, via the one or more hardware processors, an initial grid map of N*N dimension using the coordinates, wherein N is a predefined number and the initial grid map keeps on updating the grid map on receiving a new observation from the stream of egocentric images and the odometer recordings, expanding the grid map on demand;
identifying, via the one or more hardware processors, coordinates of four corners of the initial grid map;
performing, via the one or more hardware processors, a 360 degrees rotation to identify a current area;
comparing, via the one or more hardware processors, the current area with the target area;
searching, via the one or more hardware processors, for openings and free spaces in the current area to enter a newer area, if the robot identifies the current area is not the target area;
actuating, via the one or more hardware processors, the robot to a new view, if the stream of egocentric images identifies at least one of a mirror or a blockage;
backtracking, via the one or more hardware processors, the robot to the last stored landmark from amongst the plurality of landmarks, if the robot identifies a dead end; and
planning, via the one or more hardware processors, the robot path towards the identified four corners of the initial grid map sequentially, if all the landmarks are exhausted, wherein the target area lies in one of the paths.

### Claim 2 (depends on 1)

The processor implemented method of claim 1 further comprising ending the actuation of the robot if the actuation of robot crosses a predefined number of steps.

### Claim 3 (depends on 1)

The processor implemented method of claim 1 further comprising navigating the robot to at least one of a centroid point or a centroid sub-region with radius of 1 meter of the target area.

### Claim 4 (depends on 1)

The processor implemented method of claim 1, wherein the step of searching openings and free spaces comprises pretraining an object detection algorithm for additional three object classes comprising ‘open door’, ‘closed door’ and ‘free area’.

### Claim 5 (depends on 1)

The processor implemented method of claim 1, wherein the current area is identified based on the one or more objects detected over the stream of egocentric images using object-region relations, region-region transitions and learnt spatial Graph Convolutional Network (GCN) embeddings.

### Claim 6 (depends on 1)

The processor implemented method of claim 1, wherein the camera is an RGB-D (Red-Green-Blue color channels and Depth channel) camera and the output of the RGB-D camera is converted into the stream of egocentric images.

### Claim 7 (depends on 1)

The processor implemented method of claim 1, wherein the mirror is identified using one of the following:
if current and past RGB image observations across robot steps are very similar using a threshold parameter, or
utilizing Mirror3D approach to identify correct depth estimates.

### Claim 8 (depends on 1)

The processor implemented method of claim 1, wherein the backtracking is done using a visual semantic map.

### Claim 9

A system for navigation of a robot from a first area to a second area in a facility, the system comprises:
a camera, located on the robot, configured to provide a stream of egocentric images as the robot moves;
an odometer, located on the robot, configured to record coordinates of a current location of the robot in the first area;
a user interface for providing a target area as input, wherein the target area is the second area, wherein the first area and the second area comprises a plurality of objects;
one or more hardware processors;
a memory in communication with the one or more hardware processors, wherein the one or more first hardware processors are configured to execute programmed instructions stored in the one or more first memories, to:
generate a growing semantic map of the facility using the stream of egocentric images and the recording coordinates from the odometer, for storing a plurality of landmarks present in the facility, wherein the plurality of landmarks is stored at a predefined time interval;
create an initial grid map of N*N dimension using the coordinates, wherein N is a predefined number and the initial grid map keeps on updating the grid map on receiving a new observation from the stream of egocentric images and the odometer recordings, expanding the grid map on demand;
identify coordinates of four corners of the initial grid map;
perform a 360 degrees rotation to identify a current area;
compare the current area with the target area;
search for openings and free spaces in the current area to enter a newer area, if the robot identifies the current area is not the target area;
actuate the robot to a new view, if the stream of egocentric images identifies at least one of a mirror or a blockage;
backtrack the robot to the last stored landmark from amongst the plurality of landmarks, if the robot identifies a dead end; and
plan the robot path towards the identified four corners of the initial grid map sequentially, if all the landmarks are exhausted, wherein the target area lies in one of the paths.

### Claim 10 (depends on 9)

The system of claim 9 further configured to end the actuation of the robot if the actuation of robot crosses a predefined number of steps.

### Claim 11 (depends on 9)

The system of claim 9 further configured to navigate the robot to at least one of a centroid point or a centroid sub-region with radius of 1 meter of the target area.

### Claim 12 (depends on 1)

The processor implemented method of claim 1, wherein the step of searching openings and free spaces comprises pretraining an object detection algorithm for additional three object classes comprising ‘open door’, ‘closed door’ and ‘free area’.

### Claim 13 (depends on 9)

The system of claim 9, wherein the camera is an RGB-D camera and the output of the RGB-D camera is converted into the stream of egocentric images.

### Claim 14 (depends on 9)

The system of claim 9, wherein the backtracking is done using a visual semantic map.

### Claim 15

One or more non-transitory machine-readable information storage mediums comprising one or more instructions which when executed by one or more hardware processors cause:
providing, via a camera, a stream of egocentric images as the robot moves, wherein the camera is located on the robot;
recording, via an odometer, coordinates of a current location of the robot in the first area;
providing, via a user interface, a target area as input, wherein the target area is the second area, wherein the first area and the second area comprises a plurality of objects;
generating, a growing semantic map of the facility using the stream of egocentric images and the recording coordinates from the odometer, for storing a plurality of landmarks present in the facility, wherein the plurality of landmarks is stored at a predefined time interval;
creating an initial grid map of N*N dimension using the coordinates, wherein N is a predefined number and the initial grid map keeps on updating the grid map on receiving a new observation from the stream of egocentric images and the odometer recordings, expanding the grid map on demand;
identifying coordinates of four corners of the initial grid map;
performing a 360 degrees rotation to identify a current area;
comparing the current area with the target area;
searching for openings and free spaces in the current area to enter a newer area, if the robot identifies the current area is not the target area;
actuating the robot to a new view, if the stream of egocentric images identifies at least one of a mirror or a blockage;
backtracking the robot to the last stored landmark from amongst the plurality of landmarks, if the robot identifies a dead end; and
planning the robot path towards the identified four corners of the initial grid map sequentially, if all the landmarks are exhausted, wherein the target area lies in one of the paths.

### Claim 16 (depends on 15)

The one or more non-transitory machine-readable information storage mediums of claim 15, further cause ending the actuation of the robot if the actuation of robot crosses a predefined number of steps.

### Claim 17 (depends on 15)

The one or more non-transitory machine-readable information storage mediums of claim 15, further cause navigating the robot to at least one of a centroid point or a centroid sub-region with radius of 1 meter of the target area.

### Claim 18 (depends on 15)

The one or more non-transitory machine-readable information storage mediums of claim 15, further cause ‘open door’, ‘closed door’ and ‘free area’.

### Claim 19 (depends on 15)

The one or more non-transitory machine-readable information storage mediums of claim 15, wherein the current area is identified based on the one or more objects detected over the stream of egocentric images using object-region relations, region-region transitions and learnt spatial Graph Convolutional Network (GCN) embeddings.

### Claim 20 (depends on 15)

The one or more non-transitory machine-readable information storage mediums of claim 15, wherein the camera is an RGB-D (Red-Green-Blue color channels and Depth channel) camera and the output of the RGB-D camera is converted into the stream of egocentric images.
