# US 2024/0201690 A1 · Systems and methods for identifying objects in a space

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2024/0201690 A1
- Document ID: US20240201690A1
- Status: Published application
- Kind: A1
- Application number: 18/080,883
- Filed: 2022-12-14
- Published: 2024-06-20
- Earliest claimed: 2022-12-14
- Applicants: Indoor Robotics Ltd.
- Inventors: NATAN SHEMER; RUBEN MERGUI
- Web page: https://patentgrep.com/patent/US20240201690A1

## CPC

- G05D 1/0238: {using obstacle or wall sensors (G05D1/0246 and G05D1/0289 take precedence; lidar systems designed for anti-collision purposes G01S17/93)}
- G05D 1/2462: {using feature-based mapping}
- G01C 21/005: {with correlation of navigation data from several sources, e.g. map or contour matching (G01C21/30 takes precedence)}
- G01C 21/206: {specially adapted for indoor navigation}
- G05D 1/0246: {using a video camera in combination with image processing means}
- G05D 1/0274: {using mapping information stored in a memory device (navigation using map-matching G01C21/30)}
- G05D 1/243: Means capturing signals occurring naturally from the environment, e.g. ambient optical, acoustic, gravitational or magnetic signals (using passive navigation aids external to the vehicle G05D1/244; using signals from positioning sensors located off-board the vehicle G05D1/249)
- G06T 7/73: using feature-based methods
- G05D 2109/10: Land vehicles
- G05D 2111/10: Optical signals
- G05D: SYSTEMS FOR CONTROLLING OR REGULATING NON-ELECTRIC VARIABLES
- G01C: MEASURING DISTANCES, LEVELS OR BEARINGS; SURVEYING; NAVIGATION; GYROSCOPIC INSTRUMENTS; PHOTOGRAMMETRY OR VIDEOGRAMMETRY (measuring liquid level G01F; radio navigation, determining distance or velocity by use of propagation effects, e.g. Doppler effects, propagation time, of radio waves, analogous arrangements using other waves G01S)
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL

## Abstract

A method for identifying objects by a mobile robot moving in a space by obtaining a map of the space, the map including objects and objects' locations in the space, moving the robot inside the space, localizing the moving robot in the space, capturing an image in the space using an image sensor of the mobile robot, identifying an object from the captured image, computing a relative direction of the object from the camera's direction at the time of capturing the image, and determining that the object that appears in the image is the same object that appears on the map based on the relative direction, the robot's location when capturing the image, and the object's location on the map.

## Claims

### Claim 1

A method for identifying objects by a mobile robot moving in a space, the method comprising:
obtaining a map of the space, said map comprises objects and objects' locations in the space;
moving the robot inside the space;
localizing the moving robot in the space;
capturing an image in the space using an image sensor of the mobile robot;
identifying an object from the captured image;
computing a relative direction of the object from the camera's direction at the time of capturing the image; and
determining that the object that appears in the image is the same object that appears on the map based on the relative direction, the robot's location when capturing the image and the object's location on the map.

### Claim 2 (depends on 1)

The method of claim 1, further comprising computing an appearance probability that represents a probability that the object that appears in the captured image also appears on the map.

### Claim 3 (depends on 1)

The method of claim 1, wherein the map includes multiple sub-maps, the multiple sub-maps include objects located in sections of the space.

### Claim 4 (depends on 1)

The method of claim 1, wherein the map may comprise geolocation coordinates of the objects' locations.

### Claim 5 (depends on 1)

The method of claim 1, wherein the localizing comprises computing a distance and an azimuth relative to a reference point representing a location of another object whose location is known.

### Claim 6 (depends on 1)

The method of claim 1, further comprising assigning an orientation value that represents the azimuth and angle in a vertical plane of the camera when capturing the image.

### Claim 7 (depends on 1)

The method of claim 1, wherein computing a relative direction of the object from the camera's direction comprises converting objects' coordinates to angles in the camera's referential.

### Claim 8 (depends on 1)

The method of claim 1, wherein determining that the specific object is the same object that appears on the map comprises determining that the range angle is lower than a threshold angle.

### Claim 9 (depends on 8)

The method of claim 8, wherein the threshold angle differs among the objects in the space.

### Claim 10 (depends on 9)

The method of claim 9, wherein the threshold angle is updated by an operator of the mobile robot.

### Claim 11 (depends on 9)

The method of claim 9, wherein the threshold angle is automatically updated based on environmental measurements.

### Claim 12 (depends on 9)

The method of claim 9, further comprising identifying the object from the captured image as the object appearing on the map in case the difference between the relative direction extracted from the image and the estimated direction extracted from the map is smaller than the angle threshold.

### Claim 13 (depends on 1)

The method of claim 1, wherein the object is selected from a door or a window, wherein the method further comprises detecting a position of the door or the window, wherein the position is selected from a group consisting of fully open, partially closed or entirely closed.

### Claim 14 (depends on 1)

The method of claim 1, further comprising:
computing a location probability indicating a probability that the mobile robot is located in a specific area in the space, wherein determining that the specific object is the same object that appears on the map comprises determining that the range angle is lower than a threshold angle;
computing an appearance probability that represents a probability that the object that appears in the captured image also appears on the map; and
determining that the object captured in the image is the object on the map by multiplying the appearance probability by the location probability of the areas in the space in which the angle between the camera's direction and the relative location in the map satisfied the threshold angle.
