# US 12,117,844 B1 · Autonomous navigation inside trailers

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 12,117,844 B1
- Document ID: US12117844B1
- Status: Granted
- Kind: B1
- Application number: 17/544,268
- Filed: 2021-12-07
- Granted: 2024-10-15
- Earliest claimed: 2021-12-07
- Assignees: Amazon Technologies, Inc.
- Applicants: Amazon Technologies, Inc.
- Inventors: Adam Joseph Greenbaum
- Web page: https://patentgrep.com/patent/US12117844B1

## CPC

- G05D 1/0246: {using a video camera in combination with image processing means}
- G05D 1/0223: {involving speed control of the vehicle (vehicle fittings for automatically controlling, i.e. preventing speed from exceeding an arbitrarily established velocity or maintaining speed at a particular velocity, as selected by the vehicle operator B60K31/00)}
- G05D 1/0238: {using obstacle or wall sensors (G05D1/0246 and G05D1/0289 take precedence; lidar systems designed for anti-collision purposes G01S17/93)}
- G06T 7/70: Determining position or orientation of objects or cameras (camera calibration G06T7/80)
- G06V 10/25: Determination of region of interest [ROI] or a volume of interest [VOI]
- G06V 10/44: Local feature extraction by analysis of parts of the pattern, e.g. by detecting edges, contours, loops, corners, strokes or intersections; Connectivity analysis, e.g. of connected components
- G06V 20/56: exterior to a vehicle by using sensors mounted on the vehicle
- G05D: SYSTEMS FOR CONTROLLING OR REGULATING NON-ELECTRIC VARIABLES
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING

## Abstract

Systems, methods, and computer-readable media are disclosed for autonomous navigation inside trailers. In one embodiment, an example autonomous robot may include a front end, a rear end, a sensor disposed at the front end, a rear-facing camera disposed at the rear end, and a force sensor configured to detect lateral and longitudinal forces exerted on a container being transported by the autonomous robot. The autonomous robot may be configured to transport the container from a facility into a trailer, where the autonomous robot is configured to navigate inside the trailer using the rear-facing camera.

## Claims

### Claim 1

An autonomous robot configured to load containers into a semi-trailer, the autonomous robot comprising:
a rear-facing camera disposed at a rear end of the autonomous robot;
a sensor disposed at a front end of the autonomous robot;
a force sensor disposed at a top side of the autonomous robot, wherein the force sensor is disposed between the top side of the autonomous robot and a bottom side of a container being transported by the autonomous robot; and
a controller configured to guide the autonomous robot by:
determining a drop off location for the container inside the semi-trailer;
causing the autonomous robot to enter the semi-trailer through an entryway;
capturing, using the rear-facing camera, a first image of the entryway;
determining, using the first image, a first set of lines corresponding to edges of the entryway, the first set of lines comprising a first vertical line, a second vertical line, a first horizontal line, and a second horizontal line, wherein the first set of lines are connected and form a rectangular shape;
determining an area inside the first set of lines;
determining a distance between the entryway and the autonomous robot based at least in part on the area;
determining a distortion value for the rectangular shape formed by the set of lines, wherein the distortion value is relative to a default rectangle;
determining an orientation of the autonomous robot based at least in part on the distortion value;
determining, using the distance and the orientation, a current position relative to the drop off location; and
determining a path to the drop off location based at least in part on the current position.

### Claim 2 (depends on 1)

The autonomous robot of claim 1, wherein the controller is further configured to guide the autonomous robot by:
determining that the autonomous robot is within a threshold distance of the drop off location;
determining, using the force sensor, that the container is in contact with a first object in a longitudinal direction inside the semi-trailer;
determining, using the force sensor, that the container is in contact with a second object in a lateral direction inside the semi-trailer;
determining that the container is in the drop off location; and
causing the container to be unloaded.

### Claim 3 (depends on 1)

The autonomous robot of claim 1, wherein the controller is further configured to guide the autonomous robot by:
determining, using the first image, a second set of lines corresponding to intersections between the sidewalls, ceiling, and floor of the semi-trailer, wherein the second set of lines comprises a first diagonal line that connects to a first corner formed by the rectangular shape, a second diagonal line that connects to a second corner formed by the rectangular shape, a third diagonal line that connects to a third corner formed by the rectangular shape, and a fourth diagonal line that connects to a fourth corner formed by the rectangular shape; and
determining a position and orientation of the autonomous robot relative to the sidewalls of the semi-trailer based at least in part on the second set of lines.

### Claim 4 (depends on 1)

The autonomous robot of claim 1, wherein the controller is further configured to guide the autonomous robot by:
determining a first confidence level associated with the current position based at least in part on the first set of lines; and
causing a speed of the autonomous robot to be adjusted based at least in part on the first confidence level.

### Claim 5

An autonomous robot comprising:
a front end;
a rear end;
a sensor disposed at the front end;
a rear-facing camera disposed at the rear end; and
a force sensor configured to detect lateral and longitudinal forces exerted on a container being transported by the autonomous robot, wherein the autonomous robot is configured to transport the container from a facility into a trailer;
memory comprising computer-executable instructions; and
one or more computer processors configured to access the memory and execute the computer-executable instructions to:
determine a first image using the rear-facing camera;
determine, using the first image, a set of lines corresponding to edges of an entryway to the trailer, wherein the set of lines connect to form a shape;
determine an area inside the set of lines; and
determine a first distance between the entryway and the autonomous robot based at least in part on the area;
wherein the autonomous robot is configured to navigate inside the trailer using the rear-facing camera.

### Claim 6 (depends on 5)

The autonomous robot of claim 5, wherein the one or more computer processors are further configured to execute the computer-executable instructions to:
determine a position of the shape formed by the set of lines relative to a field of view of the rear-facing camera; and
determine a lateral position and angular orientation of the autonomous robot based at least in part on the position.

### Claim 7 (depends on 5)

The autonomous robot of claim 5, wherein the one or more computer processors are further configured to execute the computer-executable instructions to:
determine a parallax distortion of the shape formed by the set of lines relative to a rectangle; and
determine an orientation and a linear position of the autonomous robot based at least in part on the parallax distortion.

### Claim 8 (depends on 7)

The autonomous robot of claim 7, wherein the one or more computer processors are further configured to execute the computer-executable instructions to:
determine, using the distance and the orientation, a current position relative to a drop off location for the container; and
determine a path to the drop off location based at least in part on the current position.

### Claim 9 (depends on 5)

The autonomous robot of claim 5, wherein the one or more computer processors are further configured to execute the computer-executable instructions to:
determine that the autonomous robot is within a threshold distance of a drop off location for the container; and
detect lateral and longitudinal forces using the force sensor.

### Claim 10 (depends on 9)

The autonomous robot of claim 9, wherein the one or more computer processors are further configured to execute the computer-executable instructions to:
determine that the longitudinal force satisfies a first threshold;
determine that the lateral force satisfies a second threshold;
determine that the autonomous robot is at the drop off location; and
cause the container to be unloaded.

### Claim 11 (depends on 5)

The autonomous robot of claim 5, wherein the one or more computer processors are further configured to execute the computer-executable instructions to:
determine a second image using the rear-facing camera;
determine an updated distance at a first timestamp between the entryway and the autonomous robot based at least in part on the second image;
determine a third image using the rear-facing camera; and
determine an updated distance at a second timestamp between the entryway and the autonomous robot based at least in part on the third image;
wherein the autonomous robot is configured to navigate using closed loop control.

### Claim 12 (depends on 5)

The autonomous robot of claim 5, wherein the one or more computer processors are further configured to execute the computer-executable instructions to:
determine, using the first image, a set of diagonal lines corresponding to intersections between the sidewalls, ceiling, and floor of the trailer; and
determine a position and orientation of the autonomous robot relative to the sidewalls of the trailer based at least in part on the set of diagonal lines.

### Claim 13 (depends on 5)

The autonomous robot of claim 5, wherein the one or more computer processors are further configured to execute the computer-executable instructions to:
determine a confidence level associated with the distance based at least in part on the set of lines; and
cause a speed of the autonomous robot to be adjusted based at least in part on the confidence level.

### Claim 14 (depends on 5)

The autonomous robot of claim 5, wherein sensor is a lidar sensor, and wherein the autonomous robot further comprises:
a front-facing camera disposed at the front end;
wherein the autonomous robot is configured to tunnel underneath the container and lift the container from a bottom surface; and
wherein the autonomous robot is configured to navigate autonomously both in the facility and in the trailer.

### Claim 15

An autonomous robot comprising:
a front end;
a front-facing camera disposed at the front end;
a rear end;
a sensor disposed at the front end;
a rear-facing camera disposed at the rear end; and
a force sensor configured to detect lateral and longitudinal forces exerted on a container being transported by the autonomous robot, wherein the autonomous robot is configured to transport the container from a facility into a trailer, and to navigate inside the trailer using the rear-facing camera;
wherein the autonomous robot is configured to: (i) tunnel underneath the container and lift the container from a bottom surface, and (ii) navigate autonomously both in the facility and in the trailer.

### Claim 16 (depends on 15)

The autonomous robot of claim 15, further comprising:
memory comprising computer-executable instructions; and
one or more computer processors configured to access the memory and execute the computer-executable instructions to:
determine a first image using the rear-facing camera;
determine, using the first image, a set of lines corresponding to edges of an entryway to the trailer, wherein the set of lines connect to form a shape;
determine an area inside the set of lines; and
determine a first distance between the entryway and the autonomous robot based at least in part on the area.

### Claim 17 (depends on 16)

The autonomous robot of claim 16, wherein the one or more computer processors are further configured to execute the computer-executable instructions to:
determine a position of the shape formed by the set of lines relative to a field of view of the rear-facing camera; and
determine a lateral position and angular orientation of the autonomous robot based at least in part on the position.

### Claim 18 (depends on 16)

The autonomous robot of claim 16, wherein the one or more computer processors are further configured to execute the computer-executable instructions to:
determine a parallax distortion of the shape formed by the set of lines relative to a rectangle; and
determine an orientation and a linear position of the autonomous robot based at least in part on the parallax distortion.

### Claim 19 (depends on 18)

The autonomous robot of claim 18, wherein the one or more computer processors are further configured to execute the computer-executable instructions to:
determine, using the distance and the orientation, a current position relative to a drop off location for the container; and
determine a path to the drop off location based at least in part on the current position.

### Claim 20 (depends on 16)

The autonomous robot of claim 16, wherein the one or more computer processors are further configured to execute the computer-executable instructions to:
determine, using the first image, a set of diagonal lines corresponding to intersections between the sidewalls, ceiling, and floor of the trailer; and
determine a position and orientation of the autonomous robot relative to the sidewalls of the trailer based at least in part on the set of diagonal lines.