# US 11,113,841 B1 · Self-learning three-dimensional boundaries from images

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 11,113,841 B1
- Document ID: US11113841B1
- Status: Granted
- Kind: B1
- Application number: 16/438,028
- Filed: 2019-06-11
- Granted: 2021-09-07
- Earliest claimed: 2019-06-11
- Assignees: Amazon Technologies, Inc.
- Applicants: Amazon Technologies, Inc.
- Inventors: Lev Zelenskiy; Jon Ducrou; James Eather; Michael Neville
- Web page: https://patentgrep.com/patent/US11113841B1

## CPC

- G06Q 10/0835: Relationships between shipper or supplier and carriers
- G06T 7/246: using feature-based methods, e.g. the tracking of corners or segments
- G06T 7/74: {involving reference images or patches}
- G06T 15/20: Perspective computation
- G06V 10/255: {Detecting or recognising potential candidate objects based on visual cues, e.g. shapes}
- G06V 10/44: Local feature extraction by analysis of parts of the pattern, e.g. by detecting edges, contours, loops, corners, strokes or intersections; Connectivity analysis, e.g. of connected components
- G06V 20/52: Surveillance or monitoring of activities, e.g. for recognising suspicious objects (recognising microscopic objects G06V20/69)
- H04N 23/60: Control of cameras or camera modules
- H04N 23/90: Arrangement of cameras or camera modules, e.g. multiple cameras in TV studios or sports stadiums
- G06T 2207/20164: Salient point detection; Corner detection
- G06T 2207/30232: Surveillance
- G06T 2207/30236: Traffic on road, railway or crossing
- G06T 2207/30241: Trajectory
- G06T 2207/30252: Vehicle exterior; Vicinity of vehicle
- G06T 2207/30264: Parking
- G06T 2219/004: Annotating, labelling
- G06V 20/625: {License plates}
- G06V 2201/08: Detecting or categorising vehicles
- G06Q: INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION

## Abstract

Techniques are disclosed. The techniques include receiving, from a camera, an image showing a portion of a physical topography and target object. The techniques include determining, based on the image, a location of the target object shown in the image and a physical characteristic of the target object shown in the image, the physical characteristic being common with at least another target object. The techniques then include determining a three dimensional (3D) position of the target object in a 3D representation of the portion of the physical topography, the 3D position determined based on (i) the location of the target object, and (ii) the physical characteristic. Based on detecting a cluster of 3D positions of target objects (over several images), the techniques involve generating a 3D boundary of a zone of a portion of the physical topography.

## Claims

### Claim 1

A system, comprising:
a camera; and
a computer system communicatively coupled with the camera, the computer system comprising one or more processors and one or more computer-readable storage media storing instructions that, upon execution by the one or more processors, cause the computer system to, at least:
receive, from the camera, a first image of a first plurality of images representing a motion of a first vehicle license plate through a portion of a parking lot, the portion being within a field of view of the camera;
determine, based at least in part on the first image, a location of the first vehicle license plate shown in the first image;
determine, based at least in part on the first image, a physical characteristic of the first vehicle license plate shown in the first image, the physical characteristic being common with a second vehicle license plate;
generate a three-dimensional (3D) representation of the portion of the parking lot based at least in part on (i) the location and (ii) the physical characteristic;
subsequent to generating the 3D representation of the portion of the parking lot, receive, from the camera, a second image of a second plurality of images, the second image showing the portion of the parking lot and the second vehicle license plate;
determine a 3D position of the second vehicle license plate in the 3D representation based at least in part on (i) a second location of the second vehicle license plate shown in the second image, and (ii) the physical characteristic shown in the second image;
determine that a vehicle is parked in the parking lot, the vehicle being associated with the second vehicle license plate;
based in part on determining that the vehicle is parked, determine that the 3D position belongs to a cluster of 3D positions of one or more vehicle license plates shown in the second plurality of images; and
generate a 3D boundary of a parking space within the portion of the parking lot based at least in part on the 3D positions of the cluster, the parking space corresponding to where the vehicle is parked.

### Claim 2 (depends on 1)

The system of claim 1, wherein the motion of the first vehicle license plate corresponds to a movement in a two-dimensional (2D) plane that is substantially parallel to a ground of the parking lot.

### Claim 3 (depends on 1)

The system of claim 1, wherein the physical characteristic comprises a first vehicle license plate corner of the first vehicle license plate being at a fixed relative distance from a second vehicle license plate corner.

### Claim 4 (depends on 1)

The system of claim 1, wherein the second image of the second plurality of images is associated with a timestamp, the timestamp being used to determine that the vehicle is parked.

### Claim 5 (depends on 1)

The system of claim 1, further comprising:
a second camera, wherein the camera and the second camera have overlapping fields of view; and
the one or more computer-readable storage media storing further instructions that, when executed by the one or more processors, further cause the computer system to:
receive, from the camera, a third image showing a third vehicle license plate of another vehicle;
determine, from the third image, a unique identifier that identifies the third vehicle license plate;
determine, from the third image, a second 3D position of the third vehicle license plate in the 3D representation based at least in part on (i) a third location of the third vehicle license plate shown in the third image, and (ii) the physical characteristic shown in the third image;
determine that the another vehicle is parked in the parking space, based at least in part on (i) the second 3D position, and (ii) the 3D boundary of the parking space;
receive, from the second camera, a fourth image, the fourth image showing a second portion of the parking lot that is within a field of view of the second camera;
determine, from the fourth image, the unique identifier that identifies the third vehicle license plate; and
generate a second 3D boundary of the parking space corresponding to the second camera based at least in part on the 3D boundary of the parking space corresponding to the camera.

### Claim 6

A method implemented on a computer system, the method comprising:
receiving, from a camera, an image showing a portion of a parking lot and a vehicle license plate;
determining, based at least in part on the image, a location of the vehicle license plate shown in the image;
determining, based at least in part on the image, a physical characteristic of the vehicle license plate shown in the image, the physical characteristic being common with at least another vehicle license plate;
determining a three dimensional (3D) position of the vehicle license plate in a 3D representation of the portion of the parking lot, the 3D position determined based at least in part on (i) the location of the vehicle license plate, and (ii) the physical characteristic;
determining that the 3D position belongs to a cluster of 3D positions of one or more vehicle license plates; and
generating a 3D boundary of a parking space for a vehicle within the portion of the parking lot based at least in part on the 3D positions of the cluster.

### Claim 7 (depends on 6)

The method of claim 6, further comprising:
determining that the vehicle is parked in the parking lot, and wherein the 3D position is determined to belong to the cluster of 3D positions further based at least in part on the vehicle being parked.

### Claim 8 (depends on 7)

The method of claim 7, further comprising determining that the vehicle is parked in the parking lot by at least:
receiving an indication that the vehicle is checked-in, wherein the vehicle being checked-in corresponds to a state in which the vehicle is ready for receiving a delivery; and
based on receiving the indication, determining that the vehicle is parked.

### Claim 9 (depends on 7)

The method of claim 7, further comprising determining that the vehicle is parked in the parking lot by at least:
receiving as input a dwell time threshold;
determining a first timestamp associated with the image;
receiving, from the camera, a second image being associated with a second timestamp, the second image showing the portion of the parking lot and the vehicle license plate;
determining that the vehicle license plate is shown in the second image;
determining a second 3D position of the vehicle license plate in the 3D representation based at least in part on (i) a second location of the vehicle license plate shown in the second image, and (ii) the physical characteristic of the vehicle license plate shown in the second image; and
based at least in part on (i) a difference between the 3D position and the second 3D position, and (ii) a difference between the first timestamp and the second timestamp matching the dwell time threshold, determining that the vehicle is parked in the parking lot.

### Claim 10 (depends on 6)

The method of claim 6, further comprising determining the 3D representation by at least:
receiving, from the camera, a second image of a plurality of images representing a motion of a second vehicle license plate through the portion of the parking lot;
determining, based at least in part on the second image, a location of the second vehicle license plate shown in the second image;
determining, based at least in part on the second image, the physical characteristic of the second vehicle license plate shown in the second image that is common with the vehicle license plate; and
generating the 3D representation of the portion of the parking lot based at least in part on (i) the location and (ii) the physical characteristic.

### Claim 11 (depends on 10)

The method of claim 10, wherein determining the 3D representation further comprises:
receiving a third image of the plurality of images;
determining, based at least in part on the third image, a second location of the second vehicle license plate shown in the second image;
determining, based at least in part on the physical characteristic and utilizing a transformation model, a relative distance between the location and the second location.

### Claim 12 (depends on 10)

The method of claim 10, wherein the 3D representation of the portion of the parking lot is generated independently of an orientation of the camera.

### Claim 13 (depends on 6)

The method of claim 6, wherein the 3D boundary of the parking space corresponds to a cuboid, and wherein the cuboid is formed by a plurality of 3D coordinates of the 3D representation of the portion of the parking lot.

### Claim 14 (depends on 6)

The method of claim 6, wherein determining that the 3D position of the vehicle license plate belongs to the cluster of 3D positions comprises:
determining a distance value corresponding to a distance between an aspect of the 3D position and an aspect of the cluster of 3D positions; and
determining the 3D position of the vehicle license plate belongs to the cluster of 3D positions based at least in part on the distance value, wherein a number of positions of the cluster of 3D positions corresponds to a confidence level of the 3D boundary.

### Claim 15 (depends on 6)

The method of claim 6, further comprising:
receiving, from the camera, a second image showing the portion of the parking lot and a second vehicle license plate;
determining, based at least in part on the second image, a location of the second vehicle license plate shown in the image;
determining, based at least in part on the second image, a physical characteristic of the second vehicle license plate shown in the image, the physical characteristic being common with the at least another vehicle license plate;
determining a second 3D position of the second vehicle license plate in the 3D representation of the portion of the parking lot, the second 3D position determined based at least in part on (i) the location of the second vehicle license plate, and (ii) the physical characteristic; and
determining that a second vehicle is parked in the parking space based at least in part on (i) the second 3D position of the second vehicle license plate, and (ii) the 3D boundary.

### Claim 16 (depends on 6)

The method of claim 6, further comprising:
subsequent to generating the 3D boundary of the parking space, receiving, from the camera, a second image showing a different portion of the parking lot;
receiving an indication that a second vehicle is parked in the parking space, the indication based on an input that is independent of the second image; and
based at least in part on determining that a second vehicle license plate associated with the second vehicle is not detected within the second image, transmitting a message to an administrator device that a field of view of the camera has been changed from the portion of the parking lot to the different portion.

### Claim 17

A non-transitory computer-readable storage medium storing instructions that, upon execution on a computer system, configure the computer system to perform operations comprising:
receiving, from a camera, an image showing a portion of a physical topography and a target object, the physical topography being divided into a plurality of zones;
determining, based at least in part on the image, a location of the target object shown in the image;
determining, based at least in part on the image, a physical characteristic of the target object shown in the image, the physical characteristic being common with at least another target object;
determining a three dimensional (3D) position of the target object in a 3D representation of the portion of the physical topography, the 3D position determined based at least in part on (i) the location of the target object, and (iii) the physical characteristic;
determining that the 3D position belongs to a cluster of 3D positions of one or more target objects; and
generating a 3D boundary of a zone of the plurality of zones of the portion of the physical topography based at least in part on the 3D positions of the cluster, the zone corresponding to the 3D position of the target object.

### Claim 18 (depends on 17)

The non-transitory computer-readable storage medium of claim 17, wherein the target object corresponds to a vehicle license plate that is associated with a vehicle, the physical topography corresponds to a parking lot, and the zone corresponds to a parking space of the parking lot.

### Claim 19 (depends on 17)

The non-transitory computer-readable storage medium of claim 17, wherein the operations further comprise:
receiving, from the camera, a second image of a second plurality of images representing a motion of a second target object through the portion of the physical topography;
determining, based at least in part on the second image, a location of the second target object shown in the second image;
determining, based at least in part on the second image, the physical characteristic of the second target object shown in the second image that is common with the target object; and
generating the 3D representation of the portion of the physical topography based at least in part on (i) the location and (ii) the physical characteristic.

### Claim 20 (depends on 17)

The non-transitory computer-readable storage medium of claim 17, wherein the operations further comprise:
receiving, from the camera, a second image showing a second target object;
determining, based at least in part on the second image, a location of the second target object shown in the image;
determining, based at least in part on the second image, a physical characteristic of the second target object shown in the image, the physical characteristic being common with the at least another target object;
determining a second 3D position of the second target object in the 3D representation of the portion of the physical topography, the second 3D position determined based at least in part on (i) the location of the second target object, and (ii) the physical characteristic; and
determining that the second 3D position of the second target object belongs to a second cluster of 3D positions of one or more target objects;
determining a distance value corresponding to a distance between an aspect of the cluster of 3D positions and an aspect of the second cluster of 3D positions; and
determining that the distance value falls within a drift range of the camera, wherein generating the 3D boundary of the zone is performed based at least in part on the distance value falling within the drift range.