Back to results
US 10,157,452 B1Granted
Image processing system for image rectification
AssigneeAmazon Technologies, Inc.
InventorsJoseph Patrick Tighe, Vuong Van Le, Gerard Guy Medioni
Life of this patent
Estimating the term…
Abstract
Images may be acquired by cameras and processed to detect the presence of objects. Described are techniques to use rectified images for further processing such as object detection, object identification, and so forth. In one implementation, an image from a camera is processed to produce a rectified image having an apparent perspective from overhead. For example, the rectified image may be an image that appears to have been obtained from a virtual camera that is above the shelf and having a field-of-view that looks downward. The rectified image may be an orthonormal projection of pixels in the acquired image, relative to a plane of the surface upon which the objects are resting.
Claims26
A system comprising:
- a shelf to hold one or more of a type of item;
- a camera positioned above the shelf and having a field-of-view that includes at least a portion of the shelf at an oblique angle with respect to a plane of the shelf;
- and
- a computing device comprising:
- a memory, storing computer-executable instructions;
- and
- a hardware processor to execute the computer-executable instructions to:
- determine the type of item associated with the shelf;
- access acquired image data obtained by the camera, wherein the acquired image data is representative of a two-dimensional array of pixels, each pixel having a value and each pixel being associated with a set of coordinates within the two-dimensional array of pixels;
- access item data indicative of a height of the type of item;
- access physical layout data indicative of:
- a position of the camera with respect to the shelf, and
- a direction the camera is pointing with respect to the shelf;
- select perspective transformation data based on the height of the type of item and the physical layout data, wherein the perspective transformation data associates a first set of two-dimensional coordinates in a camera plane to a second set of two-dimensional coordinates in a virtual camera plane, wherein the virtual camera plane is parallel to and above the plane of the shelf;
- generate rectified image data by using the perspective transformation data to map the values for the pixels at the first set of two-dimensional coordinates in the acquired image data to the second set of two-dimensional coordinates;
- determine one or more features present in the rectified image data, wherein a relative size of individual tops of the type of item are within a threshold size of one another;
- and
- determine item count data indicative of a quantity of the type of item present on the shelf using the one or more features determined to be present in the rectified image data.
15 more elements
A system comprising:
- a memory, storing computer-executable instructions;
- and
- a hardware processor to execute the computer-executable instructions to:
- access acquired image data representative of a foreshortened image of an inventory location;
- determine a type of item associated with the inventory location;
- determine a height of an item plane associated with the type of item;
- based on the height, select first perspective transformation data, the first perspective transformation data associating a plurality of reference points in a common plane of a planar target that is maintained at a second height that is within a threshold distance of the height and a plurality of alternate points in a virtual camera plane;
- generate first rectified image data from the acquired image data using the first perspective transformation data;
- and
- generate first output data using the first rectified image data.
7 more elements
A method comprising:
- accessing acquired image data, wherein the acquired image data is representative of a foreshortened image of an inventory location;
- accessing physical layout data indicative of:
- a relative position of a camera with respect to the inventory location, and
- a direction of the camera with respect to the inventory location;
- selecting first perspective transformation data associated with a first height and the physical layout data, the first perspective transformation data associating a plurality of reference points in a common plane of a planar target that is maintained at a second height that is within a threshold distance of the first height and a plurality of alternate points in a second plane;
- and
- generating first rectified image data from the acquired image data using the first perspective transformation data.
4 more elements
A system comprising:
- a memory, storing computer-executable instructions;
- and
- a hardware processor to execute the computer-executable instructions to:
- access first acquired calibration image data of a planar target at a first height, the planar target comprising one or more fiducials;
- generate first perspective transformation data based on the first acquired calibration image data, the first perspective transformation data generated by:
- determining first coordinates of first pixels in the first acquired calibration image data that correspond to the one or more fiducials which act as reference points in a common plane, and
- generating a first homography matrix;
- access second acquired calibration image data of the planar target at a second height;
- and
- generate second perspective transformation data based on the second acquired calibration image data, the second perspective transformation data generated by:
- determining second coordinates of second pixels in the second acquired calibration image data that correspond to the one or more fiducials, and
- generating a second homography matrix.
9 more elements
A system comprising:
- a memory, storing computer-executable instructions;
- and
- a hardware processor to execute the computer-executable instructions to:
- access first acquired calibration image data of a planar target at a first height above an inventory location and having a first distance from a camera;
- generate first perspective transformation data based on the first acquired calibration image data, the first perspective transformation data associating a plurality of reference points in a common plane of the planar target that is maintained at the first height and a plurality of alternate points in a virtual camera plane;
- store the first perspective transformation data associated with the first height;
- access acquired image data representative of a foreshortened image of the inventory location;
- determine a type of item associated with the inventory location;
- determine a height of an item plane associated with the item;
- based on the height of the item plane being within a threshold distance of the first height, retrieve the first perspective transformation data;
- and
- generate first rectified image data using the first perspective transformation data.
9 more elements
Dependencies are inferred from the claim text.