US 9,870,521 B1Granted

Systems and methods for identifying objects

AssigneeAmazon Technologies, Inc.

InventorsMashhour Solh, Yue Liu, Gordon Lee, Bryan Taylor

Life of this patent

Estimating the term…

Loading patent…

Abstract

Systems, methods and devices for automatically cropping images taken by an electronic device in order to determine the identity of a product contained in the image are described herein. A number of different techniques may be applied to perform the automatic cropping, including a focus sweep technique in which a first image is analyzed for the presence of a human being and then a second image is taken in a plane closer to the camera than the first image. The two frames are analyzed and a resultant image is provided that avoids the regions in which the human being is present to focus on the product. In other embodiments, a motion vector calculation is made between two images in which an individual is holding a product. The motion vectors related to the human are removed and a bounding box is calculated to reduce the size of the image to include a higher percentage of the product, such that the product can be more easily identified.

Claims18

Dependent claims

A method for identifying objects, comprising:

receiving, from an electronic device, first image data representing a first image including an individual holding an object in a hand of the individual;
identifying a head of a human body of the individual represented in the first image;
determining a skin tone of the head;
identifying the hand represented within the first image by determining a portion of the first image that includes five similarly sized blocks have a color that is the skin tone;
generating a new version of the first image by subtracting the portion from the first image;
determining a region of interest associated with a location of the new version that is associated an area including the portion;
generating a cropped image by cropping the new version of the image to include the region of interest;
determining an identity of the object by:
determining a similarity value of the object located within the region of interest and a reference item stored within an object database;
and
determining that the similarity value is greater than a similarity threshold for the object;
receiving responsive text data representing a name of the object and a message;
generating responsive audio data representing the responsive text data by executing text-to-speech functionality;
and
sending the responsive audio data to the electronic device.

12 more elements

A method, comprising:

receiving audio data representing speech from an electronic device;
determining that at least one image is to be captured;
generating an instruction to cause the electronic device to capture a first image;
sending the instruction to the electronic device;
receiving, from the electronic device, first image data representing the first image;
identifying a first human body part within the first image;
determining a first location of the first human body part within the first image;
determining a first region of interest within the first image such that the first region of interest is offset from the first location;
determining an identity of an object within the first image;
and
sending information associated with the identity to the electronic device.

8 more elements

A system, comprising:

communications circuitry configured to receive first image data representing a first image taken by an image capturing component of an electronic device;
memory;
and
at least one processor operable to:
receive audio data representing speech from the electronic device prior to receiving the first image data;
determine that at least one image is to be captured;
generate an instruction to cause the electronic device to capture the first image;
send the instruction to the electronic device;
identify a first human body part within the first image;
determine a first location of the first human body part within the first image;
determine a first region of interest within the first image such that the first region of interest is offset from the first location;
determine an identity of an object within the first image;
and
send information associated with the identity to the electronic device.

11 more elements

Dependencies are inferred from the claim text.