Self checkout with visual recognition

A checkout system, comprising

• a data reader section including an optical code reader configured to read an optical code on an item and to generate a product identifier of the item;
• a collection section within which items read by the optical code reader are collected after having been read by the optical code reader;
• at least one camera disposed with a field of view of the collection section for capturing one or more images of an item within the collection section;
• a database of stored visual features of known objects;
• an image processor configured to
• a) extract a plurality of visual features from the one or more images of the item,
• b) obtain from the database a set of stored visual features corresponding to the item as identified by the optical code reader,
• c) confirm identity of the item determined by the optical code reader by comparing the extracted visual features of the item to the set of stored visual features obtained from the database;
• a transaction processor configured to execute at least one of a predetermined set of actions based on whether the identity of the item is confirmed.

A method of item checkout for a self checkout system, the system having (1) a data reader section including an optical code reader configured to read an optical code on an item and generate a product identifier of the item and (2) a collection section within which items read by the optical code reader are collected after having been read by the optical code reader, the method comprising the steps of

• by means of the optical code reader, (a) reading the optical code on the item with the optical code reader, and (b) generating a product identifier of the item;
• transferring the item into the collection section;
• by means of at least one camera disposed with a field of view of the collection section, capturing one or more images of the item that has been transferred into the collection section;
• and
• by means of a processor, (a) accessing a database of features and/or images of known objects, (b) extracting a plurality of visual features from the one or more images of the item, (c) identifying matches between the extracted visual features and the features of known objects, (d) generating a geometric transform between the extracted visual features and the features of known objects for a subset of known objects corresponding to the matches, (e) identifying one of the known objects based on a best match of the geometric transform;
• and
• executing one of a predetermined set of actions if the known object that has been identified from the extracted visual features is different than the item corresponding to the product identifier.

A method of item checkout at a checkout system, the checkout system having (1) a data reader section including an optical code reader configured to read an optical code on an item passed through or otherwise present within a read area of the optical code reader and to generate a product identifier of the item and (2) a collection section within which items having been read by the optical code reader are collected, the method comprising the steps of

• via the optical code reader, identifying items by attempting to read the optical code on an item;
• moving the item into the collection section;
• by means of at least one camera disposed with a field of view of the collection section, capturing one or more images of the item moved into the collection section;
• by means of a processor, (a) extracting a plurality of visual features from the one or more images of the item, (b) accessing a database of features and/or images of known objects and obtaining from the database a set of stored visual features corresponding to the item as identified by the optical code reader, (c) confirming identity of the item that has been moved into the collection section by comparing the extracted visual features of the item to the set of stored visual features obtained from the database;
• via a transaction processor, executing at least one of a predetermined set of actions based on whether the identity of the item is confirmed or not.