# US 8,571,298 B2 · Method and apparatus for identifying and tallying objects > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 8,571,298 B2 - Document ID: US8571298B2 - Status: Granted - Kind: B2 - Application number: 12/636,515 - Filed: 2009-12-11 - Published: 2010-06-24 - Granted: 2013-10-29 - Earliest claimed: 2008-12-23 - Assignees: Datalogic ADC, Inc. - Applicants: Alexander M. McQueen; Craig D. Cherry - Inventors: Alexander M. McQueen; Craig D. Cherry - Web page: https://patentgrep.com/patent/US8571298B2 ## CPC - G06V 20/52: Surveillance or monitoring of activities, e.g. for recognising suspicious objects (recognising microscopic objects G06V20/69) - A47F 9/046: {Arrangement of recording means in or on check-out counters (record carriers or readers per se G06K)} - G06K 7/10861: {sensing of data fields affixed to objects or articles, e.g. coded labels (postal sorting B07C3/14, conveying articles B65G47/48)} - G06Q 10/08: Logistics, e.g. warehousing, loading or distribution; Inventory or stock management - G06Q 10/087: Inventory or stock management, e.g. order filling, procurement or balancing against orders - G06Q 10/0877: {by inventory control or reporting using inventory tracking or counting} - G06Q 30/06: Buying, selling or leasing transactions - G07G 1/0036: {Checkout procedures} - G06V 2201/06: Recognition of objects for industrial automation - G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING - A47F: SPECIAL FURNITURE, FITTINGS, OR ACCESSORIES FOR SHOPS, STOREHOUSES, BARS, RESTAURANTS OR THE LIKE; PAYING COUNTERS - G06K: GRAPHICAL DATA READING (image or video recognition or understanding G06V); PRESENTATION OF DATA; RECORD CARRIERS; HANDLING RECORD CARRIERS - G06Q: INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR - G07G: REGISTERING THE RECEIPT OF CASH, VALUABLES, OR TOKENS ## Abstract An system and method for tallying objects presented for purchase preferably images the objects with a machine vision system while the objects are still, or substantially still. Images of the objects may be used to recognize the objects and to collect information about each object, such as the price. A pre-tally list may be generated and displayed to a customer showing the customer the cost of the recognized objects. A prompt on a customer display may be given to urge a customer to re-orient unrecognized objects to assist the machine vision system with recognizing such unrecognized objects. A tallying event, such as removing a recognized object from the machine vision system's field of view, preferably automatically tallies recognized objects so it is not necessary for a cashier to scan or otherwise input object information into a point of sale system. ## Claims ### Claim 1 A data reading system comprising: a machine vision system including (a) a first field of view defining a pre-transaction zone, (b) a second field of view located adjacent and downstream of the first field of view, the second field of view defining a transaction zone, and (c) an information gathering device for identifying items presented thereto; a computer system operatively connected to the machine vision system for receiving images from the machine vision system, wherein the computer system is programmed to (a) recognize objects appearing in the first field of view; (b) collect information about recognized objects in the first field of view; (c) track recognized objects when recognized objects are moved from the first field of view and into the second field of view; and (d) automatically tally recognized objects that were already previously recognized in the first field of view when the recognized objects are removed from the second field of view without requiring the previously recognized objects to be identified by the information gathering device. ### Claim 2 (depends on 1) A data reading system according to claim 1, further comprising a cashier display operatively connected to the computer system, wherein the computer system is further programmed to (e) indicate via the cashier display which objects in the second field of view have been recognized in the first field of view via the machine vision system. ### Claim 3 (depends on 1) A data reading system according to claim 1 further comprising a cashier display operatively connected to the computer system; wherein the customer display includes a first display screen; and the cashier display includes a second display screen. ### Claim 4 (depends on 3) A data reading system according to claim 3, wherein: the second display screen includes an augmented reality display viewable by a cashier. ### Claim 5 (depends on 4) A data reading system according to claim 4, wherein: the augmented reality display includes a head mounted display worn by a cashier. ### Claim 6 (depends on 4) A data reading system according to claim 4, wherein: the augmented reality display includes a see-through screen interposed between a cashier and recognized objects. ### Claim 7 (depends on 1) A data reading system according to claim 1, wherein: the machine vision system includes a first camera and a second camera; and the first field of view corresponds to the first camera and the second field of view corresponds to the second camera. ### Claim 8 (depends on 7) A data reading system according to claim 7, wherein the first and second cameras include CMOS imagers with rolling reset shutters. ### Claim 9 (depends on 7) A data reading system according to claim 7, wherein the first and second cameras include CMOS imagers with global reset shutters. ### Claim 10 (depends on 1) A data reading system according to claim 1 further comprising a customer display operatively connected to the computer system, wherein the computer system is further programmed to (e) display a pre-tally representation of recognized objects via the customer display. ### Claim 11 A data reading system comprising a machine vision system including (a) a first field of view defining a pre-transaction zone and (b) a second field of view located adjacent and downstream of the first field of view, the second field of view defining a transaction zone; a computer system operatively connected to the machine vision system for receiving images from the machine vision system; and a customer display operatively connected to the computer system, wherein the computer system is programmed to (a) recognize objects appearing in the first field of view, (b) display a pre-tally representation of recognized objects via the customer display, (c) collect information about recognized objects in the first field of view, (d) track recognized objects when recognized objects are moved, (e) automatically tally recognized objects when recognized objects are removed from the second field of view, (f) via the customer display, prompt for re-positioning objects that appear in the first field of view, but are not recognized; and (g) remove objects from the pre-tally list when recognized objects are removed from the pre-transaction zone, but do not enter the transaction zone. ### Claim 12 A data reading system comprising a machine vision system including (a) a first field of view defining a pre-transaction zone and (b) a second field of view located adjacent and downstream of the first field of view, the second field of view defining a transaction zone; a computer system operatively connected to the machine vision system for receiving images from the machine vision system; and a customer display operatively connected to the computer system, wherein the computer system is programmed to (a) recognize objects appearing in the first field of view, (b) display a pre-tally representation of recognized objects via the customer display, (c) collect information about recognized objects in the first field of view, (d) track recognized objects when recognized objects are moved, (e) automatically tally recognized objects when recognized objects are removed from the second field of view, wherein the customer display includes a plurality of colored annotations for indicating recognized objects in the pre-transaction zone and to prompt for re-positioning unrecognized objects. ### Claim 13 (depends on 12) A data reading system according to claim 12, wherein: the plurality of colored annotations included in the customer display includes light beams of a first color for indicating recognized objects and light beams of a second color to prompt for re-positioning objects. ### Claim 14 (depends on 13) A data reading system according to claim 13, further comprising lasers to generate the colored light beams. ### Claim 15 (depends on 12) A data reading system according to claim 12, further comprising a cashier display operatively connected to the computer system, wherein the cashier display includes a plurality of colored annotations for indicating recognized objects in the transaction zone. ### Claim 16 A data reading system comprising: a machine vision system including a field of view coincident with a substantially flat surface; a computer system operatively connected to the machine vision system for receiving images from the machine vision system; and an optical code reader for identifying objects presented thereto; wherein the computer system is programmed to (a) recognize an object appearing in the field of view using the images received from the machine vision system; (b) collect information about recognized objects in the field of view; (c) track recognized objects when recognized objects are moved; and (d) automatically tally recognized objects that were already recognized by the computer system using the images received from the machine vision system, without identifying the objects with the optical code reader, when the recognized objects are removed from the field of view. ### Claim 17 (depends on 16) A data reading system according to claim 16, further comprising a customer display operatively connected to the computer system, wherein the computer system is further programmed to (e) display a prompt via the customer display for re-positioning objects that appear in the field of view, but are not recognized. ### Claim 18 (depends on 17) A data reading system according to claim 17, further comprising a cashier display operatively connected to the computer system; wherein the computer system is further programmed to (f) indicate via the cashier display which objects in the field of view are recognized. ### Claim 19 (depends on 16) A data reading system according to claim 16, wherein the computer system is further programmed to (e) automatically tally recognized objects when recognized objects are removed from the field of view in a first direction. ### Claim 20 (depends on 19) A data reading system according to claim 19, wherein the computer system is further programmed to (f) display a pre-tally representation of recognized objects via the customer display; and (g) remove objects from the pre-tally list when objects are removed from the field of view in a second direction. ### Claim 21 (depends on 16) A data reading system according to claim 16, further comprising a cashier display operatively connected to the computer system; wherein the customer display includes a first display screen; and the cashier display includes a second display screen. ### Claim 22 (depends on 21) A data reading system according to claim 21, wherein: the second display screen includes an augmented reality display viewable by a cashier. ### Claim 23 (depends on 22) A data reading system according to claim 22, wherein: the augmented reality display includes a head mounted display worn by a cashier. ### Claim 24 (depends on 22) A data reading system according to claim 22, wherein: the augmented reality display includes a see-through screen interposed between a cashier and recognized objects. ### Claim 25 (depends on 16) A data reading system according to claim 16, wherein: the machine vision system includes a camera having a CMOS imager with a rolling reset shutter. ### Claim 26 (depends on 16) A data reading system according to claim 16, wherein: the machine vision system includes a camera having a CMOS imager with a global reset shutter. ### Claim 27 A data reading system comprising a machine vision system including a field of view coincident with a substantially flat surface; a computer system operatively connected to the machine vision system for receiving images from the machine vision system; and a customer display operatively connected to the computer system, wherein the computer system is programmed to (a) recognize objects appearing in the field of view; (b) collect information about recognized objects in the field of view; (c) track recognized objects when recognized objects are moved; and (d) automatically tally recognized objects when recognized objects are removed from the field of view; a cashier display operatively connected to the computer system; wherein the customer display includes a plurality of colored annotations for indicating recognized objects in the field of view and to prompt for re-orienting unrecognized objects; and wherein the cashier display includes a plurality of colored annotations for indicating recognized objects in the field of view. ### Claim 28 (depends on 27) A data reading system according to claim 27, wherein: the plurality of colored annotations included in the customer display includes light beams of one color for indicating recognized objects and light beams of a second color to prompt for re-positioning unrecognized objects. ### Claim 29 A method for tallying objects comprising: recognizing objects appearing in a field of view using a machine vision system; via a customer display, prompting a customer to re-position objects appearing in the field of view that are not recognized using the machine vision system; recognizing re-positioned objects appearing in the field of view using the machine vision system; indicating recognized objects in the field of view via the customer display; tracking recognized objects in the field of view using the machine vision system when recognized objects are moved through or out of the field of view; identifying objects not recognized by the machine vision system using an optical code reader; and automatically tallying recognized objects using a computer system operatively connected to the machine vision system when recognized objects are removed from the field of view without requiring the previously recognized objects to be identified by the optical code reader. ### Claim 30 (depends on 29) A method for tallying objects according to claim 29, further comprising: displaying via the customer display a pre-tally list of recognized objects. ### Claim 31 (depends on 29) A method for tallying objects according to claim 29, wherein automatically tallying recognized objects occurs when recognized objects are removed from the field of view in a first direction. ### Claim 32 (depends on 31) A method for tallying objects according to claim 31, further comprising: displaying via the customer display a pre-tally list of recognized objects; and removing objects from the pre-tally list when objects are removed from the field of view in a second direction. ### Claim 33 (depends on 29) A method for tallying objects according to claim 29, and further comprising: indicating recognized objects in the second field of view via a cashier display. ### Claim 34 (depends on 33) A method for tallying objects according to claim 33, further comprising: displaying via the customer display a pre-tally list of recognized objects appearing in the first field of view; and removing objects from the pre-tally list when objects are removed from the first field of view, but do not appear in the second field of view. ### Claim 35 (depends on 33) A method for tallying objects according to claim 33, further comprising: recognizing objects that were not recognized in the first field of view that appear in the second field of view using the machine vision system. ### Claim 36 A method for tallying objects comprising: moving objects along an item path through a field of view of a machine vision system, the field of view including a first view portion; via the machine vision system: recognizing objects in the first view portion, tracking both recognized and non-recognized objects being moved through the first view portion; passing non-recognized objects through a read zone of an optical code reader, the read zone being located downstream of the first view portion, and upon successfully identifying the non-recognized objects using the optical code reader: (a) signaling successful read and (b) tallying the previously non-recognized objects having been identified by the optical code reader; passing recognized objects past or through the read zone of the optical code reader and treating recognized objects moved past or through the read zone of the optical code reader as if identified by the optical code reader by (a) signaling successful read and (b) tallying the recognized objects without requiring the previously recognized objects to be identified by the optical code reader. ### Claim 37 (depends on 36) A method according to claim 36 further comprising employing double-read prevention at the optical code reader in the event a recognized object being passed through the read zone is read by the optical code reader. ### Claim 38 (depends on 36) A method according to claim 36 wherein the step of signaling successful read of the recognized object occurs as it leaves the first view portion and is moved toward the read zone of the optical code reader. ### Claim 39 (depends on 38) A data reading system according to claim 38 wherein the machine vision system comprises at least one camera (1) being positioned above the conveyor, (2) having an overhead view of the conveyor, and (3) providing the images to the computer system. ### Claim 40 A data reading system comprising: a machine vision system including a field of view; a conveyor for conveying objects through at least a first view portion of the field of view; a computer system operatively connected to the machine vision system for receiving images from the machine vision system; an optical code reader having a read zone positioned on a downstream end of the first view portion, the optical code reader being operative for reading optical codes on objects and thereby identifying objects passed through the read zone, wherein the computer system is programmed to (a) recognize objects appearing in the first view portion using the images received from the machine vision system; (b) track objects being conveyed through and out of the first view portion; and (c) automatically tally recognized objects that were already recognized by the computer system using the images received from the machine vision system, without requiring the recognized objects to be identified with the optical code reader, when the recognized objects are removed from the first view portion and passed by or into the read zone.