# US 11,908,122 B2 · System and method for performing production line product identification

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 11,908,122 B2
- Document ID: US11908122B2
- Status: Granted
- Kind: B2
- Application number: 16/939,939
- Filed: 2020-07-27
- Published: 2020-11-12
- Granted: 2024-02-20
- Earliest claimed: 2017-04-26
- Assignees: Sensors, Inc.
- Applicants: Sensors Incorporated
- Inventors: David J. Kotula
- Web page: https://patentgrep.com/patent/US11908122B2

## CPC

- G06T 7/0004: {Industrial image inspection}
- G06F 18/22: Matching criteria, e.g. proximity measures
- G06K 7/10445: {the record carriers being fixed to further objects, e.g. RFIDs fixed to packages, luggage, mail-pieces or work-pieces transported on a conveyor belt}
- G06K 7/10722: {Photodetector array or CCD scanning}
- G06K 7/10861: {sensing of data fields affixed to objects or articles, e.g. coded labels (postal sorting B07C3/14, conveying articles B65G47/48)}
- G06K 7/1413: {1D bar codes}
- G06T 7/0002: {Inspection of images, e.g. flaw detection}
- G06V 10/75: Organisation of the matching processes, e.g. simultaneous or sequential comparisons of image or video features; Coarse-fine approaches, e.g. multi-scale approaches; using context analysis; Selection of dictionaries
- G06V 20/52: Surveillance or monitoring of activities, e.g. for recognising suspicious objects (recognising microscopic objects G06V20/69)
- G06V 20/64: Three-dimensional [3D] objects
- G06V 30/242: Division of the character sequences into groups prior to recognition; Selection of dictionaries
- G06T 2207/10024: Color image
- G06T 2207/30128: Food products
- G06T 2207/30164: Workpiece; Machine component
- G06T 2207/30168: Image quality inspection
- G06V 20/68: Food, e.g. fruit or vegetables
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G06K: GRAPHICAL DATA READING (image or video recognition or understanding G06V); PRESENTATION OF DATA; RECORD CARRIERS; HANDLING RECORD CARRIERS
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING

## Abstract

In an illustrative embodiment, a system for identifying products on a production line includes image capturing devices that acquire images of containers moving along a production line at an inspection location. The system also includes a rejection device and a controller that configures the image capturing devices for image acquisition based on properties of the containers, identifies a product associated with each of the containers based on a portion of a product identification code and a portion of additional features detected in the images, and determines whether the identified product matches predetermined properties or characteristics, resulting in a pass result, otherwise a non-pass result occurs. When a non-pass result occurs, the controller outputs a signal to actuate the rejection device that removes the container from the production line.

## Claims

### Claim 1

A system for processing containers on a production line, the system comprising:
a plurality of image acquisition devices positioned proximate to an inspection location on a production line, each of the plurality of image acquisition devices adapted to capture a plurality of images of a plurality of containers transiting across the production line; and
a control device electronically coupled to the plurality of image acquisition devices, the control device including
a data storage region for storing packaging pattern data associated with a plurality of products processed on the production line,
wherein a portion of the packaging pattern data includes one or more groupings of product pattern data, wherein each of the one or more groupings of product pattern data includes a plurality of product patterns associated with a respective product of the plurality of products, and
one or more processors configured to, for each respective container of the plurality of containers transiting across the production line,
identify, from the plurality of images captured by the plurality of image acquisition devices, one or more images of the plurality of images containing features of a product of the plurality of products portrayed on the respective container,
extract, from the one or more images, the features of the product portrayed on the respective container,
determine, from the extracted features in each respective image of the one or more images, a quality score of the respective image, wherein the quality score indicates a suitability of the respective image for positively identifying the product,
identify, based on analysis of respective extracted features of an image of the one or more images having a highest quality score of the determined quality scores, a respective product grouping of the one or more groupings of product pattern data for the respective container based on a correspondence between the extracted features of the image having the highest quality score and respective pattern data from the respective product grouping,
determine, based on the respective product grouping associated with the respective container, whether a match condition exists between the product associated with the respective container and a predetermined product of the plurality of products, and
cause actuation, responsive to failing to identify the match condition, of a rejection device to remove the respective container from the production line.

### Claim 2 (depends on 1)

The system of claim 1, wherein each of the one or more groupings of product pattern data is associated with an identification code for one of the plurality of products.

### Claim 3 (depends on 1)

The system of claim 1, wherein each of the one or more groupings of product pattern data is associated with a recipe for one of the plurality of products.

### Claim 4 (depends on 1)

The system of claim 1, wherein the quality score for the respective image provides an indication of a quantity of the extracted features of the product detected in the respective image.

### Claim 5 (depends on 1)

The system of claim 1, wherein the extracted features include at least one an identification code, brand logos, product logos, ingredient lists, nutrition information, or container quantity information.

### Claim 6 (depends on 1)

The system of claim 1, wherein a product pattern of the plurality of product patterns associated with a respective pattern grouping of the one or more groupings of product pattern data includes at least one of a brand logo, a product logo, a product picture, nutrition information, an ingredient list, or a background color scheme.

### Claim 7 (depends on 6)

The system of claim 6, wherein a portion of the plurality of product patterns in a respective pattern grouping of the one or more groupings of product pattern data is associated with a holiday pattern scheme.

### Claim 8 (depends on 1)

The system of claim 1, wherein a portion of the plurality of product patterns in a respective pattern grouping of the one or more groupings of product pattern data is associated with one or more differing package shapes.

### Claim 9 (depends on 8)

The system of claim 8, wherein a first product pattern of the plurality of product patterns in the respective pattern grouping is associated with a cylindrical package shape.

### Claim 10 (depends on 9)

The system of claim 9, wherein a second product pattern of the plurality of product patterns in the respective pattern grouping is associated with a rectangular box package shape.

### Claim 11

A method comprising:
capturing, with a plurality of image acquisition devices, a plurality of images of a plurality of containers transiting past an inspection location on a production line;
for each respective container of the plurality of containers transiting past the inspection location on the production line,
identifying, by a controller communicatively coupled to the plurality of image acquisition devices from the plurality of images, one or more images of the plurality of images containing features of a product of a plurality of products portrayed on the respective container;
extracting, by the controller from the one or more images, the features of the product portrayed on the respective container;
determining, by the controller from the extracted features in each of the one or more images, a quality score of the respective image, wherein the quality score indicates a suitability of the respective image for positively identifying the product;
identifying, by the controller based on analysis of respective extracted features of an image of the one or more images having a highest quality score of the determined quality scores, a respective product grouping of one or more groupings of pattern data for the respective container based on a correspondence between the extracted features of the image having the highest quality score and respective pattern data from the respective product grouping,
wherein packaging pattern data for the one or more groupings of pattern data associated with the plurality of products processed on the production line are stored in a data storage region of the controller;
determining, by the controller based on the respective product grouping associated with the respective container, whether a match condition exists between the product associated with the respective container and a predetermined product of the plurality of products, and
causing actuation, by the controller responsive to failing to identify the match condition, of a rejection device to remove the respective container from the production line.

### Claim 12 (depends on 11)

The method of claim 11, wherein each of the one or more groupings of pattern data is associated with an identification code for one of the plurality of products.

### Claim 13 (depends on 11)

The method of claim 11, wherein each of the one or more groupings of pattern data is associated with a recipe for one of the plurality of products.

### Claim 14 (depends on 11)

The method of claim 11, wherein the quality score for the respective image provides an indication of a quantity of the extracted features of the product detected in the respective image.

### Claim 15 (depends on 11)

The method of claim 11, wherein the extracted features include at least one an identification code, brand logos, product logos, ingredient lists, nutrition information, or container quantity information.

### Claim 16 (depends on 11)

The method of claim 11, wherein a product pattern of a plurality of product patterns associated with a respective grouping of the one or more groupings of pattern data includes at least one of a brand logo, a product logo, a product picture, nutrition information, an ingredient list, or a background color scheme.

### Claim 17 (depends on 16)

The method of claim 16, wherein a portion of the plurality of product patterns in a respective grouping of the one or more groupings of pattern data is associated with a holiday pattern scheme.

### Claim 18 (depends on 11)

The method of claim 11, wherein a portion of a plurality of product patterns in a respective pattern grouping of the one or more groupings of pattern data is associated with differing package shapes.

### Claim 19 (depends on 18)

The method of claim 18, wherein a first product pattern of the plurality of product patterns in the respective pattern grouping is associated with a cylindrical package shape.

### Claim 20 (depends on 19)

The method of claim 19, wherein a second product pattern of the plurality of product patterns in the respective pattern grouping is associated with a rectangular box package shape.