# US 12,002,245 B2 · Method and system for item identification

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 12,002,245 B2
- Document ID: US12002245B2
- Status: Granted
- Kind: B2
- Application number: 17/323,943
- Filed: 2021-05-18
- Published: 2021-09-02
- Granted: 2024-06-04
- Earliest claimed: 2019-10-25
- Assignees: Mashgin Inc.
- Applicants: Mashgin Inc.
- Inventors: Abhinai Srivastava; Mukul Dhankhar
- Web page: https://patentgrep.com/patent/US12002245B2

## CPC

- G06T 9/002: {using neural networks}
- G06F 16/51: Indexing; Data structures therefor; Storage structures
- G06F 16/55: Clustering; Classification
- G06F 18/214: Generating training patterns; Bootstrap methods, e.g. bagging or boosting
- G06F 18/22: Matching criteria, e.g. proximity measures
- G06F 18/24: Classification techniques
- G06F 18/253: {of extracted features}
- G06N 3/045: Combinations of networks
- G06N 3/0464: Convolutional networks [CNN, ConvNet]
- G06N 3/08: Learning methods
- G06N 3/09: Supervised learning
- G06N 20/00: Machine learning
- G06Q 20/201: {Price look-up processing, e.g. updating}
- G06Q 20/202: {Interconnection or interaction of plural electronic cash registers [ECR] or to host computer, e.g. network details, transfer of information from host to ECR or from ECR to ECR}
- G06Q 20/203: {Inventory monitoring}
- G06Q 20/208: {Input by product or record sensing, e.g. weighing or scanner processing}
- G06V 10/764: using classification, e.g. of video objects
- G06V 10/806: {of extracted features}
- G06V 10/82: using neural networks
- G06V 20/10: Terrestrial scenes (scenes under surveillance with static cameras G06V20/52; scenes perceived from the exterior of a vehicle G06V20/56; scenes perceived from the interior of a vehicle G06V20/59)
- G07G 1/0063: {with means for detecting the geometric dimensions of the article of which the code is read, such as its size or height, for the verification of the registration}
- G06N 3/04: Architecture, e.g. interconnection topology
- G06N 3/044: Recurrent networks, e.g. Hopfield networks
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06Q: INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G07G: REGISTERING THE RECEIPT OF CASH, VALUABLES, OR TOKENS

## Abstract

The method for item identification preferably includes determining visual information for an item; calculating a first encoding using the visual information; calculating a second encoding using the first encoding; determining an item identifier for the item using the second encoding; optionally presenting information associated with the item to a user; and optionally registering a new item.

## Claims

### Claim 1

A system for item recognition, comprising:
a storage device; and
a processing system coupled to the storage device, the storage device storing software instructions that, when executed, cause the processing system to perform a method, comprising:
capturing an image of an item;
calculating an item encoding using a model trained to determine item identifiers from the image; and
determining an item identifier for the item based on a similarity between the item encoding and a set of predetermined encodings for each of a set of items.

### Claim 2 (depends on 1)

The system of claim 1, wherein the item encoding is output by an intermediate layer of the model.

### Claim 3 (depends on 1)

The system of claim 1, wherein the model comprises an item classifier and a combination classifier, wherein calculating the item encoding comprises:
determining a first encoding from an intermediate layer of the item classifier based on the image; and
determining a second encoding from an intermediate layer of the combination classifier based on the first encoding.

### Claim 4 (depends on 1)

The system of claim 1, wherein the set of predetermined encodings comprise item encodings for each of the set of items that were extracted by the model.

### Claim 5 (depends on 4)

The system of claim 4, wherein an updated set of predetermined encodings is determined each time the model is retrained at a central system, wherein a difference between a prior set of predetermined encodings and the updated set of predetermined encodings is transmitted to each local system.

### Claim 6 (depends on 5)

The system of claim 5, wherein the central system comprises a cloud computing system, remote from each local system.

### Claim 7 (depends on 5)

The system of claim 5, wherein the image is not transmitted from the central system to the local system.

### Claim 8 (depends on 1)

The system of claim 1, wherein the method is performed by a local system communicatively connected to a remote system, wherein the remote system stores:
a database of items and associated images thereof;
a set of entities and items associated therewith; and
a set of models and item encodings extracted by each model for each entity, wherein each model is trained to identify the items associated with the respective entity.

### Claim 9 (depends on 1)

The system of claim 1, wherein the system further comprises a camera configured to capture the image from at least one of: a right, left, top, front, or back side of a measurement volume.

### Claim 10 (depends on 1)

The system of claim 1, wherein the method further comprises facilitating item purchase based on the item identifier.

### Claim 11 (depends on 1)

The system of claim 1, wherein the method further comprises:
receiving a set of images and a new item identifier for a new item;
extracting a new item encoding from the set of images using the model; and
storing the new item encoding with the new item identifier, wherein the new item encoding is subsequently used to identify the new item.

### Claim 12 (depends on 11)

The system of claim 11, wherein the method is performed by a local system, wherein the method further comprises sending the set of images and the new item identifier for the new item to a central system communicatively connected to a plurality of local systems.

### Claim 13

A method for item recognition, comprising:
capturing an image of an item;
calculating an item encoding using a model trained to determine item identifiers from images; and
determining an item identifier for the item based on a similarity between the item encoding and a set of predetermined encodings for known items.

### Claim 14 (depends on 13)

The method of claim 13, further comprising, determining an updated set of predetermined encodings at a central system, wherein a difference between a prior set of predetermined encodings and the updated set of predetermined encodings is transmitted to a local system.

### Claim 15 (depends on 13)

The method of claim 13, wherein the method is performed by a local system, wherein the model is trained by a central system on images of the item and sent to the local system.

### Claim 16 (depends on 15)

The method of claim 15, wherein the central system does not send the images to the local system.

### Claim 17 (depends on 15)

The method of claim 15, wherein the model is retrained when new images of new items are received at the central system.

### Claim 18 (depends on 13)

The method of claim 13, wherein the item encoding is output by an intermediate layer of the model.

### Claim 19 (depends on 13)

The method of claim 13, further comprising determining a price for the item using the item identifier; and requesting payment from a user using the price.

### Claim 20 (depends on 13)

The method of claim 13, wherein the method is repeated at each local system of a plurality of local systems.