# US 2018/0025314 A1 · Capturing Product Details of Purchases

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2018/0025314 A1
- Document ID: US20180025314A1
- Status: Published application
- Kind: A1
- Application number: 15/700,677
- Filed: 2017-09-11
- Published: 2018-01-25
- Earliest claimed: 2015-06-17
- Assignees: MetaBrite, Inc.
- Applicants: Metabrite, Inc.
- Inventors: Court V. Lorenzini; Samuel Anthony Lucente; Roy Penn; Morgan Doocy; Shawn Rutledge
- Web page: https://patentgrep.com/patent/US20180025314A1

## CPC

- G06Q 10/087: Inventory or stock management, e.g. order filling, procurement or balancing against orders
- G06F 18/22: Matching criteria, e.g. proximity measures
- G06Q 10/08774: {using consumer inventory reporting}
- G06V 20/63: {Scene text, e.g. street names}
- G06V 30/224: of printed characters having additional code marks or containing code marks
- G06V 30/416: Extracting the logical structure, e.g. chapters, sections or page numbers; Identifying elements of the document, e.g. authors
- G06V 30/10: Character recognition
- G06Q: INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING

## Abstract

Systems, methods and computer-readable media are disclosed for capturing purchase information regarding purchased items of a consumer. Upon receiving an image of a receipt (a receipt image) regarding a list of purchased items, receipt text is generated. The receipt text is processed to identify the purchased items in the receipt image. Accordingly, an iteration is begun to iterate through the item blocks of the receipt text. An item block corresponds to a discrete item in the receipt text. The processing comprises extracting textual elements from the item block and matching the textual elements to a known product. Upon matching the textual elements to a known product, the consumer inventory associated with the consumer is updated with regard to the purchase of the known product.

## Claims

### Claim 1

A computer implemented method for capturing information of purchased items to a consumer inventory of a consumer, the method comprising each of the following as implemented on a computing device:
receiving information describing a first set of purchased items from a consumer and updating a consumer inventory corresponding to the consumer with the received information, and for each purchased item of the first set of purchased items, indicating that the item is a non-specific item in the consumer inventory corresponding to the consumer;
receiving a receipt image from the consumer of a purchase receipt regarding at least some of the first set of purchased items;
identifying a first item block from the receipt image;
matching subject matter of the first item block to a product item of a set of known product items;
updating a consumer inventory of the consumer with regard to the purchase of the product item.

### Claim 2 (depends on 1)

The method of claim 1, wherein the method further comprises repeatedly identifying a next item block of the receipt text and processing the next item block until at least a subset of the item blocks of the receipt text are processed, wherein processing a next item block comprises:
matching subject matter of the item block to a product item of a set of known products; and
updating a consumer inventory of the consumer with regard to the purchase of the matched product item.

### Claim 3 (depends on 2)

The computer-implemented method of claim 2 further comprising:
receiving a purchase list of a second set of purchased items, wherein at least one purchased item of the second set of purchased items is in the first set of purchased items;
for each purchased item in the second set of purchased items, adding the purchased item of the second set to the consumer inventory of the consumer.

### Claim 4 (depends on 3)

The computer-implemented method of claim 3, wherein adding the purchased item of the second set to the consumer inventory of the consumer comprises adding the purchased item of the second set to the consumer inventory of the consumer as a non-specific item of inventory.

### Claim 5 (depends on 4)

The computer-implemented method of claim 4, wherein updating the consumer inventory of the consumer with regard to the purchase of the product item comprises:
matching the product item to an item of inventory already added to the consumer inventory of the consumer as a non-specific item of inventory;
updating the item of inventory already added to the consumer inventory of the consumer as a non-specific item of inventory with information regarding the product item; and
converting the item of inventory already added to the consumer inventory of the consumer as a non-specific item of inventory as a specific item of inventory.

### Claim 6 (depends on 2)

The computer-implemented method of claim 2, wherein processing the next item block further comprises determining whether the next item block corresponds to a purchased item, and only upon determining that the next item block corresponds to a purchased item:
matching the subject matter of the next item block to a next product item of the set of known product items;
updating a consumer inventory of the consumer with regard to the purchase of the next product item.

### Claim 7 (depends on 6)

The computer-implemented method of claim 6 further comprising:
identifying a store from the receipt image from which the first set of purchased items was purchased; and
selecting a template corresponding to the identified store, wherein the selected template comprises information for matching subject matter of an item block corresponding to an item purchased at the identified store to a known product.

### Claim 8 (depends on 7)

The computer-implemented method of claim 7, wherein processing the next item block comprises further comprises:
determining a confidence value of a match between the subject matter of the next item block and the next product item; and
upon determining that the determined confidence value meets or exceeds a predetermined threshold, updating the consumer inventory of the consumer with regard to the purchase of the next product item.

### Claim 9 (depends on 8)

The computer-implemented method of claim 8, wherein processing the next item block comprises further comprises:
upon determining that the determined confidence value does not meet or exceed a predetermined threshold:
generating a set of potential matches for the next item block, the set of potential matches comprising a plurality of potential matches whose determined confidence value with regard to a known product item does not meet or exceed a predetermined threshold;
generating a consumer confirmation request requesting that the consumer select a match of the set of potential matches with regard to a corresponding known product item; and
transmitting the consumer confirmation request to the consumer for selection of a match of the set of potential matches with regard to a corresponding known product item.

### Claim 10 (depends on 9)

The computer-implemented method of claim 9, wherein processing the next item block comprises further comprises:
receiving a response from the consumer in regard to the consumer confirmation quest;
matching the subject matter of the next item block to a known product item according to the information in the received response from the consumer; and
updating a consumer inventory of the consumer with regard to the purchase of the known product item.

### Claim 11

(canceled)

### Claim 12

(canceled)

### Claim 13

A computer readable medium bearing computer executable instructions which, when executed on a computing system comprising at least a processor and a memory, carry out a method capturing information of purchased items to a consumer inventory of a consumer, the method comprising:
receiving a purchase list of a first set of purchased items;
for each purchased item in the first set of purchased items, adding the purchased item of the first set to the consumer inventory of the consumer, wherein the purchased item of the first set is added to the consumer inventory of the consumer as a non-specific item of inventory;
receiving a receipt image from a consumer of a purchase receipt of a second set of purchased items, wherein at least one purchased item of the second set of purchased items is in the first set of purchased items; and
repeatedly identifying a next item block of the receipt image and processing the next item block until at least some of the item blocks of the receipt image are processed, wherein processing the next item block comprises:
matching subject matter of the next item block to a known product item;
matching the known product item to an item of inventory already added to the consumer inventory of the consumer as a non-specific item of inventory;
updating the item of inventory already added to the consumer inventory of the consumer as a non-specific item of inventory with information regarding the known product item; and
converting the item of inventory already added to the consumer inventory of the consumer as a non-specific item of inventory as a specific item of inventory.

### Claim 14 (depends on 13)

The computer readable medium of claim 13, wherein processing the next item block further comprises determining whether the next item block corresponds to a purchased item, and only upon determining that the next item block corresponds to a purchased item:
matching subject matter of the next item block to a next known product item;
matching the next known product item to an item of inventory already added to the consumer inventory of the consumer as a non-specific item of inventory;
updating the item of inventory already added to the consumer inventory of the consumer as a non-specific item of inventory with information regarding the next known product item; and
converting the item of inventory already added to the consumer inventory of the consumer as a non-specific item of inventory as a specific item of inventory.

### Claim 15 (depends on 14)

The computer readable medium of claim 14 further comprising:
identifying a store from the receipt image from which the first set of purchased items was purchased; and
selecting a template corresponding to the identified store, wherein the selected template comprises information for matching subject matter of an item block corresponding to an item purchased at the identified store to a known product item; and
wherein matching subject matter to a known product item comprises matching the subject matter to a known product item according to the selected template.

### Claim 16 (depends on 15)

The computer readable medium of claim 15, wherein processing the next item block comprises further comprises:
determining a confidence value of a match between the next item block and the next known product item; and
upon determining that the determined confidence value meets or exceeds a predetermined threshold, updating the consumer inventory of the consumer with regard to the purchase of the next known product item.

### Claim 17 (depends on 16)

The computer readable medium of claim 16, wherein processing the next item block comprises further comprises:
upon determining that the determined confidence value does not meet or exceed a predetermined threshold:
generating a set of potential matches for the next item block, the set of potential matches comprising a plurality of potential matches whose determined confidence value with regard to a known product item does not meet or exceed a predetermined threshold;
generating a consumer confirmation request requesting that the consumer select a match of the set of potential matches with regard to a corresponding known product; and
transmitting the consumer confirmation request to the consumer for selection of a match of the set of potential matches with regard to a corresponding known product item.

### Claim 18 (depends on 17)

The computer readable medium of claim 17, wherein processing the next item block comprises further comprises:
receiving a response from the consumer in regard to the consumer confirmation quest;
matching the subject matter of the next item block to a known product item according to the information in the received response from the consumer; and
updating a consumer inventory of the consumer with regard to the purchase of the known product item.

### Claim 19

A computing system for capturing information of purchased items to a consumer inventory of a consumer, the system comprising a processor and a memory, wherein the processor executes instructions stored in the memory as part of or in conjunction with additional components of computing system, the additional components comprising:
a network communication component for communicating with one or more computing devices over a communication network;
a consumer interface for interacting with a consumer via the network communication component, and configured to receive at least a first list of purchased items, a receipt image regarding a second list of purchased items;
a non-specific item processor configured to process the purchased items of the first list of purchased items and update a consumer inventory associated with the consumer with the purchased items of the first list of purchased items, wherein the purchased items of the first list of purchased items are updated in the consumer inventory associated with the consumer as non-specific items; and
a receipt processor configured to process the receipt image regarding a second list of purchased items, wherein at least one purchased item of the first list of purchased items is in the second list of purchased items, and wherein processing the receipt image regarding the second list of purchased items, the receipt processor is configured to:
repeatedly identify a next item block of the receipt image and process each next item block until all item blocks of the receipt image are processed, and in processing a next item block, the receipt processor is further configured to:
match subject matter of the next item block to a known product;
match the known product to an item of inventory already added to the consumer inventory of the consumer as a non-specific item of inventory;
update the item of inventory already added to the consumer inventory of the consumer as a non-specific item of inventory with information regarding the known product; and
convert the item of inventory already added to the consumer inventory of the consumer as a non-specific item of inventory as a specific item of inventory.

### Claim 20

(canceled)

### Claim 21 (depends on 20)

The computing system of claim 20, wherein the receipt processor is further configured to process the receipt image via one or more image filters to clarify the receipt image.