# US 2021/0342807 A1 · System and methods for automatic detection of product insertions and product extraction in an open shopping cart

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2021/0342807 A1
- Document ID: US20210342807A1
- Status: Published application
- Kind: A1
- Application number: 17/376,420
- Filed: 2021-07-15
- Published: 2021-11-04
- Earliest claimed: 2019-01-16
- Assignees: Tracxone Ltd.
- Applicants: Tracxone Ltd.
- Inventors: Moshe MEIDAR; Gidon MOSHKOVITZ; Edi BAHOUS; Itai WINKLER
- Web page: https://patentgrep.com/patent/US20210342807A1

## CPC

- G06Q 30/04: Billing or invoicing
- G06Q 20/12: specially adapted for electronic shopping systems
- G06Q 20/203: {Inventory monitoring}
- G06Q 20/208: {Input by product or record sensing, e.g. weighing or scanner processing}
- G06Q 30/0282: Rating or review of business operators or products
- G06Q 30/06: Buying, selling or leasing transactions
- G06Q: INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR

## Abstract

The disclosure relates to systems and methods for automatic detection of product insertion and product extraction in an Artificial Intelligent Cart (AIC). Specifically, the disclosure relates to systems and methods of ascertaining insertion and extraction of product into and from an open shopping cart, by continuously monitoring triggered content changes.

## Claims

### Claim 1

An open shopping cart comprising:
a. a cart body, the cart body having a floor, and walls rising from the floor, forming an apically open container;
b. an inward-looking imaging module, adapted and configured to detect a first predetermined set of triggers associated with at least one of: a product insertion, and a product extraction;
c. optionally an outward-looking imaging module adapted and configured to detect a second predetermined set of triggers associated with at least one of: a product insertion, and a product extraction, wherein the second set of predetermined triggers is different than the first set of predetermined triggers; and
d. a central processing module (CPM), the CPM being in communication with the inward looking imaging module, and optionally the outward-looking imaging module.

### Claim 2 (depends on 1)

The open shopping cart of claim 1, wherein the CPM is further in communication with at least one of: a user interface module, a graphics processing unit (GPU), a data acquisition module, a product recognition module, a data fusion module, and a decision module.

### Claim 3 (depends on 2)

The open shopping cart of claim 2, wherein at least one of the user interface module, the GPU, the data fusion module, the data acquisition module, the product recognition module, and the decision module, is co-located with the CPM.

### Claim 4 (depends on 2)

The open shopping cart of claim 2, wherein at least one of the user interface module, the GPU, the data fusion module, the data acquisition module, the product recognition module, and the decision module, is located remotely to the CPM.

### Claim 5 (depends on 2)

The open shopping cart of claim 2, wherein at least one of: the inward-looking imaging module, and optionally the outward-looking imaging module comprise at least one of at least one RGB camera, at least one Infrared (IR) camera, at least one RGBD camera, and at least one depth camera.

### Claim 6 (depends on 5)

The open shopping cart of claim 5, wherein the CPM is configured to at least perform one of:
a. identify events of product insertions into the open shopping cart;
b. identify events of product extraction out from the open shopping cart;
c. provide at least one trigger associated with at least one of product insertion and product extraction to the data acquisition module and the product recognition module;
d. recognize at least one of an insertion location of the inserted product, its weight, its shape, and a heat signature parameter;
e. filter false events of product insertions and/or product extraction; and
f. detect missed events.

### Claim 7 (depends on 6)

The open shopping cart of claim 6, wherein the inward-looking imaging module comprises a plurality of cameras, positioned and configured to have an overlapping field of view, whereby an image captured by all the plurality of cameras, in combination with a signal provided by a load cell included with the open shopping cart, provides at least one of: the location of the inserted product within the shopping cart, the weight of the inserted product, the heat signature characteristic of the product, and the shape of the product.

### Claim 8 (depends on 7)

The open shopping cart of claim 7, wherein the shopping cart further comprises at least one of: a light meter, an accelerometer, an ultrasound detector, a RF transmitter/receiver, an infrared scanner, a barcode reader, a laser scanner, a camera based reader, a CCD reader, a LED scanner, a Bluetooth beacon, a near field communication module, and a wireless transceiver.

### Claim 9 (depends on 8)

The open shopping cart of claim 8, wherein at least one of: the insertion trigger and the extraction trigger is obtained by comparing a first image captured by the inward looking imaging module, to a second image captured by the inward looking imaging module.

### Claim 10 (depends on 9)

The open shopping cart of claim 9, wherein the second image is captured following at least one of: an image captured by the outward looking imaging module, and optionally weight change indicated by the load cell.

### Claim 11 (depends on 10)

The open shopping cart of claim 10, wherein the outward-looking imaging module is included with the system and is positioned and configured to capture an image, or an image sequence, of at least one of: a hand gesture of a cart user, a cart user removing the product from a store shelf, the store shelf, a user holding a product, a motion of a product on a shelf, and a motion of a product across the open shopping cart walls.

### Claim 12

A computerized method of detecting insertion and/or extraction of a product from an open shopping cart implementable in a system comprising a cart body, the cart body having a floor, and walls rising from the floor, forming an apically open container; an inward-looking imaging module, adapted and configured to detect a first predetermined set of triggers associated with at least one of a product insertion or a product extraction; optionally an outward-looking imaging module adapted and configured to detect a second predetermined set of triggers associated with at least one of a product insertion or a product extraction, wherein the second set of predetermined triggers is different than the first set of predetermined triggers; and a central processing module (CPM), the CPM being in communication with the inward-looking imaging module, and the outward-looking imaging module; the method comprising:
a. capturing a first image of the open container using the inward-looking imaging module;
b. in response to a predetermined triggering event, capturing a second image of the open container using the inward-looking imaging module; and
c. using the central processing module, comparing the first image to the second image, wherein if the second image is different than the first image, provide an indication of a product insertion or a product extraction.

### Claim 13 (depends on 12)

The method of claim 12, wherein the predetermined trigger is an increase, or decrease in weight provided by a load cell provided with the system and the indication is of product insertion or product extraction respectively.

### Claim 14 (depends on 13)

The method of claim 13, wherein the trigger further comprises capturing an image by the outward looking imaging module included with the system.

### Claim 15 (depends on 15)

The method of claim 15, wherein the image captured by the outward looking imaging module is of at least one of: a hand gesture of a cart user, a cart user removing the product from a store shelf, the store shelf, a cart user holding a product, a motion of a product on a shelf, and a motion of a product across the open shopping cart walls.

### Claim 16 (depends on 16)

The method of claim 16, wherein at least one of: the inward-looking imaging module, and the outward-looking imaging module comprise at least one of at least one RGB camera, at least one Infrared (IR) camera, at least one RGBD camera, and at least one depth camera.

### Claim 17 (depends on 17)

The method of claim 17, wherein the CPM is further in communication with at least one of: a user interface module, a graphics processing unit (GPU), a data acquisition module, a product recognition module, a data fusion module, and a decision module.

### Claim 18 (depends on 18)

The method of claim 18, further comprising at least one of:
a. providing at least one trigger associated with at least one of product insertion and product extraction to the data acquisition module and the product recognition module,
b. recognizing at least one of an insertion location of the inserted product, its weight and its shape,
c. filtering false events of product insertions and/or product extraction; and
d. detecting missed events.

### Claim 19 (depends on 19)

The method of claim 19, wherein the inward-looking imaging module comprises a plurality of cameras, positioned and configured to have an overlapping field of view, whereby the image captured by all of the plurality of cameras, optionally in combination with a signal provided by the load cell provides at least one of: the location of the inserted product within the shopping cart, optionally the weight of the inserted product, the shape of the product, and the heat signature characteristic of the product.

### Claim 20 (depends on 16)

The method of claim of claim 16, wherein the step of capturing the second image by the outward-looking imaging module further comprises capturing a plurality of images over a predetermined time.

### Claim 21 (depends on 21)

The method of claim 21, further comprising the step of: using the CPM, calculating an optical flow of a parameter and determining the flow direction of the parameter.

### Claim 22 (depends on 22)

The method of claim 22, wherein the parameter is at least one of: the hand gesture of the cart user, and the product.

### Claim 23 (depends on 22)

The method of claim 22, wherein the step of calculating the optical flow of the parameter, comprises at least one of: a gradient-based approach, a frequency-based approach, and a correlation-based approach.
