# US 2021/0334523 A1 · Shopper analysis using an acceleration sensor and imaging

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2021/0334523 A1
- Document ID: US20210334523A1
- Status: Published application
- Kind: A1
- Application number: 16/856,345
- Filed: 2020-04-23
- Published: 2021-10-28
- Earliest claimed: 2020-04-23
- Assignees: International Business Machines Corporation
- Applicants: International Business Machines Corporation
- Inventors: HIROKI SHIBAYAMA; TAKETO MATSUNAGA; KEIICHIRO SHIMADA; HAOXIANG QIU
- Web page: https://patentgrep.com/patent/US20210334523A1

## CPC

- G06V 10/764: using classification, e.g. of video objects
- G06N 20/00: Machine learning
- G06V 20/52: Surveillance or monitoring of activities, e.g. for recognising suspicious objects (recognising microscopic objects G06V20/69)
- G06V 40/10: Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
- G06V 40/173: {face re-identification, e.g. recognising unknown faces across different face tracks}
- G06V 40/23: {Recognition of whole body movements, e.g. for sport training}
- G06V 40/28: {Recognition of hand or arm movements, e.g. recognition of deaf sign language (static hand signs G06V40/113)}
- G06Q 30/0623: {by investigating goods or services}
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06Q: INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR

## Abstract

A method includes receiving, by a computing device, acceleration information from an acceleration sensor attached to a product; receiving, by the computing device, from an image capture device an image of a prospective purchaser of the product; determining, by the computing device, a movement of the prospective purchaser based on the acceleration information and the image, the movement being in relation to a movement of the product; and sending, by the computing device, display information to a digital display, the display information including information related to the product. The movement of the product is based on the acceleration information.

## Claims

### Claim 1

A method, comprising:
receiving, by a computing device, acceleration information from an acceleration sensor attached to a product;
receiving, by the computing device and from an image capture device, an image of a prospective purchaser of the product;
determining, by the computing device, a movement of the prospective purchaser based on the acceleration information and the image, the movement being in relation to a movement of the product; and
sending, by the computing device, display information to a digital display, the display information including information related to the product,
wherein, the movement of the product is based on the acceleration information.

### Claim 2 (depends on 1)

The method of claim 1, wherein the movement of the prospective purchaser is the prospective purchaser lifting the product.

### Claim 3 (depends on 1)

The method of claim 1, wherein the movement of the prospective purchaser is the prospective purchaser comparing the product to a second product.

### Claim 4 (depends on 1)

The method of claim 1, wherein the digital display is located proximate to the product and the prospective purchaser.

### Claim 5 (depends on 1)

The method of claim 1, wherein the determining a movement of the prospective purchaser comprises utilizing a machine learning model that focuses on skeletal structures of the prospective purchaser based on the image.

### Claim 6 (depends on 5)

The method of claim 5, wherein the skeletal structures of the prospective purchaser indicate a posture of the prospective purchaser.

### Claim 7 (depends on 1)

The method of claim 1, further comprising
receiving, by the computing device, second acceleration information from a second acceleration sensor attached to a second product;
receiving, by the computing device, from the image capture device a second image of the prospective purchaser;
determining, by the computing device, a second movement of the prospective purchaser based on the second acceleration information and the second image, the second movement being in relation to a movement of the second product.

### Claim 8 (depends on 7)

The method of claim 7, wherein the determining a movement of the prospective purchaser comprises utilizing a machine learning model that focuses on skeletal structures of the prospective purchaser based on the image, and
the determining a second movement of the prospective purchaser comprises utilizing the machine learning model based on the second image.

### Claim 9 (depends on 8)

The method of claim 8, wherein the sending display information to a digital display is performed in real-time relative to the determining a movement of the prospective purchaser.

### Claim 10 (depends on 8)

The method of claim 8, wherein the sending display information to a digital display is performed within one second of the determining a movement of the prospective purchaser.

### Claim 11 (depends on 8)

The method of claim 8, wherein the movement of the prospective purchaser is the prospective purchaser lifting the product, and
the second movement of the prospective purchaser is the prospective purchaser lifting the second product.

### Claim 12 (depends on 11)

The method of claim 11, wherein the determining a movement of the prospective purchaser comprises determining that the prospective purchaser moved a first hand of the prospective purchaser.

### Claim 13 (depends on 12)

The method of claim 12, wherein the determining a second movement of the prospective purchaser comprises determining that the prospective purchaser moved a second hand of the prospective purchaser.

### Claim 14 (depends on 1)

The method of claim 1, wherein the computing device includes software provided as a service in a cloud environment.

### Claim 15

A computer program product comprising one or more computer readable storage media having program instructions collectively stored on the one or more computer readable storage media, the program instructions executable to:
receive acceleration information from an acceleration sensor attached to a product, the acceleration information indicating a movement of the acceleration sensor;
receive from an image capture device an image captured by the image capture device of a prospective purchaser of the product;
determine a movement of the prospective purchaser based on the acceleration information and the image, the movement being in relation to a movement of the product; and
send information to a storage device, the information including the movement of the prospective purchaser and information related to the product,
wherein the determining a movement of the prospective purchaser comprises utilizing a machine learning model that focuses on skeletal structures of the prospective purchaser based on the image, and
the movement of the product is based on the acceleration information.

### Claim 16 (depends on 15)

The computer program product of claim 15, wherein the acceleration information includes a time stamp of when the movement of the acceleration sensor occurred,
the image includes a time stamp of when the image was captured by the image capture device, and
the determine a movement of the prospective purchaser comprises matching the time stamp of the acceleration information to the time stamp of the image.

### Claim 17 (depends on 16)

The computer program product of claim 16, wherein the information related to the product comprises identifying information of the product.

### Claim 18

A system comprising:
a processor, a computer readable memory, one or more computer readable storage media, and program instructions collectively stored on the one or more computer readable storage media, the program instructions executable to:
receive acceleration information from an acceleration sensor attached to a product;
receive from an image capturing device an image of a prospective purchaser of the product;
determine a movement of the prospective purchaser based on the acceleration information and the image, the movement being in relation to a movement of the product; and
send user display information to a digital display, the user display information suggesting to a user of the computing device that the user perform an action relative to the prospective purchaser,
wherein, the movement of the product is based on the acceleration information, and
the action suggested is based on the determined movement of the prospective purchaser.

### Claim 19 (depends on 18)

The system of claim 18, wherein the movement of the prospective purchaser is lifting the product.

### Claim 20 (depends on 18)

The system of claim 18, wherein the movement of the prospective purchaser is comparing the product to a second product.
