# US 10,373,107 B2 · Generation of actions to correct physical configurations

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 10,373,107 B2
- Document ID: US10373107B2
- Status: Granted
- Kind: B2
- Application number: 14/983,030
- Filed: 2015-12-29
- Published: 2017-06-29
- Granted: 2019-08-06
- Earliest claimed: 2015-12-29
- Assignees: Sap SE
- Applicants: Sap SE
- Inventors: Koji Kawashima; Robert Moore
- Web page: https://patentgrep.com/patent/US10373107B2

## CPC

- G06Q 10/087: Inventory or stock management, e.g. order filling, procurement or balancing against orders
- G06Q 10/08: Logistics, e.g. warehousing, loading or distribution; Inventory or stock management
- G06Q 10/083: Shipping
- G06Q 10/08724: {by shelf level inventory management, e.g. planograms}
- G06Q 10/0875: Itemisation or classification of parts, supplies or services, e.g. bill of materials
- G06Q 10/0877: {by inventory control or reporting using inventory tracking or counting}
- G06Q 20/203: {Inventory monitoring}
- G06Q: INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR

## Abstract

A system includes acquisition of an electronic image of items disposed in first locations, determination, based on the electronic image, of a visit image result comprising a first one or more linear groups, a first one or more facings per linear group and a first zero or more product identifiers per facing, determination of planogram reference data comprising a second one or more linear groups, a second one or more facings per linear group and a second zero or more product identifiers per facing, and determination of one or more corrective actions based on the visit image result and on the planogram reference data.

## Claims

### Claim 1

A computing system comprising:
a memory storing processor-executable program code; and
a processor to execute the processor-executable program code in order to cause the computing system to:
acquire an electronic image of items disposed in first locations;
determine, based on the acquired electronic image, a visit image result comprising a first one or more linear groups, a first one or more facings per linear group and a first zero or more product identifiers per facing, the visit image result including a unique identifier for the items in the acquired electronic image and indications of relative locations for those items, the linear groups including at least a portion of a shelf supporting the items in the acquired electronic image, and the facings each being a sub-portion of a linear group;
filter the determined visit image result to remove items other than those items related to a first entity;
determine planogram reference data comprising a second one or more linear groups, a second one or more facings per linear group and a second zero or more product identifiers per facing, the planogram reference data specifying expected item count, shelf configuration, shelf size, unique product identifier for the expected items and their corresponding location;
determine, based on the planogram reference data, a table of items which should be located in the facings by:
for each of the second one or more linear groups, retrieve a corresponding linear group of the table of items located in the facings;
for each of the second one or more linear groups, mark all items with a second flag which are not marked with the first flag and which are also present in the corresponding linear group of the table of items located in the facings; and
for each of the second one or more linear groups, mark all unmarked items with a third flag, the third flag indicating that an item is missing from the facings; and
determine one or more corrective actions based on the visit image result and on the planogram reference data, determination of the one or more corrective actions comprises:
for each of the second one or more linear groups, determine all items marked with the third flag;
for each of the items marked with the third flag, determine whether the item is present in the visit image result;
for each of the items marked with the third flag and not present in the visit image result, associate the item with a corresponding facing of a corrective action result table and mark the item with a flag indicating that the item is out of stock; and
for each of the items marked with the third flag and present in the visit image result, and if the item is not marked with the first flag, associate the item with a corresponding facing of a corrective action result table and mark the item with a flag indicating that the item is partially missing from the corresponding facing.

### Claim 2 (depends on 1)

A computing system according to claim 1, wherein the processor is further to execute the processor-executable program code in order to cause the computing system to:
determine, based on the visit image result, a table of items located in facings by:
creation, for each of the first one or more linear groups, of a plurality of combinations of items matching items in a corresponding one of the second one or more linear groups;
determination of a best matching combination of the plurality of combinations of items matching items for each of the first one or more linear groups; and
population of any empty facings in each of the plurality of determined best matching combinations,
wherein the one or more corrective actions are determined based on the table of items located in facings and on the planogram reference data.

### Claim 3 (depends on 2)

A computing system according to claim 2, wherein the processor is further to execute the processor-executable program code in order to cause the computing system to:
determine a table of items which should be located in the facings based on the planogram reference data and on the table of items located in the facings,
wherein the one or more corrective actions are determined based on the table of items located in the facings and on the table of items which should be located in the facings.

### Claim 4 (depends on 2)

A computing system according to claim 2, wherein population of any empty facings comprises population of any empty facings in each of the plurality of determined best matching combinations with an item and a first flag.

### Claim 5 (depends on 1)

A computing system according to claim 1, wherein determination of the one or more corrective actions comprises:
determination of at least one of: the item is out of stock; the item is partially missing from a corresponding facing; the item is present in a corresponding facings in too great a quantity; and the product has been recalled.

### Claim 6

A non-transitory computer-readable medium storing program code, the program code executable by a processor of a computing system to cause the computing system to:
acquire an electronic image of items disposed in first locations;
determine, based on the acquired electronic image, a visit image result comprising a first one or more linear groups, a first one or more facings per linear group and a first zero or more product identifiers per facing, the visit image result including a unique identifier for the items in the acquired electronic image and indications of relative locations for those items, the linear groups including at least a portion of a shelf supporting the items in the acquired electronic image, and the facings each being a sub-portion of a linear group;
filter the determined visit image result to remove items other than those items related to a first entity;
determine planogram reference data comprising a second one or more linear groups, a second one or more facings per linear group and a second zero or more product identifiers per facing, the planogram reference data specifying expected item count, shelf configuration, shelf size, unique product identifier for the expected items and their corresponding location;
determine, based on the planogram reference data, a table of items which should be located in the facings by:
for each of the second one or more linear groups, retrieve a corresponding linear group of the table of items located in the facings;
for each of the second one or more linear groups, mark all items with a second flag which are not marked with the first flag and which are also present in the corresponding linear group of the table of items located in the facings; and
for each of the second one or more linear groups, mark all unmarked items with a third flag, the third flag indicating that an item is missing from the facings; and
determine one or more corrective actions based on the visit image result and on the planogram reference data, the determination of the one or more corrective actions comprises:
for each of the second one or more linear groups, determine all items marked with the second flag;
for each of the items marked with the second flag, determine whether the item is present in the visit image result;
for each of the items marked with the second flag and not present in the visit image result, associate the item with a corresponding facing of a corrective action result table and mark the item with a flag indicating that the item is out of stock; and
for each of the items marked with the second flag and present in the visit image result, and if the item is not marked with the first flag, associate the item with a corresponding facing of a corrective action result table and mark the item with a flag indicating that the item is partially missing from the corresponding facing.

### Claim 7 (depends on 6)

A medium according to claim 6, the program code executable by a processor of a computing system to cause the computing system to:
determine, based on the visit image result, a table of items located in facings by:
creation, for each of the first one or more linear groups, of a plurality of combinations of items matching items in a corresponding one of the second one or more linear groups;
determination of a best matching combination of the plurality of combinations of items matching items for each of the first one or more linear groups; and
population of any empty facings in each of the plurality of determined best matching combinations,
wherein the one or more corrective actions are determined based on the table of items located in facings and on the planogram reference data.

### Claim 8 (depends on 7)

A medium according to claim 7, the program code executable by a processor of a computing system to cause the computing system to:
determine a table of items which should be located in the facings based on the planogram reference data and on the table of items located in the facings,
wherein the one or more corrective actions are determined based on the table of items located in the facings and on the table of items which should be located in the facings.

### Claim 9 (depends on 7)

A medium according to claim 7, wherein population of any empty facings comprises population of any empty facings in each of the plurality of determined best matching combinations with an item and a first flag.

### Claim 10 (depends on 6)

A medium according to claim 6, wherein determination of the one or more corrective actions comprises:
determination of at least one of: the item is out of stock; the item is partially missing from a corresponding facing; the item is present in a corresponding facings in too great a quantity; and
the product has been recalled.

### Claim 11

A computer-implemented method comprising:
acquiring an electronic image of items disposed in first locations;
determining, based on the acquired electronic image, a visit image result comprising a first one or more linear groups, a first one or more facings per linear group and a first zero or more product identifiers per facing, the visit image result including a unique identifier for the items in the acquired electronic image and indications of relative locations for those items, the linear groups including at least a portion of a shelf supporting the items in the acquired electronic image, and the facings each being a sub-portion of a linear group;
filtering the determined visit image result to remove items other than those items related to a first entity;
determining, based on the visit image result, a table of items located in facings by:
creating, for each of the first one or more linear groups, of a plurality of combinations of items matching items in a corresponding one of the second one or more linear groups;
determining a best matching combination of the plurality of combinations of items matching items for each of the first one or more linear groups; and
populating any empty facings in each of the plurality of determined best matching combinations;
determining planogram reference data comprising a second one or more linear groups, a second one or more facings per linear group and a second zero or more product identifiers per facing, the planogram reference data specifying expected item count, shelf configuration, shelf size, unique product identifier for the expected items and their corresponding location;
determining, based on the planogram reference data, a table of items which should be located in the facings by:
for each of the second one or more linear groups, retrieving a corresponding linear group of the table of items located in the facings;
for each of the second one or more linear groups, marking all items with a second flag which are not marked with the first flag and which are also present in the corresponding linear group of the table of items located in the facings; and
for each of the second one or more linear groups, marking all unmarked items with a third flag, the third flag indicating that an item is missing from the facings; and
determining one or more corrective actions based on the table of items located in facings and on the planogram reference data, the determining of the one or more corrective actions comprises:
for each of the second one or more linear groups, determining all items marked with the second flag;
for each of the items marked with the second flag, determining whether the item is present in the visit image result;
for each of the items marked with the second flag and not present in the visit image result, associating the item with a corresponding facing of a corrective action result table and mark the item with a flag indicating that the item is out of stock; and
for each of the items marked with the second flag and present in the visit image result, and if the item is not marked with the first flag, associating the item with a corresponding facing of a corrective action result table and mark the item with a flag indicating that the item is partially missing from the corresponding facing.

### Claim 12 (depends on 11)

A method according to claim 11, further comprising:
determining a table of items which should be located in the facings based on the planogram reference data and on the table of items located in the facings,
wherein the one or more corrective actions are determined based on the table of items located in the facings and on the table of items which should be located in the facings.

### Claim 13 (depends on 11)

A method according to claim 11, wherein population of any empty facings comprises population of any empty facings in each of the plurality of determined best matching combinations with an item and a first flag.

### Claim 14 (depends on 11)

A method according to claim 11, wherein determining the one or more corrective actions comprises:
determining at least one of: the item is out of stock; the item is partially missing from a corresponding facing; the item is present in a corresponding facings in too great a quantity; and the product has been recalled.