# US 2019/0244436 A1 · Customized augmented reality item filtering system

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2019/0244436 A1
- Document ID: US20190244436A1
- Status: Published application
- Kind: A1
- Application number: 16/250,416
- Filed: 2019-01-17
- Published: 2019-08-08
- Earliest claimed: 2018-02-06
- Assignees: Walmart Apollo, LLC; L'oreal
- Applicants: Walmart Apollo, LLC; L'oreal
- Inventors: Ian Stansell; Steven Lewis; Scott Lewis
- Web page: https://patentgrep.com/patent/US20190244436A1

## CPC

- G06F 3/011: {Arrangements for interaction with the human body, e.g. for user immersion in virtual reality (blind teaching G09B21/00)}
- G06T 19/20: Editing of three-dimensional [3D] images, e.g. changing shapes or colours, aligning objects or positioning parts
- G06F 3/0482: Interaction with lists of selectable items, e.g. menus
- G06Q 30/0631: {Recommending goods or services}
- G06Q 30/0643: {graphically representing goods, e.g. 3D product representation}
- G06T 19/006: {Mixed reality (object pose determination, tracking or camera calibration for mixed reality G06T7/00)}
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06Q: INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR

## Abstract

Examples provide customized augmented reality item filtering. An item filter analyzes item data using selection criteria and user preferences to identify high-interest item(s) and/or low-interest item(s) within a field of view (FOV) of a user device associated with a user. A high-interest item can include an item physically present within a portion of an item selection area or an un-stocked item not physically present within the portion of the item selection area. An augmented reality (AR) generator creates an AR image including a real-world image of the portion of the item selection area overlaid with graphical elements, including positive item indicators associated with high-interest items, negative item indicators associated with low-interest items, and/or un-stocked virtual items. The AR image is updated in real-time in response to user input, promotional data associated with one or more filtered item(s), and/or changes associated with the items within the FOV of the user device.

## Claims

### Claim 1

An augmented reality system for filtering items in real-time, the system comprising:
a memory;
at least one processor communicatively coupled to the memory;
a set of sensor devices generating sensor data associated with a plurality of items within a field of view (FOV) of a user device associated with a user in an item selection area;
an item filter component, implemented on the at least one processor, that analyzes item data associated with the plurality of items and the sensor data using a set of weighted selection criteria and a set of user preferences to select a set of high-interest items and a set of low-interest items for the user, the set of high-interest items comprising a set of items physically available within the FOV of the user device and a set of items physically absent from the FOV of the user device;
an overlay component, implemented on the at least one processor, that assigns a positive item indicator to each item in the set of high-interest items and a negative item indicator for each item in the set of low-interest items; and
an augmented reality (AR) generator, implemented on the at least one processor, that outputs an AR display of the FOV of the user device, the AR display comprising a real-world image of a portion of the item selection area within the FOV of the user and an AR overlay, the AR overlay comprising a set of positive item indicators associated with each item in the set of high-interest items and a set of negative item indicators associated with each item in the set of low-interest items, and a virtual display comprising a set of un-stocked virtual items physically absent from the FOV of the user device.

### Claim 2 (depends on 1)

The system of claim 1, further comprising:
a set of virtual signs within the AR display, the set of virtual signs comprising information associated with at least one item in the set of high-interest items or at least one item in the set of un-stocked virtual items.

### Claim 3 (depends on 1)

The system of claim 1, further comprising:
an un-stocked virtual items manager, implemented on the at least one processor, wherein a graphical representation of each item in the set of un-stocked virtual items comprises a status indicator, wherein the status indicator comprises at least one of an alternate location indicator associated with items currently available in another display at a different location within the item selection area, a re-stock indicator associated with items available in current inventory for re-stocking an empty item display upon request, and an order indicator associated with items available for order from a remote location for pick-up by the user or delivery to the user at a future date.

### Claim 4 (depends on 1)

The system of claim 1, further comprising:
a pop-up generator, implemented on the at least one processor, that outputs additional information associated with at least one item in the set of high-interest items within the AR display in response to receiving a user input associated with the at least one item, the additional information comprising at least one of online order information or a link to a website associated with at least one item.

### Claim 5 (depends on 1)

The system of claim 1, further comprising:
a feedback component, implemented on the at least one processor, that outputs a request for feedback associated with an arrangement of items within the AR display or feedback associated with an item within the AR display selected for purchase by the user.

### Claim 6 (depends on 1)

The system of claim 1, further comprising:
a promotions component, implemented on the at least one processor, that receives a promotional offer associated with a first item in the set of low-interest items from a remote computing device associated with a supplier in real-time, wherein the filter component further analyzes the promotional offer using the set of weighted selection criteria and the set of user preferences to determine whether to move the item from the set of low-interest items to the set of high-interest items, wherein the filter component moves the first item from the set of low-interest items to the set of high-interest items on condition the set of weighted selection criteria and the set of user preferences indicates the promotional offer increases potential interest of the user in the item.

### Claim 7 (depends on 6)

The system of claim 6, further comprising:
an item status update received from the filter component, by the overlay update component, identifying the first item as a high-interest item, wherein the AR generator generates an updated AR display in real-time, and wherein the updated AR display comprises a positive item indicator associated with the item, wherein the negative item indicator associated with the first item is removed from the updated AR display.

### Claim 8 (depends on 1)

The system of claim 1, further comprising:
a virtual shelf associated with the virtual display within the AR display, the virtual shelf comprising at least one graphical representation of at least one un-stocked item predicted to be of interest to the user based on user-provided data and historical transaction data associated with the user.

### Claim 9 (depends on 1)

The system of claim 1, wherein the AR display further comprises:
a re-stock control associated with an un-stocked item, wherein a notification is sent to an associate to re-stock an item in response to a user selection of the re-stock control in response to a determination at least one instance of the un-stocked item is available in a storage area; and
an item-order request control associated with the un-stocked item, wherein an order component places un-stocked item on order for pick-up by the user or delivery to an address provided by the user in response to the user selection of the item-order request control and completion of an item order request form.

### Claim 10 (depends on 1)

The system of claim 1, wherein the AR display further comprises:
a streaming content display node providing streaming content within the AR display, the streaming content associated with at least one item in the set of high-interest items or at least one user preference within the set of user preferences.

### Claim 11

A computer-implemented method for augmented reality item filtering, the computer-implemented method comprising:
identifying, by an analysis component, a location of a user device associated with a user and a plurality of items within a predetermined distance of the user device based on the identified location and a site-specific planogram;
identifying, by an item filter component, a set of high-interest items and a set of low-interest items from the plurality of items;
identifying, by a prediction component, a set of un-stocked items of potential interest to the user, each item in the set of un-stocked items is physically absent from a set of displays within the a of the user;
assigning, by an overlay component, a positive item indicator to each item in a set of high-interest items, a negative item indicator for each item in the set of low-interest items, and a graphical representation of each item in the set of un-stocked items; and
generating, by an AR display generator, an AR display of a portion of an item selection area within a FOV of the user device for display to the user, the AR display comprising a real-world image of a portion of the item selection area within the FOV of the user device generated by the image capture device enhanced by an AR overlay, the AR overlay comprising a set of positive item indicators associated with each item in the set of high-interest items, a set of negative item indicators for each item in the set of low-interest items, and the graphical representation of each item in the set of un-stocked items.

### Claim 12 (depends on 11)

The computer-implemented method of claim 11, further comprising:
receiving, by a promotions component, a promotional offer associated with a first item in the set of low-interest items;
analyzing, by a filter component, the promotional offer using a set of per-user selection criteria;
on condition the filter component determines the at least one item is of potential interest to the user based on the analysis of the promotion offer, dynamically removing, by the AR generator, a first negative item indicator associated with the first item; and
outputting an updated AR display to the user, the updated AR display comprising at least one positive item indicator associated with the first item.

### Claim 13 (depends on 11)

The computer-implemented method of claim 11, further comprising:
analyzing, by a feedback component, feedback associated with the set of high-interest items; and
updating the set of per-user selection criteria based on the analysis of the feedback.

### Claim 14 (depends on 11)

The computer-implemented method of claim 11, wherein the AR display comprises a restock control, and further comprising:
sending a notification to a restock manager requesting re-stock of a selected un-stocked virtual item in response to user selection of a re-stock button associated with the selected un-stocked item within the AR display;
outputting an order request to the user on condition the user activates an item-order request control within the AR display via a user interface associated with the user device, wherein a completed order request is transmitted to a remote computing device by the user device to place an order requesting an un-stocked item be delivered to an address provided by the user or picked-up by the user at a predetermined time; and
modifying contents of the AR display in response to receiving user input altering at least one graphical element within the AR display, wherein modifying the content of the AR display comprises modifying an arrangement of at least one item within the AR display or altering an appearance of at least one item within the AR display.

### Claim 15 (depends on 11)

The computer-implemented method of claim 11, further comprising:
identifying a digital output device within the predetermined distance of the user device, the digital output device attached to a physical set of shelves associated with the plurality of items; and
transmitting customized content from the user device to the digital output device for display to the user, wherein the customized content comprises information associated with at least one item in the set of high-interest items or at least one item in a set of un-stocked virtual items.

### Claim 16 (depends on 11)

The computer-implemented method of claim 11, further comprising:
analyzing item data using a set of weighted selection criteria and a set of user preferences to identify a high-interest category of items associated with the user, wherein the AR display comprises a positive category indicator associated with all items in the high-interest category within the FOV of the user device; and
analyzing the item data using the set of weighted selection criteria and the set of user preferences to identify a low-interest category of items associated with the user, wherein the AR display comprises a negative category indicator associated with all items in the low-interest category within the FOV of the user device.

### Claim 17 (depends on 11)

The computer-implemented method of claim 11, further comprising:
receiving a selection of an item within the AR display;
identifying a set of related items, wherein the set of related items comprises at least one of an item in a same brand-line as the selected item or at least one item associated with the selected item; and
updating the AR display to include the graphical representation of the set of related items within the AR display.

### Claim 18

A system for augmented reality item filtering, the system comprising:
a memory;
at least one processor communicatively coupled to the memory;
an analysis component, implemented on the at least one processor, that analyzes item data associated with a plurality of items within an item selection area and a planogram to identify a location of a user within the item selection area and identify the plurality of items within a predetermined distance of the user;
an item filter component, implemented on the at least one processor, that analyzes the plurality of items using a set of selection criteria and user preferences associated with the user to select a high-interest category of items of potential interest to the user and a low-interest category of items within the predetermined distance of the user;
an overlay component, implemented on the at least one processor, that assigns a positive category indicator to each item in the selected high-interest category and a negative category indicator to each item in the selected low-interest category;
an AR generator, implemented on the at least one processor, that displays an AR display of a FOV of the user, the AR display comprising a real-world image of a portion of the item selection area within the FOV of the user with an AR overlay enhancing the real-world image, the AR overlay comprising a positive category indicator associated with each item in the selected high-interest category within the FOV and a negative category indicator for each item in the selected low-interest category within the FOV, the AR generator updates the AR display in real-time to change an indicator associated with an identified item in the selected low-interest category from a negative category indicator to a positive category indicator on condition a promotional offer associated with the identified item is received from a set of suppliers.

### Claim 19 (depends on 18)

The system of claim 18, further comprising:
a prediction component, implemented on the at least one processor, that identifies a set of un-stocked items physically absent from the predetermined distance that is predicted to be of potential interest to the user, wherein the AR display comprises a graphical representation of each item in the set of un-stocked items within a virtual item display; and
an un-stocked items status indicator associated with each un-stocked item in the set of un-stocked items in the AR display, an un-stocked items status indicator comprising at least one of an alternate location indicator associated with items currently available in another display at a different location within the item selection area, a re-stock indicator associated with items available in current inventory for re-stocking an empty item display upon request, or an order indicator associated with items available for order from a remote location for pick-up by the user or delivery to the user at a future date.

### Claim 20 (depends on 18)

The system of claim 18, further comprising:
a digital output device receiving customized content from the user device for display to the user, wherein the customized content comprises information associated with at least one item in the set of high-interest items or at least one item in a set of un-stocked virtual items, wherein the digital output device displays the customized content upon detection of the user device within a predetermined area associated with the digital output device, and wherein the digital output device resumes display of default content upon detection of an absence of the user device within the predetermined distance.
