Automatic Generation of In-Store Product Information and Navigation Guidance, Using Augmented Reality (AR) and a Vision-and-Language Model (VLM) and Multi-Modal Artificial Intelligence (AI)

Abstract

Automatic generation of in-store product information and navigation guidance, using Augmented Reality (AR) and a Vision-and-Language Model (VLM) and multi-modal Artificial Intelligence (AI). An automated method includes: providing to the VLM images that are captured within a retailer venue by an electronic device that is a smartphone or an AR device or smart glasses; (b) automatically feeding into the VLM those images, or pre-sliced or pre-cropped image-portions of those images that were sliced or cropped using Machine Learning that performs object boundaries detection and not product recognition; (c) invoking the VLM to generate outputs of VLM analysis of content of those fed images or image-portions. The VLM outputs can be: VLM-based product recognition, VLM-based product-related information, VLM-generated virtual shopping assistance, VLM-generated in-store navigation guidance, or other VLM-generated outputs. Based on the VLM-generated outputs, the electronic device provides real-time information about products depicted in the images, VLM-generated shopping assistance, and VLM-generated in-store navigation guidance.

Dependencies are inferred from the claim text.