# US 2022/0284383 A1 · On-Shelf Image Based Barcode Reader for Inventory Management System > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 2022/0284383 A1 - Document ID: US20220284383A1 - Status: Published application - Kind: A1 - Application number: 17/685,000 - Filed: 2022-03-02 - Published: 2022-09-08 - Earliest claimed: 2021-03-04 - Assignees: Focal Systems, Inc. - Applicants: Focal Systems, Inc. - Inventors: Francois Chaubard - Web page: https://patentgrep.com/patent/US20220284383A1 ## CPC - G06K 7/1443: {locating of the code in an image} - G06Q 10/087: Inventory or stock management, e.g. order filling, procurement or balancing against orders - G06K 7/1413: {1D bar codes} - G06K 7/1417: {2D bar codes} - G06K 7/1447: {extracting optical codes from image or text carrying said optical code} - G06Q 10/08724: {by shelf level inventory management, e.g. planograms} - G06Q 10/0877: {by inventory control or reporting using inventory tracking or counting} - G06T 3/4046: using neural networks - G06T 3/4053: based on super-resolution, i.e. the output image resolution being higher than the sensor resolution - G06T 3/4076: using the original low-resolution images to iteratively correct the high-resolution images - G06V 10/225: {based on a marking or identifier characterising the area} - G06V 10/25: Determination of region of interest [ROI] or a volume of interest [VOI] - G06V 10/267: {by performing operations on regions, e.g. growing, shrinking or watersheds} - G06V 10/273: {removing elements interfering with the pattern to be recognised} - G06V 10/82: using neural networks - G06V 20/52: Surveillance or monitoring of activities, e.g. for recognising suspicious objects (recognising microscopic objects G06V20/69) - G06V 30/1448: {based on markings or identifiers characterising the document or the area} - G06K: GRAPHICAL DATA READING (image or video recognition or understanding G06V); PRESENTATION OF DATA; RECORD CARRIERS; HANDLING RECORD CARRIERS - G06Q: INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR - G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL - G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING ## Abstract An inventory visibility management system utilizes fixed or motorized cameras to scan inventory bearing shelves in a backroom or warehouse of a store, as opposed to store shelves where merchandise is available for purchase, for inventory frequently to keep the system up to date on what boxes of inventory are on the shelf, what is in those boxes and where those boxes are on the shelf. The system may identify a bounding polygon around an identifier corresponding to the product and apply the bounding polygon to a machine-learned model, which may generate a high-resolution crop of the identifier as output. The system registers the identifier to the first bounding polygon and to a location associated with cameras that captured the plurality of low-resolution images. Upon receiving a request from a client device, the system may provide the location associated with the one or more cameras to the client device. ## Claims ### Claim 1 A method comprising: receiving a plurality of low-resolution images, each of the images including at least a portion of a product, the plurality of low-resolution images having resolution lower than a predetermined threshold; pre-processing the low-resolution images resulting in a plurality of pre-processed images; identifying, in each respective one of the plurality of pre-processed images, a first bounding polygon around the product, and a second bounding polygon around an identifier corresponding to the product; applying at least the second bounding polygons from each of the pre-processed images to a machine-learned model and receiving as output from the machine-learned model a high-resolution crop of the identifier; decoding the identifier; registering the identifier to the first bounding polygon and to a location associated with one or more cameras that captured the plurality of low-resolution images; receiving a request from a client device to locate a product associated with the identifier; and in response to receiving the request, providing the location associated with the one or more cameras to the client device. ### Claim 2 (depends on 1) The method of claim 1, wherein the preprocessing further comprises removing pixels corresponding to humans from the low-resolution images prior to applying to the machine-learned model. ### Claim 3 (depends on 1) The method of claim 1, wherein the preprocessing further comprises removing occlusions blocking the product from the low-resolution images prior to applying to the machine-learned model. ### Claim 4 (depends on 1) The method of claim 1, wherein a merge algorithm is used to merge the plurality of low-resolution images into a single high-resolution image, the merge algorithm producing homography estimation for each pair of low-resolution images, wherein the high-resolution image having a resolution higher than the plurality of low-resolution images. ### Claim 5 (depends on 4) The method of claim 4, wherein the merge algorithm is applied using a CNN (convolutional neural network) or Sobel Algorithm with a classifier for each low-resolution image. ### Claim 6 (depends on 1) The method of claim 1, wherein registering the identifier comprises: extracting features from an image pair comprising a first image and a second image, the first image having an overlapping FOV (field of view) with the second image; calculating a homography and determining a quality of the homography; and registering the second image to the first image. ### Claim 7 (depends on 1) The method of claim 1, further comprising: responsive to delivery of a batch of the product, determining whether the batch of the product fits in a first location; and responsive to determining that the batch of the product does not fit in the first location, transmitting an alert to a person to divert the batch of the product to a second location for storage. ### Claim 8 (depends on 1) The method of claim 1, wherein the low-resolution images are captures by a striding camera that takes a stop each time the camera finishes traveling a predetermined distance, wherein the camera takes an image at each stop. ### Claim 9 (depends on 8) The method of claim 8, wherein the striding cameras are powered by cable or wireless charging. ### Claim 10 A non-transitory computer-readable storage medium comprising memory with executable computer instructions encoded thereon that, when executed by one or more processors, cause the one or more processors to perform operations, the instructions comprising instructions to: receive a plurality of low-resolution images, each of the images including at least a portion of a product, the plurality of low-resolution images having resolution lower than a predetermined threshold; pre-process the low-resolution images resulting in a plurality of pre-processed images; identify, in each respective one of the plurality of pre-processed images, a first bounding polygon around the product, and a second bounding polygon around an identifier corresponding to the product; apply at least the second bounding polygons from each of the pre-processed images to a machine-learned model and receiving as output from the machine-learned model a high-resolution crop of the identifier; decode the identifier; register the identifier to the first bounding polygon and to a location associated with one or more cameras that captured the plurality of low-resolution images; receive a request from a client device to locate a product associated with the identifier; and in response to receiving the request, providing the location associated with the one or more cameras to the client device. ### Claim 11 (depends on 10) The non-transitory computer-readable storage medium of claim 10, wherein the preprocessing further comprises removing pixels corresponding to humans from the low-resolution images prior to applying to the machine-learned model. ### Claim 12 (depends on 10) The non-transitory computer-readable storage medium of claim 10, wherein the preprocessing further comprises removing occlusions blocking the product from the low-resolution images prior to applying to the machine-learned model. ### Claim 13 (depends on 10) The non-transitory computer-readable storage medium of claim 10, wherein a merge algorithm is used to merge the plurality of low-resolution images into a single high-resolution image, the merge algorithm producing homography estimation for each pair of low-resolution images, wherein the high-resolution image having a resolution higher than the plurality of low-resolution images. ### Claim 14 (depends on 10) The non-transitory computer-readable storage medium of claim 10, wherein the merge algorithm is applied using a CNN (convolutional neural network) or Sobel Algorithm with a classifier for each low-resolution image. ### Claim 15 (depends on 10) The non-transitory computer-readable storage medium of claim 10, wherein registering the identifier comprises: extracting features from an image pair comprising a first image and a second image, the first image having an overlapping FOV (field of view) with the second image; calculating a homography and determining a quality of the homography; and registering the second image to the first image. ### Claim 16 (depends on 10) The non-transitory computer-readable storage medium of claim 10, further comprising: responsive to delivery of a batch of the product, determining whether the batch of the product fits in a first location; and responsive to determining that the batch of the product does not fit in the first location, transmitting an alert to a person to divert the batch of the product to a second location for storage. ### Claim 17 (depends on 10) The non-transitory computer-readable storage medium of claim 10, wherein the low-resolution images are captures by a striding camera that takes a stop each time the camera finishes traveling a predetermined distance, wherein the camera takes an image at each stop. ### Claim 18 (depends on 17) The non-transitory computer-readable storage medium of claim 17, wherein the striding cameras are powered by cable or wireless charging. ### Claim 19 A computing system comprising: a processor; and a non-transitory computer-readable storage medium storing instructions for managing inventory visibility, the instructions when executed by the processor cause the processor to perform steps including: receiving a plurality of low-resolution images, each of the images including at least a portion of a product, the plurality of low-resolution images having resolution lower than a predetermined threshold; pre-processing the low-resolution images resulting in a plurality of pre-processed images; identifying, in each respective one of the plurality of pre-processed images, a first bounding polygon around the product, and a second bounding polygon around an identifier corresponding to the product; applying at least the second bounding polygons from each of the pre-processed images to a machine-learned model and receiving as output from the machine-learned model a high-resolution crop of the identifier; decoding the identifier; registering the identifier to the first bounding polygon and to a location associated with one or more cameras that captured the plurality of low-resolution images; receiving a request from a client device to locate a product associated with the identifier; and in response to receiving the request, providing the location associated with the one or more cameras to the client device. ### Claim 20 (depends on 19) The computer system of claim 19, wherein the preprocessing further comprises removing pixels corresponding to humans from the low-resolution images prior to applying to the machine-learned model.