# US 12,183,049 B2 · Methods, systems, articles of manufacture, and apparatus to classify labels based on images using artificial intelligence > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 12,183,049 B2 - Document ID: US12183049B2 - Status: Granted - Kind: B2 - Application number: 18/492,532 - Filed: 2023-10-23 - Published: 2024-04-25 - Granted: 2024-12-31 - Earliest claimed: 2020-06-30 - Assignees: Nielsen Consumer LLC - Applicants: Nielsen Consumer LLC - Inventors: Roberto Arroyo; David Jiménez-Cabello; Javier Martínez Cebrián - Web page: https://patentgrep.com/patent/US12183049B2 ## CPC - G06V 10/25: Determination of region of interest [ROI] or a volume of interest [VOI] - G06F 18/2163: {Partitioning the feature space} - G06F 18/22: Matching criteria, e.g. proximity measures - G06F 18/24: Classification techniques - G06F 18/2413: based on distances to training or reference patterns - G06F 18/251: {of input or preprocessed data} - G06F 18/254: {of classification results, e.g. of results related to same input data} - G06N 3/045: Combinations of networks - G06N 3/0464: Convolutional networks [CNN, ConvNet] - G06N 3/08: Learning methods - G06N 3/09: Supervised learning - G06N 3/0985: Hyperparameter optimisation; Meta-learning; Learning-to-learn - G06T 3/40: Scaling of whole images or parts thereof, e.g. expanding or contracting - G06V 10/761: {Proximity, similarity or dissimilarity measures} - G06V 10/764: using classification, e.g. of video objects - G06V 10/771: Feature selection, e.g. selecting representative features from a multi-dimensional feature space - G06V 10/809: {of classification results, e.g. where the classifiers operate on the same input data} - G06V 10/82: using neural networks - G06V 20/70: Labelling scene content, e.g. deriving syntactic or semantic representations - G06V 2201/10: Recognition assisted with metadata - G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) - G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS - G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL ## Abstract Example methods, apparatus, and articles of manufacture to classify labels based on images using artificial intelligence are disclosed. An example apparatus includes a regional proposal network to determine a first bounding box for a first region of interest in a first input image of a product; and determine a second bounding box for a second region of interest in a second input image of the product; a neural network to: generate a first classification for a first label in the first input image using the first bounding box; and generate a second classification for a second label in the second input image using the second bounding box; a comparator to determine that the first input image and the second input image correspond to a same product; and a report generator to link the first classification and the second classification to the product. ## Claims ### Claim 1 An apparatus comprising: interface circuitry to receive first and second images corresponding to a product; machine readable instructions; and at least one processor circuit to execute the machine readable instructions to at least: detect a first label in the first image, the first label defined by a first bounding box; classify the first label based on the first bounding box; detect a second label in the second image, the second label defined by a second bounding box; classify the second label based on the second bounding box; and when the first classification and the second classification are associated with the product, assign the first and second classifications to the product. ### Claim 2 (depends on 1) The apparatus of claim 1, wherein the first image corresponds to a first portion of the product and the second image corresponds to a second portion of the product, the first portion to be at least partially offset relative to the second portion. ### Claim 3 (depends on 1) The apparatus of claim 1, wherein one or more of the at least one processor circuit is to detect the first label by executing the machine readable instructions to: extract a feature map from the first image, the feature map including points; and apply a region proposal network to the feature map to: generate, at respective ones of the points in the feature map, a set of anchor boxes based on predetermined anchor box sizes and anchor box ratios; identify ones of the anchor boxes that include the first label, the ones of the anchor boxes to include a respective confidence score; determine bounding box coordinates for the ones of the anchor boxes, the bounding box coordinates corresponding to positions relative to the first image; and identify the first bounding box as including the label by applying a non-maximum selection technique to the ones of the anchor boxes. ### Claim 4 (depends on 3) The apparatus of claim 3, wherein the one or more of the at least one processor circuit is to execute the machine readable instructions to train the region proposal network by configuring a first hyperparameter corresponding to the predetermined anchor box ratios and a second hyperparameter corresponding to the predetermined anchor box sizes. ### Claim 5 (depends on 4) The apparatus of claim 4, wherein the one or more of the at least one processor circuit is to execute the machine readable instructions to configure the first hyperparameter to anchor box ratios of (a) 1:2, (b) 1:1, and (c) 2:1. ### Claim 6 (depends on 4) The apparatus of claim 4, wherein the one or more of the at least one processor circuit is to execute the machine readable instructions to configure the second hyperparameter to anchor box scales of 2, 4, and 6. ### Claim 7 (depends on 1) The apparatus of claim 1, wherein one or more of the at least one processor circuit is to execute the machine readable instructions to: detect a third label in the second image, the third label defined by a third bounding box; classify the third label based on the third bounding box; and assign the third classification to the product. ### Claim 8 A non-transitory machine readable storage medium comprising instructions to cause at least one processor circuit to at least: detect a first label in a first image, the first label defined by a first bounding box, the first image to correspond to a first product; classify the first label based on the first bounding box; detect a second label in a second image, the second label defined by a second bounding box, the second image to correspond to the first product; classify the second label based on the second bounding box; and when the first classification and the second classification are associated with the product, assign the first and second classifications to the product. ### Claim 9 (depends on 8) The non-transitory machine readable storage medium of claim 8, wherein the first image corresponds to a first region of the product and the second image corresponds to a second region of the product, the first region to be at least partially different than the second region. ### Claim 10 (depends on 8) The non-transitory machine readable storage medium of claim 8, wherein the instructions cause one or more of the at least one processor circuit to: extract a feature map from the first image, the feature map including points; generate, by applying a region proposal network to the feature map, a set of anchors boxes at respective ones of the points in the feature map based on predetermined anchor box sizes and anchor box ratios; identify ones of the anchor boxes that include the first label, the ones of the anchor boxes to include a respective confidence score; determine bounding box coordinates for the ones of the anchor boxes, the bounding box coordinates corresponding to positions relative to the first image; and identify the first bounding box as including the first label by applying a non-maximum selection technique to the ones of the anchor boxes. ### Claim 11 (depends on 10) The non-transitory machine readable storage medium of claim 10, wherein the instructions cause the one or more of the at least one processor circuit to train the region proposal network by configuring a first hyperparameter corresponding to the predetermined anchor box ratios and a second hyperparameter corresponding to the predetermined anchor box sizes. ### Claim 12 (depends on 11) The non-transitory machine readable storage medium of claim 11, wherein the instructions are to cause the one or more of the at least one processor circuit to configure the first hyperparameter to anchor box ratios of (a) 1:2, (b) 1:1, and (c) 2:1. ### Claim 13 (depends on 11) The non-transitory machine readable storage medium of claim 11, wherein the instructions are to cause the one or more of the at least one processor circuit to configure the second hyperparameter to anchor box scales of 2, 4, and 6. ### Claim 14 (depends on 8) The non-transitory machine readable storage medium of claim 8, wherein the instructions are to cause the one or more of the at least one processor circuit to: detect a third label in the second image, the third label defined by a third bounding box; classify the third label based on the third bounding box; and assign the third classification to the product. ### Claim 15 A method comprising: detecting, by executing a machine readable instruction with programmable circuitry, a first label in a first image, the first label defined by a first bounding box, the first image corresponding to a product; classifying, by executing a machine readable instruction with the programmable circuitry, the first label based on the first bounding box; detecting, by executing a machine readable instruction with the programmable circuitry, a second label in a second image, the second label defined by a second bounding box the second image corresponding to the product; classifying, by executing a machine readable instruction with the programmable circuitry, the second label based on the second bounding box; and when the first classification and the second classification are associated with the product, assigning, by executing a machine readable instruction with the programmable circuitry, the first and second classifications to the product. ### Claim 16 (depends on 15) The method of claim 15, wherein the first image corresponds to a first portion of the product and the second image corresponds to a second portion of the product, the second portion to be at least partially offset from the first portion of the product. ### Claim 17 (depends on 15) The method of claim 15, wherein the detecting of the first label includes: extracting a feature map from the first image, the feature map including points; and applying a region proposal network to the feature map, wherein the applying of the region proposal network to includes: generating, at respective ones of the points in the feature map, a set of anchors boxes based on predetermined anchor box sizes and anchor box ratios; identifying ones of the anchor boxes that include the first label, the ones of the anchor boxes to include a respective confidence score; determining bounding box coordinates for the ones of the anchor boxes, the bounding box coordinates corresponding to positions relative to the first image; and identifying the first bounding box as including the label by applying a non-maximum selection technique to the ones of the anchor boxes. ### Claim 18 (depends on 17) The method of claim 17, further including training the region proposal network by configuring a first hyperparameter corresponding to the predetermined anchor box ratios and a second hyperparameter corresponding to the predetermined anchor box sizes. ### Claim 19 (depends on 18) The method of claim 18, wherein the configuring of the first hyperparameter includes tuning the first hyperparameter to generate anchor box ratios of (a) 1:2, (b) 1:1, and (c) 2:1. ### Claim 20 (depends on 18) The method of claim 18, wherein the configuring of the second hyperparameter includes tuning the second hyperparameter to generate anchor box scales of 2, 4, and 6.