# US 12,001,952 B2 · Electronic device and method for controlling electronic device > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 12,001,952 B2 - Document ID: US12001952B2 - Status: Granted - Kind: B2 - Application number: 17/281,885 - Filed: 2019-10-08 - Published: 2021-12-23 - Granted: 2024-06-04 - Earliest claimed: 2018-10-29 - Assignees: Samsung Electronics Co., Ltd. - Applicants: Samsung Electronics Co., Ltd. - Inventors: Jiman Kim; Chanjong Park; Dongha Bahn - Web page: https://patentgrep.com/patent/US12001952B2 ## CPC - G06N 3/08: Learning methods - G06F 18/214: Generating training patterns; Bootstrap methods, e.g. bagging or boosting - G06F 18/2431: Multiple classes - G06F 18/254: {of classification results, e.g. of results related to same input data} - G06N 3/02: Neural networks - G06N 3/044: Recurrent networks, e.g. Hopfield networks - G06N 3/045: Combinations of networks - G06N 3/0464: Convolutional networks [CNN, ConvNet] - G06N 3/09: Supervised learning - G06T 3/40: Scaling of whole images or parts thereof, e.g. expanding or contracting - G06V 10/764: using classification, e.g. of video objects - G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) - G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL - G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING ## Abstract Disclosed are an electronic device and a method for controlling an electronic device. Specifically, the present disclosure relates to: an electronic device configured to input an acquired image into a trained artificial intelligence model, acquire information about the image from a plurality of classifiers which are included in the artificial intelligence model and correspond to a plurality of layers classified according to higher and lower concepts of an object included in the image, train the artificial intelligence model on the basis of the information about the acquired image, and perform image recognition by using the trained artificial intelligence model; and a method for controlling an electronic device. ## Claims ### Claim 1 An electronic device comprising: a memory comprising at least one instruction; and a processor configured to be connected to the memory to control the electronic device, wherein the processor, by executing the at least one instruction, is configured to: obtain an image, and obtain information regarding the image by inputting the obtained image to a trained artificial intelligence model, and wherein the artificial intelligence model comprises a plurality of classifiers respectively corresponding to a plurality of classes divided according to higher and lower concepts of an object included in the image, and is configured to obtain information regarding the image from each of the plurality of classifiers, wherein the plurality of classes comprises a first class and a second class comprising a plurality of subcategories of each of a plurality of categories included in the first class, and wherein the processor is configured to identify a category corresponding to the object included in the image from among the plurality of categories included in the first class based on the information regarding the image obtained from a classifier corresponding to the second class. ### Claim 2 (depends on 1) The device according to claim 1, wherein the plurality of classes comprises a plurality of categories for classifying the object included in the image for each of the plurality of classes, and wherein the information regarding the image comprises information regarding a category corresponding to the object included in the image from among the plurality of categories included in each of the plurality of classes. ### Claim 3 (depends on 1) The device according to claim 1, wherein the processor is configured to: obtain a plurality of loss function values corresponding to each of the plurality of classes based on information regarding the image obtained from each of the plurality of classifiers; and train the artificial intelligence model by performing backpropagation of a total loss function value obtained by adding up the plurality of loss function values so that the total loss function value becomes a minimum value. ### Claim 4 (depends on 1) The device according to claim 1, wherein the processor is configured to: obtain a feature value of the image by inputting the image to the artificial intelligence model; and obtain information regarding the image from each of the plurality of classifiers by inputting the obtained feature value of the image to the plurality of classifiers. ### Claim 5 (depends on 1) The device according to claim 1, wherein the information regarding the image obtained from the classifier corresponding to the second class comprises information regarding a possibility that the object included in the image corresponds to each of the plurality of categories included in the second class, and wherein the processor is configured to identify a category corresponding to the object included in the image from among the plurality of categories included in the first class by obtaining information regarding a possibility that the object included in the image corresponds to each of the plurality of categories included in the first class based on the information regarding the possibility. ### Claim 6 (depends on 1) The device according to claim 1, wherein the first class comprises a plurality of categories for identifying whether a product included in the image is faulty, wherein the second class comprises a plurality of subcategories comprising a category regarding a faultiness type of the product and a category regarding a normality type of the product, and wherein the processor is configured to, obtain an appearance image of the product, obtain information regarding an image corresponding to the second class by inputting the obtained appearance image to the artificial intelligence model, and identify whether the product is faulty by obtaining information regarding an image corresponding to the first class based on the information regarding the image corresponding to the second class. ### Claim 7 (depends on 1) The device according to claim 1, wherein the artificial intelligence model comprises at least one artificial neural network among a convolutional neural network (CNN) and a recurrent neural network (RNN). ### Claim 8 (depends on 1) The device according to claim 1, wherein the processor is configured to convert the obtained image into a gray scale image and input the converted gray scale image to the artificial intelligence model. ### Claim 9 A method for controlling an electronic device, the method comprising: obtaining an image; inputting the obtained image to a trained artificial intelligence model; and obtaining information regarding the image from each of a plurality of classifiers which corresponds to each of a plurality of classes divided according to higher and lower concepts of an object included in the image and is included in the artificial intelligence model, wherein the method further comprises identifying a category, wherein the plurality of classes comprises a first class and a second class comprising a plurality of subcategories of each of a plurality of categories included in the first class, and wherein the identifying the category comprises identifying a category corresponding to the object included in the image from among the plurality of categories included in the first class based on the information regarding the image obtained from a classifier corresponding to the second class. ### Claim 10 (depends on 9) The method according to claim 9, wherein the plurality of classes comprises a plurality of categories for classifying the object included in the image for each of the plurality of classes, and wherein the information regarding the image comprises information regarding a category corresponding to the object included in the image from among the plurality of categories included in each of the plurality of classes. ### Claim 11 (depends on 9) The method according to claim 9, further comprising: training the artificial intelligence model, wherein the obtaining the information regarding the image comprises obtaining a plurality of loss function values corresponding to each of the plurality of classes based on information regarding the image obtained from each of the plurality of classifiers, and wherein the training the artificial intelligence model comprises training the artificial intelligence model by performing backpropagation of a total loss function value obtained by adding up the plurality of loss function values so that the total loss function value becomes a minimum value. ### Claim 12 (depends on 9) The method according to claim 9, further comprising: obtaining a feature value of the image by inputting the image to the artificial intelligence model, wherein the obtaining the information regarding the image comprises obtaining information regarding the image from each of the plurality of classifiers by inputting the obtained feature value of the image to the plurality of classifiers. ### Claim 13 A non-transitory computer-readable recording medium comprising a program for executing a method for controlling an electronic device, the method for controlling the electronic device comprising: obtaining an image; inputting the obtained image to a trained artificial intelligence model; and obtaining information regarding the image from each of a plurality of classifiers which corresponds to each of a plurality of classes divided according to higher and lower concepts of an object included in the image and is included in the artificial intelligence model, wherein the method further comprises identifying a category, wherein the plurality of classes comprises a first class and a second class comprising a plurality of subcategories of each of a plurality of categories included in the first class, and wherein the identifying the category comprises identifying a category corresponding to the object included in the image from among the plurality of categories included in the first class based on the information regarding the image obtained from a classifier corresponding to the second class.