# US 2020/0394451 A1 · Selecting artificial intelligence model based on input data > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 2020/0394451 A1 - Document ID: US20200394451A1 - Status: Published application - Kind: A1 - Application number: 16/899,841 - Filed: 2020-06-12 - Published: 2020-12-17 - Earliest claimed: 2019-06-12 - Assignees: Samsung Electronics Co., Ltd. - Applicants: Samsung Electronics Co., Ltd. - Inventors: Anant BAIJAL; Vivek AGARWAL; Jayoon KOO - Web page: https://patentgrep.com/patent/US20200394451A1 ## CPC - G06N 3/08: Learning methods - G06F 18/2415: based on parametric or probabilistic models, e.g. based on likelihood ratio or false acceptance rate versus a false rejection rate - G06F 18/2431: Multiple classes - G06F 18/285: {Selection of pattern recognition techniques, e.g. of classifiers in a multi-classifier system} - G06N 3/045: Combinations of networks - G06N 3/0464: Convolutional networks [CNN, ConvNet] - G06N 3/09: Supervised learning - G06N 7/01: Probabilistic graphical models, e.g. probabilistic networks - G06N 20/20: Ensemble learning - G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) ## Abstract A method of selecting an artificial intelligence (AI) model based on input data to select an AI model capable of correctly obtaining a result corresponding to data to be classified, e.g., a classification result indicating one of at least one class, from among a plurality of AI models, and a display device for performing the method. ## Claims ### Claim 1 A method of selecting an artificial intelligence model for classifying input data, the method comprising: obtaining first misclassified data of a first artificial intelligence model classifying data included in a training data set, the first misclassified data indicating first data of the training data set for which the first artificial intelligence model incorrectly classifies the first data; obtaining second misclassified data of a second artificial intelligence model classifying the data included in the training data set, the second misclassified data indicating second data of the training data set for which the second artificial intelligence model incorrectly classifies the second data; training a selection artificial intelligence model based on the first misclassified data and the second misclassified data; providing the input data to the selection artificial intelligence model; and selecting an optimal artificial intelligence model from among the first artificial intelligence model and the second artificial intelligence model to classify the input data, based on output of the selection artificial intelligence model. ### Claim 2 (depends on 1) The method of claim 1, wherein the selecting comprises selecting the optimal artificial intelligence model corresponding to a lowest value of misclassification probability of incorrectly classifying the input data determined by the selection artificial intelligence model from among a first misclassification probability of the first artificial intelligence model incorrectly classifying the input data and a second misclassification probability of the second artificial intelligence model incorrectly classifying the input data. ### Claim 3 (depends on 1) The method of claim 1, wherein the training comprises: inputting the first misclassified data and the second misclassified data to the selection artificial intelligence model. ### Claim 4 (depends on 3) The method of claim 3, wherein the training comprises training the selection artificial intelligence model about characteristics and distribution of data of the first data and the second data. ### Claim 5 (depends on 1) The method of claim 1, further comprising: providing the input data to the optimal artificial intelligence model; and obtaining a result of classifying the input data from the optimal artificial intelligence model. ### Claim 6 (depends on 1) The method of claim 1, wherein a neural network structure of the selection artificial intelligence model is different from structures of the first artificial intelligence model and the second artificial intelligence model. ### Claim 7 (depends on 1) The method of claim 1, wherein the output of the selection artificial intelligence model comprises a first misclassification probability of the first artificial intelligence model incorrectly classifying the input data and a second misclassification probability of the second artificial intelligence model incorrectly classifying the input data. ### Claim 8 (depends on 7) The method of claim 7, further comprising adjusting, based on the first misclassification probability and the second misclassification probability, a plurality of weight values applied to at least one of the first artificial intelligence model and the second artificial intelligence model comprising an ensemble artificial intelligence model. ### Claim 9 (depends on 8) The method of claim 8, wherein the adjusting of the plurality of weight values comprises adjusting the plurality of weight values applied to at least one of the first artificial intelligence model and the second artificial intelligence model, in such a manner that the plurality of weight values are reduced when misclassification probability is high. ### Claim 10 (depends on 8) The method of claim 8, further comprising: providing the input data to the ensemble artificial intelligence model; and outputting a result of classifying the input data from the ensemble artificial intelligence model. ### Claim 11 (depends on 1) The method of claim 1, wherein the first artificial intelligence model is configured to classify the input data into a class from among a plurality of classes and the second artificial intelligence model is configured to classify the input data into the class from among the plurality of classes. ### Claim 12 (depends on 1) The method of claim 1, wherein at least one of a first hyper-parameter, a first model architecture, a first training technique, or a first dataset input for training the first artificial intelligence model is different from at least one of a second hyper-parameter, a second model architecture, a second training technique, or a second dataset input for training the second artificial intelligence model. ### Claim 13 (depends on 1) The method of claim 1, wherein the first artificial intelligence model and the second artificial intelligence model are combined as an ensemble AI model configured to classify the input data. ### Claim 14 (depends on 1) The method of claim 1, wherein the first artificial intelligence model comprises at least one on-device artificial intelligence model and at least one server-based artificial intelligence model. ### Claim 15 A device for selecting an artificial intelligence model for classifying input data, the device comprising: a display; and a processor configured to execute at least one instruction to: obtain first misclassified data of a first artificial intelligence model classifying data included in a training data set, the first misclassified data indicating first data of the training data set for which the first artificial intelligence model incorrectly classifies the first data; obtain second misclassified data of a second artificial intelligence model classifying the data included in the training data set, the second misclassified data indicating second data of the training data set for which the second artificial intelligence model incorrectly classifies the second data; train a selection artificial intelligence model based on the first misclassified data and the second misclassified data; provide the input data to the selection artificial intelligence model; and select an optimal artificial intelligence model from among the first artificial intelligence model and the second artificial intelligence model to classify the input data, based on output of the selection artificial intelligence model. ### Claim 16 (depends on 15) The device of claim 15, wherein the processor is further configured to select the optimal artificial intelligence model corresponding to a lowest value of misclassification probability of incorrectly classifying the input data determined by the selection artificial intelligence model from among a first misclassification probability of the first artificial intelligence model incorrectly classifying the input data and a second misclassification probability of the second artificial intelligence model incorrectly classifying the input data. ### Claim 17 (depends on 15) The device of claim 15, wherein the processor is further configured to input the first misclassified data and the second misclassified data to the selection artificial intelligence model. ### Claim 18 (depends on 15) The device of claim 15, wherein the first artificial intelligence model is configured to classify the input data into a class from among a plurality of classes and the second artificial intelligence model is configured to classify the input data into the class from among the plurality of classes. ### Claim 19 (depends on 15) The device of claim 15, further comprising a communicator configured to communicate with a server, wherein the processor is further configured to control the communicator to receive the first misclassified data and the second misclassified data from the server. ### Claim 20 A computer-readable recording medium having recorded thereon a computer program for executing a method of selecting an artificial intelligence model for classifying input data, the method comprising: obtaining first misclassified data of a first artificial intelligence model classifying data included in a training data set, the first misclassified data indicating first data of the training data set for which the first artificial intelligence model incorrectly classifies the first data; obtaining second misclassified data of a second artificial intelligence model classifying of the data included in the training data set, the second misclassified data indicating second data of the training data set for which the second artificial intelligence model incorrectly classifies the second data; training a selection artificial intelligence model based on the first misclassified data and the second misclassified data; providing the input data to the selection artificial intelligence model; and selecting an optimal artificial intelligence model from among the first artificial intelligence model and the second artificial intelligence model to classify the input data, based on output of the selection artificial intelligence model.