# US 2020/0401855 A1 · Apparatus and method with classification > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 2020/0401855 A1 - Document ID: US20200401855A1 - Status: Published application - Kind: A1 - Application number: 16/671,639 - Filed: 2019-11-01 - Published: 2020-12-24 - Earliest claimed: 2019-06-19 - Assignees: Samsung Electronics Co., Ltd. - Applicants: Samsung Electronics Co., Ltd. - Inventors: Young-Seok KIM; Hwidong NA; Seongmin OK; Min-Joong LEE - Web page: https://patentgrep.com/patent/US20200401855A1 ## CPC - G06N 3/08: Learning methods - G06F 18/214: Generating training patterns; Bootstrap methods, e.g. bagging or boosting - G06F 18/22: Matching criteria, e.g. proximity measures - G06F 18/2321: using statistics or function optimisation, e.g. modelling of probability density functions - G06F 18/2415: based on parametric or probabilistic models, e.g. based on likelihood ratio or false acceptance rate versus a false rejection rate - G06F 18/2431: Multiple classes - G06F 18/24317: {Piecewise classification, i.e. whereby each classification requires several discriminant rules} - G06F 18/254: {of classification results, e.g. of results related to same input data} - G06N 3/045: Combinations of networks - G06N 3/0499: Feedforward networks - G06N 3/09: Supervised learning - G06V 10/761: {Proximity, similarity or dissimilarity measures} - G06V 10/764: using classification, e.g. of video objects - G06V 10/809: {of classification results, e.g. where the classifiers operate on the same input data} - G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) - G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING ## Abstract A processor-implemented classification method includes: determining a first probability vector including a first probability, for each of a plurality of classes, resulting from a classification of an input with respect to the classes; determining, based on the determined first probability vector, whether one or more of the classes represented in the first probability vector are confusing classes; adjusting, in response to one or more of the classes being the confusing classes, the determined first probability vector based on a first probability of each of the confusing classes and a maximum value of the first probabilities; determining a second probability vector including a second probability, for each of the classes, resulting from another classification of the input with respect to the classes; and performing classification on the input based on a result of a comparison between the determined second probability vector and the adjusted first probability vector. ## Claims ### Claim 1 A processor-implemented classification method, the method comprising: determining a first probability vector including a first probability, for each of a plurality of classes, resulting from a classification of an input with respect to the classes; determining, based on the determined first probability vector, whether one or more of the classes represented in the first probability vector are confusing classes; adjusting, in response to one or more of the classes being the confusing classes, the determined first probability vector based on a first probability of each of the confusing classes and a maximum value of the first probabilities; determining a second probability vector including a second probability, for each of the classes, resulting from another classification of the input with respect to the classes; and performing classification on the input based on a result of a comparison between the determined second probability vector and the adjusted first probability vector. ### Claim 2 (depends on 1) The method of claim 1, wherein the determining of whether one or more of the classes represented in the first probability vector are the confusing classes comprises: determining whether the maximum value is less than or equal to a first threshold; and in response to determining that the maximum value is less than or equal to the first threshold, determining whether one or more of the classes represented in the first probability vector are the confusing classes. ### Claim 3 (depends on 2) The method of claim 2, wherein the confusing classes are classes corresponding to first probabilities of the first feature vector that are greater than zero, other than a class corresponding to the maximum value. ### Claim 4 (depends on 1) The method of claim 1, wherein the determining of whether one or more of the classes represented in the first probability vector are the confusing classes comprises: determining whether one or more of the first probabilities of the first feature vector, other than the maximum value, are greater than or equal to a second threshold; and in response to determining that one or more of the first probabilities other than the maximum value are greater than or equal to a second threshold, determining that one or more of the classes represented in the first probability vector are the confusing classes. ### Claim 5 (depends on 1) The method of claim 1, wherein the adjusting of the determined first probability vector comprises: determining a difference between a predetermined value and the maximum value; determining, for each of the confusing classes, a ratio of the first probability of the confusing class to the determined difference; and adjusting, for each of the confusing classes, the first probability of the confusing class based on the respective ratio. ### Claim 6 (depends on 1) The method of claim 1, wherein the performing of the classification comprises: determining a similarity between the determined second probability vector and the adjusted first probability vector; and classifying the input as a class having the maximum value in response to the determined similarity being greater than or equal to a second threshold. ### Claim 7 (depends on 6) The method of claim 6, further comprising performing a function of a user terminal corresponding to the class having the maximum value, in response to the classifying of the input as the class having the maximum value. ### Claim 8 (depends on 7) The method of claim 7, wherein the function includes any one or any combination of any two or more of transmitting function, a playback function, a storing function, a removing function, and a correcting function of the user terminal. ### Claim 9 (depends on 6) The method of claim 6, further comprising: determining the input to be an out-of-domain (OOD) of the classification apparatus in response to the determined similarity being less than or equal to the second threshold. ### Claim 10 (depends on 1) The method of claim 1, wherein the determining of the first probability vector comprises determining the first probability vector using a first classification model, and the determining of the second probability vector comprises determining the second probability vector using a second classification model. ### Claim 11 (depends on 10) The method of claim 10, wherein the first classification model and the second classification model share at least a portion of weights. ### Claim 12 (depends on 1) The operation method of claim 1, wherein the input includes any one or any combination of any two or more of a text, a speech, and an image. ### Claim 13 (depends on 1) A non-transitory computer-readable storage medium storing instructions that, when executed by one or more processors, cause the one or more processors to perform the method of claim 1. ### Claim 14 A processor-implemented classification training method, the method comprising: determining, using a trained first classification model, a first probability vector including a first probability, for each of a plurality of classes, resulting from a classification of an input with respect to the classes, wherein one of the classes corresponds to a truth class of the input; determining, based on the determined first probability vector, whether one or more of the classes represented in the first probability vector are confusing classes; adjusting, in response to one or more the classes being the confusing classes, the determined first probability vector based on a first probability of each of the confusing classes and a first probability of the truth class; and training a second classification model based on the adjusted first probability vector and the input. ### Claim 15 (depends on 14) The method of claim 14, wherein the determining of whether one or more of the classes represented in the first probability vector are the confusing classes comprises: determining whether the first probability of the truth class is less than or equal to a first threshold; and in response to determining that the first probability of the truth class is less than or equal to the first threshold, determining whether one or more of the classes represented in the first probability vector are the confusing classes. ### Claim 16 (depends on 14) The method of claim 14, wherein the adjusting of the determined first probability vector comprises: determining a difference between a predetermined value and the first probability of the truth class; determining, for each of the confusing classes, a ratio of the first probability of the confusing class to the determined difference; and adjusting, for each of the confusing classes, the first probability of the confusing class based on the respective ratio. ### Claim 17 (depends on 14) The method of claim 14, wherein the trained first classification model is separate from the trained second classification model. ### Claim 18 (depends on 14) The method of claim 14, wherein the trained first classification model and the trained second classification model share at least a portion of weights. ### Claim 19 A classification apparatus comprising: one or more processors configured to: determine a first probability vector including a first probability, for each of a plurality of classes, resulting from a classification of an input with respect to the classes; determine, based on the determined first probability vector, whether one or more of the classes represented in the first probability vector are confusing classes; adjust, in response to one or more of the classes being the confusing classes, the determined first probability vector based on a first probability of each of the confusing classes and a maximum value of the first probabilities; determine a second probability vector including a second probability, for each of the classes, resulting from another classification of the input with respect to the classes; and perform classification on the input based on a result of a comparison between the determined second probability vector and the adjusted first probability vector. ### Claim 20 (depends on 19) The apparatus of claim 19, wherein, for the determining of whether one or more of the classes represented in the first probability vector are the confusing classes, the one or more processors are configured to: determine whether the maximum value is less than or equal to a first threshold; and in response to determining that the maximum value is less than or equal to the first threshold, determine whether one or more of the classes represented in the first probability vector are the confusing classes. ### Claim 21 (depends on 19) The apparatus of claim 19, wherein, for the adjusting of the determined first probability vector, the one or more processors are configured to: determine a difference between a predetermined value and the maximum value; determine, for each of the confusing classes, a ratio of the first probability of the confusing class to the determined difference; and adjust, for each of the confusing classes, the first probability of the confusing class based on the respective ratio. ### Claim 22 (depends on 19) The apparatus of claim 19, wherein, for the performing of the classification, the one or more processors are configured to: determine a similarity between the determined second probability vector and the adjusted first probability vector; and classify the input as a class having the maximum value in response to the determined similarity being greater than or equal to a second threshold. ### Claim 23 (depends on 22) The apparatus of claim 22, wherein the one or more processors are configured to determine the input to be an out-of-domain (OOD) of the classification apparatus in response to the determined similarity being less than or equal to the second threshold. ### Claim 24 (depends on 19) The apparatus of claim 19, wherein the input includes any one or any combination of any two or more of a text, a speech, and an image. ### Claim 25 (depends on 19) The apparatus of claim 19, further comprising a memory storing instructions that, when executed by the one or more processors, configure the one or more processors to perform the determining of the first probability vector, the determining of whether the classes include the confusing classes, the adjusting of the determined first probability vector, the determining of the second probability vector, and the performing of the classification. ### Claim 26 A processor-implemented classification method, the method comprising: determining a first probability that an input corresponds to a class and another first probability that the input corresponds to another class; increasing the other first probability, in response to the first probability being a maximum of the first probabilities and being below a threshold; determining a second probability that the input corresponds to the other class; and classifying the input as corresponding to the class in response to a result of a comparison between the increased first probability to the second probability being below another threshold.