# US 12,762,415 B2 · Method for evaluating performance and system thereof > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 12,762,415 B2 - Document ID: US12762415B2 - Status: Granted - Kind: B2 - Application number: 18/213,478 - Filed: 2023-06-23 - Published: 2024-03-14 - Granted: 2026-10-13 - Earliest claimed: 2022-09-13 - Assignees: Samsung Sds Co., Ltd. - Applicants: Samsung Sds Co., Ltd. - Inventors: Joon Ho Lee; Han Kyu Moon; Jae Oh Woo - Web page: https://patentgrep.com/patent/US12762415B2 ## CPC - G06N 3/088: Non-supervised learning, e.g. competitive learning - G06N 3/094: Adversarial learning - G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS ## Abstract Provided are a method for evaluating performance and a system thereof. The method according to some embodiments may include obtaining a first model trained using a labeled dataset, obtaining a second model built by performing unsupervised domain adaptation on the first model, generating pseudo labels for an evaluation dataset using the second model, wherein the evaluation dataset is an unlabeled dataset, and evaluating performance of the first model using the pseudo labels. ## Claims ### Claim 1 A method for evaluating performance, the method being performed by at least one computing device and comprising: obtaining a first model trained using a labeled dataset; obtaining a second model built by performing unsupervised domain adaptation on the first model; generating pseudo labels for an evaluation dataset using the second model, wherein the evaluation dataset is an unlabeled dataset; and evaluating performance of the first model using the pseudo labels, wherein the generating of the pseudo labels comprises: deriving adversarial noise for a data sample belonging to the evaluation dataset; generating a noisy sample by reflecting the derived adversarial noise in the data sample; and generating a pseudo label for the data sample based on a predicted label of the noisy sample obtained through the second model. ### Claim 2 (depends on 1) The method of claim 1, wherein the unsupervised domain adaptation and the generating of the pseudo labels are performed without using the labeled dataset. ### Claim 3 (depends on 1) The method of claim 1, wherein the deriving of the adversarial noise comprises: obtaining a first predicted label for the data sample through the second model; generating a noisy sample by reflecting a value of a noise parameter in the data sample; obtaining a second predicted label for the noisy sample through the second model; updating the value of the noise parameter in a direction to increase a difference between the first predicted label and the second predicted label; and calculating adversarial noise for the data sample based on the updated value of the noise parameter. ### Claim 4 (depends on 3) The method of claim 3, wherein in the updating of the value of the noise parameter, the value of the noise parameter is updated within a range that satisfies a preset size constraint condition. ### Claim 5 (depends on 3) The method of claim 3, wherein the difference between the first predicted label and the second predicted label is calculated based on Kullback-Leibler divergence. ### Claim 6 (depends on 1) The method of claim 1, wherein the noisy sample comprises a first noisy sample based on a first adversarial noise and a second noisy sample based on a second adversarial noise, wherein the first adversarial noise and the second adversarial noise are respectively derived from noise parameters having different initial values, and wherein the generating of the pseudo label for the data sample comprises generating the pseudo label for the data sample by aggregating a predicted label of the first noisy sample and a predicted label of the second noisy sample. ### Claim 7 (depends on 1) The method of claim 1, wherein the evaluating of the performance of the first model comprises: predicting labels of the evaluation dataset through the first model; and evaluating the performance of the first model by comparing the pseudo labels and the predicted labels. ### Claim 8 (depends on 1) The method of claim 1, wherein the labeled dataset is a dataset of a source domain, the evaluation dataset is a dataset of a target domain, and the method further comprising: obtaining a third model trained using a labeled dataset of the source domain; evaluating performance of the third model using the pseudo labels; and selecting a model to be applied to the target domain from among the first model and the third model based on results of evaluating the performance of the first model and evaluating the performance of the third model. ### Claim 9 (depends on 1) The method of claim 1, wherein the labeled dataset is a dataset of a first source domain, the evaluation dataset is a dataset of a target domain, and the method further comprising: obtaining a third model trained using a labeled dataset of a second source domain; evaluating performance of the third model using the pseudo labels; and selecting a model to be applied to the target domain from among the first model and the third model based on results of evaluating the performance of the first model and evaluating the performance of the third model. ### Claim 10 (depends on 1) The method of claim 1, wherein the evaluation dataset is a more recently generated dataset than the labeled dataset, and the method further comprising determining that the first model needs to be updated in response to a determination that the evaluated performance does not satisfy a predetermined condition. ### Claim 11 A system for evaluating performance, the system comprising: a memory configured to store one or more instructions; and one or more processors configured to execute the one or more stored instructions to perform: obtaining a first model trained using a labeled dataset; obtaining a second model built by performing unsupervised domain adaptation on the first model; generating pseudo labels for an evaluation dataset using the second model, wherein the evaluation dataset is an unlabeled dataset; and evaluating performance of the first model using the pseudo labels, wherein the generating of the pseudo labels comprises: deriving adversarial noise for a data sample belonging to the evaluation dataset; generating a noisy sample by reflecting the derived adversarial noise in the data sample; and generating a pseudo label for the data sample based on a predicted label of the noisy sample obtained through the second model. ### Claim 12 (depends on 11) The system of claim 11, wherein the unsupervised domain adaptation and the generating of the pseudo labels are performed without using the labeled dataset. ### Claim 13 A non-transitory computer-readable recording medium storing computer program executable by at least one processor to perform: obtaining a first model trained using a labeled dataset; obtaining a second model built by performing unsupervised domain adaptation on the first model; generating pseudo labels for an evaluation dataset using the second model, wherein the evaluation dataset is an unlabeled dataset; and evaluating performance of the first model using the pseudo labels, wherein the generating of the pseudo labels comprises: deriving adversarial noise for a data sample belonging to the evaluation dataset; generating a noisy sample by reflecting the derived adversarial noise in the data sample; and generating a pseudo label for the data sample based on a predicted label of the noisy sample obtained through the second model.