# US 2023/0274152 A1 · System and method for model configuration selection

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2023/0274152 A1
- Document ID: US20230274152A1
- Status: Published application
- Kind: A1
- Application number: 18/174,657
- Filed: 2023-02-27
- Published: 2023-08-31
- Earliest claimed: 2022-02-28
- Assignees: Saferide Technologies Ltd.
- Applicants: Saferide Technologies Ltd.
- Inventors: Sofiia KOVALETS; Stanislav BARABANOV; Yuval SHALEV; Alexander APARTSIN
- Web page: https://patentgrep.com/patent/US20230274152A1

## CPC

- G06N 3/088: Non-supervised learning, e.g. competitive learning
- G06N 3/0895: Weakly supervised learning, e.g. semi-supervised or self-supervised learning
- G06N 3/10: Interfaces, programming languages or software development kits, e.g. for simulating neural networks
- G06N 3/04: Architecture, e.g. interconnection topology
- G06N 3/0985: Hyperparameter optimisation; Meta-learning; Learning-to-learn
- G06N 20/00: Machine learning
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS

## Abstract

A model configuration selection system, the model configuration selection system comprising a processing circuitry configured to: (A) obtain: (a) one or more model configurations, each model configuration includes a set of parameters utilized to generate respective models, and (b) a training data-set comprising a plurality of unlabeled records, each unlabeled record including a collection of features describing a given state of a physical entity; (B) cluster the training data-set into two or more training data-set clusters using a clustering algorithm; (C) label (a) the unlabeled records of a subset of the training data-set clusters with a synthetic normal label, giving rise to a normal training data-set, and (b) the unlabeled records of the training data-set clusters not included in the subset with a synthetic abnormal label; (D) train, for each model configuration, using the normal training data-set, a corresponding model utilizing the corresponding set of parameters, each model capable of receiving the unlabeled records, and determining, for each of the unlabeled records, a corresponding normal label or abnormal label, wherein the normal label being indicative of conformity of the respective unlabeled record with an allowed state of the physical entity and the abnormal label being indicative of conformity of the respective unlabeled record with a disallowed state of the physical entity; (E) determine, for each model, a score, associated with an ability of the corresponding model to determine labels to the unlabeled records of the training data-set in accordance with the synthetic normal labels and with the synthetic abnormal labels; and (F) perform an action, based on the scores.

## Claims

### Claim 1

A model configuration selection system, the model configuration selection system comprising a processing circuitry configured to:
(A) obtain: (a) one or more model configurations, each model configuration includes a set of parameters utilized to generate respective models, and (b) a training data-set comprising a plurality of unlabeled records, each unlabeled record including a collection of features describing a given state of a physical entity;
(B) cluster the training data-set into two or more training data-set clusters using a clustering algorithm;
(C) label (a) the unlabeled records of a subset of the training data-set clusters with a synthetic normal label, giving rise to a normal training data-set, and (b) the unlabeled records of the training data-set clusters not included in the subset with a synthetic abnormal label, wherein the subset of the training data-set clusters is selected randomly;
(D) train, for each model configuration, using the normal training data-set, a corresponding model utilizing the corresponding set of parameters, each model capable of receiving the unlabeled records, and determining, for each of the unlabeled records, a corresponding normal label or abnormal label, wherein the normal label being indicative of conformity of the respective unlabeled record with an allowed state of the physical entity and the abnormal label being indicative of conformity of the respective unlabeled record with a disallowed state of the physical entity;
(E) determine, for each model, a score, associated with an ability of the corresponding model to determine labels to the unlabeled records of the training data-set in accordance with the synthetic normal labels and with the synthetic abnormal labels; and
(F) perform an action, based on the scores, wherein the action is changing the model configuration of the respective model by changing at least one parameter of the set of parameters, upon a second overall score calculated for a corresponding model configuration, based on the scores of the respective model is a below a second threshold, and repeating steps (D) to (F) until the second overall score is equal or above the second threshold.

### Claim 2 (depends on 1)

The model configuration selection system of claim 1, wherein the action includes also one or more of:
(a) determine, for each of the model configurations and based on the scores of the respective models, if a first overall score calculated for a corresponding model configuration is above a first threshold, or
(b) rank the model configurations based on the scores of the corresponding models .

### Claim 3 (depends on 1)

The model configuration selection system of claim 1, wherein the processing circuitry is further configured to reduce a dimension of the training data-set before clustering the training data-set into two or more training data-set clusters.

### Claim 4 (depends on 1)

The model configuration selection system of claim 1, wherein the processing circuitry is further configured to:
label each of the training data-set clusters with a synthetic unique cluster label, giving raise to uniquely labeled training data-set clusters;
train utilizing the uniquely labeled training data-set clusters, a classifier model, capable of receiving the unlabeled records, and determining, for each of the unlabeled records, a corresponding synthetic unique cluster label;
determine, for each model, an explainability score, associated with a similarity between a first explanation of a first output of the model on the unlabeled records of the training data-set and a second explanation of a second output of the classifier model on the unlabeled records of the training data-set; and
wherein the score of each model is based also on the corresponding explainability score.

### Claim 5 (depends on 1)

The model configuration selection system of claim 1, wherein the clustering algorithm is one or more of the following: dbscan, k-means, or constrained k-means.

### Claim 6

(canceled)

### Claim 7 (depends on 1)

The model configuration selection system of claim 1, wherein the model is one of: a change point detection model, or an anomaly detection model.

### Claim 8 (depends on 1)

The model configuration selection system of claim 1, wherein the training data-set is one or more of: a static data-set, or a time-series data-set.

### Claim 9 (depends on 1)

The model configuration selection system of claim 1, wherein the model configuration selection system is an unsupervised model configuration selection system.

### Claim 10 (depends on 1)

The system of claim 1, wherein the physical entity is a vehicle.

### Claim 11

A method comprising:
(A) obtaining, by a processing circuitry: (a) one or more model configurations, each model configuration includes a set of parameters utilized to generate respective models, and (b) a training data-set comprising a plurality of unlabeled records, each unlabeled record including a collection of features describing a given state of a physical entity;
(B) clustering, by the processing circuitry, the training data-set into two or more training data-set clusters using a clustering algorithm;
(C) labeling, by the processing circuitry, (a) the unlabeled records of a subset of the training data-set clusters with a synthetic normal label, giving rise to a normal training data-set, and (b) the unlabeled records of the training data-set clusters not included in the subset with a synthetic abnormal label, wherein the subset of the training data-set clusters is selected randomly;
(D) training, by the processing circuitry, for each model configuration, using the normal training data-set, a corresponding model utilizing the corresponding set of parameters, each model capable of receiving the unlabeled records, and determining, for each of the unlabeled records, a corresponding normal label or abnormal label, wherein the normal label being indicative of conformity of the respective unlabeled record with an allowed state of the physical entity and the abnormal label being indicative of conformity of the respective unlabeled record with a disallowed state of the physical entity;
(E) determining, by the processing circuitry, for each model, a score, associated with an ability of the corresponding model to determine labels to the unlabeled records of the training data-set in accordance with the synthetic normal labels and with the synthetic abnormal labels; and
(F) performing, by the processing circuitry, an action, based on the scores, wherein the action is changing the model configuration of the respective model by changing at least one parameter of the set of parameters, upon a second overall score calculated for a corresponding model configuration, based on the scores of the respective model is a below a second threshold, and repeating steps (D) to (F) until the second overall score is equal or above the second threshold.

### Claim 12 (depends on 11)

The method of claim 11, wherein the action includes also one or more of:
(a) determine, for each of the model configurations and based on the scores of the respective models, if a first overall score calculated for a corresponding model configuration is above a first threshold, or
(b) rank the model configurations based on the scores of the corresponding models .

### Claim 13 (depends on 11)

The method of claim 11, further comprising: reducing, by the processing circuitry, a dimension of the training data-set before clustering the training data-set into two or more training data-set clusters.

### Claim 14 (depends on 11)

The method of claim 11, further comprising:
labeling, by the processing circuitry, each of the training data-set clusters with a synthetic unique cluster label, giving raise to uniquely labeled training data-set clusters;
training, by the processing circuitry, utilizing the uniquely labeled training data-set clusters, a classifier model, capable of receiving the unlabeled records, and determining, for each of the unlabeled records, a corresponding synthetic unique cluster label;
determining, by the processing circuitry, for each model, an explainability score, associated with a similarity between a first explanation of a first output of the model on the unlabeled records of the training data-set and a second explanation of a second output of the classifier model on the unlabeled records of the training data-set; and
wherein the score of each model is based also on the corresponding explainability score.

### Claim 15 (depends on 11)

The method of claim 11, wherein the clustering algorithm is one or more of the following: dbscan, k-means, or constrained k-means.

### Claim 16

(canceled)

### Claim 17 (depends on 11)

The method of claim 11, wherein the model is one of: a change point detection model, or an anomaly detection model.

### Claim 18 (depends on 11)

The method of claim 11, wherein the model configuration selection system is an unsupervised model configuration selection system.

### Claim 19 (depends on 11)

The method of claim 11, wherein the physical entity is a vehicle.

### Claim 20

A non-transitory computer readable storage medium having computer readable program code embodied therewith, the computer readable program code, executable by processing circuitry of a computer to perform a method comprising:
(A) obtaining, by a processing circuitry: (a) one or more model configurations, each model configuration includes a set of parameters utilized to generate respective models, and (b) a training data-set comprising a plurality of unlabeled records, each unlabeled record including a collection of features describing a given state of a physical entity;
(B) clustering, by the processing circuitry, the training data-set into two or more training data-set clusters using a clustering algorithm;
(C) labeling, by the processing circuitry, (a) the unlabeled records of a subset of the training data-set clusters with a synthetic normal label, giving rise to a normal training data-set, and (b) the unlabeled records of the training data-set clusters not included in the subset with a synthetic abnormal label, wherein the subset of the training data-set clusters is selected randomly;
(D) training, by the processing circuitry, for each model configuration, using the normal training data-set, a corresponding model utilizing the corresponding set of parameters, each model capable of receiving the unlabeled records, and determining, for each of the unlabeled records, a corresponding normal label or abnormal label, wherein the normal label being indicative of conformity of the respective unlabeled record with an allowed state of the physical entity and the abnormal label being indicative of conformity of the respective unlabeled record with a disallowed state of the physical entity;
(E) determining, by the processing circuitry, for each model, a score, associated with an ability of the corresponding model to determine labels to the unlabeled records of the training data-set in accordance with the synthetic normal labels and with the synthetic abnormal labels; and
(F) performing, by the processing circuitry, an action, based on the score, wherein the action is changing the model configuration of the respective model by changing at least one parameter of the set of parameters, upon a second overall score calculated for a corresponding model configuration, based on the scores of the respective model is a below a second threshold, and repeating steps (D) to (F) until the second overall score is equal or above the second threshold.
