# US 2021/0232857 A1 · Electronic device and controlling method of electronic device

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2021/0232857 A1
- Document ID: US20210232857A1
- Status: Published application
- Kind: A1
- Application number: 17/158,561
- Filed: 2021-01-26
- Published: 2021-07-29
- Earliest claimed: 2020-01-28
- Assignees: Samsung Electronics Co., Ltd.
- Applicants: Samsung Electronics Co., Ltd.
- Inventors: Hyunjoo JUNG; Chiyoun PARK; Ilgu KANG
- Web page: https://patentgrep.com/patent/US20210232857A1

## CPC

- G06N 3/084: Backpropagation, e.g. using gradient descent
- G06F 18/214: Generating training patterns; Bootstrap methods, e.g. bagging or boosting
- G06F 18/217: {Validation; Performance evaluation; Active pattern learning techniques}
- G06F 18/2178: {based on feedback of a supervisor}
- G06F 18/24765: {Rule-based classification}
- G06F 18/253: {of extracted features}
- G06F 18/41: {Interactive pattern learning with a human teacher}
- G06N 3/04: Architecture, e.g. interconnection topology
- G06N 3/042: Knowledge-based neural networks; Logical representations of neural networks
- G06N 3/045: Combinations of networks
- G06N 3/0464: Convolutional networks [CNN, ConvNet]
- G06N 3/08: Learning methods
- G06N 3/09: Supervised learning
- G06N 5/013: {Automatic theorem proving}
- G06N 3/044: Recurrent networks, e.g. Hopfield networks
- G06N 3/047: Probabilistic or stochastic networks
- G06N 5/046: Forward inferencing; Production systems
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)

## Abstract

A controlling method of an electronic device are provided. The controlling method includes obtaining, a first feature value corresponding to the input data, mapping information included in the first feature value to a symbolic value corresponding to each of a logically certain information and uncertain information, tweaking a gradient corresponding to the uncertain information, obtaining a second feature value in which the first feature value and the symbolic value are merged, obtaining temporary answer information including the certain information and the uncertain information, obtaining score information related to a level the temporary answer information matches a pre-defined logic rule, training the neural network model based on the tweaked gradient when the score information is less than a pre-set threshold value, and obtaining an output data on the input data based on the obtained second feature value when the score information is greater than or equal to the pre-set threshold value.

## Claims

### Claim 1

An electronic device configured to apply a logical constraint to a neural network model, the electronic device comprising:
a memory configured to store a neural network model, a symbolic value mapping module, a merge module, a temporary answer acquisition module, a rule check module, and a training determination module; and
a processor configured to control the electronic device by executing at least one instruction stored in the memory,
wherein the processor is configured to:
obtain, based on input data being input to the neural network model, a first feature value corresponding to the input data through a feature extraction module comprised in the neural network model,
input the first feature value to the symbolic value mapping module,
map information comprised in the first feature value to a symbolic value corresponding to each of a logically certain information and uncertain information,
tweak a gradient corresponding to the uncertain information through the symbolic value mapping module for an update on the uncertain information to be possible,
obtain a second feature value in which the first feature value and the symbolic value are merged by inputting the first feature value and the symbolic value to the merge module,
obtain temporary answer information comprising the certain information and the uncertain information by inputting the first feature value and the symbolic value to the temporary answer acquisition module,
obtain a score information related to a level the temporary answer information matches a pre-defined logic rule by inputting the temporary answer information to the rule check module,
train, through the training determination module, the neural network model based on the tweaked gradient based on the score information being less than a pre-set threshold value, and
obtain an output data on the input data by inputting the obtained second feature value to a classification module comprised in the neural network model based on the score information being greater than or equal to the pre-set threshold value.

### Claim 2 (depends on 1)

The electronic device of claim 1, wherein the processor is further configured to:
train the neural network model for the uncertain information to be updated based on the tweaked gradient, and
obtain, based on the uncertain information being updated and the score information being greater than or equal to a pre-set threshold value, the output data by inputting the second feature value updated through the trained neural network model to the classification module.

### Claim 3 (depends on 1)

The electronic device of claim 1, wherein the processor is further configured to:
tweak, through the symbolic value mapping module, a gradient corresponding to the uncertain information to a value other than 0 for an update on the uncertain information to be possible, and
maintain a gradient corresponding to the certain information to 0 for an update on the certain information to not be carried out.

### Claim 4 (depends on 1)

The electronic device of claim 1,
wherein the symbolic value mapping module comprises:
a condition symbolic value mapping module configured to map the first feature value to a condition symbolic value corresponding to a pre-defined condition, and
an action symbolic value mapping module configured to map the first feature value to an action symbolic value corresponding to a pre-defined action, and
wherein the processor is further configured to:
map, through the condition symbolic value mapping module, information comprised in the first feature value to a different condition symbolic value with respect to each of the certain information and the uncertain information related to the condition, and
map, through the action symbolic value mapping module, information comprised in the first feature value to a different action symbolic value with respect to each of the certain information and the uncertain information related to the action.

### Claim 5 (depends on 4)

The electronic device of claim 4, wherein the processor is further configured to:
input the first feature value, the condition symbolic value, and the action symbolic value to the temporary answer acquisition module, and
obtain a temporary answer information comprising information on a condition comprised in the first feature value and information on an action corresponding to information on a condition comprised in the first feature value.

### Claim 6 (depends on 1)

The electronic device of claim 1, wherein the training determination module is configured to determine training of the neural network model based on first score information related to a level matching the logic rule and second score information related to a level the output data matches a learning data of the neural network model.

### Claim 7 (depends on 1)

The electronic device of claim 1, wherein the symbolic value corresponding to the certain information comprises a symbolic value corresponding to when information comprised in the first feature value is logically true and a symbolic value corresponding to when information comprised in the first feature value is logically false.

### Claim 8

A controlling method of an electronic device configured to apply a logical constraint to a neural network model, the method comprising:
obtaining, based on input data being input to the neural network model, a first feature value corresponding to the input data through a feature extraction module comprised in the neural network model;
mapping, by inputting the first feature value to a symbolic value mapping module comprised in the electronic device, information comprised in the first feature value to a symbolic value corresponding to each of a logically certain information and uncertain information;
tweaking a gradient corresponding to the uncertain information through the symbolic value mapping module for an update on the uncertain information to be possible;
obtaining a second feature value in which the first feature value and the symbolic value are merged by inputting the first feature value and the symbolic value to a merge module comprised in the electronic device;
obtaining temporary answer information comprising the certain information and the uncertain information by inputting the first feature value and the symbolic value to a temporary answer acquisition module comprised in the electronic device;
obtaining a score information related to a level the temporary answer information matches a pre-defined logic rule by inputting the temporary answer information to a rule check module comprised in the electronic device; and
obtaining, through a training determination module comprised in the electronic device, an output data on the input data by training the neural network model based on the tweaked gradient based on the score information being less than a pre-set threshold value, and inputting the obtained second feature value to a classification module comprised in the neural network model based on the score information being greater than or equal to the pre-set threshold value.

### Claim 9 (depends on 8)

The method of claim 8, wherein the controlling method of the electronic device further comprises:
training the neural network model for the uncertain information to be updated based on the tweaked gradient; and
obtaining, based on the uncertain information being updated and the score information being greater than or equal to a pre-set threshold value, the output data by inputting the second feature value updated through the trained neural network model to the classification module.

### Claim 10 (depends on 8)

The method of claim 8, wherein the tweaking the gradient comprises:
tweaking, through the symbolic value mapping module, a gradient corresponding to the uncertain information to a value other than 0 for an update on the uncertain information to be possible, and
maintaining a gradient corresponding to the certain information to 0 for an update on the certain information to not be carried out.

### Claim 11 (depends on 8)

The method of claim 8,
wherein the symbolic value mapping module comprises:
a condition symbolic value mapping module configured to map the first feature value to a condition symbolic value corresponding to a pre-defined condition, and
an action symbolic value mapping module configured to map the first feature value to an action symbolic value corresponding to a pre-defined action, and
wherein the mapping of the symbolic value comprises:
mapping, through the condition symbolic value mapping module, information comprised in the first feature value to a different condition symbolic value with respect to each of the certain information and the uncertain information related to the condition, and
mapping, through the action symbolic value mapping module, information comprised in the first feature value to a different action symbolic value with respect to each of the certain information and the uncertain information related to the action.

### Claim 12 (depends on 11)

The method of claim 11, wherein the obtaining of the temporary answer information comprises:
inputting the first feature value, the condition symbolic value, and the action symbolic value to the temporary answer acquisition module, and
obtaining temporary answer information comprising information on a condition comprised in the first feature value and information on an action corresponding to information on a condition comprised in the first feature value.

### Claim 13 (depends on 8)

The method of claim 8, wherein the training determination module is configured to determine training of the neural network model based on first score information related to a level matching the logic rule and second score information related to a level the output data matches a learning data of the neural network model.

### Claim 14 (depends on 8)

The method of claim 8, wherein the symbolic value corresponding to the certain information comprises:
a symbolic value corresponding to when information comprised in the first feature value is logically true, and
a symbolic value corresponding to when information comprised in the first feature value is logically false.

### Claim 15

A computer readable recording medium comprising a program which executes a controlling method of an electronic device, the controlling method of the electronic device comprising:
obtaining, based on input data being input to a neural network model, a first feature value corresponding to the input data through a feature extraction module comprised in the neural network model;
mapping, by inputting the first feature value to a symbolic value mapping module comprised in the electronic device, information comprised in the first feature value to a symbolic value corresponding to each of a logically certain information and uncertain information;
tweaking a gradient corresponding to the uncertain information through the symbolic value mapping module for an update on the uncertain information to be possible;
obtaining a second feature value in which the first feature value and the symbolic value are merged by inputting the first feature value and the symbolic value to a merge module comprised in the electronic device;
obtaining temporary answer information comprising the certain information and the uncertain information by inputting the first feature value and the symbolic value to a temporary answer acquisition module comprised in the electronic device;
obtaining score information related to a level the temporary answer information matches a pre-defined logic rule by inputting the temporary answer information to a rule check module comprised in the electronic device; and
obtaining, through a training determination module comprised in the electronic device, an output data on the input data by training the neural network model based on the tweaked gradient based on the score information being less than a pre-set threshold value, and inputting the obtained second feature value to a classification module comprised in the neural network model based on the score information being greater than or equal to the pre-set threshold value.

### Claim 16 (depends on 15)

The computer readable recording medium of claim 15, wherein the symbolic value mapping module comprises:
a feature conversion module configured to map the information included in the first feature value to the symbolic value, and
a gradient tweaking module configured to tweak the gradient corresponding to logically uncertain information from among the information included in the first feature value.

### Claim 17 (depends on 15)

The computer readable recording medium of claim 15, wherein the training determination module is configured to determine training of the neural network model based on first score information related to a level matching the logic rule and second score information related to a level the output data matches a learning data of the neural network model.

### Claim 18 (depends on 17)

The computer readable recording medium of claim 17, wherein the training determination module is configured to determine the training of the neural network model based on comprehensive score information that is a sum of the first score information and the second score information.

### Claim 19 (depends on 17)

The computer readable recording medium of claim 17, wherein the training determination module is configured to determine the training of the neural network model based on comprehensive score information that is a product of the first score information and the second score information.
