# US 2017/0285585 A1 · Technologies for resolving moral conflicts during autonomous operation of a machine

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2017/0285585 A1
- Document ID: US20170285585A1
- Status: Published application
- Kind: A1
- Application number: 15/089,541
- Filed: 2016-04-02
- Published: 2017-10-05
- Earliest claimed: 2016-04-02
- Inventors: John C. Weast; Tobias M. Kohlenberg; Brian D. Johnson
- Web page: https://patentgrep.com/patent/US20170285585A1

## CPC

- G05B 13/028: {using expert systems only}
- B60W 30/09: Taking automatic action to avoid collision, e.g. braking and steering
- B60W 60/0015: {specially adapted for safety}
- G06N 5/046: Forward inferencing; Production systems
- G05B: CONTROL OR REGULATING SYSTEMS IN GENERAL; FUNCTIONAL ELEMENTS OF SUCH SYSTEMS; MONITORING OR TESTING ARRANGEMENTS FOR SUCH SYSTEMS OR ELEMENTS (systems for controlling or regulating non-electric variables G05D; systems for regulating electric or magnetic variables G05F; control devices or systems insofar as characterised by mechanical features only G05G)
- B60W: CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS

## Abstract

Technologies for controlling a machine include a compute system configured to control operation of the machine. The compute system is configured to detect a moral conflict related to the operation of the machine and determine operational choices for operation of the machine to resolve the moral conflict. The compute system also determines a moral agent likely to be affected by each operational choice and one or more moral rules applicable to the moral conflict. The moral agents may be weighted based on a set of weighting rules, which may vary based on geographical location and/or other criteria. Each moral rule defines a goal to be achieved by the operation of the machine. The compute system is further configured to select one of the operational choices to resolve the conflict based on the determined moral agents and the moral rules and control the machine to perform the selected operational choice.

## Claims

### Claim 1

A compute system to control operation of a machine, the compute system comprising:
a data storage to store (i) a moral agent database that includes a plurality of moral agents, (i) a weighting rule database that includes a plurality of weighting rules, wherein each weighting rule defines a weighting factor for a corresponding moral agent, and (iii) a moral rule database that includes a plurality of moral rules, wherein each moral rule defines a goal to be achieved by the operation of the machine;
a moral conflict detection module to (i) detect a moral conflict related to the operation of the machine and (ii) determine a plurality of operational choices to resolve the moral conflict, wherein each operational choice defines a corresponding operation of the machine;
a moral agent determination module to determine, for each defined operational choice, a moral agent from the moral agent database that is to be affected by the corresponding operational choice;
a moral agent weighting module to apply a weighting factor to each determined moral agent based on one or more weighting rules of the plurality of weighting rules;
a moral agent weighting module to determine one or more moral rules from the plurality of moral rules that are applicable to the moral conflict; and
a moral decision resolution module to select an operational choice from the plurality of operational choices to resolve the moral conflict based on the weighted moral agents and the determined moral rules.

### Claim 2 (depends on 1)

The compute system of claim 1, wherein the data storage is further to store an operational database that includes a plurality of operational rules and wherein the operational rules dictate the operation of the machine, and
wherein to detect the moral conflict comprises to detect a conflict between two or more operational rules of the plurality of operational rules.

### Claim 3 (depends on 1)

The compute system of claim 1, wherein to determine the moral agents comprises to:
determine, for each operational choice, one or more real-world entities that are to be affected by the corresponding operational choice; and
match each real-world entity with a corresponding moral agent defined in the moral agent database.

### Claim 4 (depends on 1)

The compute system of claim 1, wherein to determine the one or more moral rules applicable to the moral conflict comprises to determine one or more moral rules applicable to the conflict based on at least one moral agent.

### Claim 5 (depends on 1)

The compute system of claim 1, wherein to determine the one or more moral rules applicable to the moral conflict comprises to determine one or more moral rules applicable to the conflict based on at least one operational choice of the plurality of the operational choices.

### Claim 6 (depends on 1)

The compute system of claim 1, wherein to select the operational choice comprises to select an operational choice from the plurality of operational choices based on the weighting factor applied to each moral agent.

### Claim 7 (depends on 1)

The compute system of claim 1, wherein to select the operational choice comprises to select an operational choice from the plurality of operational choices based on a number of determined moral rules satisfied by the selected operational choice.

### Claim 8 (depends on 1)

The compute system of claim 1, wherein each moral rule includes an associated importance factor that indicates a level of importance of satisfying the corresponding moral rule relative to other moral rules, and
wherein to select the operational choice comprises to select an operational choice from the plurality of operational choices based on the importance factor of each determined moral rule.

### Claim 9 (depends on 1)

The compute system of claim 1, further comprising a machine control module to control the machine to perform the selected operational choice.

### Claim 10

A method for controlling a machine, the method comprising:
detecting, by a compute system controlling operation of the machine, a moral conflict related to the operation of the machine;
determining, by the compute device, a plurality of operational choices to resolve the moral conflict, wherein each operational choice defines a corresponding operation of the machine;
determining, by the compute system and for each defined operational choice, a moral agent from a moral agent database that is to be affected by the corresponding operational choice;
applying, by the compute system, a weighting factor to each determined moral agent based on one or more weighting rules, wherein each weighting rule defines a weighting factor for a corresponding moral agent;
determining, by the compute system, one or more moral rules applicable to the moral conflict, wherein each moral rule defines a goal to be achieved by the operation of the machine; and
selecting, by the compute system, an operational choice from the plurality of operational choices to resolve the moral conflict based on the weighted moral agents and the determined moral rules.

### Claim 11 (depends on 10)

The method of claim 10, wherein detecting the moral conflict comprises detecting a conflict between two or more operational rules of the machine, wherein the operational rules dictate the operation of the machine.

### Claim 12 (depends on 10)

The method of claim 10, wherein determining the moral agents comprises:
determining, for each operational choice, one or more real-world entities affected by the corresponding operational choice; and
matching each real-world entity with a corresponding moral agent defined in the moral agent database.

### Claim 13 (depends on 10)

The method of claim 10, wherein determining the one or more moral rules applicable to the moral conflict comprises determining one or more moral rules applicable to the conflict based on at least one moral agent or at least one operational choice of the plurality of the operational choices.

### Claim 14 (depends on 10)

The method of claim 10, wherein selecting the operational choice comprises selecting an operational choice from the plurality of operational choices based on the weighting factor applied to each moral agent.

### Claim 15 (depends on 10)

The method of claim 10, wherein selecting the operational choice comprises selecting an operational choice from the plurality of operational choices based on a number of determined moral rules satisfied by the selected operational choice.

### Claim 16 (depends on 10)

The method of claim 10, wherein each moral rule includes an associated importance factor that indicates a level of importance of satisfying the corresponding moral rule relative to other moral rules, and
wherein selecting the operational choice comprises selecting an operational choice from the plurality of operational choices based on the importance factor of each determined moral rule.

### Claim 17 (depends on 10)

The method of claim 10, further comprising controlling, by the compute system, the machine to perform the selected operational choice.

### Claim 18

One or more computer-readable storage media comprising a plurality of instructions that, when executed by a compute system, cause the compute system to:
detect a moral conflict related to the operation of the machine;
determine a plurality of operational choices to resolve the moral conflict, wherein each operational choice defines a corresponding operation of the machine;
determine, for each defined operational choice, a moral agent from a moral agent database that is to be affected by the corresponding operational choice;
apply a weighting factor to each determined moral agent based on one or more weighting rules, wherein each weighting rule defines a weighting factor for a corresponding moral agent;
determine one or more moral rules applicable to the moral conflict, wherein each moral rule defines a goal to be achieved by the operation of the machine; and
select an operational choice from the plurality of operational choices to resolve the moral conflict based on the weighted moral agents and the determined moral rules.

### Claim 19 (depends on 18)

The method of claim 18, wherein to detect the moral conflict comprises to detect a conflict between two or more operational rules of the machine, wherein the operational rules dictate the operation of the machine.

### Claim 20 (depends on 18)

The method of claim 18, wherein to determine the moral agents comprises to:
determine, for each operational choice, one or more real-world entities affected by the corresponding operational choice; and
match each real-world entity with a corresponding moral agent defined in the moral agent database.

### Claim 21 (depends on 18)

The method of claim 18, wherein to determine the one or more moral rules applicable to the moral conflict comprises to determine one or more moral rules applicable to the conflict based on at least one moral agent or at least one operational choice of the plurality of the operational choices.

### Claim 22 (depends on 18)

The method of claim 18, wherein to select the operational choice comprises to select an operational choice from the plurality of operational choices based on the weighting factor applied to each moral agent.

### Claim 23 (depends on 18)

The method of claim 18, wherein to select the operational choice comprises to select an operational choice from the plurality of operational choices based on a number of determined moral rules satisfied by the selected operational choice.

### Claim 24 (depends on 18)

The method of claim 18, wherein each moral rule includes an associated importance factor that indicates a level of importance of satisfying the corresponding moral rule relative to other moral rules, and
wherein to select the operational choice comprises to select an operational choice from the plurality of operational choices based on the importance factor of each determined moral rule.

### Claim 25 (depends on 18)

The method of claim 18, wherein the plurality of instructions that, when executed by the compute system, further cause the compute system to control the machine to perform the selected operational choice.
