# US 2024/0402732 A1 · Control device, non-transitory storage medium and control method

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2024/0402732 A1
- Document ID: US20240402732A1
- Status: Published application
- Kind: A1
- Application number: 18/677,889
- Filed: 2024-05-30
- Published: 2024-12-05
- Earliest claimed: 2023-06-05
- Assignees: Toyota Jidosha Kabushiki Kaisha
- Applicants: Toyota Jidosha Kabushiki Kaisha
- Inventors: Yuki TAKAHASHI; Masateru UDATE
- Web page: https://patentgrep.com/patent/US20240402732A1

## CPC

- G05D 1/81: Handing over between on-board automatic and on-board manual control
- G05D 1/86: Monitoring the performance of the system, e.g. alarm or diagnosis modules
- B60W 50/045: {Monitoring control system parameters}
- B60W 60/0011: {involving control alternatives for a single driving scenario, e.g. planning several paths to avoid obstacles}
- B60W 60/0016: {of the vehicle or its occupants}
- G05D 1/617: Safety or protection, e.g. defining protection zones around obstacles or avoiding hazards (arrangements for controlling the position or course of two or more vehicles for avoiding collisions therebetween G05D1/693; arrangements for reacting to or preventing system or operator failure G05D1/80)
- B60W 2050/0028: {Mathematical models, e.g. for simulation}
- B60W 2050/0292: {Fail-safe or redundant systems, e.g. limp-home or backup systems}
- G05D 2101/15: using machine learning, e.g. neural networks
- G05D 2107/13: Spaces reserved for vehicle traffic, e.g. roads, regulated airspace or regulated waters
- G05D 2109/10: Land vehicles
- G05D: SYSTEMS FOR CONTROLLING OR REGULATING NON-ELECTRIC VARIABLES
- B60W: CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT

## Abstract

A control device tries derivation of a first control command using a first control model, and controls, if the derivation of the first control command by the first control model is normal, a mobile body according to the derived first control command. If an abnormality occurs in the derivation of the first control command by the first control model, the control device controls the mobile body according to a second control command derived from a second control model or a third control command originating from a manual operation by a user. The second control model is configured to be capable of normally deriving the control command in at least a part of scenes of an abnormality occurring in derivation of the control command by the first control model.

## Claims

### Claim 1

A control device comprising:
a storage configured to store a first control model and a second control model; and
a processor; wherein
the second control model is configured to be capable of normally deriving a control command in at least a part of scenes of an abnormality occurring in derivation of a control command by the first control model, by being different from the first control model in at least any of a type and input data; and
the processor is configured to execute:
trying derivation of a first control command using the first control model;
controlling, if the derivation of the first control command by the first control model is normal, movement of a mobile body according to the derived first control command; and
controlling, if an abnormality occurs in the derivation of the first control command by the first control model, the movement of the mobile body according to a second control command derived by the second control model or a third control command originating from a manual operation by a user.

### Claim 2 (depends on 1)

The control device according to claim 1, wherein
the first control model is configured with a trained machine learning model; and
the second control model is configured with a rule-based model.

### Claim 3 (depends on 1)

The control device according to claim 1, wherein
the first control model is configured with a trained machine learning model; and
the second control model is configured with such a trained machine learning model that input data is at least partially different from input data of the first control model.

### Claim 4 (depends on 1)

The control device according to claim 1, wherein
controlling the movement of the mobile body according to the second control command or the third control command is configured with:
deciding which of the second control model and the manual operation is to be adopted, according to an environment of the movement of the mobile body;
deriving, if it is decided that the second control model is to be adopted, the second control command using the second control model and controlling the movement of the mobile body according to the derived second control command; and
controlling, if it is decided that the manual operation is to be adopted, the movement of the mobile body according to the third control command originating from the manual operation.

### Claim 5 (depends on 4)

The control device according to claim 4, wherein
the mobile body is a vehicle; and
the environment of the movement of the mobile body includes a type of a road where the vehicle travels.

### Claim 6 (depends on 1)

The control device according to claim 1, wherein
controlling the movement of the mobile body according to the second control command or the third control command is configured with:
controlling, if the derivation of the second control command by the second control model is normal, the movement of the mobile body according to the derived second control command; and
controlling, if an abnormality occurs in the derivation of the second control command by the second control model, the movement of the mobile body according to the third control command originating from the manual operation.

### Claim 7 (depends on 1)

The control device according to claim 1, wherein
the abnormality occurring in the derivation of the first control command includes an output failure of the first control model occurring.

### Claim 8 (depends on 1)

The control device according to claim 1, wherein
the abnormality occurring in the derivation of the first control command includes reliability of output of the first control model being below a threshold.

### Claim 9 (depends on 1)

The control device according to claim 1, wherein
the abnormality occurring in the derivation of the first control command includes an amount of change in the output of the first control model per unit time being above a threshold.

### Claim 10

A non-transitory storage medium storing a control program for causing a control device, the control device holding a first control model and a second control model configured to be capable of normally deriving a control command in at least a part of scenes of an abnormality occurring in derivation of a control command by the first control model, by being different from the first control model in at least any of a type and input data, to execute:
trying derivation of a first control command using the first control model;
controlling, if the derivation of the first control command by the first control model is normal, movement of a mobile body according to the derived first control command; and
controlling, if an abnormality occurs in the derivation of the first control command by the first control model, the movement of the mobile body according to a second control command derived by the second control model or a third control command originating from a manual operation by a user.

### Claim 11 (depends on 10)

The non-transitory storage medium according to claim 10, wherein
the first control model is configured with a trained machine learning model; and
the second control model is configured with a rule-based model.

### Claim 12 (depends on 10)

The non-transitory storage medium according to claim 10, wherein
the first control model is configured with a trained machine learning model; and
the second control model is configured with such a trained machine learning model that input data is at least partially different from input data of the first control model.

### Claim 13 (depends on 10)

The non-transitory storage medium according to claim 10, wherein
controlling the movement of the mobile body according to the second control command or the third control command is configured with:
deciding which of the second control model and the manual operation is to be adopted, according to an environment of the movement of the mobile body;
deriving, if it is decided that the second control model is to be adopted, the second control command using the second control model and controlling the movement of the mobile body according to the derived second control command; and
controlling, if it is decided that the manual operation is to be adopted, the movement of the mobile body according to the third control command originating from the manual operation.

### Claim 14 (depends on 13)

The non-transitory storage medium according to claim 13, wherein
the mobile body is a vehicle; and
the environment of the movement of the mobile body includes a type of a road where the vehicle travels.

### Claim 15 (depends on 10)

The non-transitory storage medium according to claim 10, wherein
controlling the movement of the mobile body according to the second control command or the third control command is configured with:
controlling, if the derivation of the second control command by the second control model is normal, the movement of the mobile body according to the derived second control command; and
controlling, if an abnormality occurs in the derivation of the second control command by the second control model, the movement of the mobile body according to the third control command originating from the manual operation.

### Claim 16 (depends on 10)

The non-transitory storage medium according to claim 10, wherein
the abnormality occurring in the derivation of the first control command includes an output failure of the first control model occurring.

### Claim 17 (depends on 10)

The non-transitory storage medium according to claim 10, wherein
the abnormality occurring in the derivation of the first control command includes reliability of output of the first control model being below a threshold.

### Claim 18 (depends on 10)

The non-transitory storage medium according to claim 10, wherein
the abnormality occurring in the derivation of the first control command includes an amount of change in the output of the first control model per unit time being above a threshold.

### Claim 19

A control method for a control device, the control device holding a first control model and a second control model configured to be capable of normally deriving a control command in at least a part of scenes of an abnormality occurring in derivation of a control command by the first control model, by being different from the first control model in at least any of a type and input data, to execute:
trying derivation of a first control command using the first control model;
controlling, if the derivation of the first control command by the first control model is normal, movement of a mobile body according to the derived first control command; and
controlling, if an abnormality occurs in the derivation of the first control command by the first control model, the movement of the mobile body according to a second control command derived by the second control model or a third control command originating from a manual operation by a user.

### Claim 20 (depends on 19)

The control method according to claim 19, wherein
controlling the movement of the mobile body according to the second control command or the third control command is configured with:
deciding which of the second control model and the manual operation is to be adopted, according to an environment of the movement of the mobile body;
deriving, if it is decided that the second control model is to be adopted, the second control command using the second control model and controlling the movement of the mobile body according to the derived second control command; and
controlling, if it is decided that the manual operation is to be adopted, the movement of the mobile body according to the third control command originating from the manual operation.
