# US 2019/0244133 A1 · Learning apparatus and learning method

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2019/0244133 A1
- Document ID: US20190244133A1
- Status: Published application
- Kind: A1
- Application number: 16/343,940
- Filed: 2017-12-14
- Published: 2019-08-08
- Earliest claimed: 2016-12-26
- Assignees: Sony Corporation
- Applicants: Sony Corporation
- Inventors: Kento NAKADA; Takuya NARIHIRA; Hirotaka SUZUKI; Akihito OSATO
- Web page: https://patentgrep.com/patent/US20190244133A1

## CPC

- G06N 20/00: Machine learning
- G06N 3/006: based on simulated virtual individual or collective life forms, e.g. social simulations or particle swarm optimisation [PSO]
- G06N 99/00: Subject matter not provided for in other groups of this subclass
- G09G 5/003: {Details of a display terminal, the details relating to the control arrangement of the display terminal and to the interfaces thereto (specific for a CRT G09G1/165; for a flat panel G09G3/2092)}
- G09G 5/14: Display of multiple viewports
- G06N 7/01: Probabilistic graphical models, e.g. probabilistic networks
- G09G 2354/00: Aspects of interface with display user
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G09G: ARRANGEMENTS OR CIRCUITS FOR CONTROL OF INDICATING DEVICES USING STATIC MEANS TO PRESENT VARIABLE INFORMATION (arrangements for transferring data between digital computers and displays G06F3/14; static indicating arrangements comprising an association of a number of separate sources or light control cells G09F9/00; static indicating arrangements comprising integral associations of a number of light sources H01J, H01K, H10H, H05B33/12; scanning, transmission or reproduction of documents or the like, e.g. facsimile transmission, details thereof H04N1/00)

## Abstract

There is provided a learning apparatus and a learning method that allow a reinforcement learning model to be easily corrected on the basis of user input. A display control section causes a display section to display reinforcement learning model information regarding a reinforcement learning model. A correcting section corrects the reinforcement learning model on the basis of user input to the reinforcement learning model information. The present disclosure can be applied to, for example, a personal computer (PC) and the like that correct a reinforcement learning model on the basis of input from a user and perform reinforcement learning of a movement policy of an agent using the corrected reinforcement learning model.

## Claims

### Claim 1

A learning apparatus comprising:
a display control section configured to cause a display section to display reinforcement learning model information regarding a reinforcement learning model; and
a correcting section configured to correct the reinforcement learning model on a basis of user input to the reinforcement learning model information.

### Claim 2 (depends on 1)

The learning apparatus according to claim 1, wherein the reinforcement learning model information includes policy information indicating a policy learned by the reinforcement learning model.

### Claim 3 (depends on 1)

The learning apparatus according to claim 1, wherein the reinforcement learning model information includes reward function information indicating a reward function used in the reinforcement learning model.

### Claim 4 (depends on 1)

The learning apparatus according to claim 1, wherein the user input includes teaching of a policy.

### Claim 5 (depends on 4)

The learning apparatus according to claim 4, wherein, in a case where an objective function is improved by adding a basis function of a reward function used in the reinforcement learning model, the correcting section adds the basis function of the reward function.

### Claim 6 (depends on 1)

The learning apparatus according to claim 1, wherein the user input includes teaching of a reward function.

### Claim 7 (depends on 6)

The learning apparatus according to claim 6, wherein, in a case where a difference between the reward function taught as the user input and a reward function of the reinforcement learning model corrected on the basis of the user input is decreased by adding a basis function of the reward function used in the reinforcement learning model, the correcting section adds the basis function of the reward function.

### Claim 8 (depends on 1)

The learning apparatus according to claim 1, wherein the display control section superimposes the reinforcement learning model information on environment information indicating an environment and causes the display section to display the reinforcement learning model information superimposed on the environment information.

### Claim 9

A learning method comprising:
a display control step of a learning apparatus causing a display section to display reinforcement learning model information regarding a reinforcement learning model; and
a correcting step of the learning apparatus correcting the reinforcement learning model on a basis of user input to the reinforcement learning model information.