# US 2023/0405803 A1 · Information processing device, robot manipulating system and robot manipulating method

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2023/0405803 A1
- Document ID: US20230405803A1
- Status: Published application
- Kind: A1
- Application number: 18/241,001
- Filed: 2023-08-31
- Published: 2023-12-21
- Earliest claimed: 2018-08-10
- Assignees: Kawasaki Jukogyo Kabushiki Kaisha
- Applicants: Kawasaki Jukogyo Kabushiki Kaisha
- Inventors: Yasuhiko HASHIMOTO; Masayuki KAMON; Shigetsugu TANAKA; Yoshihiko MARUYAMA
- Web page: https://patentgrep.com/patent/US20230405803A1

## CPC

- A63F 13/42: by mapping the input signals into game commands, e.g. mapping the displacement of a stylus on a touch screen to the steering angle of a virtual vehicle
- B25J 9/0084: {comprising a plurality of manipulators}
- A63F 13/20: Input arrangements for video game devices
- A63F 13/65: automatically by game devices or servers from real world data, e.g. measurement in live racing competition
- A63F 13/67: adaptively or by learning from player actions, e.g. skill level adjustment or by storing successful combat sequences for re-use
- A63F 13/90: Constructional details or arrangements of video game devices not provided for in groups A63F13/20 or A63F13/25, e.g. housing, wiring, connections or cabinets
- B25J 11/003: {Manipulators for entertainment}
- B25J 13/06: Control stands, e.g. consoles, switchboards
- A63F: CARD, BOARD, OR ROULETTE GAMES; INDOOR GAMES USING SMALL MOVING PLAYING BODIES; VIDEO GAMES; GAMES NOT OTHERWISE PROVIDED FOR
- B25J: MANIPULATORS; CHAMBERS PROVIDED WITH MANIPULATION DEVICES ({manipulators specially adapted for use in surgery A61B34/70; manipulators used in cleaning hollow articles B08B9/04}; manipulators associated with rolling mills B21B39/20; manipulators associated with forging machines B21J13/10; {manipulators associated with picking-up and placing mechanisms B23P19/007}; means for holding wheels or parts thereof B60B30/00; {vehicles with ground-engaging propulsion means, e.g. walking members B62D57/02, B62D57/032; devices for picking-up and depositing articles or materials between conveyors B65G47/90, B65G47/91; manipulators with gripping or holding means for transferring packages B65H67/065}; cranes B66C; {manipulators used in the protection or supervision of pipe-line installations F17D5/00; walking equipment adapted for nuclear steam-generators F22B37/006}; manipulators specially adapted for, or associated with, nuclear reactors G21C; {apparatus used for handling wafers during manufacture or treatment of semiconductor H10P72/50})

## Abstract

A robot manipulating system includes a game terminal having a game computer, a game controller, and a display configured to display a virtual space, a robot configured to perform a work in a real space based on robot control data, and an information processing device configured to mediate between the game terminal and the robot. The information processing device supplies game data associated with a content of work to the game terminal, acquires game manipulation data including a history of an input of manipulation accepted by the game controller while a game program to which the game data is reflected is executed, converts the game manipulation data into the robot control data based on a given conversion rule, and supplies the robot control data to the robot.

## Claims

### Claim 1

An information processing device configured to mediate between a game terminal having a game computer configured to execute a game program, a game controller configured to accept an input of manipulation into the game computer, and a display configured to display a virtual space outputted from the game computer, and a robot configured to perform a work in a real space based on robot control data, comprising:
a game data feeding part configured to supply game data associated with the content of work to the game terminal;
a game manipulation data acquiring part configured to acquire game manipulation data including a history of the input of the manipulation accepted by the game controller while the game program to which the game data is reflected is executed;
a first converting part configured to convert the game manipulation data into the robot control data based on a given conversion rule; and
a robot control data feeding part configured to supply the robot control data to the robot.

### Claim 2 (depends on 1)

The information processing device of claim 1, wherein the first converting part converts the game manipulation data into the robot control data by inputting the game manipulation data into a first learned model where a relation between the game manipulation data and the robot control data is learned.

### Claim 3 (depends on 2)

The information processing device of claim 2, further comprising a first learning part configured to learn the relation between the game manipulation data and the robot control data by using first learned data including the game manipulation data associated with the content of work and the robot control data, and generate the first learned model.

### Claim 4 (depends on 1)

The information processing device of claim 1, further comprising a second converting part configured to convert the content of work into the game data based on a given conversion rule.

### Claim 5 (depends on 4)

The information processing device of claim 4, wherein the second converting part converts the content of work into the game data by inputting the content of work into a second learned model where a relation between the content of work and the game data is learned.

### Claim 6 (depends on 5)

The information processing device of claim 5, further comprising a second learning part configured to learn the relation between the content of work and the game data by using second learned data including the content of work and the game data associated with the content of work, and generate the second learned model.

### Claim 7 (depends on 1)

A robot manipulating system, comprising:
a game terminal having a game computer configured to execute a game program, a game controller configured to accept an input of manipulation into the game computer, and a display configured to display a virtual space outputted from the game computer;
a robot configured to perform a work in a real space based on robot control data; and
the information processing device of claim 1 configured to mediate between the game terminal and the robot.

### Claim 8

A method of manipulating a robot configured to perform a work in a real space based on robot control data by using a game terminal having a game computer configured to execute a game program, a game controller configured to accept an input of manipulation into the game computer, and a display configured to display a virtual space outputted from the game computer, the method comprising the steps of:
supplying game data associated with the content of work to the game terminal;
acquiring game manipulation data including a history of the input of the manipulation accepted by the game controller while the game program to which the game data is reflected is executed;
converting the game manipulation data into the robot control data based on a given conversion rule; and
supplying the robot control data to the robot.

### Claim 9 (depends on 8)

The method of claim 8, wherein the converting the game manipulation data includes converting the game manipulation data into the robot control data by inputting the game manipulation data into a first learned model where a relation between the game manipulation data and the robot control data is learned.

### Claim 10 (depends on 9)

The method of claim 9, further comprising generating the first learned model by learning the relation between the game manipulation data and the robot control data by using first learned data including the game manipulation data associated with the content of work and the robot control data.

### Claim 11 (depends on 8)

The method of claim 8, further comprising converting the content of work into the game data based on a given conversion rule.

### Claim 12 (depends on 11)

The method of claim 11, wherein the converting the content of work includes converting the content of work into the game data by inputting the content of work into a second learned model where a relation between the content of work and the game data is learned.

### Claim 13 (depends on 12)

The method of claim 12, further comprising generating the second learned model by learning the relation between the content of work and the game data by using second learned data including the content of work and the game data associated with the content of work.
