# US 11,908,337 B2 · Information processing device, intermediation device, simulation system, and information processing method > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 11,908,337 B2 - Document ID: US11908337B2 - Status: Granted - Kind: B2 - Application number: 17/267,420 - Filed: 2019-08-09 - Published: 2021-09-23 - Granted: 2024-02-20 - Earliest claimed: 2018-08-10 - Assignees: Kawasaki Jukogyo Kabushiki Kaisha - Applicants: Kawasaki Jukogyo Kabushiki Kaisha - Inventors: Yasuhiko Hashimoto; Masayuki Kamon; Shigetsugu Tanaka; Yoshihiko Maruyama - Web page: https://patentgrep.com/patent/US11908337B2 ## CPC - G09B 19/00: Teaching not covered by other main groups of this subclass (teaching or practice apparatus for gun-aiming or gun-laying F41G3/26) - G09B 19/24: Use of tools - A63F 13/20: Input arrangements for video game devices - A63F 13/23: for interfacing with the game device, e.g. specific interfaces between game controller and console - A63F 13/25: Output arrangements for video game devices - A63F 13/28: responding to control signals received from the game device for affecting ambient conditions, e.g. for vibrating players' seats, activating scent dispensers or affecting temperature or light - A63F 13/50: Controlling the output signals based on the game progress - A63F 13/65: automatically by game devices or servers from real world data, e.g. measurement in live racing competition - A63F 13/67: adaptively or by learning from player actions, e.g. skill level adjustment or by storing successful combat sequences for re-use - B25J 9/0081: {with leader teach-in means} - B25J 9/1605: {Simulation of manipulator lay-out, design, modelling of manipulator} - B25J 9/161: {Hardware, e.g. neural networks, fuzzy logic, interfaces, processor} - B25J 9/163: {learning, adaptive, model based, rule based expert control} - B25J 9/1661: {characterised by task planning, object-oriented languages} - B25J 9/1671: {characterised by simulation, either to verify existing program or to create and verify new program, CAD/CAM oriented, graphic oriented programming systems} - B25J 9/1697: {Vision controlled systems} - B25J 13/006: {by means of a wireless system for controlling one or several manipulators} - B25J 13/02: Hand grip control means {(handles or pedals for crane control B66C13/56; for measuring the force applied to control members G01L5/22; hand-held casings for switching devices, e.g. joy-sticks H01H9/0214)} - B25J 13/06: Control stands, e.g. consoles, switchboards - B25J 13/065: {comprising joy-sticks} - B25J 13/082: {Grasping-force detectors (in general G01L5/16, G01L5/22)} - B25J 13/088: {with position, velocity or acceleration sensors} - B25J 19/021: {Optical sensing devices} - G09B 5/02: with visual presentation of the material to be studied, e.g. using film strip - G09B 19/16: Control of vehicles or other craft (simulators G09B9/02) - G05B 2219/40099: Graphical user interface for robotics, visual robot user interface - G05B 2219/40515: Integration of simulation and planning - G09B: EDUCATIONAL OR DEMONSTRATION APPLIANCES; APPLIANCES FOR TEACHING, OR COMMUNICATING WITH, THE BLIND, DEAF OR MUTE; MODELS; PLANETARIA; GLOBES; MAPS; DIAGRAMS (devices for psychotechnics or for testing reaction times A61B5/16; games, sports, amusements A63; projectors, projector screens G03B) - A63F: CARD, BOARD, OR ROULETTE GAMES; INDOOR GAMES USING SMALL MOVING PLAYING BODIES; VIDEO GAMES; GAMES NOT OTHERWISE PROVIDED FOR - B25J: MANIPULATORS; CHAMBERS PROVIDED WITH MANIPULATION DEVICES ({manipulators specially adapted for use in surgery A61B34/70; manipulators used in cleaning hollow articles B08B9/04}; manipulators associated with rolling mills B21B39/20; manipulators associated with forging machines B21J13/10; {manipulators associated with picking-up and placing mechanisms B23P19/007}; means for holding wheels or parts thereof B60B30/00; {vehicles with ground-engaging propulsion means, e.g. walking members B62D57/02, B62D57/032; devices for picking-up and depositing articles or materials between conveyors B65G47/90, B65G47/91; manipulators with gripping or holding means for transferring packages B65H67/065}; cranes B66C; {manipulators used in the protection or supervision of pipe-line installations F17D5/00; walking equipment adapted for nuclear steam-generators F22B37/006}; manipulators specially adapted for, or associated with, nuclear reactors G21C; {apparatus used for handling wafers during manufacture or treatment of semiconductor H10P72/50}) - G05B: CONTROL OR REGULATING SYSTEMS IN GENERAL; FUNCTIONAL ELEMENTS OF SUCH SYSTEMS; MONITORING OR TESTING ARRANGEMENTS FOR SUCH SYSTEMS OR ELEMENTS (systems for controlling or regulating non-electric variables G05D; systems for regulating electric or magnetic variables G05F; control devices or systems insofar as characterised by mechanical features only G05G) ## Abstract An information processing device converts first information for manipulating a robot model inputted into a manipulation terminal connected with a simulation device configured to execute a simulation for causing the robot model to perform a simulated operation and operated by a remote user of the simulation device, into second information for manipulating the robot model of the simulation device, operates the simulation device according to the second information, and causes the manipulation terminal to present information on the operation of the robot model of the simulation device configured to operate according to the second information. ## Claims ### Claim 1 An information processing device, wherein the information processing device is adapted to: select one of a simulation computer and a simulation robot as a selected simulation device based on selection information received from a manipulation terminal; convert first information for manipulating a robot model inputted into the manipulation terminal connected with the selected simulation device configured to execute a simulation for causing the robot model to perform a simulated operation and operated by a remote user of the selected simulation device, into second information for manipulating the robot model of the selected simulation device; operate the selected simulation device according to the second information; and cause the manipulation terminal to present information on the simulated operation of the robot model of the selected simulation device configured to operate according to the second information. ### Claim 2 (depends on 1) The information processing device of claim 1, wherein the information processing device is adapted to: determine, based on request information that is inputted into the manipulation terminal and is information on the simulation requested to be executed, the selected simulation device corresponding to the simulation requested in the request information among a plurality of simulation devices; and enable an execution of the simulation using the selected simulation device and the manipulation terminal. ### Claim 3 (depends on 1) The information processing device of claim 1, wherein the information processing device is adapted to: evaluate an operation result of the robot model based on information on the operation of the robot model of the selected simulation device; and register the evaluation of the operation result as a user evaluation that is an evaluation of the user who operates the manipulation terminal. ### Claim 4 (depends on 3) The information processing device of claim 3, wherein the information processing device is adapted to: determine a simulation executable by a user corresponding to the user evaluation and the selected simulation device based on the registered user evaluation; and cause the manipulation terminal to present information on the determined simulation and selected simulation device. ### Claim 5 (depends on 1) The information processing device of claim 1, wherein the simulated operation of the robot model is operation simulating a work performed by an actual robot. ### Claim 6 (depends on 1) The information processing device of claim 1, wherein the information processing device is adapted to: when the information processing device selects the simulation computer as the selected simulation device enable an execution of the simulation using the manipulation terminal and the simulation computer to cause an imaginary robot model to perform the simulated operation; and cause the manipulation terminal to present information on the operation of the imaginary robot model generated by the selected simulation device. ### Claim 7 (depends on 1) The information processing device of claim 1, wherein the information processing device is adapted to: when the information processing device selects the simulation robot as the selected simulation device enable an execution of the simulation using the manipulation terminal and the simulation robot to cause an actual robot to perform the simulated operation as the robot model; and cause the manipulation terminal to present information on an image of the robot captured by an imaging device as information on the operation of the robot model. ### Claim 8 (depends on 7) The information processing device of claim 7, wherein the information processing device is adapted to: convert the image of the robot into an image of an imaginary robot model; and cause the manipulation terminal to present information on operation of the imaginary robot model in the converted image. ### Claim 9 (depends on 1) The information processing device of claim 1, wherein the manipulation terminal is connected with the selected simulation device through a communication network. ### Claim 10 (depends on 1) The information processing device of claim 1, wherein the manipulation terminal is at least any of a game machine, a game controller, a remote controller dedicated for a robot, a personal data assistant (PDA), a smart device, and a personal computer. ### Claim 11 (depends on 1) An intermediation device, comprising the information processing device of claim 1, wherein the intermediation device mediates between the manipulation terminal and the selected simulation device through a communication network. ### Claim 12 (depends on 1) A simulation system, comprising: the information processing device of claim 1; the selected simulation device; and an intermediation device configured to mediate between the manipulation terminal and the selected simulation device through a communication network. ### Claim 13 (depends on 12) The simulation system of claim 12, further comprising the manipulation terminal. ### Claim 14 An information processing method, comprising the steps of: selecting one of a simulation computer and a simulation robot as a selected simulation device based on selection information received from a manipulation terminal; converting first information for manipulating a robot model inputted into the manipulation terminal connected with the selected simulation device configured to execute a simulation for causing the robot model to perform a simulated operation and operated by a remote user of the selected simulation device, into second information for manipulating the robot model of the selected simulation device; causing the selected simulation device to operate according to the second information; and causing the manipulation terminal to present information on the simulated operation of the robot model of the selected simulation device configured to operate according to the second information.