# US 2004/0243281 A1 · Robot behavior control system, behavior control method, and robot device

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2004/0243281 A1
- Document ID: US20040243281A1
- Status: Published application
- Kind: A1
- Application number: 10/477,574
- Filed: 2004-05-21
- Published: 2004-12-02
- Earliest claimed: 2002-03-15
- Assignees: Sony Corporation
- Inventors: Masahiro Fujita; Tsuyoshi Takagi; Rika Horinaka; Shinya Ohtani
- Web page: https://patentgrep.com/patent/US20040243281A1

## CPC

- G06N 3/008: based on physical entities controlled by simulated intelligence so as to replicate intelligent life forms, e.g. based on robots replicating pets or humans in their appearance or behaviour
- B25J 9/1602: {characterised by the control system, structure, architecture}
- B25J 9/161: {Hardware, e.g. neural networks, fuzzy logic, interfaces, processor}
- B25J 13/003: {by means of an audio-responsive input (audible safety signals B25J19/061)}
- G05B 2219/33052: Subsumption architecture, behavioral modules in layers, override older ones
- G05B 2219/39254: Behaviour controller, robot have feelings, learns behaviour
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- B25J: MANIPULATORS; CHAMBERS PROVIDED WITH MANIPULATION DEVICES ({manipulators specially adapted for use in surgery A61B34/70; manipulators used in cleaning hollow articles B08B9/04}; manipulators associated with rolling mills B21B39/20; manipulators associated with forging machines B21J13/10; {manipulators associated with picking-up and placing mechanisms B23P19/007}; means for holding wheels or parts thereof B60B30/00; {vehicles with ground-engaging propulsion means, e.g. walking members B62D57/02, B62D57/032; devices for picking-up and depositing articles or materials between conveyors B65G47/90, B65G47/91; manipulators with gripping or holding means for transferring packages B65H67/065}; cranes B66C; {manipulators used in the protection or supervision of pipe-line installations F17D5/00; walking equipment adapted for nuclear steam-generators F22B37/006}; manipulators specially adapted for, or associated with, nuclear reactors G21C; {apparatus used for handling wafers during manufacture or treatment of semiconductor H10P72/50})
- G05B: CONTROL OR REGULATING SYSTEMS IN GENERAL; FUNCTIONAL ELEMENTS OF SUCH SYSTEMS; MONITORING OR TESTING ARRANGEMENTS FOR SUCH SYSTEMS OR ELEMENTS (systems for controlling or regulating non-electric variables G05D; systems for regulating electric or magnetic variables G05F; control devices or systems insofar as characterised by mechanical features only G05G)

## Abstract

A situated behavior layer is formed from a tree structure of schemas, and a parent schema calls a Monitor function of a child schema using an external stimulus and an internal state as arguments whereas the child schema returns an AL value as a return value. The child schema calls a Monitor function of its child schema in order to calculate an AL value of the child schema itself. AL values from sub trees are returned to a root schema, and evaluation of behaviors and execution of a behavior are performed concurrently. Further, emotions are divided into a plurality of layers depending upon the significance of presence thereof, and it is determined which one of a plurality of such determined motions should be selectively performed depending upon an external environment and an internal state at the time. While a behavior is selected in each of the layers, such behavior is developed preferentially beginning with that of a comparatively low layer so that an instinctive behavior such as reflex and a higher order behavior such as selection of a motion by use of memory can be developed without a contradiction on a single individual.

## Claims

### Claim 1

A behavior control system for a robot that operates autonomously, comprising:
a plurality of behavior description sections for describing motions of a machine body of said robot;
an external environment recognition section for recognizing an external environment of said machine body;
an internal state management section for managing an internal state of said robot in response to the recognized external environment and/or a result of execution of a behavior; and
a behavior evaluation section for evaluating execution of behaviors described in said behavior description sections in response to the external environment and/or the internal state.

### Claim 2 (depends on 1)

A behavior control system for a robot according to claim 1, wherein
said external environment recognition section performs at least one of visual recognition of the outside, auditory recognition of sound generated outside said robot and contact recognition of contact applied from the outside to said robot.

### Claim 3 (depends on 1)

A behavior control system for a robot according to claim 1, wherein
said internal state management section manages an instinct model and/or a feeling model of said robot.

### Claim 4 (depends on 1)

A behavior control system for a robot according to claim 1, wherein
said behavior description sections are arranged in a tree structure form in accordance with implementation levels of the machine body motions, and
said behavior evaluation section simultaneously and concurrently evaluates a plurality of ones of said behavior description sections from above to below in the tree structure.

### Claim 5 (depends on 1)

A behavior control system for a robot according to claim 1, further comprising:
a behavior selection section for selectively executing, when evaluation of each of said behavior description sections is performed by said behavior evaluation section in response to new recognition by said external environment recognition section and/or a variation of the internal state by said internal state management section, a behavior described in the behavior description section based on the evaluation values.

### Claim 6 (depends on 5)

A behavior control system for a robot according to claim 5, further comprising:
a resource management section for managing conflict of a resource on said machine body when behaviors described in a plurality of ones of said behavior description sections are executed simultaneously,
said behavior selection section permits said resource management section to simultaneously select two or more ones of said behavior description sections in response to a conflict relationship of a resource.

### Claim 7 (depends on 5)

A behavior control system for a robot according to claim 5, wherein,
when a result of execution of evaluation for each of said behavior description sections by said behavior evaluation section based on new recognition by said external environment recognition section reveals appearance of a behavior description section which exhibits an evaluation value higher than that of the behavior being currently executed, said behavior selection section stops the behavior being currently executed and preferentially executes the behavior described in the behavior description section having the higher evaluation value.

### Claim 8 (depends on 7)

A behavior control system for a robot according to claim 7, wherein
said behavior selection section causes the behavior stopped once to be resumed after the behavior executed preferentially comes to an end.

### Claim 9 (depends on 5)

A behavior control system for a robot according to claim 5, wherein
said behavior selection section can successively select a same one of said behavior description sections in response to a variation of each different external environment, and
said behavior control system further comprises a working space management section for allocating an individual working space for each external environment every time the behavior described in the behavior description section is executed.

### Claim 10

A behavior control system for a robot that operates autonomously, comprising:
one or more behavior modules each including a state machine which describes a motion of a machine body of said robot and a behavior evaluator for evaluating an activity level of a current machine body motion state of said state machine and a machine body resource to be used upon starting of the machine body motion state; and
a behavior state control section for issuing an instruction to calculate an activity level and a use resource to said behavior evaluator of each of said behavior modules, selecting one of said behavior modules to be activated in response to the activity levels and the use resources of said behavior modules and issuing an instruction to said state machine of the selected behavior module to execute the state machine thereby to control behavior states of said behavior modules.

### Claim 11 (depends on 10)

A behavior control system for a robot according to claim 10, wherein
said behavior evaluator evaluates the activity level of said state machine in response to an external environment of said machine body and/or an internal state of said robot.

### Claim 12 (depends on 10)

A behavior control system for a robot according to claim 10, wherein
said behavior modules are arranged in a tree structure form in accordance with implementation levels of the machine body motions, and
said behavior state control section is mounted in each of said behavior modules and issues an instruction to evaluate the activity level and the use resource, an instruction to select a behavior module and an instruction to execute the state machine to another one of said behavior modules which is positioned in a lower order in the tree structure.

### Claim 13 (depends on 10)

A behavior control system for a robot according to claim 10, wherein
said behavior state control section places one of said behavior modules whose activity level exhibits a drop from an active state into a standby state, and places another one of said behavior modules whose activity level exhibits a rise from a standby state into an active state.

### Claim 14 (depends on 13)

A behavior control system for a robot according to claim 13, further comprising:
means for storing, when any of said behavior modules is placed from an active state into a standby state, data necessary to resume the behavior module and activating a behavior necessary to place the behavior model into a standby state; and
means for regenerating, when the behavior model is placed back from the standby state into the active state, the stored data to initialize the state of the behavior module and activating a motion necessary to place the behavior module into the active state.

### Claim 15

A behavior control system for a robot that operates autonomously, comprising:
one or more behavior modules each formed from a combination of a command for causing said robot to move and a behavior evaluator for evaluating a resource of a machine body of said robot necessary to execute the command;
means for detecting a machine body resource or resources in a released state; and
means for selectively activating those of said behavior modules which can be executed with the released machine body resources in accordance with a predetermined priority order;
two or more ones of said behavior modules with which conflict of a resource does not occur being capable of being executed concurrently.

### Claim 16

A behavior control system for a robot that operates autonomously in response to an internal state thereof, comprising:
an internal state management section for managing emotions each of which is an index to the internal state in a hierarchical structure having a plurality of layers; and
a behavior selection section for selectively executing a behavior for satisfying one of the emotions in each of said layers.

### Claim 17 (depends on 16)

A behavior control system for a robot according to claim 16, wherein
said internal state management section divides the emotions into a layer of primary emotions necessary for individual preservation and another layer of secondary emotions which vary depending upon excess/deficiency of the primary emotions, and further divides the primary emotions into layers including an innate reflexive or physiological layer and an associative layer based on dimensions.

### Claim 18 (depends on 17)

A behavior control system for a robot according to claim 17, wherein
said behavior selection section preferentially selects a behavior which satisfies a comparatively low order primary emotion.

### Claim 19 (depends on 18)

A behavior control system for a robot according to claim 18, wherein
said behavior selection section suppresses selection of a behavior which satisfies a comparatively low order primary emotion where a comparatively high order primary emotion exhibits significant shortage when compared with the comparatively low order primary emotion.

### Claim 20 (depends on 16)

A behavior control system for a robot according to claim 16, further comprising:
an external environment recognition section for recognizing a variation of an external environment of said robot,
said behavior selection section selecting a behavior based on an index to the external environment in addition to the index to the internal state.

### Claim 21 (depends on 16)

A behavior control system for a robot according to claim 16, wherein
said internal state management section changes the index to the internal state in response to passage of time.

### Claim 22 (depends on 16)

A behavior control system for a robot according to claim 16, wherein
said internal state management section changes the index to the internal state in response to execution of a behavior selected by said behavior selection section.

### Claim 23 (depends on 16)

A behavior control system for a robot according to claim 16, further comprising:
an external environment recognition section for recognizing a variation of an external environment of said robot,
said internal state management section changing the index to the internal state in response to a variation of the external environment.

### Claim 24 (depends on 16)

A behavior control system for a robot according to claim 16, further comprising:
an external environment recognition section for recognizing a variation of an external environment of said robot and an associative memory section for associatively storing a variation of the internal state from the external environment,
said internal state management section changing the index to the internal state based on a variation of the internal state recalled from the external environment by said associative memory section.

### Claim 25 (depends on 24)

A behavior control system for a robot according to claim 24, wherein
said associative memory section associatively stores a variation of the internal state for each object article to be recognized.

### Claim 26

A behavior control method for a robot which operates autonomously, a plurality of behavior description modules which describe motions of a machine body of said robot being prepared, said behavior control method comprising:
an external environment recognition step of recognizing an external environment of said machine body;
an internal state management step of managing an internal state of said robot in response to the recognized external environment and/or a result of execution of a behavior; and
a behavior evaluation step of evaluating execution of behaviors described in said behavior description modules in response to the external environment and/or the internal state.

### Claim 27 (depends on 26)

A behavior control method for a robot according to claim 26, wherein,
at the external environment recognition step, at least one of visual recognition of the outside, auditory recognition of sound generated outside said robot and contact recognition of contact applied from the outside to said robot is performed.

### Claim 28 (depends on 26)

A behavior control method for a robot according to claim 26, wherein,
at the internal state management step, an instinct model and/or a feeling model of said robot is managed.

### Claim 29 (depends on 26)

A behavior control method for a robot according to claim 26, wherein
said behavior description modules are arranged in a tree structure form in accordance with implementation levels of the machine body motions, and,
at the behavior evaluation step, a plurality of ones of said behavior description modules are simultaneously and concurrently evaluated from above to below in the tree structure.

### Claim 30 (depends on 26)

A behavior control method for a robot according to claim 26, further comprising:
a behavior selection step of selectively executing, when evaluation of each of said behavior description modules is performed at the behavior evaluation step in response to new recognition at the external environment recognition step and/or a variation of the internal state at the internal state management step, a behavior described in the behavior description module based on the evaluation values.

### Claim 31 (depends on 30)

A behavior control method for a robot according to claim 30, further comprising:
a resource management step of managing conflict of a resource on said machine body when behaviors described in a plurality of ones of said behavior description section are executed simultaneously,
simultaneous selection of two or more ones of said behavior description section in response to a conflict relationship of a resource being permitted at the behavior selection step.

### Claim 32 (depends on 30)

A behavior control method for a robot according to claim 30, wherein,
when a result of execution of evaluation of each of said behavior description modules at the behavior evaluation step based on new recognition at the external environment recognition step reveals appearance of a behavior description module which exhibits an evaluation value higher than that of the behavior being currently executed, at the behavior selection step, the behavior being currently executed is stopped and the behavior described in the behavior description section having the higher evaluation value is preferentially executed.

### Claim 33 (depends on 32)

A behavior control method for a robot according to claim 32, wherein
said behavior selection step causes the behavior stopped once to be resumed after the behavior executed preferentially comes to an end.

### Claim 34 (depends on 30)

A behavior control method for a robot according to claim 30, wherein,
at the behavior selection step, a same one of said behavior description modules can be successively selected in response to a variation of each different external environment, and
said behavior control method further comprises a working space management step of allocating an individual working space for each external environment every time the behavior described in the behavior description module is executed.

### Claim 35

A behavior control method for a robot that operates autonomously in response to an internal state thereof, comprising:
an internal state management step of managing emotions each of which is an index to the internal state in a hierarchical structure having a plurality of layers; and
a behavior selection step of selectively executing a behavior for satisfying one of the emotions in each of said layers.

### Claim 36 (depends on 35)

A behavior control method for a robot according to claim 35, wherein,
at the internal state management step, the emotions are divided into a layer of primary emotions necessary for individual preservation and another layer of secondary emotions which vary depending upon excess/deficiency of the primary emotions, and the primary emotions are further divided into layers including an innate reflexive or physiological layer and an associative layer based on dimensions to handle the internal state.

### Claim 37 (depends on 35)

A behavior control method for a robot according to claim 35, wherein,
at the behavior selection step, a behavior which satisfies a comparatively low order primary emotion is preferentially selected.

### Claim 38 (depends on 35)

A behavior control method for a robot according to claim 35, wherein,
at the behavior selection step, selection of a behavior which satisfies a comparatively low order primary emotion is suppressed where a comparatively high order primary emotion exhibits significant shortage when compared with the comparatively low order primary emotion.

### Claim 39 (depends on 35)

A behavior control method for a robot according to claim 35, further comprising:
an external environment recognition step of recognizing a variation of an external environment of said robot,
a behavior being selected, at the behavior selection step, based on an index to the external environment in addition to the index to the internal state.

### Claim 40 (depends on 35)

A behavior control method for a robot according to claim 35, wherein,
at the internal state management step, the index to the internal state is changed in response to passage of time.

### Claim 41 (depends on 35)

A behavior control method for a robot according to claim 35, wherein,
at the internal state management step, the index to the internal state is changed in response to execution of a behavior selected at the behavior selection step.

### Claim 42 (depends on 35)

A behavior control method for a robot according to claim 35, further comprising:
an external environment recognition step of recognizing a variation of an external environment of said robot,
the index to the internal state being changed, at the internal state management step, in response to a variation of the external environment.

### Claim 43 (depends on 35)

A behavior control method for a robot according to claim 35, further comprising:
an external environment recognition step of recognizing a variation of an external environment of said robot and an associative memory step of associatively storing a variation of the internal state from the external state,
the index to the internal state being changed, at the internal state management step, based on a variation of the internal environment recalled from the external environment by the associative memory.

### Claim 44 (depends on 43)

A behavior control method for a robot according to claim 43, wherein,
at the associative memory step, a variation of the internal state is associatively stored for each object article to be recognized.

### Claim 45

A robot apparatus that generates a motion based on an internal input or an external input, comprising:
one or more behavior modules for determining a behavior of said robot;
behavior module management means for managing said behavior modules;
state storage means for storing current states of said behavior modules; and
one or more state machines corresponding to the states stored in said state storage means for outputting a behavior instruction based on an external or internal input;
said behavior module management means managing said behavior modules such that, when outputting of a behavior instruction from any of said state machines is stopped and behavior outputting is to be resumed thereafter, said behavior module management means causes the state machine to resume the behavior outputting after said behavior module management means causes said robot to execute a predetermined behavior based on the states stored in said state storage means.

### Claim 46 (depends on 45)

A robot apparatus according to claim 45, wherein
said state storage means and said state machines are provided in said behavior modules, and
said behavior module management means is accommodated in another behavior module which cooperates with said behavior modules to form a hierarchical structure and is positioned in a comparatively higher layer in the hierarchical structure.

### Claim 47

A robot apparatus that operates autonomously, comprising:
a behavior module including means for discriminating a situation based on an external stimulus and an internal environment to calculate an activity level and means for outputting a behavior in response to an input and a state based on a predetermined state machine; and
state setting means for setting a state of said behavior module.

### Claim 48 (depends on 47)

A robot apparatus according to claim 47, wherein
said state setting means sets said behavior module to one of ready, active and standby states in response to the activity level.

### Claim 49 (depends on 47)

A robot apparatus according to claim 47, wherein
two or more two behavior modules are arranged in a hierarchical structure, and
said state setting means is configured such that one of said behavior modules which is positioned in a comparatively high layer in the hierarchical structure selects another one of said behavior modules which is positioned in a lower layer in the hierarchical structure.
