# US 10,369,699 B2 · Executing software applications on a robot > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 10,369,699 B2 - Document ID: US10369699B2 - Status: Granted - Kind: B2 - Application number: 15/300,724 - Filed: 2015-04-17 - Published: 2017-04-20 - Granted: 2019-08-06 - Earliest claimed: 2014-04-17 - Assignees: Softbank Robotics Europe - Applicants: Softbank Robotics Europe - Inventors: Bruno Maisonnier; Jérôme Monceaux; David Houssin; Gabriele Barbieri; Taylor Veltrop - Web page: https://patentgrep.com/patent/US10369699B2 ## CPC - B25J 9/1694: {characterised by use of sensors other than normal servo-feedback from position, speed or acceleration sensors, perception control, multi-sensor controlled systems, sensor fusion} - G10L 15/02: Feature extraction for speech recognition; Selection of recognition unit - G10L 15/1815: {Semantic context, e.g. disambiguation of the recognition hypotheses based on word meaning} - G10L 15/22: Procedures used during a speech recognition process, e.g. man-machine dialogue - G10L 25/51: for comparison or discrimination - G10L 2015/223: {Execution procedure of a spoken command} - B25J: MANIPULATORS; CHAMBERS PROVIDED WITH MANIPULATION DEVICES ({manipulators specially adapted for use in surgery A61B34/70; manipulators used in cleaning hollow articles B08B9/04}; manipulators associated with rolling mills B21B39/20; manipulators associated with forging machines B21J13/10; {manipulators associated with picking-up and placing mechanisms B23P19/007}; means for holding wheels or parts thereof B60B30/00; {vehicles with ground-engaging propulsion means, e.g. walking members B62D57/02, B62D57/032; devices for picking-up and depositing articles or materials between conveyors B65G47/90, B65G47/91; manipulators with gripping or holding means for transferring packages B65H67/065}; cranes B66C; {manipulators used in the protection or supervision of pipe-line installations F17D5/00; walking equipment adapted for nuclear steam-generators F22B37/006}; manipulators specially adapted for, or associated with, nuclear reactors G21C; {apparatus used for handling wafers during manufacture or treatment of semiconductor H10P72/50}) - G10L: SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING ## Abstract A method of handling a software application on a robot comprising a plurality of installed software application, a software application being associated with a predefined semantic description, the method comprises the steps of extracting one or more patterns from an audio dialog with a human user; an audio dialog comprising sentences and a pattern comprising predefined sentences; comparing one or more patterns with the semantic descriptions of the software applications; selecting a software application based on the performed comparisons; and executing the selected software application. Described developments comprise software application execution rules, predefined or dynamically defined execution rules, the audio listing of the installed applications depending on environmental parameters, the optional use of complimentary display means and the installation of a missing application. Associated systems are described. ## Claims ### Claim 1 A method of handling a software application on a humanoid robot, said robot comprising a plurality of installed software applications, a software application being associated with a predefined semantic description, the method comprising the steps of: extracting one or more patterns from an audio dialog with a human user, an audio dialog comprising sentences and a pattern comprising predefined sentences; comparing said one or more patterns with the semantic descriptions of said software applications, the semantic descriptions comprising one or more pre-fabricated dialogs associated with a software application, said pre-fabricated dialogs comprising variations describing the software application; selecting a software application based on the performed comparisons; executing the selected software application, wherein at least one installed software application is associated with one or more execution rules and the step of selecting a software application comprises verifying said one or more rules; the humanoid robot does not comprise a graphical user interface, the dialog comprises a request by the user of the robot to pronounce a list of installed software applications, and the list to be pronounced by the robot is dependent on parameters associated with the perceived environment of the robot. ### Claim 2 (depends on 1) The method of claim 1, wherein said one or more rules are predefined or are dynamically defined. ### Claim 3 (depends on 1) The method of claim 1, wherein said one or more rules are locally retrievable on the robot or are remotely accessible via a network connection. ### Claim 4 (depends on 1) The method of claim 1, wherein the audio dialog comprises a response by the user to a question raised by the robot. ### Claim 5 (depends on 4) The method of claim 4, wherein the question is raised by the robot to disambiguate a sentence of the dialog or a pattern extracted from said dialog. ### Claim 6 (depends on 1) The method of claim 1, wherein the parameters are selected from the group comprising the age of a user, the gender of a user, the height of a user, the number of users in the vicinity of the robot, the mood of a user, the presence or the absence of a smile of a user, the ambient audio level, the current date, the current time, the current location and combinations thereof. ### Claim 7 (depends on 1) The method of claim 1, wherein the step of selecting a software application further comprises associating an execution probability value to each installed software application depending on said comparisons. ### Claim 8 (depends on 1) The method of claim 1, further comprising receiving the confirmation of the user before executing the selected software application. ### Claim 9 (depends on 1) The method of claim 1, further comprising displaying one or more representations associated with one or more of the installed software applications during the dialog with the user. ### Claim 10 (depends on 1) The method of claim 1, further comprising identifying a missing application in the robot, retrieving and installing said missing software application in the robot during the dialog with the user. ### Claim 11 (depends on 1) A non-transitory computer readable medium comprising instructions that when executed by a suitable computer device cause the computer device to automatically perform the steps of the method according to claim 1. ### Claim 12 (depends on 1) A system comprising means adapted to carry out the steps of the method according to claim 1.