# US 10,319,381 B2 · Iteratively updating parameters for dialog states > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 10,319,381 B2 - Document ID: US10319381B2 - Status: Granted - Kind: B2 - Application number: 15/896,738 - Filed: 2018-02-14 - Published: 2018-06-21 - Granted: 2019-06-11 - Earliest claimed: 2015-11-12 - Assignees: Semantic Machines, Inc. - Applicants: Semantic Machines, Inc. - Inventors: Jacob Daniel Andreas; Daniel Lawrence Roth; Jesse Daniel Eskes Rusak; Andrew Robert Volpe; Steven Andrew Wegmann; Taylor Darwin Berg-Kirkpatrick; Pengyu Chen; Jordan Rian Cohen; Laurence Steven Gillick; David Leo Wright Hall; Daniel Klein; Michael Newman; Adam David Pauls - Web page: https://patentgrep.com/patent/US10319381B2 ## CPC - G10L 15/22: Procedures used during a speech recognition process, e.g. man-machine dialogue - G06F 40/169: Annotation, e.g. comment data or footnotes - G06F 40/186: Templates - G06F 40/35: Discourse or dialogue representation - G06N 3/006: based on simulated virtual individual or collective life forms, e.g. social simulations or particle swarm optimisation [PSO] - G06N 3/0442: characterised by memory or gating, e.g. long short-term memory [LSTM] or gated recurrent units [GRU] - G06N 3/0464: Convolutional networks [CNN, ConvNet] - G06N 3/082: modifying the architecture, e.g. adding, deleting or silencing nodes or connections - G06N 3/0895: Weakly supervised learning, e.g. semi-supervised or self-supervised learning - G06N 3/09: Supervised learning - G06N 3/092: Reinforcement learning - G10L 15/063: {Training} - G10L 15/1822: {Parsing for meaning understanding} - G10L 15/26: Speech to text systems (G10L15/08 takes precedence) - G10L 25/51: for comparison or discrimination - G06N 3/08: Learning methods - G10L 2015/223: {Execution procedure of a spoken command} - G10L: SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) - G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS ## Abstract An interaction assistant conducts multiple turn interaction dialogs with a user in which context is maintained between turns, and the system manages the dialog to achieve an inferred goal for the user. The system includes a linguistic interface to a user and a parser for processing linguistic events from the user. A dialog manager of the system is configured to receive alternative outputs from the parser, and selecting an action and causing the action to be performed based on the received alternative outputs. The system further includes a dialog state for an interaction with the user, and the alternative outputs represent alternative transitions from a current dialog state to a next dialog state. The system further includes a storage for a plurality of templates, and wherein each dialog state is defined in terms of an interrelationship of one or more instances of the templates. ## Claims ### Claim 1 A method for determining parameter values for a plurality of components of an interaction system, the system being configured to process ordered pluralities of events, the events including linguistic events including speech utterances, and application related events; the processing of events including parsing of linguistic events via automatic speech recognition to produce text-based representations of the speech utterances, determining an ordered plurality of dialog states, and determining an ordered plurality of output actions from an ordered plurality of events corresponding to the ordered plurality of dialog states; and the method comprising: collecting a training data set including one or more ordered pluralities of events and, for each ordered plurality of events, a corresponding ordered plurality of output actions; repeating an iteration, each iteration including processing an ordered plurality of events and a corresponding ordered plurality of output actions by: processing the ordered plurality of events using current parameter values of the system, the processing including determining an ordered plurality of dialog states from the ordered plurality of events, determining an ordered plurality of output actions from the ordered plurality of dialog states, and using a comparison of the determined ordered plurality of output actions and the collected ordered plurality of output actions to update trainable parameter values of the plurality of components of the interaction system; and completing the repeating of the iterations upon reaching a stopping condition; and setting the trainable parameter values for the plurality of components of the interaction system using a result of the iterations. ### Claim 2 (depends on 1) The method of claim 1 wherein the components of the system include at least one of an automated speech recognizer, a parser, and a dialog manager for selecting actions proposed by the parser. ### Claim 3 (depends on 2) The method of claim 2 wherein the trainable parameter values of the components include neural network parameters and/or machine learning parameters. ### Claim 4 (depends on 1) The method of claim 1, wherein collecting the training data set includes: recognizing an exemplary linguistic event during natural language interaction with a human user; presenting the exemplary linguistic event to a human trainer; and receiving, from the human trainer, an exemplary output action to associate with the exemplary linguistic event. ### Claim 5 (depends on 1) The method of claim 1, wherein updating the trainable parameter values of the plurality of components of the interaction system includes selecting a best weighted execution trace and incrementing the trainable parameter values based at least on a comparison between the best weighted execution trace and the ordered plurality of output actions. ### Claim 6 (depends on 1) The method of claim 1, wherein updating the trainable parameters of the interaction system includes: assessing a comparison score based on comparing the determined ordered plurality of output actions to an exemplary ordered plurality of output actions from the training data set; and updating the trainable parameters of the interaction system based on the assessed comparison score to improve a likelihood of generating a correct ordered plurality of output actions with regard to a future ordered plurality of events. ### Claim 7 A method for training a plurality of components of an interaction system, the method comprising: collecting a training data set including one or more ordered pluralities of events, and, for each ordered plurality of events, a corresponding ordered plurality of output actions, wherein collecting the training data set includes: recognizing an exemplary linguistic event during natural language interaction with a human user; presenting the exemplary linguistic event to a human trainer; and receiving, from the human trainer, an exemplary output action to associate with the exemplary linguistic event; and for an exemplary ordered plurality of events and a corresponding exemplary ordered plurality of output actions: generating an ordered plurality of output actions based on the exemplary ordered plurality of events and current parameter values for trainable parameters of the interaction system; comparing the generated ordered plurality of output actions to the exemplary ordered plurality of output actions; and updating the trainable parameters of the interaction system based on the comparison. ### Claim 8 (depends on 7) The method of claim 7, wherein receiving the exemplary output action from the human trainer includes presenting the human trainer with a plurality of candidate output action plans, and receiving input from the human trainer indicating a most appropriate candidate output action plan. ### Claim 9 (depends on 7) The method of claim 7, wherein the ordered plurality of events includes a speech utterance, and the method further comprises performing automatic speech recognition to produce a text-based representation of the speech utterance. ### Claim 10 (depends on 7) The method of claim 7, wherein the components of the system include at least one of an automated speech recognizer, a parser, and a dialog manager for selecting actions proposed by the parser. ### Claim 11 (depends on 10) The method of claim 10, wherein the trainable parameter values of the components include neural network parameters and/or machine learning parameters. ### Claim 12 (depends on 7) The method of claim 7, wherein updating the trainable parameter values of the plurality of components of the interaction system includes selecting a best weighted execution trace and incrementing the trainable parameter values based at least on a comparison between the best weighted execution trace and the ordered plurality of output actions. ### Claim 13 (depends on 7) The method of claim 7, wherein updating the trainable parameters of the interaction system includes: assessing a comparison score based on comparing the determined ordered plurality of output actions to an exemplary ordered plurality of output actions from the training data set; and updating the trainable parameters of the interaction system based on the assessed comparison score to improve a likelihood of generating a correct ordered plurality of output actions with regard to a future ordered plurality of events. ### Claim 14 A method for training a plurality of components of an interaction system, the method comprising: collecting a training data set including one or more ordered pluralities of events, and, for each ordered plurality of events, a corresponding ordered plurality of output actions; for an exemplary ordered plurality of events and a corresponding exemplary ordered plurality of output actions: generating an ordered plurality of output actions based on the exemplary ordered plurality of events and current parameter values for trainable parameters of the interaction system; assessing a comparison score based on comparing the generated ordered plurality of output actions to the exemplary ordered plurality of output actions; and updating the trainable parameters of the interaction system based on the assessed comparison score to improve a likelihood of generating a correct ordered plurality of output actions with regard to a future ordered plurality of events. ### Claim 15 (depends on 14) The method of claim 14, wherein the ordered plurality of events includes a speech utterance, and the method further comprises performing automatic speech recognition to produce a text-based representation of the speech utterance. ### Claim 16 (depends on 14) The method of claim 14, wherein the components of the system include at least one of an automated speech recognizer, a parser, and a dialog manager for selecting actions proposed by the parser. ### Claim 17 (depends on 16) The method of claim 16, wherein the trainable parameter values of the components include neural network parameters and/or machine learning parameters. ### Claim 18 (depends on 14) The method of claim 14, wherein updating the trainable parameter values of the plurality of components of the interaction system includes selecting a best weighted execution trace and incrementing the trainable parameter values based at least on a comparison between the best weighted execution trace and the ordered plurality of output actions. ### Claim 19 (depends on 14) The method of claim 14, wherein collecting the training data set includes: recognizing an exemplary linguistic event during natural language interaction with a human user; presenting the exemplary linguistic event to a human trainer; and receiving, from the human trainer, an exemplary output action to associate with the exemplary linguistic event. ### Claim 20 (depends on 19) The method of claim 19, wherein receiving the exemplary output action from the human trainer includes presenting the human trainer with a plurality of candidate output action plans, and receiving input from the human trainer indicating a most appropriate candidate output action plan.