# US 12,190,864 B1 · Interest-based conversational recommendation system > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 12,190,864 B1 - Document ID: US12190864B1 - Status: Granted - Kind: B1 - Application number: 18/734,961 - Filed: 2024-06-05 - Granted: 2025-01-07 - Earliest claimed: 2024-06-05 - Assignees: Roku, Inc. - Applicants: Roku, Inc. - Inventors: Fei Xiao; Amit Verma; Rohit Mahto; Rameen Mahdavi; Nam Vo; Zidong Wang; Lian Liu; Jose Sanchez; Pulkit Aggarwal; Atishay Jain; Abhishek Bambha; Ronica Jethwa - Web page: https://patentgrep.com/patent/US12190864B1 ## CPC - G10L 15/063: {Training} - G06N 3/006: based on simulated virtual individual or collective life forms, e.g. social simulations or particle swarm optimisation [PSO] - G06N 3/044: Recurrent networks, e.g. Hopfield networks - G06N 3/045: Combinations of networks - G06N 3/08: Learning methods - G06N 7/01: Probabilistic graphical models, e.g. probabilistic networks - G10L 15/1822: {Parsing for meaning understanding} - G10L 2015/0635: {updating or merging of old and new templates; Mean values; Weighting} - G10L: SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING - G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS ## Abstract Disclosed herein are system, method and/or computer program product embodiments, and/or combinations thereof, for training a conversational recommendation system. An embodiment generates a probabilistic pseudo-user neural network model based on at least one interest probability distribution corresponding to a pseudo-user profile. The embodiment trains, using the pseudo-user neural network model, the conversational recommendation system to learn a recommendation policy, where the conversational recommendation system includes an interest-exploration engine and a prompt-decision engine. The training includes performing an iterative learning process that includes selecting an interest-exploration strategy based on one or more of the following: an interest-exploration policy, an earlier pseudo-user response generated by the pseudo-user neural network model, content data, and pseudo-user interaction history. The embodiment then generates, using the trained conversational recommendation system, a real-time recommendation having high play probability based on the minimal number of iterations of conversation between a user and the trained conversational recommendation system. ## Claims ### Claim 1 A computer-implemented method for training a conversational recommendation system for generating an output, having a high play-probability, based on a minimal number of iterations of conversation, comprising: generating, by at least one computer processor, a probabilistic pseudo-user neural network model based on at least one interest probability distribution corresponding to a pseudo-user profile; training, using the probabilistic pseudo-user neural network model, the conversational recommendation system to learn a recommendation policy, wherein the conversational recommendation system comprises an interest-exploration engine and a prompt-decision engine, and wherein the training includes performing one or more iterations of an iterative learning process, including: selecting, by the interest-exploration engine, an interest-exploration strategy based on one or more of the following: an interest-exploration policy, an earlier pseudo-user response generated by the probabilistic pseudo-user neural network model, content data, and pseudo-user interaction history; selecting, by the prompt-decision engine, an interest prompt based on a prompt-decision policy and the selected interest-exploration strategy; generating, by the probabilistic pseudo-user neural network model, another pseudo-user response based on the selected interest prompt; updating a reward function, corresponding to the interest-exploration engine and the prompt-decision engine, based on the another pseudo-user response; and updating, using a reinforcement-learning method, the interest-exploration policy and the prompt-decision policy based on at least the updated reward function; and generating, using the trained conversational recommendation system, a real-time recommendation having the high play-probability based on the minimal number of iterations of conversation between a user and the trained conversational recommendation system. ### Claim 2 (depends on 1) The method of claim 1, further comprising: terminating the iterative learning process if the another pseudo-user response comprises accepting to play a recommended media content corresponding to the selected interest prompt. ### Claim 3 (depends on 1) The method of claim 1, wherein the recommendation policy corresponds to the interest-exploration policy and the prompt-decision policy that cumulatively result in generating a pseudo-user response of accepting to play a recommended media content in a minimal number of iterations of the iterative learning process. ### Claim 4 (depends on 1) The method of claim 1, wherein the at least one interest probability distribution further comprises a long-term interest probability distribution and a short-term interest probability distribution corresponding to the pseudo-user profile. ### Claim 5 (depends on 1) The method of claim 1, wherein the another pseudo-user response comprises accepting to play a recommended media content corresponding to the selected interest prompt, quitting a conversation session with the conversational recommendation system, or generating a further pseudo-user response. ### Claim 6 (depends on 5) The method of claim 5, wherein the updating the reward function further comprises: incrementing the reward function by a predetermined value if the another pseudo-user response comprises accepting to play the recommended media content corresponding to the selected interest prompt; decrementing the reward function by a first value if the another pseudo-user response comprises quitting the conversation session with the conversational recommendation system; and decrementing the reward function by a second value if the another pseudo-user response comprises generating the further pseudo-user response. ### Claim 7 (depends on 1) The method of claim 1, wherein the selecting the interest-exploration strategy further comprises: extracting a current interest from the pseudo-user interaction history using named entity recognition; and performing an interest prediction based on the current interest. ### Claim 8 (depends on 1) The method of claim 1, wherein the selecting the interest-exploration strategy further comprises: selecting the interest-exploration strategy from a plurality of candidate interest-exploration strategies, including one or more of the following: exploration via an area target, exploration via a point target, exploration via a filtered target, exploration via a popular target, and exploration via a similar target. ### Claim 9 (depends on 1) The method of claim 1, wherein a response generated by the probabilistic pseudo-user neural network model is processed by an automatic speech recognition module and a natural language understanding module before being received by the interest exploration engine. ### Claim 10 (depends on 1) The method of claim 1, wherein an output of the prompt decision engine corresponding to the selected interest prompt is processed by a large language model and a text to speech module before being received by the probabilistic pseudo-user neural network model. ### Claim 11 A system, comprising: one or more memories; and at least one processor each coupled to at least one of the memories and configured to perform operations comprising: generating a probabilistic pseudo-user neural network model based on at least one interest probability distribution corresponding to a pseudo-user profile; training, using the probabilistic pseudo-user neural network model, a conversational recommendation system to learn a recommendation policy, wherein the conversational recommendation system comprises an interest-exploration engine and a prompt-decision engine, and wherein the training includes performing one or more iterations of an iterative learning process, including: selecting, by the interest-exploration engine, an interest-exploration strategy based on one or more of the following: an interest-exploration policy, an earlier pseudo-user response generated by the probabilistic pseudo-user neural network model, content data, and pseudo-user interaction history; selecting, by the prompt-decision engine, an interest prompt based on a prompt-decision policy and the selected interest-exploration strategy; generating, by the probabilistic pseudo-user neural network model, another pseudo-user response based on the selected interest prompt; updating a reward function, corresponding to the interest-exploration engine and the prompt-decision engine, based on the another pseudo-user response; and updating, using a reinforcement-learning method, the interest-exploration policy and the prompt-decision policy based on at least the updated reward function; and generating, using the trained conversational recommendation system, a real-time recommendation having a high play-probability based on a minimal number of iterations of conversation between a user and the trained conversational recommendation system. ### Claim 12 (depends on 11) The system of claim 11, the operations further comprising: terminating the iterative learning process if the another pseudo-user response comprises accepting to play a recommended media content corresponding to the selected interest prompt. ### Claim 13 (depends on 11) The system of claim 11, wherein the recommendation policy corresponds to the interest-exploration policy and the prompt-decision policy that cumulatively result in generating a pseudo-user response of accepting to play a recommended media content in a minimal number of iterations of the iterative learning process. ### Claim 14 (depends on 11) The system of claim 11, wherein the at least one interest probability distribution further comprises a long-term interest probability distribution and a short-term interest probability distribution corresponding to the pseudo-user profile. ### Claim 15 (depends on 11) The system of claim 11, wherein the another pseudo-user response comprises accepting to play a recommended media content corresponding to the selected interest prompt, quitting a conversation session with the conversational recommendation system, or generating a further pseudo-user response. ### Claim 16 A non-transitory computer-readable medium having instructions stored thereon that, when executed by at least one computing device, cause the at least one computing device to perform operations comprising: generating, by at least one computer processor, a probabilistic pseudo-user neural network model based on at least one interest probability distribution corresponding to a pseudo-user profile; training, using the probabilistic pseudo-user neural network model, a conversational recommendation system to learn a recommendation policy, wherein the conversational recommendation system comprises an interest-exploration engine and a prompt-decision engine, and wherein the training includes performing one or more iterations of an iterative learning process, including: selecting, by the interest-exploration engine, an interest-exploration strategy based on one or more of the following: an interest-exploration policy, an earlier pseudo-user response generated by the probabilistic pseudo-user neural network model, content data, and pseudo-user interaction history; selecting, by the prompt-decision engine, an interest prompt based on a prompt-decision policy and the selected interest-exploration strategy; generating, by the probabilistic pseudo-user neural network model, another pseudo-user response based on the selected interest prompt; updating a reward function, corresponding to the interest-exploration engine and the prompt-decision engine, based on the another pseudo-user response; and updating, using a reinforcement-learning method, the interest-exploration policy and the prompt-decision policy based on at least the updated reward function; and generating, using the trained conversational recommendation system, a real-time recommendation having a high play-probability based on a minimal number of iterations of conversation between a user and the trained conversational recommendation system. ### Claim 17 (depends on 16) The non-transitory computer-readable medium of claim 16, the operations further comprising: terminating the iterative learning process if the another pseudo-user response comprises accepting to play a recommended media content corresponding to the selected interest prompt. ### Claim 18 (depends on 16) The non-transitory computer-readable medium of claim 16, wherein the recommendation policy corresponds to the interest-exploration policy and the prompt-decision policy that cumulatively result in generating a pseudo-user response of accepting to play a recommended media content in a minimal number of iterations of the iterative learning process. ### Claim 19 (depends on 16) The non-transitory computer-readable medium of claim 16, wherein the at least one interest probability distribution further comprises a long-term interest probability distribution and a short-term interest probability distribution corresponding to the pseudo-user profile. ### Claim 20 (depends on 16) The non-transitory computer-readable medium of claim 16, wherein the another pseudo-user response comprises accepting to play a recommended media content corresponding to the selected interest prompt, quitting a conversation session with the conversational recommendation system, or generating a further pseudo-user response.