# US 2021/0240779 A1 · Routing of cases using unstructured input and natural language processing > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 2021/0240779 A1 - Document ID: US20210240779A1 - Status: Published application - Kind: A1 - Application number: 17/239,295 - Filed: 2021-04-23 - Published: 2021-08-05 - Earliest claimed: 2018-09-24 - Assignees: salesforce.com, inc. - Applicants: salesforce.com, inc. - Inventors: Sitaram ASUR; Aditya SAKHUJA; Hui S. FISHER; Anjan GOSWAMI; Khoa LE - Web page: https://patentgrep.com/patent/US20210240779A1 ## CPC - G06F 16/90332: {Natural language query formulation or dialogue systems} - G06F 9/453: {Help systems} - G06F 16/3329: Natural language query formulation - G06F 40/295: Named entity recognition - G06F 40/30: Semantic analysis - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) ## Abstract A method is provided for generating a ranked list of candidate responders. In some embodiments, the method includes receiving a question from a user and generating a question feature vector representing an intent of the question and a first skill set inferred from the question. The method also includes for one or more candidate responders, generating a candidate feature vector representing a skill set and questions associated with the respective candidate responder; computing a reputation score based on questions and user feedback associated with the respective candidate responder; and computing, based on the question feature vector, candidate feature vector, and reputation score, a probability score representing a prediction of the quality of an answer that would be provided by the respective candidate responder if the input question were routed to the respective candidate responder. The method further includes generating a ranked list of candidate responders using the computed probability scores. ## Claims ### Claim 1 A computing device comprising: a memory containing machine readable medium storing machine executable code; and one or more processors coupled to the memory and configurable to execute the machine executable code to cause the one or more processors to: receive a text input question from a user; generate a question feature vector representing an intent of the text input question and a first skill set inferred from the text input question; for each of one or more candidate responders: compute a reputation score based on a set of input questions previously answered by a respective candidate responder and user feedback for the respective candidate responder using a set of weights, wherein the weights are provided by domain experts or are learned in a training process; compute, based on the question feature vector and the reputation score, a probability score representing a prediction of a quality of an answer that would be provided by the respective candidate responder to the text input question if the text input question were routed to the respective candidate responder; generate a ranked list of candidate responders using the probability score; and select a responder in response to the text input question based on the ranked list of candidate responders. ### Claim 2 (depends on 1) The computing device of claim 1, wherein the machine executable code further causes the one or more processors to: generate a candidate feature vector representing a skill set of the respective candidate responder, a first set of input questions previously answered by the respective candidate responder, and metadata profile information provided by the respective candidate responder, the metadata profile information including one or more candidate skill sets and one or more candidate skill levels corresponding to the one or more candidate skill sets. ### Claim 3 (depends on 1) The computing device of claim 1, wherein the question feature vector represents a topic of the input question. ### Claim 4 (depends on 1) The computing device of claim 1, wherein the candidate feature vector represents at least one of metadata profile information or inferred topics derived from a candidate responder's posts. ### Claim 5 (depends on 1) The computing device of claim 1, wherein the machine executable code further causes the one or more processors to: for each of one or more candidate responders, collect activity information for the respective candidate responder, wherein the probability score is computed based on the activity information. ### Claim 6 (depends on 3) The computing device of claim 3, wherein the activity information includes at least one of the candidate responder's average lag time in responding to a set of questions, posting activity and frequency, or login activity. ### Claim 7 (depends on 1) The computing device of claim 1, wherein the machine executable code further causes the one or more processors to: receive user feedback indicating a quality of one or more answers previously provided by a candidate responder to one or more questions, wherein the reputation score is based on the user feedback. ### Claim 8 (depends on 7) The computing device of claim 7, wherein the machine executable code further causes the one or more processors to: receive an indication that a user has selected a user-selectable option that when selected provides the user feedback. ### Claim 9 (depends on 1) The computing device of claim 1, wherein the machine executable code further causes the one or more processors to: send a notification of the input question to a first listed candidate responder in the ranked list of candidate responders; and identify a schedule for sending notifications to candidate responders, wherein the notification is sent in accordance with the schedule. ### Claim 10 (depends on 9) The computing device of claim 9, wherein the machine executable code further causes the one or more processors to: determine that a time period has elapsed before the first candidate responder has responded to the input question; and in response to a determination that the time period has elapsed, send the notification to the second listed candidate responder in the ranked list of candidate responders. ### Claim 11 A method executed by one or more processors, the method comprising: receiving a text input question from a user; generating a question feature vector representing an intent of the text input question and a first skill set inferred from the text input question; for each of one or more candidate responders: computing a reputation score based on a set of input questions previously answered by a respective candidate responder and user feedback for the respective candidate responder using a set of weights, wherein the weights are provided by domain experts or are learned in a training process; computing, based on the question feature vector and the reputation score, a probability score representing a prediction of a quality of an answer that would be provided by the respective candidate responder to the text input question if the text input question were routed to the respective candidate responder; generating a ranked list of candidate responders using the probability score; and selecting a responder in response to the text input question based on the ranked list of candidate responders. ### Claim 12 (depends on 11) The method of claim 11, further comprising: generating a candidate feature vector representing a skill set of the respective candidate responder, a first set of input questions previously answered by the respective candidate responder, and metadata profile information provided by the respective candidate responder, the metadata profile information including one or more candidate skill sets and one or more candidate skill levels corresponding to the one or more candidate skill sets. ### Claim 13 (depends on 11) The method of claim 11, wherein the question feature vector represents a topic of the input question. ### Claim 14 (depends on 11) The method of claim 11, wherein the candidate feature vector represents at least one of metadata profile information or inferred topics derived from a candidate responder's posts. ### Claim 15 (depends on 11) The method of claim 11, further comprising: for each of one or more candidate responders, collecting activity information for the respective candidate responder, wherein the probability score is computed based on the activity information. ### Claim 16 (depends on 13) The method of claim 13, wherein the activity information includes at least one of the candidate responder's average lag time in responding to a set of questions, posting activity and frequency, or login activity. ### Claim 17 (depends on 11) The method of claim 11, further comprising: receiving user feedback indicating a quality of one or more answers previously provided by a candidate responder to one or more questions, wherein the reputation score is based on the user feedback. ### Claim 18 (depends on 17) The method of claim 17, further comprising: receiving an indication that a user has selected a user-selectable option that when selected provides the user feedback. ### Claim 19 (depends on 11) The method of claim 11, further comprising: sending a notification of the input question to a first listed candidate responder in the ranked list of candidate responders; identifying a schedule for sending notifications to candidate responders, wherein the notification is sent in accordance with the schedule; determining that a time period has elapsed before the first candidate responder has responded to the input question; and in response to a determination that the time period has elapsed, sending the notification to the second listed candidate responder in the ranked list of candidate responders. ### Claim 20 A non-transitory machine-readable medium comprising executable code which when executed by one or more processors associated with a computing device are adapted to cause the one or more processors to perform a method comprising: receiving a text input question from a user; generating a question feature vector representing an intent of the text input question and a first skill set inferred from the text input question; for each of one or more candidate responders: computing a reputation score based on a set of input questions previously answered by a respective candidate responder and user feedback for the respective candidate responder using a set of weights, wherein the weights are provided by domain experts or are learned in a training process; computing, based on the question feature vector and the reputation score, a probability score representing a prediction of a quality of an answer that would be provided by the respective candidate responder to the text input question if the text input question were routed to the respective candidate responder; generating a ranked list of candidate responders using the probability score; and selecting a responder in response to the text input question based on the ranked list of candidate responders. receiving user feedback indicating a quality of one or more answers previously provided by a candidate responder to one or more questions, wherein the reputation score is based on the user feedback.