# US 10,122,860 B1 · Techniques for estimating expected performance in a task assignment system > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 10,122,860 B1 - Document ID: US10122860B1 - Status: Granted - Kind: B1 - Application number: 15/645,277 - Filed: 2017-07-10 - Granted: 2018-11-06 - Earliest claimed: 2017-07-10 - Assignees: Afiniti Europe Technologies Limited - Applicants: Afiniti, Ltd. - Inventors: Zia Chishti; Ittai Kan; Vikash Khatri - Web page: https://patentgrep.com/patent/US10122860B1 ## CPC - H04M 3/5232: {Call distribution algorithms} - G06Q 10/06311: {Scheduling, planning or task assignment for a person or group} - H04M 3/5175: {Call or contact centers supervision arrangements} - H04M 2203/401: Performance feedback - H04M 2203/402: Agent or workforce management - H04M: TELEPHONIC COMMUNICATION (circuits for controlling other apparatus via a telephone cable and not involving telephone switching apparatus G08) - G06Q: INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR ## Abstract Techniques for estimating expected performance of a task assignment strategy in a task assignment system are disclosed. In one particular embodiment, the techniques may be realized as a method comprising receiving, by at least one computer processor communicatively coupled to a task assignment system, a plurality of historical agent task assignments; determining, by the at least one computer processor, a sample of the plurality based on a strategy for pairing agents with tasks; determining, by the at least one computer processor, an expected performance of the strategy based on the sample; outputting, by the at least one computer processor, the expected performance; and optimizing, by the at least one computer processor, the performance of the task assignment system based on the expected performance. ## Claims ### Claim 1 A method comprising: receiving, by at least one switch configured to perform task assignment operations in a task assignment system, a plurality of historical agent task assignments; partitioning, by at least one computer processor communicatively coupled to and configured to operate in the task assignment system, the plurality of historical agent task assignments into a first subset of historical agent task assignments and a holdout subset of historical agent task assignments different from the first subset; determining, by the at least one computer processor, a sample of the plurality of historical agent task assignments by sampling the plurality of historical agent task assignments according to a strategy for pairing agents with tasks, wherein the strategy is generated based on the first subset, wherein the sample is a subset of the holdout subset; determining, by the at least one computer processor, an expected performance of the strategy based on the sample, wherein determining the expected performance is based on the holdout subset; and outputting, by the at least one computer processor, the expected performance for use in pairing agents with tasks in the task assignment system based upon the strategy; wherein actual performance of the task assignment system is optimized by using the strategy based on the expected performance. ### Claim 2 (depends on 1) The method of claim 1, wherein the task assignment system is a contact center, and wherein the strategy assigns contacts to contact center agents. ### Claim 3 (depends on 1) The method of claim 1, further comprising: oversampling, by the at least one computer processor, the plurality of historical agent task assignments, wherein at least one of the plurality of oversampled historical agent task assignment is included in each of a plurality of samples. ### Claim 4 (depends on 1) The method of claim 1, further comprising: determining, by the at least one computer processor, a bias in the sample; and accounting, by the at least one computer processor, for the bias in the expected performance. ### Claim 5 (depends on 4) The method of claim 4, wherein the bias is attributable to an overrepresentation of a subset of agents in the sample or an overrepresentation of a subset of task types in the sample. ### Claim 6 (depends on 1) The method of claim 1, wherein determining the expected performance comprises: determining, by the at least one computer processor, a plurality of samples of the plurality of historical agent task assignments. ### Claim 7 (depends on 1) The method of claim 1, further comprising: over-representing in the holdout subset historical agent task assignments attributable to a second strategy for pairing agents with tasks different from the strategy. ### Claim 8 (depends on 1) The method of claim 1, wherein determining the expected performance is based on a plurality of holdout subsets. ### Claim 9 (depends on 8) The method of claim 8, further comprising determining a standard error associated with the expected performance based on the plurality of holdout subsets. ### Claim 10 (depends on 1) The method of claim 1, wherein the sample is associated with an amount of performance difference between expected performance of the sample and peak performance of the strategy. ### Claim 11 A method comprising: receiving, by at least one switch configured to perform task assignment operations in a task assignment system, a plurality of historical agent task assignments; partitioning, by at least one computer processor communicatively coupled to and configured to operate in the task assignment system, the plurality of historical agent task assignments into a first subset of historical agent task assignments and a holdout subset of historical agent task assignments different from the first subset; determining, by the at least one computer processor, a weighting of at least one of the plurality of historical agent task assignments by weighting the plurality of historical agent task assignments according to a strategy for pairing agents with tasks, wherein the strategy is generated based on the first subset; determining, by the at least one computer processor, an expected performance of the strategy based on the weighting, wherein determining the expected performance is based on the holdout subset; and outputting, by the at least one computer processor, the expected performance for use in pairing agents with tasks in the task assignment system based upon the strategy; wherein actual performance of the task assignment system is optimized using the strategy based on the expected performance. ### Claim 12 (depends on 11) The method of claim 11, wherein the task assignment system is a contact center, and wherein the strategy assigns contacts to contact center agents. ### Claim 13 (depends on 11) The method of claim 11, wherein at least one weighting is zero or epsilon. ### Claim 14 (depends on 11) The method of claim 11, further comprising: determining, by the at least one computer processor, a bias in the sample; and accounting, by the at least one computer processor, for the bias in the expected performance. ### Claim 15 (depends on 14) The method of claim 14, wherein the bias is attributable to an overweighting of a subset of agents or an overweighting of a subset of task types. ### Claim 16 (depends on 11) The method of claim 11, wherein determining the expected performance comprises combining at least two weighted outcomes corresponding to at least two weighted pairings in the sample. ### Claim 17 A system comprising: at least one computer processor communicatively coupled to and configured to perform task assignment operations in a task assignment system, wherein the at least one computer processor is configured to: receive a plurality of historical agent task assignments at a switch configured to perform task assignment operations in the task assignment system; partition the plurality of historical agent task assignments into a first subset of historical agent task assignments and a holdout subset of historical agent task assignments different from the first subset; determine a sample of the plurality of historical agent task assignments by sampling the plurality of historical agent task assignments according to a strategy for pairing agents with tasks, wherein the strategy is generated based on the first subset, wherein the sample is a subset of the holdout subset; determine an expected performance of the strategy based on the sample, wherein determining the expected performance is based on the holdout subset; and output the expected performance for use in pairing agents with tasks in the task assignment system based upon the strategy; wherein actual performance of the task assignment system is optimized by using the strategy based on the expected performance. ### Claim 18 (depends on 17) The system of claim 17 wherein the task assignment system is a contact center, and wherein the strategy assigns contacts to contact center agents. ### Claim 19 (depends on 17) The system of claim 17, wherein the at least one computer processor is further configured to: oversample the plurality of historical agent task assignments, wherein at least one of the plurality of oversampled historical agent task assignments is included in each of a plurality of samples. ### Claim 20 (depends on 17) The system of claim 17, wherein the at least one computer processor is further configured to: determine a bias in the sample; and account for the bias in the expected performance. ### Claim 21 (depends on 20) The system of claim 20, wherein the bias is attributable to an overrepresentation of a subset of agents in the sample or an overrepresentation of a subset of task types in the sample. ### Claim 22 (depends on 17) The system of claim 17, wherein determining the expected performance comprises determining a plurality of samples of the plurality of historical agent task assignments. ### Claim 23 An article of manufacture comprising: a non-transitory processor readable medium; and instructions stored on the medium; wherein the instructions are configured to be readable from the medium by at least one computer processor communicatively coupled to and configured to perform task assignment operations in a task assignment system and thereby cause the at least one computer processor to operate so as to: receive a plurality of historical agent task assignments at a switch configured to perform task assignment operations in the task assignment system; partition the plurality of historical agent task assignments into a first subset of historical agent task assignments and a holdout subset of historical agent task assignments different from the first subset; determine a sample of the plurality of historical agent task assignments by sampling the plurality of historical agent task assignments according to a strategy for pairing agents with tasks, wherein the strategy is generated based on the first subset, wherein the sample is a subset of the holdout subset; determine an expected performance of the strategy based on the sample, wherein determining the expected performance is based on the holdout subset; and output the expected performance for use in pairing agents with tasks in the task assignment system based upon the strategy; wherein actual performance of the task assignment system is optimized by using the strategy based on the expected performance. ### Claim 24 (depends on 23) The article of manufacture of claim 23, wherein the task assignment system is a contact center, and wherein the strategy assigns contacts to contact center agents. ### Claim 25 (depends on 23) The article of manufacture of claim 23, wherein the at least one computer processor is further caused to operate so as to: oversample the plurality of historical agent task assignments, wherein at least one of the plurality of oversampled historical agent task assignments is included in each of a plurality of samples. ### Claim 26 (depends on 23) The article of manufacture of claim 23, wherein the at least one computer processor is further caused to operate so as to: determine a bias in the sample; and account for the bias in the expected performance. ### Claim 27 (depends on 26) The article of manufacture of claim 26, wherein the bias is attributable to an overrepresentation of a subset of agents in the sample or an overrepresentation of a subset of task types in the sample. ### Claim 28 (depends on 23) The article of manufacture of claim 23, wherein determining the expected performance comprises determining a plurality of samples of the plurality of historical agent task assignments.