# US 12,602,313 B2 · Smart mechanism to evaluate performance of test engineers > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 12,602,313 B2 - Document ID: US12602313B2 - Status: Granted - Kind: B2 - Application number: 18/367,515 - Filed: 2023-09-13 - Published: 2025-01-30 - Granted: 2026-04-14 - Earliest claimed: 2023-07-28 - Assignees: Dell Products L.P. - Applicants: Dell Products L.P. - Inventors: Huijuan Fan; Steven Wenguo Shen - Web page: https://patentgrep.com/patent/US12602313B2 ## CPC - G06F 11/3692: {for test results analysis} - G06F 11/3698: Environments for analysis, debugging or testing of software - G06F 11/3616: {using software metrics} - G06F 11/3688: {for test execution, e.g. scheduling of test suites} - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) ## Abstract Methods, system, and non-transitory processor-readable storage medium for a resource evaluation system are provided herein. An example method includes obtaining defect data obtained in response to executing software on a plurality of tester systems. The resource evaluation system determines a performance weighted metric associated with a defect detection resource and the defect data. Using the performance weighted metric, the resource evaluation system determines a defect scope delta percentage associated with the defect detection resource and the defect data. The resource evaluation system then determines a defect detection resource score based on the defect scope delta percentage. ## Claims ### Claim 1 A method, comprising: obtaining, by a resource evaluation system configured to evaluate performance of test engineers, defect data obtained in response to executing software on a plurality of tester systems, wherein the defect data comprises defect priorities associated with defects detected by individual engineers; determining, by the resource evaluation system, a performance weighted metric associated with a defect detection resource and the defect data; using the performance weighted metric, determining a defect scope delta percentage associated with the defect detection resource and the defect data; and determining a defect detection resource score based on the defect scope delta percentage, wherein the defect detection resource score evaluates individual test engineer performance, wherein the performance comprises productivity of the test engineer and the test engineer's skill breadth and depth of knowledge, wherein the method is implemented by at least one processing device comprising a processor coupled to a memory. ### Claim 2 (depends on 1) The method of claim 1 wherein obtaining, by the resource evaluation system, defect data comprises: executing the software on a plurality of tester systems; obtaining defect data resulting from the software execution; associating the defect data with the defect detection resource; and storing the defect data in a defect database. ### Claim 3 (depends on 1) The method of claim 1 wherein the performance weighted metric comprises a defect priority associated with the defect data. ### Claim 4 (depends on 3) The method of claim 3 wherein the defect priority is associated with the defect detection resource, wherein the defect detection resource generated the defect data. ### Claim 5 (depends on 1) The method of claim 1 wherein the performance weighted metric comprises a priority weight associated with a defect priority associated with the defect data. ### Claim 6 (depends on 1) The method of claim 1 wherein the performance weighted metric comprises a plurality of priority weights each associated with a respective defect priority in a plurality of defect priorities associated with the defect data, wherein the plurality of priority weights, when aggregated, equal one. ### Claim 7 (depends on 1) The method of claim 1 wherein the performance weighted metric comprises a plurality of defects detected associated with a defect priority associated with the defect data, wherein the plurality of defects detected is weighted with a priority weight associated with a defect priority associated with the defect data. ### Claim 8 (depends on 7) The method of claim 7 wherein the performance weighted metric comprises aggregating a plurality of weighted plurality of defects detected, wherein each of the plurality of weighted plurality of defects detected is associated with a respective defect priority and weighted with a respective priority weight. ### Claim 9 (depends on 1) The method of claim 1 wherein determining a defect scope delta percentage comprises: calculating a percentage difference between the performance weighted metric and a historical performance weighted metric associated with a plurality of defect detection resources. ### Claim 10 (depends on 9) The method of claim 9 wherein the historical performance weighted metric comprises a plurality of defects detected associated with the defect detection resource. ### Claim 11 (depends on 9) The method of claim 9 wherein the historical performance weighted metric comprises a plurality of defects detected associated with a second defect detection resource, wherein the second defect detection resource is associated with a defect priority associated with the defect data. ### Claim 12 (depends on 9) The method of claim 9 wherein the historical performance weighted metric comprises a plurality of defects detected associated with a second defect detection resource, wherein the second defect detection resource is associated with a second defect priority associated with the defect data. ### Claim 13 (depends on 12) The method of claim 12 wherein, when determining the defect scope delta percentage, an impact factor is applied to the second defect priority associated with the defect data. ### Claim 14 (depends on 1) The method of claim 1 wherein determining a defect detection resource score comprises: determining a test case percentage associated with the defect detection resource, wherein the test case percentage indicates a percentage of test cases the defect detection resource executed with respect to a plurality of defect detection resources. ### Claim 15 (depends on 1) The method of claim 1 wherein determining a defect detection resource score comprises: normalizing the defect scope delta percentage and the test case percentage. ### Claim 16 (depends on 15) The method of claim 15 further comprising: weighting the defect scope delta percentage with a defect weight; and weighting the test case percentage with a test case weight. ### Claim 17 (depends on 16) The method of claim 16 wherein the defect weight and test case weight, when added together equal one. ### Claim 18 (depends on 16) The method of claim 16 further comprising: tuning at least one of the defect weight and the test case weight. ### Claim 19 A system comprising: at least one processing device comprising a processor coupled to a memory; the at least one processing device being configured: to obtain, by a resource evaluation system configured to evaluate performance of test engineers, defect data obtained in response to executing software on a plurality of tester systems, wherein the defect data comprises defect priorities associated with defects detected by individual engineers; to determine, by the resource evaluation system, a performance weighted metric associated with a defect detection resource and the defect data; to use the performance weighted metric, determining a defect scope delta percentage associated with the defect detection resource and the defect data; and to determine a defect detection resource score based on the defect scope delta percentage, wherein the defect detection resource score evaluates individual test engineer performance, wherein the performance comprises productivity of the test engineer and the test engineer's skill breadth and depth of knowledge. ### Claim 20 A computer program product comprising a non-transitory processor-readable storage medium having stored therein program code of one or more software programs, wherein the program code when executed by at least one processing device causes said at least one processing device: to obtain, by a resource evaluation system configured to evaluate performance of test engineers, defect data obtained in response to executing software on a plurality of tester systems, wherein the defect data comprises defect priorities associated with defects detected by individual engineers; to determine, by the resource evaluation system, a performance weighted metric associated with a defect detection resource and the defect data; to use the performance weighted metric, determining a defect scope delta percentage associated with the defect detection resource and the defect data; and to determine a defect detection resource score based on the defect scope delta percentage, wherein the defect detection resource score evaluates individual test engineer performance, wherein the performance comprises productivity of the test engineer and the test engineer's skill breadth and depth of knowledge.