# US 11,307,974 B2 · Horizontally scalable distributed system for automated firmware testing and method thereof > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 11,307,974 B2 - Document ID: US11307974B2 - Status: Granted - Kind: B2 - Application number: 17/012,923 - Filed: 2020-09-04 - Published: 2022-03-10 - Granted: 2022-04-19 - Earliest claimed: 2020-09-04 - Assignees: SK hynix Inc. - Applicants: SK hynix Inc. - Inventors: Aleh Karatkevich; Kiryl Smalouski; Farhad Jabiyev; Raman Piatrovich; Uladzislau Savik; Mikalai Zialenkin - Web page: https://patentgrep.com/patent/US11307974B2 ## CPC - G06F 11/3688: {for test execution, e.g. scheduling of test suites} - G06F 9/546: {Message passing systems or structures, e.g. queues} - G06F 9/547: {Remote procedure calls [RPC]; Web services} - G06F 11/328: {Computer systems status display (G06F11/327 takes precedence)} - G06F 11/3676: {for coverage analysis} - G06F 11/3684: {for test design, e.g. generating new test cases} - G06F 11/3692: {for test results analysis} - H04L 67/02: based on web technology, e.g. hypertext transfer protocol [HTTP] - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) - H04L: TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION (arrangements common to telegraphic and telephonic communication H04M) ## Abstract A system and method for automated firmware testing. The system includes test stations for testing firmware products. The stations split into pools, with each pool including multiple test stations. The system also includes multiple execution instances, each execution instance for executing tests corresponding to the associated pool. Each of competing test stations delivers a test start event to a corresponding execution instance. The corresponding execution instance receives test start events from the competing test stations, and executes a run test command on a select test station among the competing test stations such that the select test station performs test execution based on a test sequence. ## Claims ### Claim 1 A system comprising: a plurality of test stations for testing a plurality of firmware products, which stations are split into multiple pools, each pool including multiple test stations; and multiple execution instances, each execution instance for executing tests corresponding to an associated pool, wherein each of competing test stations among the multiple test stations delivers a competing test start event, comprising a request to start testing, to a corresponding execution instance, wherein the corresponding execution instance receives competing test start events from the competing test stations in the associated pool, and executes a run test command on a select test station among the competing test stations such that the select test station performs test execution based on a test sequence, wherein the multiple test stations in each pool compete for tests, and wherein the plurality of test stations is split into the multiple pools based on set characteristics, wherein the test stations in a given pool have at least one of the set characteristics in common. ### Claim 2 (depends on 1) The system of claim 1, wherein the select test station is a test station which firstly delivered a corresponding test start event to the corresponding execution instance, among the competing test stations. ### Claim 3 (depends on 2) The system of claim 2, wherein the corresponding execution instance includes: state queues; an agent manager coupled to the multiple test stations, for storing state information on the multiple test stations in the state queues, based on a state of each of the multiple test stations; and a test engine for determining the select test station based on the state information, calling the agent manager to execute the run test command on the select test station. ### Claim 4 (depends on 3) The system of claim 3, wherein each of the multiple test stations is in an idle, running or completed state with respect to test execution, and wherein the state queues include an idle queue, a running queue, and a completed queue for storing idle, running and completed states of each of the multiple test stations, respectively. ### Claim 5 (depends on 3) The system of claim 3, wherein the agent manager is coupled to the multiple test stations through a remote procedure call (RPC) protocol, and is coupled to the test engine through a hypertext transfer protocol (HTTP) interface. ### Claim 6 (depends on 1) The system of claim 1, wherein the corresponding execution instance is ready to be executed in response to a start test sequence command from a user interface. ### Claim 7 (depends on 6) The system of claim 6, wherein the corresponding execution instance receives a test completed event and a test result from the select test station, and provides the user interface with the test result in response to the test completed event. ### Claim 8 (depends on 7) The system of claim 7, wherein the corresponding execution instance includes an incoming queue for storing the start test sequence command, and an external queue for storing the test result. ### Claim 9 (depends on 1) The system of claim 1, wherein the set characteristics include hardware used in a memory system, an operating system used in the memory system, and an amount of available memory and memory device of the memory system. ### Claim 10 A method for operating a test system comprising: splitting a plurality of test stations for testing a plurality of firmware products into multiple pools, each pool including multiple test stations; delivering, by each of competing test stations among the multiple test stations, a competing test start event, comprising a request to start testing, to a corresponding execution instance among multiple execution instances, each execution instance for executing tests corresponding to an associated pool; receiving, by the corresponding execution instance, competing test start events from the competing test stations in the associated pool; and executing, by the corresponding execution instance, a run test command on a select test station among the competing test stations such that the select test station performs test execution based on a test sequence, wherein the multiple test stations in each pool compete for tests, and wherein the splitting of the plurality of test stations comprises: splitting the plurality of test stations into the multiple pools based on set characteristics, wherein the test stations in a given pool have at least one of the set characteristics in common. ### Claim 11 (depends on 10) The method of claim 10, wherein the select test station is a test station which firstly delivered a corresponding test start event to the corresponding execution instance, among the competing test stations. ### Claim 12 (depends on 11) The method of claim 11, wherein the corresponding execution instance includes: state queues; an agent manager coupled to the multiple test stations, for storing state information on the multiple test stations in the state queues, based on a state of each of the multiple test stations; and a test engine for determining the select test station based on the state information, calling the agent manager to execute the run test command on the select test station. ### Claim 13 (depends on 12) The method of claim 12, wherein each of the multiple test stations is in an idle, running or completed state with respect to test execution, and wherein the state queues include an idle queue, a running queue, and a completed queue for storing idle, running and completed states of each of the multiple test stations, respectively. ### Claim 14 (depends on 12) The method of claim 12, wherein the agent manager is coupled to the multiple test stations through a remote procedure call (RPC) protocol, and is coupled to the test engine through a hypertext transfer protocol (HTTP) interface. ### Claim 15 (depends on 10) The method of claim 10, wherein the corresponding execution instance is ready to be executed in response to a start test sequence command from a user interface. ### Claim 16 (depends on 15) The method of claim 15, further comprising: receiving, by the corresponding execution instance, a test completed event and a test result from the select test station, and providing the user interface with the test result in response to the test completed event. ### Claim 17 (depends on 16) The method of claim 16, wherein the corresponding execution instance includes an incoming queue for storing the start test sequence command, and an external queue for storing the test result. ### Claim 18 (depends on 10) The method of claim 10, wherein the characteristics include hardware used in a memory system, an operating system used in the memory system, and an amount of available memory and memory device of the memory system.