Testing networked system using abnormal node failure

A system of comprising:

• at least one hardware processor;
• and
• a non-transitory computer-readable medium storing executable instructions that, when executed, cause the at least one processor to perform operations comprising:
• during a first period of time, repeatedly transmitting simulated requests to a networked system on which a software application is implemented using a plurality of nodes, the networked system being configured to respond to the simulated requests using the plurality of nodes;
• during the first period of time, terminating and restarting one or more nodes randomly selected until each one of the plurality of nodes in the plurality of nodes has been terminated and restarted at least once during the first period of time;
• for each one of the simulated requests transmitted during the first period of time, determining a corresponding response time of the networked system in responding to the simulated request, the corresponding response time comprising an amount of time between a first time at which the simulated request was transmitted to the networked system and a second time at which the networked system responded to the simulated request;
• storing the corresponding response times for the simulated requests in a database;
• and
• performing a function using at least a portion of the corresponding response times for the simulated requests, the function comprising:
• identifying the simulated requests for which the corresponding response times are above a threshold response time;
• determining whether the identified simulated requests include multiple types of requests above a threshold number of types of requests;
• determining a type of problem affecting the networked system based on the determining of whether the identified simulated requests include multiple types of requests above the threshold number of types of requests;
• generating a recommendation for modifying the networked system based on the determining of the type of problem affecting the networked system;
• and
• causing the generated recommendation to be displayed on a computing device.

A non-transitory machine-readable storage medium tangibly embodying a set of instructions that, when executed by at least one hardware processor, causes the at least one processor to perform operations comprising:

• during a first period of time, repeatedly transmitting simulated requests to a networked system on which a software application is implemented using a plurality of nodes, the networked system being configured to respond to the simulated requests using the plurality of nodes;
• during the first period of time, terminating and restarting one or more nodes randomly selected until each one of the plurality of nodes in the plurality of nodes has been terminated and restarted at least once during the first period of time;
• for each one of the simulated requests transmitted during the first period of time, determining a corresponding response time of the networked system in responding to the simulated request, the corresponding response time comprising an amount of time between a first time at which the simulated request was transmitted to the networked system and a second time at which the networked system responded to the simulated request;
• storing the corresponding response times for the simulated requests in a database;
• and
• performing a function using at least a portion of the corresponding response times for the simulated requests, the function comprising:
• identifying the simulated requests for which the corresponding response times are above a threshold response time;
• determining whether the identified simulated requests include multiple types of requests above a threshold number of types of requests;
• determining a type of problem affecting the networked system based on the determining of whether the identified simulated requests include multiple types of requests above the threshold number of types of requests;
• generating a recommendation for modifying the networked system based on the determining of the type of problem affecting the networked system;
• and
• causing the generated recommendation to be displayed on a computing device.