Performance metrics gathering from multiple video game platforms

A method for performance testing a video game, comprising:

• providing a metrics gathering framework run by a processor of a test-hosting computer;
• with the metrics gathering framework, generating a test instruction defining a performance metric for the video game;
• transmitting a message including the test instruction to a plurality of game platforms running the video game and communicatively linked to the testing framework;
• receiving from each of the game platforms a value of the performance metric, wherein the value of the performance metric is an indicator of operational performance of the game platform during the running of the video game;
• and
• for each of the game platforms, comparing the received value with a tolerance limit for the performance metric and when the tolerance limit is exceeded, issuing a request to the game platform for additional performance data,
• wherein the operation of each of the game platforms during collection of the value of the performance metric is substantially similar to operations during collection of the additional performance data, whereby the running of the video game is not modified by performance testing.

A system for testing video games, comprising:

• a plurality of video game platforms running a video game;
• and
• a test-hosting computer system communicatively linked to each of the game platforms and providing a metrics gathering framework,
• wherein the metrics gathering framework generates a plurality of messages each defining at least one performance metric corresponding to operation of the game platforms while running the video game and transmits the messages for concurrent delivery to the game platforms and
• wherein an automation module is provided on each of the game platforms to, in response to receipt of one or more of the messages from the metrics gathering framework, communicate values for the performance metrics gathered from the game platform to the metrics gathering framework,
• wherein the metrics gathering framework, for each of the game platforms, compares at least one of the values of the performance metrics to a performance tolerance and, when the tolerance is exceeded, the metrics gathering framework generates a message to the game platforms requesting additional performance data, and
• wherein the operation of each of the game platforms during collection of the values of the performance metrics is substantially similar to operations during collection of the additional performance data, whereby the running of the video game is not modified by performance testing.