Network Impairment Simulation Framework for Verification of Real Time Interactive Media Streaming Systems

A method comprising:

• at a computing system having one or more processors and memory:
• obtaining a network-enabled test application;
• displaying, in a first user interface of the computing system, a plurality of affordances associated with a respective one of a plurality of predefined network simulation scenarios, each of the network simulation scenarios embodying a specific combination of dynamic network parameters, wherein the dynamic network parameters for a respective predefined simulation scenario are based on measurements of a real world communication environment corresponding to the respective simulation scenario;
• receiving a user selection of a first affordance associated with a first network simulation scenario;
• and
• in response to the user selection of the first affordance:
• encoding a video stream associated with a plurality of specific operations of the network-enabled test application;
• simulating transmission of the encoded video stream in a simulated test network for the first network simulation scenario with associated first dynamic network parameters;
• determining one or more performance aspects of the network-enabled test application based on a plurality of time-varying quality metrics for the filtered data stream;
• and
• associating the one or more performance aspects of the network-enabled test application with the first simulation scenario.

A computing system comprising one or more processors and memory coupled to the one or more processors, the memory storing one or more programs configured to be executed by the one or more processors, the one or more programs including instructions for:

• obtaining a first network simulation scenario selected from a plurality of predefined network simulation scenarios, each of the network simulation scenarios embodying a specific combination of dynamic network parameters, wherein the dynamic network parameters for a respective predefined simulation scenario are based on measurements of a real world communication environment corresponding to the respective simulation scenario;
• executing a network-enabled test application;
• encoding a video stream associated with a plurality of specific operations of the network-enabled test application;
• simulating transmission of the encoded video stream in a simulated test network for the first network simulation scenario with associated first dynamic network parameters;
• generating a filtered data stream that reflects simulated real time effects of the simulated test network on transmission of the encoded data stream based on the first dynamic network parameters;
• and
• determining a plurality of time-varying quality metrics for the filtered data stream.