Simulation architecture for safety testing of aircraft monitoring software

A simulation system for evaluating performance of a monitoring system of an aircraft in a simulated environment, the simulation system comprising:

• a memory;
• and
• at least one processor coupled to the memory, the at least one processor being configured to execute instructions stored in the memory to perform steps comprising:
• obtaining, from the memory, a reference scenario associated with the aircraft;
• generating, for the reference scenario, a plurality of conditional scenario variations, each of the plurality of conditional scenario variations comprising a set of environmental state values relating to the simulated environment, wherein the environmental state values are randomly selected based on one or more ranges of possible values;
• executing, for each of the plurality of conditional scenario variations, (a) generating, based on the set of environmental state values, simulated environment data, (b) inputting, to the monitoring system, the simulated environment data, (c) receiving, from the monitoring system, instructions for controlling the aircraft based on the simulated environment data, (d) updating the simulated environment data for the respective conditional scenario variation based on the received instructions, and (e) storing, in a memory, log data in accordance with the updated simulated environment data;
• generating metrics based on the log data stored for each of the plurality of conditional scenario variations, wherein at least one metric of the generated metrics characterizes a performance of the monitoring system in maintaining a threshold distance from a detected object external to the aircraft;
• comparing the generated metrics to one or more predefined safety standards;
• and
• determining whether the monitoring system conforms to the one or more predefined safety standards based on the comparison.