Systems and methods for testing, certifying, and recertifying safety models
AssigneeReynolds & Moore, LLC
InventorSamuel Erik Reynolds
Life of this patent
Estimating the term…
Abstract
The present invention claims methods and systems for testing, certification, and recertification of safety models that cannot be explained or evaluated using traditional software assessment methodology due to the creation or development of software source code using machine learning or artificial intelligence models. Various embodiments of the invention implements fundamental changes to the traditional testing paradigm utilized in environments subject to safety and risk mitigation standards by providing systems and methods for the efficient virtual and physical testing of software developed by machine learning or artificial intelligence in safety testing, certification, and recertification processes and procedures.
Claims19
A system for certifying a safety model wherein virtual test results and physical test results are correlated to enable the certification of a new version of said safety model by using only virtual testing comprising:
- a hardware processor that is configured to accept a request to approve said safety model for use in said environment requiring safety compliance approval, virtually test said safety model using adversarial attacks configured to emulate said environment requiring safety compliance approval, physically test said safety model for compliance with safety standards, and analyze test results to determine whether said safety model should be approved for use in said environment requiring safety compliance approval;
- a memory connected to said hardware processor, said memory configured to store safety test protocols, test results, certification histories, and safety certificates;
- and
- a database within said memory wherein safety model datasets, tested safety model, and test correlation data are maintained.
The system for certifying a safety model of claim 1 wherein said hardware processor is further configured to send said test results to a certification authority.
- The system for certifying a safety model of claim 3 wherein said hardware processor is further configured to receive a safety certificate from said certification authority.
A method for approving a safety model for use in an environment requiring safety compliance wherein the results of a set of virtual tests and a set of physical tests to which said safety model is subjected are correlated so that future versions of said safety model can be approved by passing said set of virtual tests comprising:
- accepting a request to approve said safety model for use in said environment requiring safety compliance approval;
- virtually testing said safety model using adversarial attacks configured to emulate said environment requiring safety compliance approval;
- physically testing said safety model for compliance with safety standards;
- and
- analyzing test results to determine whether said safety model should be approved for use in said environment requiring safety compliance approval.
A method for recertifying a safety model wherein a previous version of said safety model was certified as safety compliant by passing a set of virtual tests and a set of physical tests and wherein recertification of said safety model is based on meeting or exceeding said set of virtual tests comprising:
- accepting a request to recertify said safety model;
- virtually testing said safety model using the virtual testing process used for said previous version of said safety model;
- and
- analyzing virtual test results by comparing said virtual test results with virtual test results from the certification of said previous version of said safety model, wherein said virtual test results from the certification of said previous version of said safety model are calibrated to physical test results from said certification of said previous version of said safety model.
Dependencies are inferred from the claim text.