# US 11,907,376 B2 · Compliance verification testing using negative validation

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 11,907,376 B2
- Document ID: US11907376B2
- Status: Granted
- Kind: B2
- Application number: 17/229,158
- Filed: 2021-04-13
- Published: 2022-10-13
- Granted: 2024-02-20
- Earliest claimed: 2021-04-13
- Assignees: Saudi Arabian Oil Company
- Applicants: Saudi Arabian Oil Company
- Inventors: John Gwilliams; Sultan Saadaldean Alsharif
- Web page: https://patentgrep.com/patent/US11907376B2

## CPC

- G06F 21/577: {Assessing vulnerabilities and evaluating computer system security}
- G06N 20/00: Machine learning
- G06Q 50/265: {Personal security, identity or safety}
- G06F 2221/034: Test or assess a computer or a system
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06Q: INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR

## Abstract

Methods and systems, including computer programs encoded on a computer storage medium, implement compliance testing to evaluate controls used to protect assets of a target system. A respective first score is generated for each control based on compliance tests performed to detect each of the controls at the target system. A compliance model is generated that integrates machine-learning algorithms to classify inputs corresponding to a compliance test and to enable predictive analytics of the compliance model using the classified inputs. The compliance model derives a negative compliance test (nCT) for each of the compliance tests by applying the predictive analytics to a data set that includes the first score for each control. An nCT is performed for each control detected at the target system and a second score is generated for each nCT. An assurance score characterizing effectiveness of the control is generated based on the first and second scores.

## Claims

### Claim 1

A computer-implemented method of compliance testing to evaluate controls used to protect assets of a target system, the method comprising:
generating a respective first score for each control based on a plurality of compliance tests that are performed to detect each of the controls at the target system;
generating a compliance model that integrates machine-learning algorithms to:
i) classify inputs corresponding to a compliance test, and
ii) enable predictive analytics of the compliance model using the classified inputs;
deriving, by the compliance model, a negative compliance test for each of the plurality of compliance tests by applying the predictive analytics to a data set comprising the respective first score for each of the controls;
for each control detected at the target system:
performing a negative compliance test that corresponds to a compliance test performed to detect the control;
generating a second score for the negative compliance test that is performed; and
based on a comparison of a first score for the control and the second score, generating an assurance score that characterizes effectiveness of the control at protecting an asset of the target system.

### Claim 2 (depends on 1)

The method of claim 1, wherein deriving a negative compliance test comprises:
processing, based on the predictive analytics of the compliance model, information in the data set that describes assessment of a first control used to protect the asset of the target system;
in response to processing the information, computing, by the compliance model, inferences that are indicative of a particular negative compliance test to be performed to assess a protection feature of the first control; and
based on the computed inferences, deriving a first negative compliance test to assess effectiveness the first control and the protection feature at protecting the asset.

### Claim 3 (depends on 2)

The method of claim 2, further comprising:
negatively validating effectiveness of the first control and the protection feature at protecting the asset based on an outcome of the first negative compliance test.

### Claim 4 (depends on 2)

The method of claim 2, wherein the predictive analytics of the compliance model comprises a negative validation algorithm that is applied to each first score to validate a respective accuracy of the first score.

### Claim 5 (depends on 4)

The method of claim 4, wherein:
a first control comprises a response function that is responsive to a particular incident; and
the negative validation algorithm is applied to a corresponding first score for the first control to negatively validate efficacy of the response function.

### Claim 6 (depends on 5)

The method of claim 5, wherein:
the particular incident indicates an attempt at unauthorized access of the asset; and
the response function of the first control protects the asset by precluding unauthorized access.

### Claim 7 (depends on 1)

The method of claim 1, wherein generating the assurance score comprises:
determining a measure of effectiveness of the control at precluding unauthorized access to an asset of the target system; and
generating the assurance score based on the determined measure of effectiveness.

### Claim 8 (depends on 1)

The method of claim 1, wherein the assurance score quantifies the measure of effectiveness of the control at precluding unauthorized access to the asset of the target system.

### Claim 9 (depends on 1)

The method of claim 1, wherein:
the first score is a positive compliance score that indicates presence of a given control configured to protect the asset; and
the second score is a negative compliance score that assess a capability of the given control to protect the asset.

### Claim 10 (depends on 1)

The method of claim 1 further comprising:
generating a percent compliant based on the respective first score for each of the controls;
applying the percent compliant to a boundary condition that defines a risk profile of an entity that includes the target system; and
determining an asset rating that characterizes protection of the asset relative to the boundary condition in response to applying the percent compliant to the boundary condition.

### Claim 11

A computing system for implementing compliance testing to evaluate controls used to protect assets of a target system, the system comprising one or more processing devices and one or more non-transitory machine-readable storage devices storing instructions that are executable by the one or more processing devices to cause performance of operations comprising:
generating a respective first score for each control based on a plurality of compliance tests that are performed to detect each of the controls at the target system;
generating a compliance model that integrates machine-learning algorithms to:
i) classify inputs corresponding to a compliance test, and
ii) enable predictive analytics of the compliance model using the classified inputs;
deriving, by the compliance model, a negative compliance test for each of the plurality of compliance tests by applying the predictive analytics to a data set comprising the respective first score for each of the controls;
for each control detected at the target system:
performing a negative compliance test that corresponds to a compliance test performed to detect the control;
generating a second score for the negative compliance test that is performed; and
based on a comparison of a first score for the control and the second score, generating an assurance score that characterizes effectiveness of the control at protecting an asset of the target system.

### Claim 12 (depends on 11)

The computing system of claim 11, wherein deriving a negative compliance test comprises:
processing, based on the predictive analytics of the compliance model, information in the data set that describes assessment of a first control used to protect the asset of the target system;
in response to processing the information, computing, by the compliance model, inferences that are indicative of a particular negative compliance test to be performed to assess a protection feature of the first control; and
based on the computed inferences, deriving a first negative compliance test to assess effectiveness the first control and the protection feature at protecting the asset.

### Claim 13 (depends on 12)

The computing system of claim 12, wherein the operations further comprise:
negatively validating effectiveness of the first control and the protection feature at protecting the asset based on an outcome of the first negative compliance test.

### Claim 14 (depends on 12)

The computing system of claim 12, wherein the predictive analytics of the compliance model comprises a negative validation algorithm that is applied to each first score to validate a respective accuracy of the first score.

### Claim 15 (depends on 14)

The computing system of claim 14, wherein:
a first control comprises a response function that is responsive to a particular incident; and
the negative validation algorithm is applied to a corresponding first score for the first control to negatively validate efficacy of the response function.

### Claim 16 (depends on 15)

The computing system of claim 15, wherein:
the particular incident indicates an attempt at unauthorized access of the asset; and
the response function of the first control protects the asset by precluding unauthorized access.

### Claim 17 (depends on 11)

The computing system of claim 11, wherein generating the assurance score comprises:
determining a measure of effectiveness of the control at precluding unauthorized access to an asset of the target system; and
generating the assurance score based on the determined measure of effectiveness.

### Claim 18 (depends on 11)

The computing system of claim 11, wherein the assurance score quantifies the measure of effectiveness of the control at precluding unauthorized access to the asset of the target system.

### Claim 19 (depends on 11)

The computing system of claim 11, wherein:
the first score is a positive compliance score that indicates presence of a given control configured to protect the asset; and
the second score is a negative compliance score that assess a capability of the given control to protect the asset.

### Claim 20

A non-transitory machine-readable storage device storing instructions for implementing compliance testing to evaluate controls used to protect assets of a target system, the instructions being executable by a processing device to cause performance of operations comprising:
generating a respective first score for each control based on a plurality of compliance tests that are performed to detect each of the controls at the target system;
generating a compliance model that integrates machine-learning algorithms to:
i) classify inputs corresponding to a compliance test, and
ii) enable predictive analytics of the compliance model using the classified inputs;
deriving, by the compliance model, a negative compliance test for each of the plurality of compliance tests by applying the predictive analytics to a data set comprising the respective first score for each of the controls;
for each control detected at the target system:
performing a negative compliance test that corresponds to a compliance test performed to detect the control;
generating a second score for the negative compliance test that is performed; and
based on a comparison of a first score for the control and the second score, generating an assurance score that characterizes effectiveness of the control at protecting an asset of the target system.