# US 6,862,696 B1 · System and method for software certification > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 6,862,696 B1 - Document ID: US6862696B1 - Status: Granted - Kind: B1 - Application number: 09/847,624 - Filed: 2001-05-03 - Granted: 2005-03-01 - Earliest claimed: 2000-05-03 - Assignees: Cigital - Inventors: Jeffrey M. Voas; Anup K. Ghosh - Web page: https://patentgrep.com/patent/US6862696B1 ## CPC - G06F 11/3466: {Performance evaluation by tracing or monitoring} - G06F 11/3672: {Test management} - G06F 2201/865: Monitoring of software - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) ## Abstract A method and method that builds accurate operational profiles for COTS software. The systems and methods disclosed allow software vendors to detect misused and unused features; identify common machine configurations for a given piece of software or software component; monitor changing user habits as new software version are released; derive more accurate testing methods for in-house testing purposes; and create user manuals which focus on those features most frequently used, or misused, by users. The disclosed system and method provides the tools enabling a software certification laboratory (SCL) to gather detailed usage data and failure data for a software application as it is used in the field. With this data the SCL can confidently issue certificates of reliability for software products. ## Claims ### Claim 1 A method for certifying a software product's reliability, said method comprising: establishing a software certification laboratory; receiving the software product at the software certification laboratory; instrumenting the software product, by the software certification laboratory, to collect a plurality of usage data and a plurality of failure data when the software product is being executed; digitally signing the software product to be verified; verifying the digital signature of the software product; employing a residual testing tool in the software product that collects information about the software product behavior as the software product is used; providing the software product to a plurality of users; receiving, at the software certification laboratory, the plurality of usage data and the plurality of failure data from each of the plurality of users; analyzing the plurality of usage data and the plurality of failure data; building an operational profile according to a result from the analyzing; and issuing a certificate of reliability to the software product based on the result from the analyzing. ### Claim 2 (depends on 1) The method of claim 1, wherein the software certification laboratory is independent of a vendor that provides the software product. ### Claim 3 (depends on 1) The method of claim 1, wherein the operational profile is returned from the software certification laboratory to a vendor that provides the software product for reference. ### Claim 4 (depends on 1) The method of claim 1, wherein the signature of the software product is signed by a software product vendor. ### Claim 5 (depends on 1) The method of claim 1, wherein the signature of the software is verified by the software certification laboratory. ### Claim 6 (depends on 1) The method of claim 1, wherein the certification is presented by computing a Mean Time to Failure (MTTF) for the software product, wherein the MTTF is given by MTTF = ∑ i = 1 N ⁢ ⁢ t i N , wherein N is a number of instrumented versions of the software product, and ti represents the time for each version before encountering a failure. ### Claim 7 A system for certifying a software product's reliability, comprising: a software product received from a product vendor; and a software certification laboratory for instrumenting the software product to collect information about the software behavior when the software product is used, wherein the software product, after instrumented by the software certification laboratory, is used by a plurality of users, and the information collected by the software products includes a plurality of usage data and a plurality of failure data from each of the plurality of users, wherein the software certification laboratory receives and analyzes the plurality of usage data and the plurality of failure data from each of the plurality of users collected by the software product, builds an operational profile according to a result from the analyzing, and issues a certificate of reliability to the software product, and wherein the software certification laboratory is further configured to: digitally sign the software product to be verified; verify the digital signature of the software product; and employ a residual testing tool in the software product that collects information about the software product behavior as the software product is used. ### Claim 8 (depends on 7) The system of claim 7, wherein the software certification laboratory instruments the software product with a residual testing tool that collects information about the software behavior when the software product is used. ### Claim 9 (depends on 8) The system of claim 8, wherein the software product is provided by a software vendor that is independent from the software certification laboratory. ### Claim 10 (depends on 9) The system of claim 9, wherein the vendor provides a digital signature on the software product. ### Claim 11 (depends on 10) The system of claim 10, wherein the software certification laboratory verifies the digital signature of the software product. ### Claim 12 (depends on 7) The system of claim 7, wherein the operational profile is returned from the software certification laboratory to a vendor that provides the software product for reference. ### Claim 13 (depends on 7) The system of claim 7, wherein the collected data is scrubbed of personal information of the plurality of users. ### Claim 14 (depends on 7) The system of claim 7, wherein the software certification laboratory instructs the software product to collect the information periodically. ### Claim 15 (depends on 7) The system of claim 7, wherein the certification is presented by computing a Mean Time to Failure (MTTF) for the software product, wherein the MTTF is given by MTTF = ∑ i = 1 N ⁢ ⁢ t i N , wherein N is the number of instrumented versions of the software product, and ti represents the time for each version before encountering a failure. ### Claim 16 A method for certifying a software product's reliability, comprising: employing a residual testing tool to the software, wherein the residual testing tool is configured to collect information about the software product behavior as the software product is used; digitally signing the software product; distributing the signed software product to a software certification laboratory, wherein the software certification laboratory verifies the digital signature of the software product and instruments the residual testing tool employed in the software product to collect information about the software product behavior; distributing the verified software product to a plurality of users, wherein the residual testing tool collects a plurality of usage data and a plurality of failure data from each of the plurality of users while the software product is used; reporting the collected usage and failure data to the software certification laboratory; and analyzing the collected data to issues a certificate of reliability to the software product. ### Claim 17 (depends on 16) The method of claim 16, wherein the software certification laboratory further builds an operational profile according to a result from the analyzing. ### Claim 18 (depends on 16) The method of claim 16, wherein the certification is presented by computing a Mean Time to Failure (MTTF) for the software product, wherein the MTTF is given by MTTF = ∑ i = 1 N ⁢ ⁢ t i N , wherein N is a number of instrumented versions of the software product, and ti represents the time for each version before encountering a failure.