# US 2022/0035728 A1 · System for discovering semantic relationships in computer programs > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 2022/0035728 A1 - Document ID: US20220035728A1 - Status: Published application - Kind: A1 - Application number: 17/468,166 - Filed: 2021-09-07 - Published: 2022-02-03 - Earliest claimed: 2018-05-31 - Assignees: The Ultimate Software Group, Inc. - Applicants: The Ultimate Software Group, Inc. - Inventors: Brian R. Muras; Keith A. Briggs; David Adamo; John A. Maliani; Robert L. Vanderwall; Michael L. Mattera; Dionny Santiago; Tariq King - Web page: https://patentgrep.com/patent/US20220035728A1 ## CPC - G06F 11/3608: {using formal methods, e.g. model checking, abstract interpretation (theorem proving G06N5/013)} - G06F 40/216: using statistical methods - G06F 16/288: {Entity relationship models} - G06F 40/30: Semantic analysis - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) ## Abstract A system for discovering semantic relationships in computer programs is disclosed. In particular, the system may synergistically identify and validate semantic relationships, concepts, and groupings associated with data elements within a static or dynamic, time varying, source input. The system may utilize feature extractors to extract features from the input and reasoners to develop associations using data from multiple feature set types, and, can thus generate reliable, robust, and complete sets of semantic relationships from the input. The system may generate hypotheses associated with the relationships, concepts, and groupings, and validate the hypotheses by testing an application under evaluation by the system and observing the outputs generated from the testing. Information pertaining to validated or invalidated hypotheses may be provided to a learning engine to maximize reasoning and performance in subsequent discovery processes by adjusting models, vocabularies, dictionaries, parameters utilized by the system in identifying the relationships, concepts, and groupings. ## Claims ### Claim 1 A system, comprising: a memory that stores instructions; and a processor that executes the instructions to perform operations, the operations comprising: analyzing, for a discovery process, information provided by a source, wherein the information is associated with an application under evaluation by the system and is extracted from an interaction conducted with the application under evaluation, from the source, or a combination thereof; generating, based on a data element in the information, a hypothesis associated with a first concept associated with the data element, a first relationship associated with the data element, a first grouping associated with the data element, or a combination thereof; testing the hypothesis against the application under evaluation to confirm or reject the hypothesis; and training, based on the testing of the hypothesis and based on a confirmation or a rejection of the hypothesis, a model to enhance determination of a second concept, a second relationship, a second grouping, or a combination thereof, for a subsequent discovery process. ### Claim 2 (depends on 1) The system of claim 1, wherein the data element, the first grouping, the first relationship, the hypothesis, the information, the first concept, or a combination thereof, a function related to the application under evaluation, a product related to the application under evaluation, a computing environment associated with the application under evaluation, an application related to the application under evaluation, a version of the application under evaluation, a previous release of the application under evaluation, or a combination thereof. ### Claim 3 (depends on 1) The system of claim 1, wherein the operations further comprise generating, based on the information, a constraint suggestion for the application under evaluation. ### Claim 4 (depends on 3) The system of claim 3, wherein the operations further comprise generating the hypothesis based on the constraint suggestion. ### Claim 5 (depends on 1) The system of claim 1, wherein the operations further comprise extracting a feature from the information by utilizing a feature extractor, and wherein the operations further comprise determining the first concept associated with the data element, the first relationship associated with the data element, the first grouping associated with the data element, or a combination thereof, based on the feature extracted from the information. ### Claim 6 (depends on 1) The system of claim 1, wherein the operations further comprise testing the application under evaluation to validate the hypothesis and generating an output based on the testing of the application under evaluation, based on testing a version of the application under evaluation, based on testing associated with another application, or a combination thereof. ### Claim 7 (depends on 1) The system of claim 1, wherein the operations further comprise detecting a condition associated with the system, associated with the application under evaluation, or a combination thereof, and wherein the operations further comprise detecting a conflict associated with the application under evaluation, a defect associated with the application, or a combination thereof. ### Claim 8 (depends on 7) The system of claim 7, wherein the operations further comprise generating, by utilizing a code generator, software code for resolving the condition, the conflict, the defect, or a combination thereof. ### Claim 9 (depends on 8) The system of claim 8, wherein the operations further comprise incorporating the software code into the application under evaluation to modify the application under evaluation. ### Claim 10 (depends on 6) The system of claim 6, wherein the operations further comprise generating a report based on the testing of the application under evaluation, wherein the report provides information associated with a successful path taken through the application under evaluation, information associated with a defect detected while testing the application under evaluation, information associated with a conflict detected while testing the application under evaluation. an indication relating to re-testing of the hypothesis against the application under evaluation after training the model, or a combination thereof. ### Claim 11 (depends on 1) The system of claim 1, wherein the operations further comprise training, based on the testing, the model to enhance generation of a future hypothesis for the application under evaluation, an application associated with the application under evaluation, a computing environment associated with the application under valuation, or a combination thereof. ### Claim 12 (depends on 1) The system of claim 1, wherein the operations further comprise determining if a confidence level for the hypothesis satisfies a threshold. ### Claim 13 (depends on 12) The system of claim 12, wherein the operations further comprise filtering the hypothesis from a plurality of hypotheses if the confidence level for the hypothesis satisfies the threshold. ### Claim 14 A method, comprising: analyzing, for a discovery process, information provided by a source, wherein the information is associated with an application under evaluation by a system and is extracted from an interaction conducted with the application under evaluation, from the source, or a combination thereof; generating, based on a data element in the information and by utilizing instructions from a memory that are executed by a processor of the system, a hypothesis associated with a first concept associated with the data element, a first relationship associated with the data element, a first grouping associated with the data element, or a combination thereof; testing the hypothesis against the application under evaluation to confirm or reject the hypothesis; and training, based on the testing of the hypothesis and based on a confirmation or a rejection of the hypothesis, a model to enhance determination of a second concept, a second relationship, a second grouping, or a combination thereof, for a subsequent discovery process. ### Claim 15 (depends on 14) The method of claim 14, further comprising testing the hypothesis against a different application to confirm or reject the hypothesis with respect to the different application. ### Claim 16 (depends on 15) The method of claim 15, further comprising training the model to enhance determination of a third concept, a third relationship, a third grouping, or a combination thereof, based on the testing of the hypothesis against the different application, a different release, a different application version associated with the application under evaluation, the application under evaluation, or a combination thereof. ### Claim 17 (depends on 14) The method of claim 14, further comprising requesting an input from a device, a computing system, an application, a user, or a combination thereof to confirm or reject the hypothesis to facilitate confirming or rejecting the hypothesis generated based on testing of the hypothesis against the application under evaluation. ### Claim 18 (depends on 14) The method of claim 14, further comprising extracting a feature from the information provided by the source by utilizing a feature selection evaluator that measures a cost, a benefit, or a combination thereof, or individual or feature groups. ### Claim 19 (depends on 14) The method of claim 14, further comprising determining whether the feature does or does not correlate with a validated result obtained from testing the hypothesis on the application under evaluation. ### Claim 20 (depends on 14) The method of claim 14, further comprising verifying a determination associated with a confidence for the hypothesis based on analyzing a previous release of the application under evaluation, another version of the application under evaluation, or a combination thereof. ### Claim 21 (depends on 20) The method of claim 20, further comprising adjusting the confidence for the hypothesis based on the verifying of the determination. ### Claim 22 (depends on 14) The method of claim 14, further comprising enhancing the determination based on the confirmation or the rejection of the hypothesis by adjusting a parameter, a vocabulary, a dictionary, or a combination thereof. ### Claim 23 A non-transitory computer-readable device comprising instructions, which when loaded and executed by a processor, cause the processor to perform operations comprising: analyzing, for a discovery process, information provided by a source, wherein the information is associated with an application under evaluation by a system and is extracted from an interaction conducted with the application under evaluation, from the source, or a combination thereof; generating, based on a data element in the information, a hypothesis associated with a first concept associated with the data element, a first relationship associated with the data element, a first grouping associated with the data element, or a combination thereof; testing the hypothesis against the application under evaluation to confirm or reject the hypothesis; and training, based on the testing of the hypothesis and based on a confirmation or a rejection of the hypothesis, a model to enhance determination of a second concept, a second relationship, a second grouping, or a combination thereof, for a subsequent discovery process. ### Claim 24 (depends on 23) The non-transitory computer-readable device of claim 23, wherein the operations further comprise utilizing the confirmation or the rejection of the hypothesis to determine a feature that does not correlate with a validated result generated based on testing the application under evaluation.