# US 12,118,441 B2 · Knowledge augmented sequential decision-making under uncertainty > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 12,118,441 B2 - Document ID: US12118441B2 - Status: Granted - Kind: B2 - Application number: 17/548,672 - Filed: 2021-12-13 - Published: 2023-06-15 - Granted: 2024-10-15 - Earliest claimed: 2021-12-13 - Assignees: International Business Machines Corporation - Applicants: International Business Machines Corporation - Inventors: Radu Marinescu - Web page: https://patentgrep.com/patent/US12118441B2 ## CPC - G06N 10/40: Physical realisations or architectures of quantum processors or components for manipulating qubits, e.g. qubit coupling or qubit control - G06N 20/00: Machine learning - G06F 40/40: Processing or translation of natural language (natural language analysis G06F40/20; semantic analysis G06F40/30) - G06N 7/00: Computing arrangements based on specific mathematical models - G06F 40/279: Recognition of textual entities - G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) ## Abstract One or more systems, devices, computer program products and/or computer-implemented methods of use provided herein relate to outputting an optimal decision policy base on informal knowledge input. A system can comprise a memory that stores computer executable components, and a processor that executes the computer executable components stored in the memory, wherein the computer executable components can comprise an analysis component that analyzes an input dataset comprising a constraint in a natural language form, and an augmentation component that generates an influence mapping comprising a constraint variable based on the constraint input. In an embodiment, an input dataset employed to support the influence mapping can comprise time-stamped tuple data comprising a state, an action and a reward. In an embodiment, an inference engine can generate an output policy in response to the constraint input and which output policy can be based on the constraint input and constraint variable. ## Claims ### Claim 1 A system, comprising: a memory that stores computer executable components; and a processor that executes at least one of the computer executable components that: trains, using a data structure comprising tuples, an optimization model to generate a decision policy for a problem, wherein the tuples respectively comprise a state variable, an action variable, and a reward variable; analyzes a desired constraint input provided in a natural language form; transforms the desired constraint input provided in the natural language form into a constraint based deterministic probability function; generates an influence mapping based on the optimization model, the data structure comprising the tuples, and the constraint based deterministic probability function; and retrains, using the influence mapping, the optimization model to generate an updated decision policy for the problem. ### Claim 2 (depends on 1) The system of claim 1, wherein an input dataset employed to support the influence mapping comprises time-stamped tuple data comprising a state, an action and a reward. ### Claim 3 (depends on 2) The system of claim 2, wherein the at least one of the computer executable components further: expands the influence mapping over a specified time horizon. ### Claim 4 (depends on 3) The system of claim 3, wherein the at least one of the computer executable components further: generates an output policy in response to the desired constraint input and which output policy is based on the input dataset and the constraint variable. ### Claim 5 (depends on 1) The system of claim 1, wherein the at least one of the computer executable components further: accepts input data including an additional desired constraint input in the natural language form, in response to generation of an output policy based on the desired constraint input. ### Claim 6 (depends on 5) The system of claim 5, wherein the at least one of the computer executable components further: transforms the additional desired constraint input provided in the natural language form into an additional constraint based deterministic probability function; updates the influence mapping based on the optimization model, the data structure comprising the tuples, and the additional constraint based deterministic probability function; and retrains, using the updated influence mapping, the optimization model to generate another updated decision policy for the problem. ### Claim 7 (depends on 1) The system of claim 1, wherein the at least one of the computer executable components further: receives, via a chat bot interface, during a chat conversation with a user, the desired constraint input in the natural language form. ### Claim 8 A computer-implemented method, comprising: training, by a system operatively coupled to a processor, using a data structure comprising tuples, an optimization model to generate a decision policy for a problem, wherein the tuples respectively comprise a state variable, an action variable, and a reward variable; analyzing, by the system, a desired constraint input provided in a natural language form; transforming, by the system, the desired constraint input provided in the natural language form into a constraint based deterministic probability function; generating, by the system, an influence mapping based on the optimization model, the data structure comprising the tuples, and the constraint based deterministic probability function; and retraining, by the system, using the influence mapping, the optimization model to generate an updated decision policy for the problem. ### Claim 9 (depends on 8) The computer-implemented method of claim 8, wherein an input dataset employed to support the influence mapping comprises time-stamped tuple data comprising a state, an action and a reward. ### Claim 10 (depends on 9) The computer-implemented method of claim 9, further comprising: generating, by the system, an output policy in response to the desired constraint input and which output policy is based on the input dataset and the constraint variable. ### Claim 11 (depends on 8) The computer-implemented method of claim 8, further comprising: expanding, by the system, the influence mapping over a specified time horizon. ### Claim 12 (depends on 8) The computer-implemented method of claim 8, further comprising: accepting, by the system, input data including an additional desired constraint input in the natural language form, in response to generation of an output policy based on the desired constraint input. ### Claim 13 (depends on 12) The computer-implemented method of claim 12, further comprising: transforming, by the system, the additional desired constraint input provided in the natural language form into an additional constraint based deterministic probability function; updating, by the system, the influence mapping based on the optimization model, the data structure comprising the tuples, and the additional constraint based deterministic probability function; and retraining, by the system, using the updated influence mapping, the optimization model to generate another updated decision policy for the problem. ### Claim 14 (depends on 8) The computer-implemented method of claim 8, further comprising: receiving, by the system, via a chat bot interface, during a chat conversation with a user, the desired constraint input in the natural language form. ### Claim 15 A computer program product facilitating a process to generate a decision policy, the computer program product comprising a computer readable storage medium having program instructions embodied therewith, the program instructions executable by a processor to cause the processor to: train, by the processor, using a data structure comprising tuples, an optimization model to generate the decision policy for a problem, wherein the tuples respectively comprise a state variable, an action variable, and a reward variable; transform, by the processor, the desired constraint input provided in the natural language form into a constraint based deterministic probability function; analyze, by the processor, a desired constraint input provided in a natural language form; generate, by the processor, an influence mapping based on the optimization model, the data structure comprising the tuples, and the constraint based deterministic probability function; and retrain, by the processor, using the influence mapping, the optimization model to generate an updated decision policy for the problem. ### Claim 16 (depends on 15) The computer program product of claim 15, wherein an input dataset employed to support the influence mapping comprises time-stamped tuple data comprising a state, an action and a reward. ### Claim 17 (depends on 16) The computer program product of claim 16, wherein the program instructions are further executable by the processor to cause the processor to: generate, by the processor, an output policy in response to the desired constraint input and which output policy is based on the input dataset and the constraint variable. ### Claim 18 (depends on 15) The computer program product of claim 15, wherein the program instructions are further executable by the processor to cause the processor to: expand, by the processor, the influence mapping over a specified time horizon. ### Claim 19 (depends on 15) The computer program product of claim 15, wherein the program instructions are further executable by the processor to cause the processor to: accept, by the processor, via a chat bot interface during a chat conversation with a user, input data including an additional desired constraint input in the natural language form, in response to generation of an output policy based on the desired constraint input. ### Claim 20 (depends on 19) The computer program product of claim 19, wherein the program instructions are further executable by the processor to cause the processor to: transform, by the processor, the additional desired constraint input provided in the natural language form into an additional constraint based deterministic probability function; update, by the processor, the influence mapping based on the optimization model, the data structure comprising the tuples, and the additional constraint based deterministic probability function; and retrain, by the processor, using the updated influence mapping, the optimization model to generate another updated decision policy for the problem.