# US 10,769,325 B2 · Simulation augmented reality system for emergent behavior > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 10,769,325 B2 - Document ID: US10769325B2 - Status: Granted - Kind: B2 - Application number: 16/825,605 - Filed: 2020-03-20 - Published: 2020-07-09 - Granted: 2020-09-08 - Earliest claimed: 2016-04-12 - Assignees: Dassault Systemes Simulia Corp. - Applicants: Dassault Systemes Simulia Corp. - Inventors: Alexander Jacobus Maria Van der Velden; Robert Vincent Combier - Web page: https://patentgrep.com/patent/US10769325B2 ## CPC - G06F 30/20: Design optimisation, verification or simulation (optimisation, verification or simulation of circuit designs G06F30/30) - G06T 19/006: {Mixed reality (object pose determination, tracking or camera calibration for mixed reality G06T7/00)} - G06F 3/147: using display panels - G06T 2210/61: Scene description - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) - G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL ## Abstract Embodiments of the present invention provide an augmented reality by defining a model representing a real-world system. After defining the model, a plurality of model simulations are performed using the defined model which produce predicted field data that is stored in memory. In turn, data from one or more sensors in the real-world system is received and the defined model is calibrated using the received field data relative to the stored predicted field data. Then, an augmented reality of the real-world system is provided using the calibrated model. ## Claims ### Claim 1 A computer implemented method for providing an augmented reality experience, the method comprising: defining in computer memory, a model of a real-world system; performing a plurality of simulations of the real-world system using the defined model, each simulation of the plurality of simulations including one or more simulation input parameters for that simulation, performing each simulation of the plurality of simulations producing predicted field data for that simulation; storing the predicted field data and the associated one or more simulation input parameters for each simulation of the plurality of simulations; receiving field data from one or more sensors in the real-world system; calibrating the model of the real-world system using the received field data relative to the predicted field data, resulting in a calibrated model; providing the augmented reality experience of the real-world system using the determined values of the one or more calibrated input parameters as inputs to the calibrated model of the real-world system, the augmented reality experience including a presentation of the received field data and simulation data; and illustrating a state of the real-world system at a future time, wherein the state of the real-world system at a future time is determined to be the nearest failure condition to the current state of the real-world system. ### Claim 2 (depends on 1) The method of claim 1, wherein illustrating the state of the real-world system at a future time further includes using one or more counterexamples. ### Claim 3 (depends on 1) The method of claim 1, wherein calibrating the model further includes determining values of one or more calibrated simulation input parameters of the model based on the plurality of simulations and the one or more simulation input parameters associated with each of the plurality of simulations such that the output of the model minimizes differences between the field data from the one or more sensors and the predicted field data. ### Claim 4 (depends on 1) The method of claim 1 wherein defining the model of the real-world system includes defining one or more model parameters, wherein the plurality of simulations are performed for one or more variations of the one or more model parameters, the method further comprising storing the one or more variations of the one or more model parameters. ### Claim 5 (depends on 1) The method of claim 1 wherein the augmented reality is provided via an augmented reality headset. ### Claim 6 (depends on 1) The method of claim 1 further comprising: predicting one or more points of failure of the real-world system by performing a simulation of the real-world system using the calibrated model; and indicating the one or more predicted points of failure in the augmented reality. ### Claim 7 (depends on 1) The method of claim 1 further comprising performing one or more potential action simulations of the real-world system using the defined model with potential actions to identify one or more potential actions that can be taken to avoid the nearest failure condition. ### Claim 8 (depends on 7) The method of claim 7 further comprising illustrating the one or more actions that can be taken to avoid nearest failure condition. ### Claim 9 (depends on 8) The method of claim 8, wherein illustrating the one or more actions is performed in response to a user request. ### Claim 10 (depends on 7) The method of claim 7 wherein identifying one or more potential actions that can be taken to avoid the nearest failure condition further includes using one or more previously performed simulations. ### Claim 11 (depends on 1) The method of claim 1 wherein illustrating the state of the real-world system at a future time depicts at least one of: the real-world system if no action is performed; the real-world system if a user proposed action is performed; and the real-world system if a system recommended action is performed. ### Claim 12 (depends on 1) The method of claim 1 wherein the performed plurality of simulations, simulates one or more emergent behaviors of the real-world system. ### Claim 13 (depends on 1) The method of claim 1 wherein the augmented reality illustrates physical field data of the real-world system determined using the calibrated model. ### Claim 14 (depends on 13) The method of claim 13 wherein the physical field data includes at least one of: temperature; density; gas flow; probable fracture location; likelihood of fracture; velocity; stress; and strain. ### Claim 15 A computer system for providing an augmented reality experience, the computer system comprising: a processor; and a memory with computer code instructions stored thereon, the processor and the memory, with the computer code instructions being configured to cause the system to: define in computer system memory, a model of a real-world system; perform a plurality of simulations of the real-world system using the defined model, each simulation of the plurality of simulations including one or more simulation input parameters for that simulation, performing each simulation of the plurality of simulations producing predicted field data for that simulation; store the predicted field data and the associated one or more simulation input parameters for each simulation of the plurality of simulations; receive field data from one or more sensors in the real-world system; calibrate the model of the real-world system using the received field data relative to the predicted field data, resulting in a calibrated model; and provide the augmented reality experience of the real-world system illustrating a state of the real-world system at a future time using the determined values of the one or more calibrated input parameters as inputs to the calibrated model of the real-world system, the augmented reality experience including a presentation of the received field data and simulation data the state of the real-world system at the future time determined to be the nearest failure condition to the current state of the real-world system. ### Claim 16 (depends on 15) The system of claim 15, wherein illustrating the state of the real-world system at a future time further includes using one or more counterexamples, wherein calibrating the model further includes determining values of one or more calibrated simulation input parameters of the model based on the plurality of simulations and the one or more simulation input parameters associated with each of the plurality of simulations such that the output of the model minimizes differences between the field data from the one or more sensors and the predicted field data. ### Claim 17 (depends on 15) The system of claim 15 wherein the computer code instructions further cause the system to perform one or more potential action simulations of the real-world system using the defined model with potential actions to identify one or more potential actions that can be taken to avoid the nearest failure condition. ### Claim 18 (depends on 17) The system of claim 17 wherein the computer code instructions further cause the system to illustrate the one or more actions that can be taken to avoid nearest failure condition. ### Claim 19 (depends on 17) The system of claim 17 wherein identifying one or more potential actions that can be taken to avoid the nearest failure condition further includes using one or more previously performed simulations. ### Claim 20 A non-transitory computer program product for providing an augmented reality experience, the computer program product executed by a server in communication across a network with one or more clients and comprising: a computer readable medium, the computer readable medium comprising program instructions which, when executed by a processor causes: defining in computer memory, a model of a real-world system; performing a plurality of simulations of the real-world system using the defined model, each simulation of the plurality of simulations including one or more simulation input parameters for that simulation, performing each simulation of the plurality of simulations producing predicted field data for that simulation; storing the predicted field data and the associated one or more simulation input parameters for each simulation of the plurality of simulations; receiving field data from one or more sensors in the real-world system; calibrating the model of the real-world system using the received field data relative to the predicted field data, resulting in a calibrated model; and providing the augmented reality experience of the real-world system using the determined values of the one or more calibrated input parameters as inputs to the calibrated model of the real-world system, the state of the real-world system at a future time is determined to be the nearest failure condition to the current state of the real-world system.