# US 12,223,575 B2 · AR-based visualization of accidental situation in multi-machine environment > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 12,223,575 B2 - Document ID: US12223575B2 - Status: Granted - Kind: B2 - Application number: 18/048,559 - Filed: 2022-10-21 - Published: 2024-04-25 - Granted: 2025-02-11 - Earliest claimed: 2022-10-21 - Assignees: International Business Machines Corporation - Applicants: International Business Machines Corporation - Inventors: Sarbajit K. Rakshit; Saraswathi Sailaja Perumalla; Ashwini Padubidri Chandrase - Web page: https://patentgrep.com/patent/US12223575B2 ## CPC - G06T 13/00: Animation - G06V 20/20: in augmented reality scenes - G06Q 10/0635: Risk analysis of enterprise or organisation activities - G06T 19/006: {Mixed reality (object pose determination, tracking or camera calibration for mixed reality G06T7/00)} - G06T 2215/16: Using real world measurements to influence rendering - G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL - G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING - G06Q: INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR ## Abstract An embodiment for augmented reality (AR)-based visualization of an accidental situation in a multi-machine environment is provided. The embodiment may include receiving real-time and historical data relating to an activity. The embodiment may also include identifying a context of the activity and at least one property of one or more objects associated with the activity. The embodiment may further include identifying one or more items of safety equipment used by a user. The embodiment may also include assigning a compatibility score to the user. The embodiment may further include in response to determining the compatibility score is not adequate, executing a digital twin simulation of an avatar of the user. The embodiment may also include creating a visual animation of a resultant state of the avatar of the user consistent with the executed digital twin simulation. The embodiment may further include displaying the visual animation to the user. ## Claims ### Claim 1 A computer-based method of augmented reality (AR)-based visualization of an accidental situation in a multi-machine environment, the method comprising: receiving real-time and historical data relating to an activity in a multi-machine environment; identifying a context of the activity and at least one property of one or more objects associated with the activity based on the real-time data; identifying one or more items of safety equipment used by a user in the multi-machine environment; assigning a compatibility score to the user based on the identified one or more items of safety equipment; determining whether the compatibility score is adequate to perform a predicted action of the user without an accident based on the historical data; in response to determining the compatibility score is not adequate, executing a digital twin simulation of an avatar of the user having the identified one or more items of safety equipment based on the predicted action of the user and the historical data, wherein the avatar of the user is wearing the identified one or more items of safety equipment in a same manner as the user in the multi-machine environment, and wherein the digital twin simulation is executed based on determining the user is within a pre-defined distance of at least one machine in the multi-machine environment; creating a visual animation of a resultant state of the avatar of the user consistent with the executed digital twin simulation; and displaying the visual animation to the user via an AR device. ### Claim 2 (depends on 1) The computer-based method of claim 1, wherein displaying the visual animation to the user via the AR device further comprises: performing a preventive measure. ### Claim 3 (depends on 2) The computer-based method of claim 2, wherein the preventive measure includes displaying a boundary via the AR device as a digital image overlay in the multi-machine environment, wherein the boundary indicates a safe area for the user to perform the predicted action. ### Claim 4 (depends on 3) The computer-based method of claim 3, wherein the preventive measure includes assigning the activity to a machine within the displayed boundary in response to the user selecting the resultant state of the avatar. ### Claim 5 (depends on 1) The computer-based method of claim 1, further comprising: identifying a required compatibility score for the user to perform the predicted action without the accident based on the historical data. ### Claim 6 (depends on 5) The computer-based method of claim 5, further comprising: recommending one or more updates to the one or more items of safety equipment used by the user based on the required compatibility score. ### Claim 7 (depends on 6) The computer-based method of claim 6, wherein the recommended update is selected from a group consisting of an additional item of safety equipment and a modification to the item of safety equipment. ### Claim 8 A computer system, the computer system comprising: one or more processors, one or more computer-readable memories, one or more computer-readable tangible storage medium, and program instructions stored on at least one of the one or more computer-readable tangible storage medium for execution by at least one of the one or more processors via at least one of the one or more computer-readable memories, wherein the computer system is capable of performing a method comprising: receiving real-time and historical data relating to an activity in a multi-machine environment; identifying a context of the activity and at least one property of one or more objects associated with the activity based on the real-time data; identifying one or more items of safety equipment used by a user in the multi-machine environment; assigning a compatibility score to the user based on the identified one or more items of safety equipment; determining whether the compatibility score is adequate to perform a predicted action of the user without an accident based on the historical data; in response to determining the compatibility score is not adequate, executing a digital twin simulation of an avatar of the user having the identified one or more items of safety equipment based on the predicted action of the user and the historical data, wherein the avatar of the user is wearing the identified one or more items of safety equipment in a same manner as the user in the multi-machine environment, and wherein the digital twin simulation is executed based on determining the user is within a pre-defined distance of at least one machine in the multi-machine environment; creating a visual animation of a resultant state of the avatar of the user consistent with the executed digital twin simulation; and displaying the visual animation to the user via an AR device. ### Claim 9 (depends on 8) The computer system of claim 8, wherein displaying the visual animation to the user via the AR device further comprises: performing a preventive measure. ### Claim 10 (depends on 9) The computer system of claim 9, wherein the preventive measure includes displaying a boundary via the AR device as a digital image overlay in the multi-machine environment, wherein the boundary indicates a safe area for the user to perform the predicted action. ### Claim 11 (depends on 10) The computer system of claim 10, wherein the preventive measure includes assigning the activity to a machine within the displayed boundary in response to the user selecting the resultant state of the avatar. ### Claim 12 (depends on 8) The computer system of claim 8, further comprising: identifying a required compatibility score for the user to perform the predicted action without the accident based on the historical data. ### Claim 13 (depends on 12) The computer system of claim 12, further comprising: recommending one or more updates to the one or more items of safety equipment used by the user based on the required compatibility score. ### Claim 14 (depends on 13) The computer system of claim 13, wherein the recommended update is selected from a group consisting of an additional item of safety equipment and a modification to the item of safety equipment. ### Claim 15 A computer program product, the computer program product comprising: one or more computer-readable tangible storage medium and program instructions stored on at least one of the one or more computer-readable tangible storage medium, the program instructions executable by a processor capable of performing a method, the method comprising: receiving real-time and historical data relating to an activity in a multi-machine environment; identifying a context of the activity and at least one property of one or more objects associated with the activity based on the real-time data; identifying one or more items of safety equipment used by a user in the multi-machine environment; assigning a compatibility score to the user based on the identified one or more items of safety equipment; determining whether the compatibility score is adequate to perform a predicted action of the user without an accident based on the historical data; in response to determining the compatibility score is not adequate, executing a digital twin simulation of an avatar of the user having the identified one or more items of safety equipment based on the predicted action of the user and the historical data, wherein the avatar of the user is wearing the identified one or more items of safety equipment in a same manner as the user in the multi-machine environment, and wherein the digital twin simulation is executed based on determining the user is within a pre-defined distance of at least one machine in the multi-machine environment; creating a visual animation of a resultant state of the avatar of the user consistent with the executed digital twin simulation; and displaying the visual animation to the user via an AR device. ### Claim 16 (depends on 15) The computer program product of claim 15, wherein displaying the visual animation to the user via the AR device further comprises: performing a preventive measure. ### Claim 17 (depends on 16) The computer program product of claim 16, wherein the preventive measure includes displaying a boundary via the AR device as a digital image overlay in the multi-machine environment, wherein the boundary indicates a safe area for the user to perform the predicted action. ### Claim 18 (depends on 17) The computer program product of claim 17, wherein the preventive measure includes assigning the activity to a machine within the displayed boundary in response to the user selecting the resultant state of the avatar. ### Claim 19 (depends on 15) The computer program product of claim 15, further comprising: identifying a required compatibility score for the user to perform the predicted action without the accident based on the historical data. ### Claim 20 (depends on 19) The computer program product of claim 19, further comprising: recommending one or more updates to the one or more items of safety equipment used by the user based on the required compatibility score.