# US 12,179,785 B2 · Augmented reality incident projection based on vehicle movements > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 12,179,785 B2 - Document ID: US12179785B2 - Status: Granted - Kind: B2 - Application number: 18/171,979 - Filed: 2023-02-21 - Published: 2024-08-22 - Granted: 2024-12-31 - Earliest claimed: 2023-02-21 - Assignees: International Business Machines Corporation - Applicants: International Business Machines Corporation - Inventors: Tushar Agrawal; Martin G. Keen; Jeremy R. Fox; Sarbajit K. Rakshit - Web page: https://patentgrep.com/patent/US12179785B2 ## CPC - B60W 50/14: Means for informing the driver, warning the driver or prompting a driver intervention - B60K 35/00: Instruments specially adapted for vehicles; Arrangement of instruments in or on vehicles - B60K 35/28: characterised by the type of the output information, e.g. video entertainment or vehicle dynamics information; characterised by the purpose of the output information, e.g. for attracting the attention of the driver - B60K 35/29: Instruments characterised by the way in which information is handled, e.g. showing information on plural displays or prioritising information according to driving conditions - B60W 30/0956: {the prediction being responsive to traffic or environmental parameters} - B60W 50/0097: {Predicting future conditions} - G06N 20/00: Machine learning - G06V 20/58: Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads - G08G 1/166: {for active traffic, e.g. moving vehicles, pedestrians, bikes} - B60K 2360/171: Vehicle or relevant part thereof displayed - B60K 2360/177: Augmented reality - B60K 2360/1868: according to driving situations - B60W 2050/0028: {Mathematical models, e.g. for simulation} - B60W 2050/146: {Display means} - B60W 2556/10: Historical data - B60W 2556/65: Data transmitted between vehicles - B60W: CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT - B60K: ARRANGEMENT OR MOUNTING OF PROPULSION UNITS OR OF TRANSMISSIONS IN VEHICLES; ARRANGEMENT OR MOUNTING OF PLURAL DIVERSE PRIME-MOVERS IN VEHICLES; AUXILIARY DRIVES FOR VEHICLES; INSTRUMENTATION OR DASHBOARDS FOR VEHICLES; ARRANGEMENTS IN CONNECTION WITH COOLING, AIR INTAKE, GAS EXHAUST OR FUEL SUPPLY OF PROPULSION UNITS IN VEHICLES - G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS - G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING - G08G: TRAFFIC CONTROL SYSTEMS (guiding railway traffic, ensuring the safety of railway traffic B61L; arrangement of road signs or traffic signals E01F9/00; radar or analogous systems, sonar systems, lidar systems specially adapted for traffic control G01S13/91, G01S15/88, G01S17/88; {radar or analogous systems, sonar systems, lidar systems specially adapted for anti-collision purposes G01S13/93, G01S15/93, G01S17/93}) ## Abstract A computer-implemented method, a computer system and a computer program product project incident information within an augmented reality environment based on vehicle movements. The method includes determining a predicted path of a vehicle based on telemetry data associated with the vehicle. The method also includes obtaining driving conditions associated with a surrounding environment using a sensor. In addition, the method includes identifying an object in the predicted path of the vehicle. The method further includes determining a collision zone on the object and a corresponding collision zone on the vehicle based on physical attributes of the object and the predicted path of the vehicle. Lastly, the method includes generating a display of the driving conditions using an augmented reality device, where the display indicates the collision zone on the object and includes a virtual model of the vehicle that indicates the corresponding collision zone on the vehicle. ## Claims ### Claim 1 A computer-implemented method for projecting incident information within an augmented reality environment based on vehicle movements, the method comprising: determining a predicted path of a vehicle based on telemetry data associated with the vehicle; obtaining driving conditions associated with a surrounding environment using a sensor, wherein the driving conditions are selected from a group consisting of: image data, audio data and text data; identifying an object in the predicted path of the vehicle; determining a collision zone on the object and a corresponding collision zone on the vehicle based on physical attributes of the object and the predicted path of the vehicle; and generating a display of the driving conditions using an augmented reality device, wherein the display indicates the collision zone on the object and includes a virtual model of the vehicle, wherein the virtual model of the vehicle indicates the corresponding collision zone on the vehicle. ### Claim 2 (depends on 1) The computer-implemented method of claim 1, wherein the virtual model of the vehicle includes information based on a determination of a vehicle damage level in a potential collision with the object. ### Claim 3 (depends on 2) The computer-implemented method of claim 2, further comprising: creating a digital twin instance for the object; simulating a collision between the object and the vehicle using the digital twin instance; and updating the vehicle damage level in the potential collision with the object based on a digital twin simulation output. ### Claim 4 (depends on 1) The computer-implemented method of claim 1, further comprising: forming a communications network between the vehicle and a second vehicle; and obtaining the driving conditions associated with the surrounding environment from the second vehicle. ### Claim 5 (depends on 4) The computer-implemented method of claim 4, wherein the display includes a second virtual model of the second vehicle that indicates a second collision zone on the second vehicle. ### Claim 6 (depends on 1) The computer-implemented method of claim 1, wherein a machine learning model that predicts a path of a moving vehicle based on vehicle data and historical driving information associated with a driver is used to determine the predicted path of the vehicle. ### Claim 7 (depends on 1) The computer-implemented method of claim 1, wherein the augmented reality device is worn by a driver of the vehicle. ### Claim 8 A computer system for projecting incident information within an augmented reality environment based on vehicle movements, the computer system comprising: a processor set; one or more computer-readable storage media; and program instructions stored on the one or more computer-readable storage media to cause the processor set to: determine a predicted path of a vehicle based on telemetry data associated with the vehicle; obtain driving conditions associated with a surrounding environment using a sensor, wherein the driving conditions are selected from a group consisting of: image data, audio data and text data; identify an object in the predicted path of the vehicle; determine a collision zone on the object and a corresponding collision zone on the vehicle based on physical attributes of the object and the predicted path of the vehicle; and generate a display of the driving conditions using an augmented reality device, wherein the display indicates the collision zone on the object and includes a virtual model of the vehicle, wherein the virtual model of the vehicle indicates the corresponding collision zone on the vehicle. ### Claim 9 (depends on 8) The computer system of claim 8, wherein the virtual model of the vehicle includes information based on a determination of a vehicle damage level from a potential collision with the object. ### Claim 10 (depends on 9) The computer system of claim 9, further comprising program instructions stored on the one or more computer-readable storage media to cause the processor set to: create a digital twin instance for the object; simulate a collision between the object and the vehicle using the digital twin instance; and update the vehicle damage level in the potential collision with the object based on a digital twin simulation output. ### Claim 11 (depends on 8) The computer system of claim 8, further comprising program instructions stored on the one or more computer-readable storage media to cause the processor set to: form a communications network between the vehicle and a second vehicle; and obtain the driving conditions associated with the surrounding environment from the second vehicle. ### Claim 12 (depends on 11) The computer system of claim 11, wherein the display includes a second virtual model of the second vehicle that indicates a second collision zone on the second vehicle. ### Claim 13 (depends on 8) The computer system of claim 8, wherein a machine learning model that predicts a path of a moving vehicle based on vehicle data and historical driving information associated with a driver is used to determine the predicted path of the vehicle. ### Claim 14 (depends on 8) The computer system of claim 8, wherein the augmented reality device is worn by a driver of the vehicle. ### Claim 15 A computer program product for projecting incident information within an augmented reality environment based on vehicle movements, the computer program product comprising: one or more non-transitory computer-readable storage media; and program instructions stored on the one or more computer-readable storage media to: determine a predicted path of a vehicle based on telemetry data associated with the vehicle; obtain driving conditions associated with a surrounding environment using a sensor, wherein the driving conditions are selected from a group consisting of: image data, audio data and text data; identify an object in the predicted path of the vehicle; determine a collision zone on the object and a corresponding collision zone on the vehicle based on physical attributes of the object and the predicted path of the vehicle; and generate a display of the driving conditions using an augmented reality device, wherein the display indicates the collision zone on the object and includes a virtual model of the vehicle, wherein the virtual model of the vehicle indicates the corresponding collision zone on the vehicle. ### Claim 16 (depends on 15) The computer program product of claim 15, wherein the virtual model of the vehicle includes information based on a determination of a vehicle damage level from a potential collision with the object. ### Claim 17 (depends on 16) The computer program product of claim 16, further comprising program instructions stored on the one or more computer-readable storage media to: create a digital twin instance for the object; simulate a collision between the object and the vehicle using the digital twin instance; and update the vehicle damage level in the potential collision with the object based on a digital twin simulation output. ### Claim 18 (depends on 15) The computer program product of claim 15, further comprising program instructions stored on the one or more computer-readable storage media to: form a communications network between the vehicle and a second vehicle; and obtain the driving conditions associated with the surrounding environment from the second vehicle. ### Claim 19 (depends on 18) The computer program product of claim 18, wherein the display includes a second virtual model of the second vehicle that indicates a second collision zone on the second vehicle. ### Claim 20 (depends on 15) The computer program product of claim 15, wherein a machine learning model that predicts a path of a moving vehicle based on vehicle data and historical driving information associated with a driver is used to determine the predicted path of the vehicle.