# US 12,008,781 B1 · Synchronizing vehicle telematics data with infrastructure data pertaining to a road segment > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 12,008,781 B1 - Document ID: US12008781B1 - Status: Granted - Kind: B1 - Application number: 17/088,393 - Filed: 2020-11-03 - Granted: 2024-06-11 - Earliest claimed: 2020-05-20 - Assignees: State Farm Mutual Automobile Insurance Company - Applicants: State Farm Mutual Automobile Insurance Company - Inventors: Alexander Cardona; Kip Wilson; David Frank; Phillip Michael Wilkowski; Nolan White - Web page: https://patentgrep.com/patent/US12008781B1 ## CPC - G06F 18/251: {of input or preprocessed data} - B60W 30/08: {Active safety systems} predicting or avoiding probable or impending collision {or attempting to minimise its consequences} - B60W 30/182: Selecting between different operative modes, e.g. comfort and performance modes - B60W 40/06: Road conditions - B60W 40/105: Speed - G01C 21/3461: {Preferred or disfavoured areas, e.g. dangerous zones, toll or emission zones, intersections, manoeuvre types or segments such as motorways, toll roads or ferries} - G01C 21/3484: {Personalized, e.g. from learned user behaviour or user-defined profiles} - G01C 21/3602: {Input other than that of destination using image analysis, e.g. detection of road signs, lanes, buildings, real preceding vehicles using a camera} - G05D 1/0088: {characterized by the autonomous decision making process, e.g. artificial intelligence, predefined behaviours (using knowledge based models G06N5/00)} - G05D 1/43: Control of position or course in two dimensions [2D] - G06F 3/0484: for the control of specific functions or operations, e.g. selecting or manipulating an object, an image or a displayed text element, setting a parameter value or selecting a range - G06F 16/29: Geographical information databases - G06F 16/54: Browsing; Visualisation therefor - G06F 16/5866: {using information manually generated, e.g. tags, keywords, comments, manually generated location and time information} - G06F 16/587: using geographical or spatial information, e.g. location - G06N 20/00: Machine learning - G06Q 10/20: Administration of product repair or maintenance - G06Q 50/26: Government or public services (business processes related to the transportation industry G06Q50/40) - G06T 7/20: Analysis of motion (motion estimation for coding, decoding, compressing or decompressing digital video signals H04N19/43, H04N19/51) - G06T 7/292: Multi-camera tracking - G06T 7/70: Determining position or orientation of objects or cameras (camera calibration G06T7/80) - G06V 20/44: {Event detection} - G06V 20/52: Surveillance or monitoring of activities, e.g. for recognising suspicious objects (recognising microscopic objects G06V20/69) - G06V 20/54: of traffic, e.g. cars on the road, trains or boats - G06V 20/56: exterior to a vehicle by using sensors mounted on the vehicle - G06V 20/58: Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads - G06V 20/584: {of vehicle lights or traffic lights} - G06V 20/588: {Recognition of the road, e.g. of lane markings; Recognition of the vehicle driving pattern in relation to the road} - G07C 5/008: {communicating information to a remotely located station (transmission systems for measured values G08C)} - G08G 1/0112: {from the vehicle, e.g. floating car data [FCD]} - G08G 1/052: with provision for determining speed or overspeed {(speed measuring in general G01P)} - G11B 27/10: Indexing; Addressing; Timing or synchronising; Measuring tape travel - G11B 27/34: Indicating arrangements {(indicating means incorporated in magazine or cassette G11B23/046 and G11B23/0875; indicating measured values in general G01D)} - H04N 7/18: Closed-circuit television [CCTV] systems, i.e. systems in which the video signal is not broadcast - H04W 4/44: for communication between vehicles and infrastructures, e.g. vehicle-to-cloud [V2C] or vehicle-to-home [V2H] - B60W 2030/082: {Vehicle operation after collision} - B60W 2420/403: Image sensing, e.g. optical camera - B60W 2420/408: Radar; Laser, e.g. lidar - B60W 2520/10: Longitudinal speed - B60W 2552/20: Road profile, i.e. the change in elevation or curvature of a plurality of continuous road segments - B60W 2552/53: Road markings, e.g. lane marker or crosswalk - B60W 2556/50: of positioning data, e.g. GPS [Global Positioning System] data - B60W 2556/55: using telemetry - G01C 21/3815: {Road data} - G01C 21/3848: {Data obtained from both position sensors and additional sensors} - G06Q 10/10: Office automation; Time management - G06Q 40/08: Insurance - G06T 2207/30252: Vehicle exterior; Vicinity of vehicle - H04L 67/12: specially adapted for proprietary or special-purpose networking environments, e.g. medical networks, sensor networks, networks in vehicles or remote metering networks - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) - B60W: CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT - G01C: MEASURING DISTANCES, LEVELS OR BEARINGS; SURVEYING; NAVIGATION; GYROSCOPIC INSTRUMENTS; PHOTOGRAMMETRY OR VIDEOGRAMMETRY (measuring liquid level G01F; radio navigation, determining distance or velocity by use of propagation effects, e.g. Doppler effects, propagation time, of radio waves, analogous arrangements using other waves G01S) - G05D: SYSTEMS FOR CONTROLLING OR REGULATING NON-ELECTRIC VARIABLES - G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS - G06Q: INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR - G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL - G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING - G07C: TIME OR ATTENDANCE REGISTERS; REGISTERING OR INDICATING THE WORKING OF MACHINES; GENERATING RANDOM NUMBERS; VOTING OR LOTTERY APPARATUS; ARRANGEMENTS, SYSTEMS OR APPARATUS FOR CHECKING NOT PROVIDED FOR ELSEWHERE - G08G: TRAFFIC CONTROL SYSTEMS (guiding railway traffic, ensuring the safety of railway traffic B61L; arrangement of road signs or traffic signals E01F9/00; radar or analogous systems, sonar systems, lidar systems specially adapted for traffic control G01S13/91, G01S15/88, G01S17/88; {radar or analogous systems, sonar systems, lidar systems specially adapted for anti-collision purposes G01S13/93, G01S15/93, G01S17/93}) - G11B: INFORMATION STORAGE BASED ON RELATIVE MOVEMENT BETWEEN RECORD CARRIER AND TRANSDUCER (recording measured values in a way that does not require playback through a transducer G01D9/00; recording or playback apparatus using mechanically marked tape, e.g. punched paper tape, or using unit records, e.g. punched or magnetically marked cards G06K; transferring data from one type of record carrier to another G06K1/18; circuits for coupling output of reproducer to radio receiver H04B1/20; gramophone pick-ups or like acoustic electromechanical transducers or circuits therefor H04R) - H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION - H04W: WIRELESS COMMUNICATION NETWORKS (broadcast communication H04H; communication systems using wireless links for non-selective communication, e.g. wireless extensions H04M1/72) - H04L: TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION (arrangements common to telegraphic and telephonic communication H04M) ## Abstract Techniques for collecting, synchronizing, and displaying various types of data relating to a road segment enable, via one or more local or remote processors, servers, transceivers, and/or sensors, (i) enhanced and contextualized analysis of vehicle events by way of synchronizing different data types, relating to a monitored road segment, collected via various different types of data sources; (ii) enhanced and contextualized analysis of filed insurance claims pertaining to a vehicle incident at a road segment; (iii) advantageous machine learning techniques for predicting a level of risk assumed for a given vehicle event or a given road segment; (iv) techniques for accounting for region-specific driver profiles when controlling autonomous vehicles; and/or (v) improved techniques for providing a GUI to display collected data in a meaningful and contextualized manner. ## Claims ### Claim 1 A computer-implemented method for synchronizing data from vehicle sensors with data from infrastructure devices to facilitate improved analysis or display of information about road segments, the method comprising, via one or more local or remote processors, servers, transceivers, or sensors: (A) detecting a plurality of road segment parameters associated with a road segment during a time-period, wherein the plurality of road segment parameters includes first and second sets of parameters existing during the time-period, wherein said detecting includes: (i) detecting the first set of road segment parameters via one or more vehicle sensors; and (ii) detecting the second set of road segment parameters via one or more infrastructure devices disposed relative to the road segment such that the road segment is observable by the one or more infrastructure devices; (B) timestamping the plurality of road segment parameters; (C) time-synchronizing each of the plurality of road segment parameters to one another according to a common clock based on a given event represented in the set of parameters, and such that the plurality of road segment parameters is associated with a set of timestamps synchronized to the common clock; (D) storing the plurality of road segment parameters; and (E) performing one or more of: (i) displaying the plurality of road segment parameters via a graphic user interface according to a chronological order determined based upon a set of timestamps, or (ii) analyzing the plurality of road segment parameters to identify one or more vehicle events, each characterized by a subset of the plurality of road segment parameters. ### Claim 2 (depends on 1) The computer-implemented method of claim 1, further comprising: detecting one or more images; timestamping each of the one or more images; time-synchronizing the one or more images to the plurality of road segment parameters; and storing the one or more images; and wherein displaying the plurality of road segment parameters further includes displaying the one or more images simultaneously with the plurality of road segment parameters; and wherein analyzing the plurality of road segment parameters to identify the one or more vehicle events further includes analyzing the one or more images. ### Claim 3 (depends on 1) The computer-implemented method of claim 1, wherein detecting the first set of road segment parameters comprises detecting one or more of position, heading, or speed of a vehicle in which the vehicle sensors are disposed. ### Claim 4 (depends on 1) The computer-implemented method of claim 1, wherein the vehicle sensors are disposed in a first vehicle and wherein detecting the first set of road segment parameters comprises detecting one or more of position, heading, or speed of a second vehicle. ### Claim 5 (depends on 1) The computer-implemented method of claim 1, wherein the one or more vehicle sensors includes a first set of vehicle sensors in a first vehicle and a second set of vehicle sensors in a second vehicle. ### Claim 6 (depends on 1) The computer-implemented method of claim 1, wherein detecting the second set of road segment parameters via the one or more infrastructure devices comprises: detecting a health status of an infrastructure component. ### Claim 7 (depends on 1) The computer-implemented method of claim 1, wherein detecting the second set of road segment parameters via the one or more infrastructure devices comprises: detecting an operational status of an infrastructure component. ### Claim 8 (depends on 7) The computer-implemented method of claim 7, wherein the infrastructure component is a traffic signal and wherein the operational status indicates that the streetlight was green, yellow, or red. ### Claim 9 (depends on 1) The computer-implemented method of claim 1, wherein the infrastructure device is a radar device or a motion sensing device. ### Claim 10 (depends on 1) The computer-implemented method of claim 1, wherein time-synchronizing each of the plurality of road segment parameters to one another comprises: analyzing the first set of road segment parameters to identify: (i) at least one road segment parameter in the first set of road segment parameters corresponding to an occurrence of an event, and (ii) a first time period, according to a first clock associated with the one or more vehicle sensors, during which the event took place; analyzing the second set of road segment parameters to identify: (i) at least one road segment parameter in the second set of road segment parameters corresponding to the occurrence of the event, and (ii) a second time period, according to a second clock associated with the one or more infrastructure devices, during which the event took place; and setting the timestamps for the plurality of road segment parameters according to the common clock such that timestamps for the first set of road segment parameters matches timestamps for the second set of road segment parameters with respect to a time at which the event took place. ### Claim 11 A system for synchronizing data from vehicle sensors with data from infrastructure devices to facilitate improved analysis or display of information about road segments, the system comprising: one or more vehicle sensors; one or more infrastructure devices disposed relative to the road segment such that the road segment is observable by the one or more infrastructure devices; a computer system coupled to the one or more vehicle sensors and the one or more infrastructure devices, the computer system comprising one or more processors and a computer-readable memory storing instructions that, when executed by the one or more processors, cause the computer system to: (A) detect a plurality of road segment parameters associated with a road segment during a time-period, wherein the plurality of road segment parameters includes first and second sets of parameters existing during the time-period, including: (i) detect the first set of road segment parameters via the one or more vehicle sensors; and (ii) detect the second set of road segment parameters via the one or more infrastructure devices; (B) timestamp the plurality of road segment parameters; (C) time-synchronize each of the plurality of road segment parameters to one another according to a common clock based on a given event represented in the set of parameters, and such that the plurality of road segment parameters is associated with a set of timestamps synchronized to the common clock; (D) store the plurality of road segment parameters; and (E) perform one or more of: (i) display the plurality of road segment parameters via a graphic user interface according to a chronological order determined based upon a set of timestamps, or (ii) analyze the plurality of road segment parameters to identify one or more vehicle events, each characterized by a subset of the plurality of road segment parameters. ### Claim 12 (depends on 11) The system of claim 11, wherein the system further includes an image sensor coupled to the computer system and wherein the computer system is further configured to: detect one or more images via the image sensor; timestamp each of the one or more images; time-synchronize the one or more images to the plurality of road segment parameters; and store the one or more images; and wherein displaying the plurality of road segment parameters further includes displaying the one or more images simultaneously with the plurality of road segment parameters; and wherein analyzing the plurality of road segment parameters to identify the one or more vehicle events further includes analyzing the one or more images. ### Claim 13 (depends on 11) The system of claim 11, wherein the first set of road segment parameters include one or more of position, heading, or speed of a vehicle in which the vehicle sensors are disposed. ### Claim 14 (depends on 11) The system of claim 11, wherein the vehicle sensors are disposed in a first vehicle and wherein the first set of road segment parameters comprises one or more of position, heading, or speed of a second vehicle. ### Claim 15 (depends on 11) The system of claim 11, wherein the one or more vehicle sensors includes a first set of vehicle sensors in a first vehicle and a second set of vehicle sensors in a second vehicle. ### Claim 16 (depends on 11) The system of claim 11, wherein the second set of road segment comprises a health status of an infrastructure component. ### Claim 17 (depends on 11) The system of claim 11, wherein the second set of road segment parameters comprises an operational status of an infrastructure component. ### Claim 18 (depends on 17) The system of claim 17, wherein the infrastructure component is a traffic signal and wherein the operational status indicates that the streetlight was green, yellow, or red. ### Claim 19 (depends on 11) The system of claim 11, wherein the infrastructure device is a radar device or a motion sensing device. ### Claim 20 (depends on 11) The system of claim 11, wherein causing the computer system to time-synchronize each of the plurality of road segment parameters to one another comprises causing the computer system to: analyze the first set of road segment parameters to identify: (i) at least one road segment parameter in the first set of road segment parameters corresponding to an occurrence of an event, and (ii) a first time period, according to a first clock associated with the one or more vehicle sensors, during which the event took place; analyze the second set of road segment parameters to identify: (i) at least one road segment parameter in the second set of road segment parameters corresponding to the occurrence of the event, and (ii) a second time period, according to a second clock associated with the one or more infrastructure devices, during which the event took place; and set the timestamps for the plurality of road segment parameters according to the common clock such that timestamps for the first set of road segment parameters matches timestamps for the second set of road segment parameters with respect to a time at which the event took place.