Methods and systems for streaming autonomous vehicle log data

A system for streaming log data of an autonomous vehicle for visualization, the system comprising one or more processors and memory storing instructions that, when executed by the one or more processors, cause the system to:

• receive autonomous vehicle log data for an event associated with operation of the autonomous vehicle;
• generate a primary data file for the event comprising one or more groupings of the autonomous vehicle log data, wherein each grouping of the one or more groupings corresponds to a particular data type of one or more data types of autonomous vehicle log data and each grouping of the one or more groupings is divided into one or more segments associated with one or more predefined time intervals;
• generate a metadata file for the event comprising one or more timestamps and one or more data locations, wherein each timestamp and data location corresponds to a start point or an end point of one of the one or more segments in the primary data file;
• receive a request specifying:
• at least one data type of the one or more data types of autonomous vehicle log data to be streamed;
• and
• at least one temporal parameter, wherein the at least one temporal parameter corresponds to at least one grouping of the one or more groupings in the primary data file;
• retrieve one or more portions of the autonomous vehicle log data from the primary data file based on the request, the one or more timestamps in the metadata file, and the one or more data locations in the metadata file;
• and
• transmit via a network communication interface the one or more portions of the autonomous vehicle log data to a data visualization platform to generate a visualization based on the specified data types of the one or more data types of autonomous vehicle log data.

A non-transitory computer readable storage medium storing instructions for streaming log data of an autonomous vehicle for visualization, wherein the instructions, when executed by one or more processors of an electronic device, cause the device to:

• receive autonomous vehicle log data for an event associated with operation of the autonomous vehicle;
• generate a primary data file for the event comprising one or more groupings of the autonomous vehicle log data, wherein each grouping of the one or more groupings corresponds to a particular data type of one or more data types of autonomous vehicle log data and each grouping of the one or more groupings is divided into one or more segments associated with one or more predefined time intervals;
• generate a metadata file for the event comprising one or more timestamps and one or more data locations, wherein each timestamp and data location corresponds to a start point or an end point of one of the one or more segments in the primary data file;
• receive a request specifying:
• at least one data type of the one or more data types of autonomous vehicle log data to be streamed;
• and
• at least one temporal parameter, wherein the at least one temporal parameter corresponds to at least one grouping of the one or more groupings in the primary data file;
• retrieve one or more portions of the autonomous vehicle log data from the primary data file based on the request, the one or more timestamps in the metadata file, and the one or more data locations in the metadata file;
• and
• transmit via a network communication interface the one or more portions of the autonomous vehicle log data to a data visualization platform to generate a visualization based on the specified data types of the one or more data types of autonomous vehicle log data.