Multimodal vehicle sensor fusion and streaming

A system for providing data streams from a vehicle to multiple clients, comprising:

• a vehicle including a plurality of sensors, wherein the plurality of sensors generate sensor data, and wherein the plurality of sensors include image sensors and the sensor data includes image data;
• a plurality of endpoint clients including a respective plurality of client front ends;
• and
• a streaming platform including:
• a streaming manager configured to:
• receive a first data stream request from a first endpoint client of the plurality of endpoint clients, wherein the first data stream request includes a vehicle identification and a view identifier indicating a specific camera of the plurality of sensors,
• assign the first endpoint client to a streaming server,
• receive a second data stream request from a second endpoint client of the plurality of endpoint clients,
• determine that the second data stream request can be fulfilled by the streaming server by determining that the second data stream request specifies the same vehicle identification and the same view identifier as the data stream being received by the streaming server, and
• assign the second endpoint client to the streaming server;
• and
• the streaming server configured to:
• allocate a first port for receiving a data stream from the vehicle, wherein the data stream includes the sensor data,
• allocate a second port for transmitting the data stream to the first endpoint client,
• receive the data stream including the sensor data from the vehicle, and
• transmit the data stream to the first endpoint client and the second endpoint client.

One or more non-transitory computer-readable media storing instructions executable to perform operations, the operations comprising:

• receiving a first data stream request from a first endpoint client including a vehicle identification number and a view identifier indicating a specific camera of a plurality of sensors;
• identifying a vehicle corresponding to the vehicle identification number and identifying one or more vehicle sensors corresponding to the view identifier;
• assigning the first endpoint client to a streaming server;
• receiving a second data stream request from a second endpoint client;
• determining that the second data stream request can be fulfilled by the streaming server by determining that the second data stream request specifies the same vehicle identification and the same view identifier as the data stream being received by the streaming server;
• allocating, at the streaming server, a first port for receiving a data stream from the vehicle, wherein the data stream includes sensor data from the one or more vehicle sensors;
• allocating, at the streaming server, a second port for transmitting the data stream to the first endpoint client;
• receiving, at the streaming server, the data stream including the sensor data from the vehicle, wherein the data stream is transmitted from the vehicle at a first bit rate, wherein the first bit rate is variable;
• and
• transmitting the sensor data from the streaming server to the first endpoint client at the first bit rate.