# US 2025/0220080 A1 · Multimodal vehicle sensor fusion and streaming > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 2025/0220080 A1 - Document ID: US20250220080A1 - Status: Published application - Kind: A1 - Application number: 18/403,527 - Filed: 2024-01-03 - Published: 2025-07-03 - Earliest claimed: 2024-01-03 - Assignees: GM Cruise Holdings LLC - Applicants: GM Cruise Holdings LLC - Inventors: Vinayaka Dattatraya; Christopher Gorham; Curtis Boushley; Yu Chen; Steven Sturkop - Web page: https://patentgrep.com/patent/US20250220080A1 ## CPC - H04L 67/12: specially adapted for proprietary or special-purpose networking environments, e.g. medical networks, sensor networks, networks in vehicles or remote metering networks - H04L 47/25: with rate being modified by the source upon detecting a change of network conditions - G08G 1/20: {Monitoring the location of vehicles belonging to a group, e.g. fleet of vehicles, countable or determined number of vehicles} - H04L: TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION (arrangements common to telegraphic and telephonic communication H04M) - G08G: TRAFFIC CONTROL SYSTEMS (guiding railway traffic, ensuring the safety of railway traffic B61L; arrangement of road signs or traffic signals E01F9/00; radar or analogous systems, sonar systems, lidar systems specially adapted for traffic control G01S13/91, G01S15/88, G01S17/88; {radar or analogous systems, sonar systems, lidar systems specially adapted for anti-collision purposes G01S13/93, G01S15/93, G01S17/93}) ## Abstract Systems and methods for encrypting and streaming AV sensor data and AV derived data. In particular, systems and methods are provided for receiving AV sensor data at a streaming platform and providing the received data to multiple endpoints. In some examples, the multiple endpoints are multiple viewers viewing the AV sensor data. In various examples, an AV transmits the AV sensor data to the streaming platform, and the AV sensor data can be transmitted from the streaming platform to multiple endpoints without any effect on AV central processing unit or bandwidth constraints. The AV sensor data can include multiple data streams that are layered and compressed for efficient streaming, and the streaming platform can receive real-time streamed data from the AV even as bandwidth and connection constraints vary. In various examples, the bit rate of the video ingestion from the AV can be changed based on bandwidth availability. ## Claims ### Claim 1 A system for providing data streams from a vehicle to multiple clients, comprising: a vehicle including a plurality of sensors, wherein the plurality of sensors generate sensor data, and wherein the plurality of sensors include image sensors and the sensor data includes image data; a plurality of endpoint clients including a respective plurality of client front ends; and a streaming platform including: a streaming manager configured to: receive a first data stream request from a first endpoint client of the plurality of endpoint clients, wherein the first data stream request includes a vehicle identification and a view identifier, assign the first endpoint client to a streaming server, receive a second data stream request from a second endpoint client of the plurality of endpoint clients, determine that the second data stream request can be fulfilled by the streaming server, and assign the second endpoint client to the streaming server; and the streaming server configured to: allocate a first port for receiving a data stream from the vehicle, wherein the data stream includes the sensor data, allocate a second port for transmitting the data stream to the first endpoint client, receive the data stream including the sensor data from the vehicle, and transmit the data stream to the first endpoint client and the second endpoint client. ### Claim 2 (depends on 1) The system of claim 1, wherein the first port has corresponding first port information, and wherein the streaming server is further configured to transmit the first port information to the streaming manager, and wherein the streaming manager is further configured to provide the first port information and a streaming server IP address to the vehicle. ### Claim 3 (depends on 2) The system of claim 2, wherein the streaming server is further configured to provide a streaming server authentication certificate to the vehicle, wherein the vehicle is configured to: validate the streaming server authentication certificate, encrypt the data stream including the sensor data, and transmit an encrypted data stream to the streaming server. ### Claim 4 (depends on 3) The system of claim 3, wherein the vehicle is further configured to transmit the encrypted data stream at a first bit rate, wherein the first bit rate is variable, and wherein the first bit rate is based on vehicle network bandwidth. ### Claim 5 (depends on 4) The system of claim 4, wherein the streaming server is further configured to transmit a request to the vehicle to adjust the first bit rate. ### Claim 6 (depends on 1) The system of claim 1, wherein the second port has corresponding second port information, and wherein the streaming server is further configured to transmit the second port information to the streaming manager, and wherein the streaming manager is further configured to provide the second port information and a streaming server IP address to the endpoint client. ### Claim 7 (depends on 1) The system of claim 1, wherein the streaming manager is further configured to determine that the second data stream request can be fulfilled by the streaming server by determining that the second data stream request can be fulfilled using the data stream. ### Claim 8 (depends on 7) The system of claim 7, wherein the streaming server is further configured to allocate a third port for transmitting the data stream to the second endpoint client, transmit the data stream to the first endpoint client via the second port, and transmit the data stream to the second endpoint client via the third port. ### Claim 9 (depends on 1) The system of claim 1, wherein the streaming manager further includes a streamer database including a list of current streamers including the streaming server, and a list of currently active streams. ### Claim 10 A method for providing data streams from a vehicle to multiple clients, comprising: receiving a data stream request from an endpoint client including a vehicle identification number and a view identifier; identifying a vehicle corresponding to the vehicle identification number and identifying one or more vehicle sensors corresponding to the view identifier; assigning the endpoint client to a streaming server; allocating, at the streaming server, a first port for receiving a data stream from the vehicle, wherein the data stream includes sensor data from the one or more vehicle sensors; allocating, at the streaming server, a second port for transmitting the data stream to the endpoint client; receiving, at the streaming server, the data stream including the sensor data from the vehicle, wherein the data stream is transmitted from the vehicle at a first bit rate, wherein the first bit rate is variable; and transmitting the data stream from the streaming server to the endpoint client at the first bit rate. ### Claim 11 (depends on 10) The method of claim 10, wherein the endpoint client is a first endpoint client, and further comprising transmitting the data stream from the streaming server to a second endpoint client at the first bit rate. ### Claim 12 (depends on 10) The method of claim 10, further comprising changing the first bit rate based on signal bandwidth on the vehicle. ### Claim 13 (depends on 10) The method of claim 10, further comprising receiving, at a streaming manager, a heartbeat signal from the streaming server, wherein the heartbeat signal includes information on a streaming server condition and a list of streaming server streams. ### Claim 14 (depends on 10) The method of claim 10, further comprising displaying the sensor data in an interface at the client endpoint. ### Claim 15 (depends on 10) The method of claim 10, further comprising: transmitting first port information to the streaming manager, and providing, by the streaming manager, the first port information and a streamer IP address to the vehicle. ### Claim 16 One or more non-transitory computer-readable media storing instructions executable to perform operations, the operations comprising: receiving a data stream request from an endpoint client including a vehicle identification number and a view identifier; identifying a vehicle corresponding to the vehicle identification number and identifying one or more vehicle sensors corresponding to the view identifier; assigning the endpoint client to a streaming server; allocating, at the streaming server, a first port for receiving a data stream from the vehicle, wherein the data stream includes sensor data from the one or more vehicle sensors; allocating, at the streaming server, a second port for transmitting the data stream to the endpoint client; receiving, at the streaming server, the data stream including the sensor data from the vehicle, wherein the data stream is transmitted from the vehicle at a first bit rate, wherein the first bit rate is variable; and transmitting the sensor data from the streaming server to the endpoint client at the first bit rate. ### Claim 17 (depends on 16) The one or more non-transitory computer-readable media of claim 16, wherein the endpoint client is a first endpoint client, and the operations further comprising transmitting the data stream from the streaming server to a second endpoint client at the second bit rate. ### Claim 18 (depends on 16) The one or more non-transitory computer-readable media of claim 16, the operations further comprising changing the first bit rate based on signal bandwidth on the vehicle. ### Claim 19 (depends on 16) The one or more non-transitory computer-readable media of claim 16, the operations further comprising receiving, at a streaming manager, a heartbeat signal from the streaming server, wherein the heartbeat signal includes information on a streaming server condition and a list of streaming server streams. ### Claim 20 (depends on 16) The one or more non-transitory computer-readable media of claim 16, the operations further comprising: transmitting first port information to a streaming manager, and providing, by the streaming manager, the first port information and a streamer IP address to the vehicle.