Systems and methods for vehicle-to-vehicle communications for improved autonomous vehicle operations

An autonomous vehicle computing system comprising:

• a machine-learned aggregation model configured to aggregate a plurality of intermediate environmental representations from a first autonomous vehicle and a second autonomous vehicle, the autonomous vehicle computing system being onboard the second autonomous vehicle;
• a machine-learned perception model configured to generate autonomy outputs;
• one or more processors;
• and
• one or more tangible, non-transitory, computer readable media that store instructions that are executable by the one or more processors to cause the computing system to perform operations, the operations comprising:
• obtaining a first compressed intermediate environmental representation output by one or more intermediate layers of a machine-learned perception model of the first autonomous vehicle, wherein the first compressed intermediate environmental representation is indicative of at least a portion of an environment of the second autonomous vehicle;
• generating a first decompressed intermediate environmental representation by decompressing the first compressed intermediate environmental representation;
• obtaining a second intermediate environmental representation output by one or more intermediate layers of a machine-learned perception model of the second autonomous vehicle;
• determining, using the machine-learned aggregation model, an updated intermediate environmental representation based at least in part on the first decompressed intermediate environmental representation and the second intermediate environmental representation,
• wherein the machine-learned aggregation model comprises a graph neural network, the graph neural network comprising a plurality of nodes each corresponding to a respective autonomous vehicle of a plurality of autonomous vehicle within the environment of the second autonomous vehicle, wherein the plurality of autonomous vehicles comprises the first autonomous vehicle,
• wherein at least one node of the machine-learned aggregation model is configured to be updated based on a change to the plurality of autonomous vehicles that are within the environment of the second autonomous vehicle;
• and
• generating an autonomy output for the second autonomous vehicle based at least in part on the updated intermediate environmental representation.

An autonomous vehicle comprising:

• one or more processors;
• and
• one or more tangible, non-transitory, computer readable media that store instructions that are executable by the one or more processors to cause the one or more processors to perform operations, the operations comprising:
• obtaining a first compressed intermediate environmental representation output by one or more intermediate layers of a machine-learned perception model of another autonomous vehicle, wherein the first compressed intermediate environmental representation is indicative of at least a portion of an environment of the autonomous vehicle;
• generating a first decompressed intermediate environmental representation by decompressing the first compressed intermediate environmental representation;
• generating a second intermediate environmental representation output by one or more intermediate layers of a machine-learned perception model of the autonomous vehicle;
• determining, using a machine-learned aggregation model onboard the autonomous vehicle, an updated intermediate environmental representation based at least in part on the first decompressed intermediate environmental representation and the second intermediate environmental representation,
• wherein the machine-learned aggregation model comprises a graph neural network, and wherein the graph neural network comprises a plurality of nodes, each respective node of the graph neural network corresponding to a respective autonomous vehicle of a plurality of autonomous vehicles within the environment of the autonomous vehicle, wherein the plurality of autonomous vehicles comprises the other autonomous vehicle,
• wherein at least one node of the machine-learned aggregation model is configured to be updated based on a change to the plurality of autonomous vehicles that are within the environment of the autonomous vehicle;
• and
• generating an autonomy output for the autonomous vehicle based at least in part on the updated intermediate environmental representation, wherein the autonomy output is indicative of an object within the environment of the autonomous vehicle and one or more predicted future locations of the object;
• and
• generating a motion plan for the autonomous vehicle based at least in part on the autonomy output.