# US 2023/0054626 A1 · Persisting Predicted Objects for Robustness to Perception Issues in Autonomous Driving > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 2023/0054626 A1 - Document ID: US20230054626A1 - Status: Published application - Kind: A1 - Application number: 17/404,553 - Filed: 2021-08-17 - Published: 2023-02-23 - Earliest claimed: 2021-08-17 - Assignees: Argo AI, LLC - Applicants: Argo AI, LLC - Inventors: Neal Seegmiller; Ed Venator; Christopher Cunningham; Patrick Barone; Xi Cai - Web page: https://patentgrep.com/patent/US20230054626A1 ## CPC - B60W 60/0027: {using trajectory prediction for other traffic participants} - B60W 60/0011: {involving control alternatives for a single driving scenario, e.g. planning several paths to avoid obstacles} - B60W 2420/403: Image sensing, e.g. optical camera - B60W 2420/408: Radar; Laser, e.g. lidar - B60W 2554/402: Type - B60W 2554/4042: Longitudinal speed - B60W 2556/10: Historical data - B60W 2556/15: Data age - B60W 2556/50: of positioning data, e.g. GPS [Global Positioning System] data - B60W: CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT ## Abstract Systems and methods for operating an autonomous vehicle (AV) are provided. The method includes detecting one or more objects in an environment, predicting a first set of predicted object trajectories comprising one or more trajectories for each of the detected one or more objects, generating a plurality of candidate AV trajectories for the AV, scoring each of the candidate AV trajectories according to a cost function, using the scoring to select a final AV trajectory for execution, determining which of the predicted object trajectories affected the final AV trajectory and which did not do so, adding the predicted object trajectories that affected the final AV trajectory to a persisted prediction cache, excluding from the persisted prediction cache any predicted object trajectories that did not affect the final AV trajectory, and executing the final AV trajectory to cause the AV to move along the final AV trajectory. ## Claims ### Claim 1 A method of operating an autonomous vehicle, method comprising: detecting one or more objects in an environment of an autonomous vehicle; predicting a first set of predicted object trajectories comprising one or more trajectories for each of the detected one or more objects; using a motion planning model to generate a plurality of candidate autonomous vehicle trajectories for the autonomous vehicle, wherein at least one predicted object trajectory from a persisted prediction cache is used as an input to the motion planning model for each of the candidate autonomous vehicle trajectories; scoring each of the candidate autonomous vehicle trajectories according to a cost function; and using the scoring to select a final autonomous vehicle trajectory for execution, wherein the final autonomous vehicle trajectory causes the autonomous vehicle to execute a maneuver. ### Claim 2 (depends on 1) The method of claim 1, further comprising updating the persisted prediction cache with any predicted object trajectories from the first set that affected the final autonomous vehicle trajectory. ### Claim 3 (depends on 1) The method of claim 1, further comprising using at least one of the predicted object trajectories from the first set is also used as an input to the motion planning model. ### Claim 4 (depends on 1) The method of claim 1, further comprising excluding from the persisted prediction cache any predicted object trajectories that did not affect the final autonomous vehicle trajectory. ### Claim 5 (depends on 4) The method of claim 4, wherein excluding from the persisted prediction cache any predicted object trajectories that did not affect the final autonomous vehicle trajectory comprises storing, to the persisted prediction cache, only predicted object trajectories that affected the final autonomous vehicle trajectory. ### Claim 6 (depends on 1) The method of claim 1, further comprising comparing an age of each persisted predicted object trajectory in the persisted prediction cache against an age limit, and removing each persisted predicted object trajectory in the persisted prediction cache that has an age greater than the age limit. ### Claim 7 (depends on 1) The method of claim 1, further comprising determining an object type for each object in the one or more objects. ### Claim 8 A system comprising: one or more sensors coupled to an autonomous vehicle and configured to detect one or more objects in an environment of the autonomous vehicle; and a computing device comprising a processor and memory, wherein the memory includes instructions that, when executed by the processor, will cause the processor to: predict a first set of predicted object trajectories comprising one or more trajectories for each of the detected one or more objects, use a motion planning model to generate a plurality of candidate autonomous vehicle trajectories for the autonomous vehicle, wherein at least one predicted object trajectory from a persisted prediction cache is used as an input to the motion planning model for each of the candidate autonomous vehicle trajectories, score each of the candidate autonomous vehicle trajectories according to a cost function, use the scoring to select a final autonomous vehicle trajectory for execution, determine which of the predicted object trajectories affected the final autonomous vehicle trajectory and which did not do so, add the predicted object trajectories that affected the final autonomous vehicle trajectory to the persisted prediction cache, exclude from the persisted prediction cache any predicted object trajectories that did not affect the final autonomous vehicle trajectory, and using a motion planning system of the autonomous vehicle, execute the final autonomous vehicle trajectory to cause the autonomous vehicle to move along the final autonomous vehicle trajectory. ### Claim 9 (depends on 8) The system of claim 8, further comprising instructions to also use at least one of the predicted object trajectories from the first set as an input to the motion planning model. ### Claim 10 (depends on 8) The system of claim 8, further comprising instructions to exclude from the persisted prediction cache any predicted object trajectories that did not affect the final autonomous vehicle trajectory comprises adding all of the predicted object trajectories to the cache, and then remove from the cache any predicted object trajectories that did not affect the final autonomous vehicle trajectory. ### Claim 11 (depends on 10) The system of claim 10, wherein the instructions to exclude from the persisted prediction cache any predicted object trajectories that did not affect the final autonomous vehicle trajectory comprise instructions to add to the cache only the predicted object trajectories that affected the final autonomous vehicle trajectory, and not any predicted object trajectories that did not affect the final autonomous vehicle trajectory. ### Claim 12 (depends on 8) The system of claim 8, further comprising instructions to cause the processor to compare an age of each persisted predicted object trajectory in the persisted prediction cache against an age limit. ### Claim 13 (depends on 12) The system of claim 12, wherein the instructions to cause the processor to exclude each persisted predicted object trajectory in the persisted prediction cache that has an age greater than the age limit. ### Claim 14 (depends on 8) The system of claim 8, further comprising instructions to cause the processor to determine an object type for each object in the one or more objects. ### Claim 15 A computer-readable medium comprising programming instructions that are configured to cause a processor to: receive information relating to one or more objects that one or more sensors coupled to an autonomous vehicle detected in an environment of the autonomous vehicle; predict a first set of predicted object trajectories comprising one or more trajectories for each of the detected one or more objects; use a motion planning model to generate a plurality of candidate autonomous vehicle trajectories for the autonomous vehicle, wherein at least one predicted object trajectory from a persisted prediction cache is used as an input to the motion planning model for each of the candidate autonomous vehicle trajectories; score each of the candidate autonomous vehicle trajectories according to a cost function; use the scoring to select a final autonomous vehicle trajectory for execution; determine which of the predicted object trajectories affected the final autonomous vehicle trajectory and which did not do so; add the predicted object trajectories that affected the final autonomous vehicle trajectory to the persisted prediction cache; exclude from the persisted prediction cache any predicted object trajectories that did not affect the final autonomous vehicle trajectory; and using a motion planning system of the autonomous vehicle, execute the final autonomous vehicle trajectory to cause the autonomous vehicle to move along the final autonomous vehicle trajectory. ### Claim 16 (depends on 15) The computer-readable medium of claim 15, further comprising instructions to also use at least one of the predicted object trajectories from the first set as an input to the motion planning model. ### Claim 17 (depends on 15) The computer-readable medium of claim 15, further comprising instructions to exclude from the persisted prediction cache any predicted object trajectories that did not affect the final autonomous vehicle trajectory comprises adding all of the predicted object trajectories to the cache, and then remove from the cache any predicted object trajectories that did not affect the final autonomous vehicle trajectory. ### Claim 18 (depends on 17) The computer-readable medium of claim 17, wherein the instructions to exclude from the persisted prediction cache any predicted object trajectories that did not affect the final autonomous vehicle trajectory comprise instructions to add to the cache only the predicted object trajectories that affected the final autonomous vehicle trajectory, and not any predicted object trajectories that did not affect the final autonomous vehicle trajectory. ### Claim 19 (depends on 15) The computer-readable medium of claim 15, further comprising instructions to cause the processor to compare an age of each persisted predicted object trajectory in the persisted prediction cache against an age limit. ### Claim 20 (depends on 19) The computer-readable medium of claim 19, wherein the instructions to cause the processor to exclude each persisted predicted object trajectory in the persisted prediction cache that has an age greater than the age limit.