# US 2025/0085115 A1 · Transformer framework for trajectory prediction > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 2025/0085115 A1 - Document ID: US20250085115A1 - Status: Published application - Kind: A1 - Application number: 18/501,362 - Filed: 2023-11-03 - Published: 2025-03-13 - Earliest claimed: 2023-09-13 - Assignees: Tusimple, Inc. - Applicants: TuSimple, Inc. - Inventors: Hao XIAO; Yiqian GAN; Ethan ZHANG; Xin YE; Yizhe ZHAO; Zhe HUANG; Lingting GE; Robert August ROSSI, JR. - Web page: https://patentgrep.com/patent/US20250085115A1 ## CPC - G01C 21/30: Map- or contour-matching - G01C: MEASURING DISTANCES, LEVELS OR BEARINGS; SURVEYING; NAVIGATION; GYROSCOPIC INSTRUMENTS; PHOTOGRAMMETRY OR VIDEOGRAMMETRY (measuring liquid level G01F; radio navigation, determining distance or velocity by use of propagation effects, e.g. Doppler effects, propagation time, of radio waves, analogous arrangements using other waves G01S) ## Abstract A computer-implemented method of trajectory prediction includes obtaining a first cross-attention between a vectorized representation of a road map near a vehicle and information obtained from a rasterized representation of an environment near the vehicle by processing through a first cross-attention stage; obtaining a second cross-attention between a vectorized representation of a vehicle history and information obtained from the rasterized representation by processing through a second cross-attention stage; operating a scene encoder on the first cross-attention and the second cross-attention; operating a trajectory decoder on an output of the scene encoder; obtaining one or more trajectory predictions by performing one or more queries on the trajectory decoder. ## Claims ### Claim 1 A computer-implemented method of trajectory prediction, comprising: determining a first cross-attention between a vectorized representation of a road map near a vehicle and information obtained from a rasterized representation of an environment near the vehicle by processing through a first cross-attention stage; determining a second cross-attention between a vectorized representation of a vehicle history and information obtained from the rasterized representation by processing through a second cross-attention stage; operating a scene encoder on the first cross-attention and the second cross-attention; operating a trajectory decoder on an output of the scene encoder; generating one or more trajectory predictions by performing one or more queries on the trajectory decoder. ### Claim 2 (depends on 1) The computer-implemented method of claim 1, wherein the information obtained from the rasterized representation comprises a multi-sourced, multi-grained feature map. ### Claim 3 (depends on 2) The computer-implemented method of claim 2, wherein the information is further based on a lidar point cloud obtained from a sensor located on the vehicle. ### Claim 4 (depends on 1) The computer-implemented method of claim 1, wherein the rasterized representation comprises traffic signal information near the vehicle. ### Claim 5 (depends on 1) The computer-implemented method of claim 1, wherein the information comprises a raw camera image obtained by a camera on the vehicle. ### Claim 6 (depends on 1) The computer-implemented method of claim 1, wherein the scene encoder comprises N encoding layers, where N is a positive integer. ### Claim 7 (depends on 1) The computer-implemented method of claim 1, wherein the trajectory decoder comprises M encoding layers, where N is a positive integer. ### Claim 8 (depends on 1) The computer-implemented method of claim 1, wherein the generating the one or more trajectory predictions includes generating a probability associated with each trajectory prediction. ### Claim 9 (depends on 8) The computer-implemented method of claim 8, wherein a Gaussian mixture model is used for generating the probability associated with each trajectory prediction. ### Claim 10 (depends on 1) The computer-implemented method of claim 1, wherein the scene encoder generates at least one token used for a query that is responsive to a pedestrian pose or a pedestrian gaze. ### Claim 11 An apparatus comprising one or more processors configured to implement a method, the processor configured to: determine a first cross-attention between a vectorized representation of a road map near a vehicle and information obtained from a rasterized representation of an environment near the vehicle by processing through a first cross-attention stage; determine a second cross-attention between a vectorized representation of a vehicle history and information obtained from the rasterized representation by processing through a second cross-attention stage; operate a scene encoder on the first cross-attention and the second cross-attention; operate a trajectory decoder on an output of the scene encoder; generate one or more trajectory predictions by performing one or more queries on the trajectory decoder. ### Claim 12 (depends on 11) The apparatus of claim 11, wherein the information obtained from the rasterized representation comprises a multi-sourced, multi-grained feature map. ### Claim 13 (depends on 12) The apparatus of claim 12, wherein the information is further based on a lidar point cloud obtained from a sensor located on the vehicle. ### Claim 14 (depends on 11) The apparatus of claim 11, wherein the rasterized representation comprises traffic signal information near the vehicle. ### Claim 15 (depends on 11) The apparatus of claim 11, wherein the information comprises a raw camera image obtained by a camera on the vehicle. ### Claim 16 A non-transitory computer-storage medium having process-executable code that, upon execution, causes one or more processor to implement a method, comprising: determining a first cross-attention between a vectorized representation of a road map near a vehicle and information obtained from a rasterized representation of an environment near the vehicle by processing through a first cross-attention stage; determining a second cross-attention between a vectorized representation of a vehicle history and information obtained from the rasterized representation by processing through a second cross-attention stage; operating a scene encoder on the first cross-attention and the second cross-attention; operating a trajectory decoder on an output of the scene encoder; generating one or more trajectory predictions by performing one or more queries on the trajectory decoder. ### Claim 17 (depends on 16) The non-transitory computer-storage medium of claim 16, wherein the scene encoder comprises N encoding layers, where N is a positive integer. ### Claim 18 (depends on 16) The non-transitory computer-storage medium of claim 16, wherein the trajectory decoder comprises M encoding layers, where N is a positive integer. ### Claim 19 (depends on 16) The non-transitory computer-storage medium of claim 16, wherein the generating the one or more trajectory predictions includes generating a probability associated with each trajectory prediction. ### Claim 20 (depends on 19) The non-transitory computer-storage medium of claim 19, wherein a Gaussian mixture model is used for generating the probability associated with each trajectory prediction.