Apparatus, system and method for fusing sensor data to do sensor translation

A sensor system comprising an image sensor and a light detection and ranging (LiDAR) sensor, comprising:

• a first input for receiving image data from the image sensor associated with an image scene of a landscape;
• a second input for receiving LiDAR data from the LiDAR sensor associated with a LiDAR scene of the landscape, wherein the LiDAR scene and image scene of the landscape overlap;
• a memory, coupled to the input for storing the first sensor data;
• and
• a processor, operatively coupled to the memory, wherein the processor and memory are configured to
• apply one or more machine-learning models to (i) the image data to identify image features of interest, and to (ii) the LiDAR data to identify LiDAR features of interest,
• fuse features of the LiDAR features of interest and the image features of interest, and
• generate LiDAR data based on the fused features, wherein the generated LiDAR data has a depth map and a LiDAR location mask representative of the landscape.

A method of operating a sensor system comprising an image sensor and a light detection and ranging (LiDAR) sensor, comprising:

• receiving image data from the image sensor associated with an image scene of a landscape;
• receiving LiDAR data from the LiDAR sensor associated with a LiDAR scene of the landscape, wherein the LiDAR scene and image scene of the landscape overlap;
• applying one or more machine-learning models to (i) the image data to identify image features of interest in the image data, and (ii) the LiDAR data to identify LiDAR features of interest in the LiDAR data;
• fusing the LiDAR features of interest and the image points of interest, utilizing an attention mechanism, and generating an output;
• and
• generating new LiDAR data based on the fusing output, wherein the generated new LiDAR data comprises a depth map and a LiDAR location mask representative of the landscape.