Techniques to implement transformers with multi-task neural networks

An apparatus, comprising:

• one or more controllers associated with one or more memories, wherein the one or more controllers are configured to cause the apparatus to:
• receive an image;
• generate, using a feature extractor having one or more convolutional layers and taking the image as a first input, one or more representation vectors corresponding to the one or more convolutional layers;
• apply one or more self-attention based transformers to a second input that is based on the one or more representation vectors to obtain an indication of one or more objects in the image;
• apply the one or more self-attention based transformers to a third input that is based on the one or more representation vectors to an indication of one or more drivable areas in the image;
• and
• output the indication of the one or more objects in the image and the indication of the one or more drivable areas in the image.

A non-transitory computer-readable medium storing code, the code comprising instructions executable by one or more processors to:

• receive an image;
• generating, used a feature extractor having one or more convolutional layers and taking the image as a first input, one or more representation vectors corresponding to the one or more convolutional layers;
• apply one or more self-attention based transformers to a second input that is based on the one or more representation vectors to obtain an indication of one or more objects in the image;
• apply the one or more self-attention based transformers to a third input that is based on the one or more representation vectors to obtain an indication of one or more drivable areas in the image;
• and
• output the indication of the one or more objects in the image and the indication of the one or more drivable areas in the image.