Bar code detection and authentication for contactless transactions using wearable multimedia device

A method comprising:

• capturing, using a wide field-of-view camera of a wearable multimedia device worn by a user, an image of a scene that includes a plurality of unlabeled objects and a user gesture, wherein the wearable multimedia device comprises a housing, the wide field-of-view camera embedded in the housing, a depth sensor, one or more processors, and a memory storing instructions that are executed by the one or more processors;
• capturing, by the one or more processors using the depth sensor, depth data of the scene;
• performing, by the one or more processors, semantic segmentation on the image of the scene to predict an object mask for each unlabeled object;
• performing, by the one or more processors, instance segmentation on pixel data within the each object mask;
• labeling, by the one or more processors based on the instance segmentation, that one of the plurality of unlabeled objects is a contactless terminal device;
• associating, by the one or more processors with sensor fusion, an intent of the user to engage with the contactless terminal device based at least in part on the user gesture, the depth data and the labeled contactless terminal device, where the user gesture is the user pointing in a direction of the contactless terminal device;
• establishing, by the one or more processors, a communication channel between the wearable multimedia device and the contactless terminal device;
• receiving, by the one or more processors using the communication channel, data from the contactless terminal device;
• responsive to the received data, sending, using the communication channel, authentication credentials to the contactless terminal device;
• receiving, by the one or more processors using the communication channel, access from the contactless terminal device based at least in part on the authentication credentials;
• and
• responsive to the received access to the contactless terminal device, performing, by the one or more processors, an interaction with the contactless terminal device.