Method and system for real time cloud image recognition using camera feed

A computer-implemented method for performing distributed image recognition on a hybrid recognition pipeline, the computer-implemented method comprising:

• receiving, at a communication device, at least one image of a physical environment captured by one or more sensors integrated within the communication device;
• extracting, at the communication device, one or more global features and one or more local features from the received image;
• transmitting, at the communication device, the one or more global features to a cloud server configured with a reference vector database storing a plurality of pre-computed feature descriptors;
• executing, at the cloud server, a global similarity search operation comprising comparing the one or more global features with a plurality of pre-computed feature descriptors stored in the reference vector database, calculate one or more global similarity scores representing a degree of similarity between the one or more global features and the plurality of pre-computed feature descriptors, and identifying, based on the calculated one or more global similarity scores, one or more candidate reference images ranked according to a corresponding degree of similarity with the one or more global features;
• transmitting, to the communication device by the cloud server, one or more sets of pre-computed descriptors corresponding to the one or more candidate reference images and the one or more global similarity scores corresponding to the one or more sets of pre-computed descriptors of the one or more candidate reference images;
• performing local descriptor matching, at the communication device, between the one or more sets of pre-computed descriptors associated with the one or more candidate reference images and the one or more local features to generate one or more refined similarity scores;
• fusing, at the communication device, the one or more refined similarity scores with the one or more global similarity scores to determine an initial recognition result;
• executing continuous refinement of the initial recognition result based on newly extracted local features transmitted from the communication device to the cloud server;
• generate, using the one or more processors of the communication device, one or more final recognition outputs corresponding to at least one identified physical object based on the re-ranked results received from the cloud server;
• and
• display, using a display module operatively coupled to the communication device, the one or more final recognition outputs in real time.

A non-transitory computer-readable storage medium storing computer-executable instructions that, when executed by one or more processors of a computing device, cause the computing device to perform a method for performing distributed image recognition on a hybrid recognition pipeline, the method comprising:

• receiving, at a communication device, at least one image of a physical environment captured by one or more sensors integrated within the communication device;
• extracting, at the communication device, one or more global features and one or more local features from the received image;
• transmitting, at the communication device, the one or more global features to a cloud server configured with a reference vector database storing a plurality of pre-computed feature descriptors;
• executing, at the cloud server, a global similarity search operation comprising comparing the one or more global features with a plurality of pre-computed feature descriptors stored in the reference vector database, calculate one or more global similarity scores representing a degree of similarity between the one or more global features and the plurality of pre-computed feature descriptors, and identifying, based on the calculated one or more global similarity scores, one or more candidate reference images ranked according to a corresponding degree of similarity with the one or more global features;
• transmitting, to the communication device by the cloud server, one or more sets of pre-computed descriptors corresponding to the one or more candidate reference images and the one or more global similarity scores corresponding to the one or more sets of pre-computed descriptors of the one or more candidate reference images;
• performing local descriptor matching, at the communication device, between the one or more sets of pre-computed descriptors associated with the one or more candidate reference images and the one or more local features to generate one or more refined similarity scores;
• fusing, at the communication device, the one or more refined similarity scores with the one or more global similarity scores to determine an initial recognition result;
• executing continuous refinement of the initial recognition result based on newly extracted local features transmitted from the communication device to the cloud server;
• generate, using the one or more processors of the communication device, one or more final recognition outputs corresponding to at least one identified physical object based on the re-ranked results received from the cloud server;
• and
• display, using a display module operatively coupled to the communication device, the one or more final recognition outputs in real time.