Object recognition for improving interfaces on an eyewear device and other wearable and mobile devices

An eyewear device, including:

• a frame;
• an image display to present a graphical user interface to a user;
• a camera connected to the frame to capture an image of a scene with an unknown object;
• a processor coupled to the eyewear device and connected to the camera;
• a memory accessible to the processor;
• and
• programming in the memory, wherein execution of the programming by the processor causes the processor to perform operations including:
• capturing, via the camera, the image of the scene with the unknown object;
• determining a recognized object-based adjustment of an application with which the user is interacting by:
• executing firmware programming adapted to extract object features of the unknown object from the captured image of the scene, the unknown object features including a gradient, an edge, a contour, a ridge, a color, a corner, a blob, or a combination thereof and to compare the extracted unknown object features against a recognized object database to match the unknown object to a recognized object in the recognized object database, wherein each recognized object has a recognized object model that includes multiple recognized object features, and
• executing application level programming that retrieves the recognized object-based adjustment of the application for the recognized object;
• and
• producing visible output to the user via the graphical user interface presented on the image display of the eyewear device based on the recognized object-based adjustment of the application.

A non-transitory computer-readable medium comprising instructions that when executed by one or more processors caused the one or more processors to perform operations comprising:

• capturing, via a camera, an image of a scene with an unknown object;
• determining a recognized object-based adjustment of an application with which a user is interacting by:
• extracting object features of the unknown object from the captured image of the scene, the unknown object features including a gradient, an edge, a contour, a ridge, a color, a corner, a blob, or a combination thereof;
• comparing the extracted unknown object features against a recognized object database to match the unknown object to a recognized object in the recognized object database, wherein each recognized object has a recognized object model that includes multiple recognized object features;
• and
• retrieving the recognized object-based adjustment of the application for the recognized object;
• and
• producing visible output to the user via a graphical user interface presented on an image display of a wearable device or a mobile device based on the recognized object-based adjustment of the application.