Speech-enabled augmented reality

A computer-implemented method, comprising:

• receiving, by a camera of a device, an image;
• showing the image on a display of the device;
• recognizing one or more virtual objects in the image;
• determining, based on a relevance score, a respective probability that a user will interact with the one or more virtual objects;
• determining a relevant object based on the respective probability exceeding a predetermined threshold from the one or more virtual objects in the image;
• and
• overlaying, in the image, text indicating a corresponding key phrase associated with the relevant object on the display.

A computer system, comprising:

• at least one processor;
• a display;
• at least one camera;
• and
• memory including instructions that, when executed by the at least one processor, cause the computer system to:
• receive, by the at least one camera, an image;
• recognize one or more virtual objects in the image;
• determining, based on a relevance score, a respective probability that a user will interact with the one or more virtual objects;
• determine at least one relevant object based on the respective probability exceeding a predetermined threshold from the one or more virtual objects in the image;
• overlay, in the image, text indicating a corresponding key phrase associated with the at least one relevant object on the display;
• receive speech audio from the user;
• infer a key phrase associated with a relevant object based on the speech audio;
• and
• enable an interaction session with the user, wherein the user can obtain information related to the relevant object.