Methods and systems for reidentification in a user interface

A computer-implemented method (200) for performing reidentification check for a user interface (UI) element, in a current UI, the method comprising:

• retrieving (S210), for the UI element, at least one of a stored first local description and a stored first global description, based on a concept identifier representing the UI element, wherein each of the at least one of the stored first local description and the stored first global description is generated based on a plurality of reference descriptions, each of the plurality of reference descriptions being associated with a UI element corresponding to a type of UI element corresponding to the concept identifier, wherein the stored first local description comprises a description component that is specific to a respective UI, and wherein the stored first global description comprises a description component that is not specific to a UI;
• and
• querying (S220) a representation of the current UI using the at least one first stored local description and first stored global description to determine whether the respective description can be used to reidentify the UI element;
• enabling, via the current UI, an interactive prompt requesting user identification when the UI element cannot be reidentified using the respective description;
• and
• enabling a user input to identify the UI element via a highlighting action on the current UI in response to the interactive prompt.

A system (300) for performing description mining for a concept, the system comprising:

• at least one processor;
• and
• at least one memory storing instructions that, when executed by the at least one processor, cause the system to:
• acquire a plurality of reference descriptions, wherein each of the plurality of reference descriptions is associated with a user interface (UI) element corresponding to a type of UI element represented by the concept;
• generate, based on the plurality of reference descriptions, at least one of a local description for the concept and a global description for the concept, wherein the local description for the concept comprises a description component that is specific to a respective UI, and wherein the global description for the concept comprises a description component that is not specific to a UI;
• and
• store the at least one of generated local description and global description such that it is associated with a concept identifier corresponding to the concept,
• execute workflow comprising an action involving the UI element, wherein executing the workflow comprises:
• retrieving, for the UI element, at least one of the stored local description and the stored global description, based on the concept identifier;
• and
• querying a representation of a current UI using the at least one of stored local description and stored global description to determine whether the respective description can be used to reidentify the UI element;
• enable, via the current UI, an interactive prompt requesting user identification when the UI element cannot be reidentified using the respective description;
• and
• enable a user input to identify the UI element via a highlighting action on the current UI in response to the interactive prompt.

A system (400) for performing reidentification check for a user interface (UI) element, in a current UI, the system comprising:

• at least one processor;
• and
• at least one memory storing instructions that, when executed by the at least one processor, cause the system to:
• retrieve, for the UI element, at least one of a stored first local description and a stored first global description, based on a concept identifier representing the UI element, wherein each of the at least one of the stored first local description and the stored first global description is generated based on a plurality of reference descriptions, each of the plurality of reference descriptions being associated with a UI element corresponding to a type of UI element corresponding to the concept identifier, wherein the stored first local description comprises a description component that is specific to a respective UI, and wherein the stored first global description comprises a description component that is not specific to a UI;
• and
• query a representation of the current UI using the at least one first stored local description and first stored global description to determine whether the respective description can be used to reidentify the UI element,
• enable, via the current UI, an interactive prompt requesting user identification when the UI element cannot be reidentified using the respective description;
• and
• enable a user input to identify the UI element via a highlighting action on the current UI in response to the interactive prompt.