Method and apparatus for monitoring usage of at least one application executed within an operating system

A method for training a model capable of determining, based on a screenshot of a menu screen of at least one application whose usage is to be monitored, a type of screen of the menu screen and identifying at least one data field based on the type of screen of the menu screen, the training method being executed by a learning engine and comprising:

• receiving, as an input, at least one identified screenshot of all the menu screens that can be displayed during the execution of said application, and for a given type of menu screen, at least a list of data fields included in said menu screen;
• determining at least one generic layout corresponding to a type of menu screen based on at least one layout of the at least one menu screen by analysing a layout of at least one screenshot, said layout being compared in order to find similarities with other layouts already analysed, such as a number of data fields included in a menu screen along with information about these data fields;
• and
• delivering, to a monitoring application, a data-structure including a given type of menu screen and a list of data fields identifying the type of menu screen.

An electronic device embedding a monitoring application capable of monitoring usage of at least one application executed within an operating system of the electronic device, the electronic device comprising at least one processor configured to, upon detection of an input on a input control interface of the electronic device:

• acquire a screenshot of a menu screen currently displayed by the monitored application;
• run a trained model on the screenshot, the trained model being capable of determining the type of menu screen of the currently displayed menu screen and retrieving data from at least one data field identified based on the type of menu screen of the currently displayed menu screen from the screenshot;
• and
• deliver a consolidated set of data obtained from the screenshot, to a remote electronic device for data processing, including a type of menu screen of the currently displayed menu screen, and data retrieved from at least one data field identified based on the type of menu screen of the currently displayed menu screen.