Sharing and exchanging information onboard displays or instruments and an offboard device

A portable computing device for interfacing with an electronic system having a graphical user interface, the portable computing device comprising:

• one or more sensors configured to capture data on the graphical user interface of the electronic system, wherein at least one of the one or more sensors is an optical sensor configured to capture image data;
• at least one processor configured to recognize at least one component of the graphical user interface state of the electronic system, wherein the at least one component of the user interface state further comprises structure and content of the graphical user interface;
• a database coupled to the at least one processor;
• wherein the processor is further configured to:
• match the at least one recognized component with one or more entries in the database;
• and
• perform an action associated with the matched component.

A non-transitory computer readable medium storing a program having instructions stored thereon, executable by a processor, to perform a method for interfacing between first and second electronic systems, the method comprising:

• capturing data on a graphical user interface state of the first electronic system using one or more sensors of the second electronic system, wherein at least one type of the data captured with the one or more sensors is image data captured using an optical sensor;
• recognizing at least one component of the graphical user interface state of the first electronic system, wherein the at least one component of the user interface state further comprises structure and content of the graphical user interface;
• matching the at least one recognized component with one or more entries in a database on the second electronic system;
• and
• performing an action on the second electronic system associated with the matched component.