Method and computing apparatus for displaying an operation assistance screen of precision machining equipment

A non-transitory computer-readable medium storing instructions for generating a real-time operation assistance screen for precision machining equipment, the instructions when executed by one or more processors, cause a computing apparatus to:

• receive user input to designate a region of interest and select one or more execution target recipes, the region of interest being at least a partial region of a display on which an operation user interface (UI) of the precision machining equipment is displayed, and the one or more execution target recipes being from among a plurality of previously generated recipes;
• capture, in real time, an image of the region of interest on the display displaying the operation UI of the precision machining equipment;
• generate a real time assistance screen of the precision machining equipment based on the captured image of the region of interest and the execution target recipe;
• and
• display the generated real-time assistance screen on the display,
• wherein each of the execution target recipes comprises one or more activities for generating real-time assistance output based on an input image and a predefined workflow,
• wherein the real-time assistance screen comprises the real-time assistance output, and
• wherein the capturing the image, the generating the real time assistance screen, and the displaying the generated real-time assistance screen comprise no direct information exchange with a program associated with the operation UI of the precision machining equipment.

A computing apparatus for controlling precision machining equipment, comprising:

• memory which loads an operation assistance screen display program of the precision machining equipment;
• and
• at least one processor which executes the operation assistance screen display program of the precision machining equipment loaded into the memory,
• wherein the operation assistance screen display program comprises:
• receiving code configured to cause the at least one processor to receive user input to designate a region of interest and select one or more execution target recipes, the region of interest being at least a partial region of a display on which an operation user interface (UI) of the precision machining equipment is displayed, and the one or more execution target recipes being from among a plurality of previously generated recipes;
• capturing code configured to cause the at least one processor to capture, in real time, an image of the region of interest on the display displaying the operation UI of the precision machining equipment;
• generating code configured to cause the at least one processor to generate a real time assistance screen of the precision machining equipment based on the captured image of the region of interest and the execution target recipe;
• and
• displaying code configured to cause the at least one processor to display the generated real-time assistance screen on the display,
• wherein each of the execution target recipes comprises one or more activities for generating real-time assistance output based on an input image and a predefined workflow,
• wherein the real-time assistance screen comprises the real-time assistance output, and
• wherein the capturing the image, the generating the real time assistance screen, and the displaying the generated real-time assistance screen comprise no direct information exchange with a program associated with the operation UI of the precision machining equipment.