# US 2018/0024730 A1 · Custom widgets based on graphical user interfaces of applications > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 2018/0024730 A1 - Document ID: US20180024730A1 - Status: Published application - Kind: A1 - Application number: 15/213,518 - Filed: 2016-07-19 - Published: 2018-01-25 - Earliest claimed: 2016-07-19 - Assignees: International Business Machines Corporation - Applicants: International Business Machines Corporation - Inventors: Emmanuel JAMBOU - Web page: https://patentgrep.com/patent/US20180024730A1 ## CPC - G06F 3/04847: Interaction techniques to control parameter settings, e.g. interaction with sliders or dials - G06F 9/451: Execution arrangements for user interfaces - G06F 3/0482: Interaction with lists of selectable items, e.g. menus - G06F 3/04855: Interaction with scrollbars - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) ## Abstract Systems, methods, and computer program products to perform an operation comprising receiving input specifying a first portion of a graphical user interface (GUI) of a first application executing on a first computing device, wherein the GUI is outputted on a display device, rendering the GUI of the first application on a first virtual display of the first computing device, and not on the first display device, and generating a first widget which displays the first portion of the GUI of the first application on the display device, wherein a unique identifier of the first widget is associated with a unique identifier of the first virtual display. ## Claims ### Claim 1 A method, comprising: receiving input specifying a first portion of a graphical user interface (GUI) of a first application executing on a first computing device, wherein the GUI is outputted on a first display device and wherein the specified first portion is less than the entire GUI; rendering the GUI of the first application on a first virtual display of the first computing device; removing, based on the rendering, the GUI of the first application from the first display device; and generating a first widget which displays the first portion of the GUI of the first application on the first display device, wherein the first widget includes one or more interactive elements that expose an underlying functionality of the first portion of the GUI. ### Claim 2 (depends on 1) The method of claim 1, further comprising: receiving input specifying a second portion of the GUI, different than the first portion of the GUI; and providing a scrollbar on the first widget which allows a user to scroll between the first portion of the GUI and the second portion of the GUI, wherein the first widget is of a size that is smaller than a collective size of the first portion of the GUI and the second portion of the GUI. ### Claim 3 (depends on 1) The method of claim 1, wherein a unique identifier of the first widget is associated with a unique identifier of the first virtual display, the method further comprising: receiving user input by the first widget displayed on the first display device; transmitting an indication of: (i) a location on the widget where the user input was received, (ii) a type of the user input, and (iii) the unique identifier of the first widget, to the first virtual display, wherein the first virtual display is identified based on the association between the unique identifier of the first virtual display and the unique identifier of the first widget; mapping, by the first virtual display, the received location on the widget to a corresponding location on the GUI of the first application; and providing the user input to the GUI of the first application on the first virtual display. ### Claim 4 (depends on 3) The method of claim 3, further comprising: generating, by the first application, an output responsive to the received user input; transmitting, by the first virtual display to the first widget, the output; and refreshing the first widget to display the received output on the first display device. ### Claim 5 (depends on 4) The method of claim 4, wherein a portion of the received output is not visible due to a size of the first widget, the method further comprising: increasing the size of the first widget to display the received output including the portion of the output that was not visible. ### Claim 6 (depends on 3) The method of claim 3, further comprising: determining that the received user input is associated with a first rule, of a plurality of rules for the first widget; determining that the first rule is associated with a first response, wherein the first response specifies to: minimize the first widget such that the first widget is not displayed on the first display device; and display the GUI of the first application on the first display device. ### Claim 7 (depends on 1) The method of claim 1, further comprising: receiving input specifying a first portion of a GUI of a second application, wherein the GUI of the second application is outputted on the first display device; rendering the GUI of the second application on a second virtual display, and not on the first display device; generating a second widget which displays the first portion of the GUI of the second application on the first display device, wherein a unique identifier of the second widget is associated with a unique identifier of the second virtual display; and outputting the first and second widgets on a second display device, wherein the second display device is of a second computing device, communicably coupled to the first computing device. ### Claim 8 A system, comprising: one or more computer processors; a memory containing a program which when executed by the processors performs an operation comprising: receiving input specifying a first portion of a graphical user interface (GUI) of a first application executing on a first computing device, wherein the GUI is outputted on a first display device of the system and wherein the specified first portion is less than the entire GUI; rendering the GUI of the first application on a first virtual display of the first computing device; removing, based on the rendering, the GUI of the first application from the first display device; and generating a first widget which displays the first portion of the GUI of the first application on the first display device, wherein the first widget includes one or more interactive elements that expose an underlying functionality of the first portion of the GUI. ### Claim 9 (depends on 8) The system of claim 8, the operation further comprising: receiving input specifying a second portion of the GUI, different than the first portion of the GUI; and providing a scrollbar on the first widget which allows a user to scroll between the first portion of the GUI and the second portion of the GUI, wherein the first widget is of a size that is smaller than a collective size of the first portion of the GUI and the second portion of the GUI. ### Claim 10 (depends on 8) The system of claim 8, wherein a unique identifier of the first widget is associated with a unique identifier of the first virtual display, the operation further comprising: receiving user input by the first widget displayed on the first display device; transmitting an indication of: (i) a location on the widget where the user input was received, (ii) a type of the user input, and (iii) the unique identifier of the first widget, to the first virtual display, wherein the first virtual display is identified based on the association between the unique identifier of the first virtual display and the unique identifier of the first widget; mapping, by the first virtual display, the received location on the widget to a corresponding location on the GUI of the first application; and providing the user input to the GUI of the first application on the first virtual display. ### Claim 11 (depends on 10) The system of claim 10, the operation further comprising: generating, by the first application, an output responsive to the received user input; transmitting, by the first virtual display to the first widget, the output; and refreshing the first widget to display the received output on the first display device. ### Claim 12 (depends on 11) The system of claim 11, wherein a portion of the received output is not visible due to a size of the first widget, the operation further comprising: increasing the size of the first widget to display the received output including the portion of the output that was not visible. ### Claim 13 (depends on 10) The system of claim 10, the operation further comprising: determining that the received user input is associated with a first rule, of a plurality of rules for the first widget; determining that the first rule is associated with a first response, wherein the first response specifies to: minimize the first widget such that the first widget is not displayed on the first display device; and display the GUI of the first application on the first display device. ### Claim 14 (depends on 8) The system of claim 8, the operation further comprising: receiving input specifying a first portion of a GUI of a second application, wherein the GUI of the second application is outputted on the first display device; rendering the GUI of the second application on a second virtual display, and not on the first display device; generating a second widget which displays the first portion of the GUI of the second application on the first display device, wherein a unique identifier of the second widget is associated with a unique identifier of the second virtual display; and outputting the first and second widgets on a second display device, wherein the display device is of a second system, communicably coupled to the first system. ### Claim 15 A computer program product, comprising: a computer-readable storage medium having computer-readable program code embodied therewith, the computer-readable program code executable by one or more computer processors to perform an operation comprising: receiving input specifying a first portion of a graphical user interface (GUI) of a first application executing on a first computing device, wherein the GUI is outputted on a first display device and wherein the specified first portion is less than the entire GUI; rendering the GUI of the first application on a first virtual display of the first computing device; removing, based on the rendering, the GUI of the first application from the first display device; and generating a first widget which displays the first portion of the GUI of the first application on the first display device, wherein the first widget includes one or more interactive elements that expose an underlying functionality of the first portion of the GUI. ### Claim 16 (depends on 15) The computer program product of claim 15, the operation further comprising: receiving input specifying a second portion of the GUI, different than the first portion of the GUI; and providing a scrollbar on the first widget which allows a user to scroll between the first portion of the GUI and the second portion of the GUI, wherein the first widget is of a size that is smaller than a collective size of the first portion of the GUI and the second portion of the GUI. ### Claim 17 (depends on 15) The computer program product of claim 15, wherein a unique identifier of the first widget is associated with a unique identifier of the first virtual display, the operation further comprising: receiving user input by the first widget displayed on the first display device; transmitting an indication of: (i) a location on the widget where the user input was received, (ii) a type of the user input, and (iii) the unique identifier of the first widget, to the first virtual display, wherein the first virtual display is identified based on the association between the unique identifier of the first virtual display and the unique identifier of the first widget; mapping, by the first virtual display, the received location on the widget to a corresponding location on the GUI of the first application; and providing the user input to the GUI of the first application on the first virtual display. ### Claim 18 (depends on 17) The computer program product of claim 17, the operation further comprising: generating, by the first application, an output responsive to the received user input; transmitting, by the first virtual display to the first widget, the output; and refreshing the first widget to display the received output on the first display device. ### Claim 19 (depends on 18) The computer program product of claim 18, wherein a portion of the received output is not visible due to a size of the first widget, the operation further comprising: increasing the size of the first widget to display the received output including the portion of the output that was not visible. ### Claim 20 (depends on 15) The computer program product of claim 15, the operation further comprising: receiving input specifying a first portion of a GUI of a second application, wherein the GUI of the second application is outputted on the first display device; rendering the GUI of the second application on a second virtual display, and not on the first display device; generating a second widget which displays the first portion of the GUI of the second application on the first display device, wherein a unique identifier of the second widget is associated with a unique identifier of the second virtual display; and outputting the first and second widgets on a second display device, wherein the second display device is of a second computing device, communicably coupled to the first computing device.