# US 12,216,959 B2 · Systems and methods for initiating and interacting with a companion-display mode for an electronic device with a touch-sensitive display > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 12,216,959 B2 - Document ID: US12216959B2 - Status: Granted - Kind: B2 - Application number: 18/235,312 - Filed: 2023-08-17 - Published: 2023-12-07 - Granted: 2025-02-04 - Earliest claimed: 2019-04-16 - Assignees: Apple Inc. - Applicants: Apple Inc. - Inventors: Raymond S. Sepulveda; Chun Kin Minor Wong; John O. Louch; Christopher N. Ryan; Kevin J. Van Vechten; Patrick L Coffman - Web page: https://patentgrep.com/patent/US12216959B2 ## CPC - G06F 3/0488: using a touch-screen or digitiser, e.g. input of commands through traced gestures - G06F 3/147: using display panels - G06F 3/0416: {Control or interface arrangements specially adapted for digitisers} - G06F 3/04817: using icons (graphical or visual programming using iconic symbols G06F8/34) - G06F 3/0482: Interaction with lists of selectable items, e.g. menus - G06F 3/04845: for image manipulation, e.g. dragging, rotation, expansion or change of colour - G06F 3/04847: Interaction techniques to control parameter settings, e.g. interaction with sliders or dials - G06F 3/0486: Drag-and-drop - G06F 3/04883: for inputting data by handwriting, e.g. gesture or text - G06F 3/1438: {using more than one graphics controller} - G06F 3/1454: {involving copying of the display data of a local workstation or window to a remote workstation or window so that an actual copy of the data is displayed simultaneously on two or more displays, e.g. teledisplay} - G09G 5/08: Cursor circuits - G06F 21/30: Authentication, i.e. establishing the identity or authorisation of security principals - G06F 2203/04803: Split screen, i.e. subdividing the display area or the window area into separate subareas - G06F 2203/04808: Several contacts: gestures triggering a specific function, e.g. scrolling, zooming, right-click, when the user establishes several contacts with the surface simultaneously; e.g. using several fingers or a combination of fingers and pen - G09G 2370/16: Use of wireless transmission of display information - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) - G09G: ARRANGEMENTS OR CIRCUITS FOR CONTROL OF INDICATING DEVICES USING STATIC MEANS TO PRESENT VARIABLE INFORMATION (arrangements for transferring data between digital computers and displays G06F3/14; static indicating arrangements comprising an association of a number of separate sources or light control cells G09F9/00; static indicating arrangements comprising integral associations of a number of light sources H01J, H01K, H10H, H05B33/12; scanning, transmission or reproduction of documents or the like, e.g. facsimile transmission, details thereof H04N1/00) ## Abstract An example method includes, at a first electronic device that includes a display, displaying, on the display, a user interface that is associated with an application, the user interface displayed with a control user interface element for changing a display property of the user interface. The method includes, detecting an input directed to the control user interface element. The method also includes that in response to detecting the input, and while continuing to display the user interface, concurrently displaying on the display: a first selectable option for changing the display property of the user interface on the display of the first electronic device, and a second selectable option for requesting display at a second electronic device, distinct from the first electronic device, of a user interface that includes content from the user interface. ## Claims ### Claim 1 A method, comprising: at a first electronic device that includes a touch-sensitive display: operating the first electronic device in a companion-display mode in which user interfaces generated by a second electronic device are displayed at the first electronic device, wherein the second electronic device is separate from the first electronic device; and while operating in the companion-display mode: displaying, on the touch-sensitive display of the first electronic device, a user interface generated by the second electronic device; detecting, at the first electronic device, a gesture using an input object; and in response to detecting the gesture: in accordance with determining that the input object is one or more user fingers, performing a first operation on the touch-sensitive display, wherein the first operation is an operation that is performed by the first electronic device; and in accordance with determining that the input object is a stylus, performing a second operation, distinct from the first operation, on the touch-sensitive display, wherein the second operation is an operation that is performed via the second electronic device. ### Claim 2 (depends on 1) The method of claim 1, wherein: the gesture includes a contact near a top edge of the touch-sensitive display and movement of the contact in a downward direction away from the top edge of the touch-sensitive display, performing the first operation includes displaying, on the touch-sensitive display, a user interface generated by the first electronic device, and performing the second operation includes displaying, on the touch-sensitive display, a status bar generated by the second electronic device. ### Claim 3 (depends on 2) The method of claim 2, wherein the user interface generated by the first electronic device is a settings user interface when the contact near the top edge is also near a corner of the touch-sensitive display. ### Claim 4 (depends on 3) The method of claim 3, wherein the user interface generated by the first electronic device is a user interface that includes electronic notifications when the contact near the top edge is at a predefined distance away from the corner of the touch-sensitive display. ### Claim 5 (depends on 1) The method of claim 1, wherein: the gesture includes a contact near a right edge of the touch-sensitive display of the first electronic device and movement of the contact in a leftward direction away from the right edge of the touch-sensitive display, performing the first operation includes overlaying, on the touch-sensitive display of the first electronic device, a user interface for an application executing on the first electronic device on a portion of the user interface generated by the second electronic device, and performing the second operation includes overlaying, on the touch-sensitive display of the first electronic device, a user interface that includes electronic notifications on a portion of the user interface generated by the second electronic device. ### Claim 6 (depends on 1) The method of claim 1, wherein: the gesture includes a contact near a bottom edge of the touch-sensitive display and movement of the contact in an upward direction away from the bottom edge of the touch-sensitive display, performing the first operation includes overlaying, on the touch-sensitive display of the first electronic device, a first dock on the user interface generated by the second electronic device, the first dock including user interface objects each for opening an application installed on the first electronic device; and performing the second operation includes overlaying, on the touch-sensitive display of the first electronic device, a second dock on the user interface generated by the second electronic device, the second dock including a first user interface object for displaying a user interface of a first application on the second electronic device and a second user interface object for displaying a user interface of a second application on the second electronic device. ### Claim 7 (depends on 1) The method of claim 1, wherein: the gesture includes movement of the input object over the user interface generated by the second electronic device, performing the first operation includes scrolling content within the user interface generated by the second electronic device, and performing the second operation includes moving a cursor within the user interface generated by the second electronic device. ### Claim 8 (depends on 7) The method of claim 7, wherein the gesture is made by the stylus as it is hovering above the touch-sensitive display, wherein as the stylus is hovering above the touch-sensitive display it remains within a threshold distance of the touch-sensitive display, but does not touch the touch-sensitive display. ### Claim 9 (depends on 1) The method of claim 1, wherein: the gesture includes a two-finger tap from a user's fingers on the touch-sensitive display, performing the first operation includes displaying, on the touch-sensitive display of the first electronic device, a menu associated with a right-click operation within the user interface generated by the second electronic device. ### Claim 10 (depends on 1) The method of claim 1, wherein: the gesture includes a single tap on the touch-sensitive display, performing the second operation includes performing an operation associated with a single click within the user interface generated by the second electronic device. ### Claim 11 (depends on 1) The method of claim 1, wherein: the gesture includes two contacts over content on the touch-sensitive display, followed by rotational movement of the two contacts relative to one another; and performing the first operation includes rotating content in the user interface generated by the second electronic device. ### Claim 12 (depends on 1) The method of claim 1, wherein inputs provided by the stylus are processed by sending data to the second electronic device, and the method includes: detecting, at the stylus, input corresponding to a double-tap on the stylus and, in response, ceasing to send data regarding inputs provided by the stylus to the second electronic device. ### Claim 13 (depends on 1) The method of claim 1, wherein: the gesture is a pinch or de-pinch gesture; and performing the first operation includes resizing content, on the touch-sensitive display of the first electronic device, within the user interface generated by the second electronic device in accordance with the pinch or de-pinch gesture. ### Claim 14 A non-transitory computer-readable storage medium storing executable instructions that, when executed by one or more processors of a first electronic device with a touch-sensitive display, cause the first electronic device to: operate the first electronic device in a companion-display mode in which user interfaces generated by a second electronic device are displayed at the first electronic device, wherein the second electronic device is separate from the first electronic device; and while operating in the companion-display mode: display, on the touch-sensitive display of the first electronic device, a user interface generated by the second electronic device; detect, at the first electronic device, a gesture using an input object; and in response to detecting the gesture: in accordance with determining that the input object is one or more user fingers, perform a first operation on the touch-sensitive display, wherein the first operation is an operation that is performed by the first electronic device; and in accordance with determining that the input object is a stylus, perform a second operation, distinct from the first operation, on the touch-sensitive display, wherein the second operation is an operation that is performed via the second electronic device. ### Claim 15 (depends on 14) The non-transitory computer-readable storage medium of claim 14, wherein: the gesture includes a contact near a top edge of the touch-sensitive display and movement of the contact in a downward direction away from the top edge of the touch-sensitive display, performing the first operation includes displaying, on the touch-sensitive display, a user interface generated by the first electronic device, and performing the second operation includes displaying, on the touch-sensitive display, a status bar generated by the second electronic device. ### Claim 16 (depends on 15) The non-transitory computer-readable storage medium of claim 15, wherein the user interface generated by the first electronic device is a settings user interface when the contact near the top edge is also near a corner of the touch-sensitive display. ### Claim 17 (depends on 16) The non-transitory computer-readable storage medium of claim 16, wherein the user interface generated by the first electronic device is a user interface that includes electronic notifications when the contact near the top edge is at a predefined distance away from the corner of the touch-sensitive display. ### Claim 18 (depends on 14) The non-transitory computer-readable storage medium of claim 14, wherein: the gesture includes a contact near a right edge of the touch-sensitive display of the first electronic device and movement of the contact in a leftward direction away from the right edge of the touch-sensitive display, performing the first operation includes overlaying, on the touch-sensitive display of the first electronic device, a user interface for an application executing on the first electronic device on a portion of the user interface generated by the second electronic device, and performing the second operation includes overlaying, on the touch-sensitive display of the first electronic device, a user interface that includes electronic notifications on a portion of the user interface generated by the second electronic device. ### Claim 19 (depends on 14) The non-transitory computer-readable storage medium of claim 14, wherein: the gesture includes a contact near a bottom edge of the touch-sensitive display and movement of the contact in an upward direction away from the bottom edge of the touch-sensitive display, performing the first operation includes overlaying, on the touch-sensitive display of the first electronic device, a first dock on the user interface generated by the second electronic device, the first dock including user interface objects each for opening an application installed on the first electronic device; and performing the second operation includes overlaying, on the touch-sensitive display of the first electronic device, a second dock on the user interface generated by the second electronic device, the second dock including a first user interface object for displaying a user interface of a first application on the second electronic device and a second user interface object for displaying a user interface of a second application on the second electronic device. ### Claim 20 (depends on 14) The non-transitory computer-readable storage medium of claim 14, wherein: the gesture includes movement of the input object over the user interface generated by the second electronic device, performing the first operation includes scrolling content within the user interface generated by the second electronic device, and performing the second operation includes moving a cursor within the user interface generated by the second electronic device. ### Claim 21 (depends on 20) The non-transitory computer-readable storage medium of claim 20, wherein the gesture is made by the stylus as it is hovering above the touch-sensitive display, wherein as the stylus is hovering above the touch-sensitive display it remains within a threshold distance of the touch-sensitive display, but does not touch the touch-sensitive display. ### Claim 22 (depends on 14) The non-transitory computer-readable storage medium of claim 14, wherein: the gesture includes a two-finger tap from a user's fingers on the touch-sensitive display, performing the first operation includes displaying, on the touch-sensitive display of the first electronic device, a menu associated with a right-click operation within the user interface generated by the second electronic device. ### Claim 23 (depends on 14) The non-transitory computer-readable storage medium of claim 14, wherein: the gesture includes a single tap on the touch-sensitive display, performing the second operation includes performing an operation associated with a single click within the user interface generated by the second electronic device. ### Claim 24 (depends on 14) The non-transitory computer-readable storage medium of claim 14, wherein: the gesture includes two contacts over content on the touch-sensitive display, followed by rotational movement of the two contacts relative to one another; and performing the first operation includes rotating content in the user interface generated by the second electronic device. ### Claim 25 (depends on 14) The non-transitory computer-readable storage medium of claim 14, wherein inputs provided by the stylus are processed by sending data to the second electronic device, and the instructions, when executed by the one or more processors of the first electronic device, cause the first electronic device to: detect, at the stylus, input corresponding to a double-tap on the stylus and, in response, cease to send data regarding inputs provided by the stylus to the second electronic device. ### Claim 26 (depends on 14) The non-transitory computer-readable storage medium of claim 14, wherein: the gesture is a pinch or de-pinch gesture; and performing the first operation includes resizing content, on the touch-sensitive display of the first electronic device, within the user interface generated by the second electronic device in accordance with the pinch or de-pinch gesture. ### Claim 27 A first electronic device, comprising: one or more processors; a touch-sensitive display; and memory storing one or more programs that are configured for execution by the one or more processors, the one or more programs including instructions for: operating the first electronic device in a companion-display mode in which user interfaces generated by a second electronic device are displayed at the first electronic device, wherein the second electronic device is separate from the first electronic device; and while operating in the companion-display mode: displaying, on the touch-sensitive display of the first electronic device, a user interface generated by the second electronic device; detecting, at the first electronic device, a gesture using an input object; and in response to detecting the gesture: in accordance with determining that the input object is one or more user fingers, performing a first operation on the touch-sensitive display, wherein the first operation is an operation that is performed by the first electronic device; and in accordance with determining that the input object is a stylus, performing a second operation, distinct from the first operation, on the touch-sensitive display, wherein the second operation is an operation that is performed via the second electronic device. ### Claim 28 (depends on 27) The first electronic device of claim 27, wherein: the gesture includes a contact near a top edge of the touch-sensitive display and movement of the contact in a downward direction away from the top edge of the touch-sensitive display, the instructions for performing the first operation include instructions for displaying, on the touch-sensitive display, a user interface generated by the first electronic device, and the instructions for performing the second operation include instructions for displaying, on the touch-sensitive display, a status bar generated by the second electronic device. ### Claim 29 (depends on 28) The first electronic device of claim 28, wherein the user interface generated by the first electronic device is a settings user interface when the contact near the top edge is also near a corner of the touch-sensitive display. ### Claim 30 (depends on 29) The first electronic device of claim 29, wherein the user interface generated by the first electronic device is a user interface that includes electronic notifications when the contact near the top edge is at a predefined distance away from the corner of the touch-sensitive display. ### Claim 31 (depends on 27) The first electronic device of claim 27, wherein: the gesture includes a contact near a right edge of the touch-sensitive display of the first electronic device and movement of the contact in a leftward direction away from the right edge of the touch-sensitive display, the instructions for performing the first operation include instructions for overlaying, on the touch-sensitive display of the first electronic device, a user interface for an application executing on the first electronic device on a portion of the user interface generated by the second electronic device, and the instructions for performing the second operation include instructions for overlaying, on the touch-sensitive display of the first electronic device, a user interface that includes electronic notifications on a portion of the user interface generated by the second electronic device. ### Claim 32 (depends on 27) The first electronic device of claim 27, wherein: the gesture includes a contact near a bottom edge of the touch-sensitive display and movement of the contact in an upward direction away from the bottom edge of the touch-sensitive display, the instructions for performing the first operation include instructions for overlaying, on the touch-sensitive display of the first electronic device, a first dock on the user interface generated by the second electronic device, the first dock including user interface objects each for opening an application installed on the first electronic device; and the instructions for performing the second operation include instructions for overlaying, on the touch-sensitive display of the first electronic device, a second dock on the user interface generated by the second electronic device, the second dock including a first user interface object for displaying a user interface of a first application on the second electronic device and a second user interface object for displaying a user interface of a second application on the second electronic device. ### Claim 33 (depends on 27) The first electronic device of claim 27, wherein: the gesture includes movement of the input object over the user interface generated by the second electronic device, the instructions for performing the first operation include instructions for scrolling content within the user interface generated by the second electronic device, and the instructions for performing the second operation include instructions for moving a cursor within the user interface generated by the second electronic device. ### Claim 34 (depends on 33) The first electronic device of claim 33, wherein the gesture is made by the stylus as it is hovering above the touch-sensitive display, wherein as the stylus is hovering above the touch-sensitive display it remains within a threshold distance of the touch-sensitive display, but does not touch the touch-sensitive display. ### Claim 35 (depends on 27) The first electronic device of claim 27, wherein: the gesture includes a two-finger tap from a user's fingers on the touch-sensitive display, the instructions for performing the first operation include instructions for displaying, on the touch-sensitive display of the first electronic device, a menu associated with a right-click operation within the user interface generated by the second electronic device. ### Claim 36 (depends on 27) The first electronic device of claim 27, wherein: the gesture includes a single tap on the touch-sensitive display, the instructions for performing the second operation include instructions for performing an operation associated with a single click within the user interface generated by the second electronic device. ### Claim 37 (depends on 27) The first electronic device of claim 27, wherein: the gesture includes two contacts over content on the touch-sensitive display, followed by rotational movement of the two contacts relative to one another; and performing the first operation includes rotating content in the user interface generated by the second electronic device. ### Claim 38 (depends on 27) The first electronic device of claim 27, wherein inputs provided by the stylus are processed by sending data to the second electronic device, and the one or more programs further include instructions for: detecting, at the stylus, input corresponding to a double-tap on the stylus and, in response, ceasing to send data regarding inputs provided by the stylus to the second electronic device. ### Claim 39 (depends on 27) The first electronic device of claim 27, wherein: the gesture is a pinch or de-pinch gesture; and the instructions for performing the first operation include instructions for resizing content, on the touch-sensitive display of the first electronic device, within the user interface generated by the second electronic device in accordance with the pinch or de-pinch gesture. ### Claim 40 A method, comprising: at a first electronic device that includes a touch-sensitive display: operating the first electronic device in a companion-display mode in which user interfaces generated by a second electronic device are displayed at the first electronic device, wherein the second electronic device is separate from the first electronic device; and while operating in the companion-display mode: displaying, on the touch-sensitive display of the first electronic device, a user interface generated by the second electronic device; detecting, at the first electronic device, a gesture using an input object, the gesture including a contact near a bottom edge of the touch-sensitive display and movement of the contact in an upward direction away from the bottom edge of the touch-sensitive display; and in response to detecting the gesture: in accordance with determining that the input object is one or more user fingers, performing a first operation on the touch-sensitive display of the first electronic device, the first operation including overlaying, on the touch-sensitive display, a first dock on the user interface generated by the second electronic device, the first dock including user interface objects each for opening an application installed on the first electronic device; and in accordance with determining that the input object is a stylus, performing a second operation on the touch-sensitive display of the first electronic device, wherein the second operation is distinct from the first operation and the second operation includes overlaying, on the touch-sensitive display, a second dock on the user interface generated by the second electronic device, the second dock including a first user interface object for displaying a user interface of a first application on the second electronic device and a second user interface object for displaying a user interface of a second application on the second electronic device. ### Claim 41 A first electronic device, comprising: one or more processors; a touch-sensitive display; and memory storing one or more programs that are configured for execution by the one or more processors, the one or more programs including instructions for: operating the first electronic device in a companion-display mode in which user interfaces generated by a second electronic device are displayed at the first electronic device, wherein the second electronic device is separate from the first electronic device; and while operating in the companion-display mode: displaying, on the touch-sensitive display of the first electronic device, a user interface generated by the second electronic device; detecting, at the first electronic device, a gesture using an input object, the gesture including a contact near a bottom edge of the touch-sensitive display and movement of the contact in an upward direction away from the bottom edge of the touch-sensitive display; and in response to detecting the gesture: in accordance with determining that the input object is one or more user fingers, performing a first operation on the touch-sensitive display of the first electronic device, the first operation including overlaying, on the touch-sensitive display, a first dock on the user interface generated by the second electronic device, the first dock including user interface objects each for opening an application installed on the first electronic device; and in accordance with determining that the input object is a stylus, performing a second operation on the touch-sensitive display of the first electronic device, wherein the second operation is distinct from the first operation and the second operation includes overlaying, on the touch-sensitive display, a second dock on the user interface generated by the second electronic device, the second dock including a first user interface object for displaying a user interface of a first application on the second electronic device and a second user interface object for displaying a user interface of a second application on the second electronic device. ### Claim 42 A non-transitory computer-readable storage medium storing executable instructions that, when executed by one or more processors of a first electronic device with a touch-sensitive display, cause the first electronic device to: operate the first electronic device in a companion-display mode in which user interfaces generated by a second electronic device are displayed at the first electronic device, wherein the second electronic device is separate from the first electronic device; and while operating in the companion-display mode: display, on the touch-sensitive display of the first electronic device, a user interface generated by the second electronic device; detect, at the first electronic device, a gesture using an input object, the gesture including a contact near a bottom edge of the touch-sensitive display and movement of the contact in an upward direction away from the bottom edge of the touch-sensitive display; and in response to detecting the gesture: in accordance with determining that the input object is one or more user fingers, perform a first operation on the touch-sensitive display of the first electronic device, the first operation including overlaying, on the touch-sensitive display, a first dock on the user interface generated by the second electronic device, the first dock including user interface objects each for opening an application installed on the first electronic device; and in accordance with determining that the input object is a stylus, perform a second operation on the touch-sensitive display of the first electronic device, wherein the second operation is distinct from the first operation and the second operation includes overlaying, on the touch-sensitive display, a second dock on the user interface generated by the second electronic device, the second dock including a first user interface object for displaying a user interface of a first application on the second electronic device and a second user interface object for displaying a user interface of a second application on the second electronic device. ### Claim 43 A method, comprising: at a first electronic device that includes a touch-sensitive display: operating the first electronic device in a companion-display mode in which user interfaces generated by a second electronic device are displayed at the first electronic device, wherein the second electronic device is separate from the first electronic device; and while operating in the companion-display mode: displaying, on the touch-sensitive display of the first electronic device, a user interface generated by the second electronic device; detecting, at the first electronic device, a gesture using an input object, the gesture including movement of the input object over a second user interface generated by the second electronic device; and in response to detecting the gesture: in accordance with determining that the input object is one or more user fingers, performing a first operation on the touch-sensitive display of the first electronic device, the first operation including scrolling content within the user interface generated by the second electronic device; and in accordance with determining that the input object is a stylus, performing a second operation on the touch-sensitive display of the first electronic device, wherein the second operation is distinct from the first operation and includes moving a cursor within the user interface generated by the second electronic device. ### Claim 44 A first electronic device, comprising: one or more processors; a touch-sensitive display; and memory storing one or more programs that are configured for execution by the one or more processors, the one or more programs including instructions for: operating the first electronic device in a companion-display mode in which user interfaces generated by a second electronic device are displayed at the first electronic device, wherein the second electronic device is separate from the first electronic device; and while operating in the companion-display mode: displaying, on the touch-sensitive display of the first electronic device, a user interface generated by the second electronic device; detecting, at the first electronic device, a gesture using an input object, the gesture including movement of the input object over a second user interface generated by the second electronic device; and in response to detecting the gesture: in accordance with determining that the input object is one or more user fingers, performing a first operation on the touch-sensitive display of the first electronic device, the first operation including scrolling content within the user interface generated by the second electronic device; and in accordance with determining that the input object is a stylus, performing a second operation on the touch-sensitive display of the first electronic device, wherein the second operation is distinct from the first operation and includes moving a cursor within the user interface generated by the second electronic device. ### Claim 45 A non-transitory computer-readable storage medium storing executable instructions that, when executed by one or more processors of a first electronic device with a touch-sensitive display, cause the first electronic device to: operate the first electronic device in a companion-display mode in which user interfaces generated by a second electronic device are displayed at the first electronic device, wherein the second electronic device is separate from the first electronic device; and while operating in the companion-display mode: display, on the touch-sensitive display of the first electronic device, a user interface generated by the second electronic device; detect, at the first electronic device, a gesture using an input object, the gesture including movement of the input object over a second user interface generated by the second electronic device; and in response to detecting the gesture: in accordance with determining that the input object is one or more user fingers, perform a first operation on the touch-sensitive display of the first electronic device, the first operation including scrolling content within the user interface generated by the second electronic device; and in accordance with determining that the input object is a stylus, perform a second operation on the touch-sensitive display of the first electronic device, wherein the second operation is distinct from the first operation and includes moving a cursor within the user interface generated by the second electronic device.