# US 2014/0208223 A1 · User interface including content from a remote device

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2014/0208223 A1
- Document ID: US20140208223A1
- Status: Published application
- Kind: A1
- Application number: 14/222,302
- Filed: 2014-03-21
- Published: 2014-07-24
- Earliest claimed: 2008-05-13
- Assignees: Apple Inc.
- Applicants: Apple Inc.
- Inventors: Emily Clark Schubert; Peter T. Langenfeld
- Web page: https://patentgrep.com/patent/US20140208223A1

## CPC

- G06F 3/04842: Selection of displayed objects or displayed text elements (G06F3/0482 takes precedence)
- G06F 3/048: Interaction techniques based on graphical user interfaces [GUI]
- G06F 3/0482: Interaction with lists of selectable items, e.g. menus
- G06F 3/0484: for the control of specific functions or operations, e.g. selecting or manipulating an object, an image or a displayed text element, setting a parameter value or selecting a range
- G06F 9/452: {Remote windowing, e.g. X-Window System, desktop virtualisation (protocols for virtual reality H04L67/131)}
- H04L 67/025: for remote control or remote monitoring of applications
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- H04L: TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION (arrangements common to telegraphic and telephonic communication H04M)

## Abstract

A user interface displayed by an electronic device can include a first user-interface portion generated by the electronic device and a second user interface portion generated by a remote device. The electronic device can detect received input and determine whether the input corresponds to an interactive element in the first user-interface portion or an interactive element in the second user interface portion. If the input corresponds to an interactive element in the first user-interface portion, the electronic device can generate a response to the input. If the input corresponds to an interactive element in the second user-interface portion, the electronic device can transmit information indicative of the input to the remote device.

## Claims

### Claim 1

An electronic device, comprising:
one or more processors;
a display;
one or more input devices;
memory; and
one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors, the one or more programs including instructions for:
displaying, on the display, a user interface, wherein the user interface includes a first user-interface portion generated by the electronic device and a second user-interface portion generated by a remote device;
detecting input from an input device of the one or more input devices; and
in response to detecting the input:
determining whether the input corresponds to an interactive element in the first user-interface portion or an interactive element in the second user-interface portion;
in accordance with a determination that the input corresponds to an interactive element in the first user-interface portion, generating a response to the input; and
in accordance with a determination that the input corresponds to an interactive element in the second user-interface portion, transmitting information indicative of the input to the remote device.

### Claim 2 (depends on 1)

The device of claim 1, wherein the one or more programs include instructions for:
after transmitting the information indicative of the input to the remote device, receiving, from the remote device, remote response information for responding to the input; and
responding to the input in accordance with the remote response information.

### Claim 3 (depends on 2)

The device of claim 2, wherein responding to the input in accordance with the remote response information includes updating the second user-interface portion in accordance with the remote response information.

### Claim 4 (depends on 1)

The device of claim 1, wherein:
the one or more input devices include a touchscreen on which the user interface is displayed;
detecting the input includes detecting a contact on the touchscreen; and
determining whether the input corresponds to an interactive element in the first user-interface portion or an interactive element in the second user-interface portion includes determining whether the contact was detected on a portion of the touchscreen that displays the first user-interface portion or on a portion of the touchscreen that displays the second user-interface portion.

### Claim 5 (depends on 4)

The device of claim 4, wherein the information indicative of the input transmitted to the remote device includes pixel coordinates of a location on the touchscreen.

### Claim 6 (depends on 1)

The device of claim 1, wherein the information indicative of the input transmitted to the remote device includes information identifying a control with which an interaction occurred.

### Claim 7 (depends on 1)

The device of claim 1, wherein the information indicative of the input transmitted to the remote device includes information indicating an operation associated with the input.

### Claim 8

A method comprising:
at an electronic device with one or more processors, a display, one or more input devices, memory:
displaying, on the display, a user interface, wherein the user interface includes a first user-interface portion generated by the electronic device and a second user-interface portion generated by a remote device;
detecting input from an input device of the one or more input devices; and
in response to detecting the input:
determining whether the input corresponds to an interactive element in the first user-interface portion or an interactive element in the second user-interface portion;
in accordance with a determination that the input corresponds to an interactive element in the first user-interface portion, generating a response to the input; and
in accordance with a determination that the input corresponds to an interactive element in the second user-interface portion, transmitting information indicative of the input to the remote device.

### Claim 9 (depends on 8)

The method of claim 8, further comprising:
after transmitting the information indicative of the input to the remote device, receiving, from the remote device, remote response information for responding to the input; and
responding to the input in accordance with the remote response information.

### Claim 10 (depends on 9)

The method of claim 9, wherein responding to the input in accordance with the remote response information includes updating the second user-interface portion in accordance with the remote response information.

### Claim 11 (depends on 8)

The method of claim 8, wherein the one or more input devices include a touchscreen on which the user interface is displayed;
detecting the input includes detecting a contact on the touchscreen; and
determining whether the input corresponds to an interactive element in the first user-interface portion or an interactive element in the second user-interface portion includes determining whether the contact was detected on a portion of the touchscreen that displays the first user-interface portion or on a portion of the touchscreen that displays the second user-interface portion.

### Claim 12 (depends on 11)

The method of claim 11, wherein the information indicative of the input transmitted to the remote device includes pixel coordinates of a location on the touchscreen.

### Claim 13 (depends on 8)

The method of claim 8, wherein the information indicative of the input transmitted to the remote device includes information identifying a control with which an interaction occurred.

### Claim 14 (depends on 8)

The method of claim 8, wherein the information indicative of the input transmitted to the remote device includes information indicating an operation associated with the input.

### Claim 15

A non-transitory computer readable storage medium storing one or more programs, the one or more programs comprising instructions, which when executed by an electronic device with one or more processors, a display, one or more input devices, and memory, cause the device to:
display, on the display, a user interface, wherein the user interface includes a first user-interface portion generated by the device and a second user-interface portion generated by a remote device;
detect input from an input device of the one or more input devices; and
in response to detecting the input:
determine whether the input corresponds to an interactive element in the first user-interface portion or an interactive element in the second user-interface portion;
in accordance with a determination that the input corresponds to an interactive element in the first user-interface portion, generate a response to the input; and
in accordance with a determination that the input corresponds to an interactive element in the second user-interface portion, transmit information indicative of the input to the remote device.

### Claim 16 (depends on 15)

The non-transitory computer readable storage medium of claim 15, wherein the one or more programs include instructions for:
after transmitting the information indicative of the input to the remote device, receiving, from the remote device, remote response information for responding to the input; and
responding to the input in accordance with the remote response information.

### Claim 17 (depends on 16)

The non-transitory computer readable storage medium of claim 16, wherein responding to the input in accordance with the remote response information includes updating the second user-interface portion in accordance with the remote response information.

### Claim 18 (depends on 15)

The non-transitory computer readable storage medium of claim 15, wherein:
the one or more input devices include a touchscreen on which the user interface is displayed;
detecting the input includes detecting a contact on the touchscreen; and
determining whether the input corresponds to an interactive element in the first user-interface portion or an interactive element in the second user-interface portion includes determining whether the contact was detected on a portion of the touchscreen that displays the first user-interface portion or on a portion of the touchscreen that displays the second user-interface portion.

### Claim 19 (depends on 15)

The non-transitory computer readable storage medium of claim 15, wherein the information indicative of the input transmitted to the remote device includes pixel coordinates of a location on the touchscreen.

### Claim 20 (depends on 15)

The non-transitory computer readable storage medium of claim 15, wherein the information indicative of the input transmitted to the remote device includes Information identifying a control with which an interaction occurred.

### Claim 21 (depends on 15)

The non-transitory computer readable storage medium of claim 15, wherein the information indicative of the input transmitted to the remote device includes Information indicating an operation associated with the input.

### Claim 22

An electronic device, comprising:
one or more processors;
memory; and
one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors, the one or more programs including instructions for:
while the device is coupled to an accessory that includes a display and an input device, receiving a request to provide a user interface for display at the accessory; and
in response to receiving the request:
generating a first user-interface portion for display in a first region of the display; and
initiating display of the first user-interface portion in the first region of the display of the accessory, wherein the first user-interface portion is displayed in the first region of the display while a second user-interface portion generated by the accessory is concurrently displayed in a second region of the display that is distinct from the first region of the display; and
after initiating display of the first user-interface portion in the first region of the display of the accessory, receiving input that is indicative of interaction with one or more controls in the first region of the display; and
in response to receiving the input:
generating an updated first user-interface portion in accordance with the input; and
initiating display of the updated first user-interface portion in the first region of the display.

### Claim 23 (depends on 22)

The device of claim 22, wherein the one or more programs include instructions for transmitting information identifying a portion of the display corresponding to the first region of the display.

### Claim 24 (depends on 22)

The device of claim 22, wherein generating a first user-interface portion includes generating placeholder data for the second user-interface portion.

### Claim 25 (depends on 22)

The device of claim 22, wherein the request includes at least one of configuration, environment, and status information.

### Claim 26 (depends on 22)

The device of claim 22, wherein the first user-interface portion is adapted to the configuration of the first region of the display.

### Claim 27 (depends on 22)

The device of claim 22, wherein the one or more programs include instructions for:
after receiving the request, transmitting a list of remote user interface image types;
receiving a selection of a user interface image type from the list of remote user interface image types; and
generating the first user-interface portion using the selected user interface image type.

### Claim 28 (depends on 22)

The device of claim 22, wherein the one or more programs include instructions for:
receiving a request to stop providing the user interface for display at the accessory; and;
in response to receiving the request to stop providing the user interface for display at the accessory; stop generating the updated first user-interface portion.
