# US 12,223,228 B2 · User interfaces for audio media control > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 12,223,228 B2 - Document ID: US12223228B2 - Status: Granted - Kind: B2 - Application number: 18/234,613 - Filed: 2023-08-16 - Published: 2023-12-07 - Granted: 2025-02-11 - Earliest claimed: 2019-05-31 - Assignees: Apple Inc. - Applicants: Apple Inc. - Inventors: Taylor G. Carrigan; Patrick L Coffman; Christopher P. Foss - Web page: https://patentgrep.com/patent/US12223228B2 ## CPC - G06F 3/167: {Audio in a user interface, e.g. using voice commands for navigating, audio feedback} - H04M 1/72412: using two-way short-range wireless interfaces - G06F 3/017: {Gesture based interaction, e.g. based on a set of recognized hand gestures (interaction based on gestures traced on a digitiser G06F3/04883)} - G06F 3/0481: based on specific properties of the displayed interaction object or a metaphor-based environment, e.g. interaction with desktop elements like windows or icons, or assisted by a cursor's changing behaviour or appearance - G06F 3/0482: Interaction with lists of selectable items, e.g. menus - G06F 3/04842: Selection of displayed objects or displayed text elements (G06F3/0482 takes precedence) - G06F 3/04883: for inputting data by handwriting, e.g. gesture or text - G06F 3/165: {Management of the audio stream, e.g. setting of volume, audio stream path} - H04M 1/72415: for remote control of appliances - H04M 1/72442: for playing music files - H04M 1/72457: according to geographic location - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) - H04M: TELEPHONIC COMMUNICATION (circuits for controlling other apparatus via a telephone cable and not involving telephone switching apparatus G08) ## Abstract A method includes receiving data indicating a current media playback state of an external media playback device. The method includes, in response to receiving the data and in accordance with a determination that a set of external media playback device control criteria are met, displaying, via the display generation component, a user interface for controlling media playback at the external media playback device that includes one or more selectable user interface objects based on whether the data indicates that the external media playback device is currently performing a media playback operation. ## Claims ### Claim 1 A method, comprising: at a computer system that is in communication with a display generation component: displaying, via the display generation component, a user interface that includes: a first selectable user interface object for controlling a first set of one or more media playback devices; a second selectable user interface object for controlling a second set of one or more media playback devices that are different from the first set of one or more media playback devices; and an indication that the first selectable user interface object is selected; while the user interface includes the indication, receiving user input corresponding to a selection of the second selectable user interface object; and in response to receiving the user input corresponding to the selection of the second selectable user interface object and in accordance with a determination that a set of criteria is met, wherein the set of criteria includes a criterion that is met when the indication that the first selectable user interface object is selected and an indication that the second selectable user interface object is selected are concurrently displayed for a predetermined period of time: displaying, in the user interface, a third selectable user interface object for controlling a third set of one or more media playback devices that includes the first set of one or more media playback devices and the second set of one or more media playback devices; and ceasing to display the first selectable user interface object and the second selectable user interface object. ### Claim 2 (depends on 1) The method of claim 1, further comprising: while displaying the third selectable user interface object, receiving a first user input corresponding to a selection of the third selectable user interface object; and in response to receiving the first user input corresponding to the selection of the third selectable user interface object: in accordance with the user interface including an indication that the third selectable user interface object is selected, ceasing to display, in the user interface the indication that the third selectable user interface object is selected; and in accordance with the user interface not including the indication that the third selectable user interface object is selected, displaying, in the user interface, the indication that the third selectable user interface object is selected. ### Claim 3 (depends on 1) The method of claim 1, wherein: the first selectable user interface object includes an identifier associated with the first set of one or more media playback devices; the second selectable user interface object includes an identifier associated with the second set of one or more media playback devices; and displaying the third selectable user interface object includes displaying an identifier associated with the third set of one or more media playback devices, wherein the identifier associated with the third set of one or more media playback devices includes a portion that is representative of the identifier associated with the first set of one or more media playback devices and a portion that is representative of the identifier associated with the second set of one or more media playback devices. ### Claim 4 (depends on 1) The method of claim 1, further comprising: while displaying the third selectable user interface object, receiving a second user input corresponding to a selection of the third selectable user interface object; and in response to receiving the second user input corresponding to the selection of the third selectable user interface object: displaying the first selectable user interface object for controlling the first set of one or more media playback devices and the indication that the first selectable user interface object is selected; and displaying the second selectable user interface object for controlling the second set of one or more media playback devices and the indication that the second selectable user interface object is selected. ### Claim 5 (depends on 4) The method of claim 4, further comprising: while displaying the first selectable user interface object for controlling the first set of one or more media playback devices and the indication that the first selectable user interface object is selected and displaying the second selectable user interface object for controlling the second set of one or more media playback devices and the indication that the second selectable user interface object is selected, receiving one or more user inputs corresponding to one or more selections of the first selectable user interface object; and in response to receiving the one or more user inputs: in accordance with a determination that the last of the one or more selections of the first selectable user interface object caused the computer system to cease to display the indication that that the first selectable user interface object is selected, continuing to display the first selectable user interface object and the second selectable user interface object without displaying the third selectable user interface object. ### Claim 6 (depends on 1) The method of claim 1, wherein: the user interface includes a fourth selectable user interface object for controlling a fourth set of one or more media playback devices; and the method further comprises: while displaying the third selectable user interface object and the fourth selectable user interface object, receiving a user input corresponding to selection of the fourth selectable user interface object; and in response to receiving the user input corresponding to selection of the fourth selectable user interface object: displaying, in the user interface, a fifth selectable user interface object for controlling the first set of one or more media playback devices, the second set of one or more media playback devices, and the fourth set of one or more media playback devices; and ceasing to display the third selectable user interface object and the fourth selectable user interface object. ### Claim 7 (depends on 1) The method of claim 1, further comprising: in response to receiving the user input corresponding to the selection of the second selectable user interface object, transmitting, to a second computer system different from the computer system, grouping information that indicates a grouping of the first selectable user interface object and the second selectable user interface object. ### Claim 8 (depends on 1) The method of claim 1, further comprising: after displaying, in the user interface, the third selectable user interface object, receiving data corresponding to an indication that the first set of one or more media playback devices are connected to the computer system; and in response to receiving the data, displaying a notification that indicates that the first set of one or more media playback devices and the second set of one or more media playback devices are connected to the computer system. ### Claim 9 (depends on 1) The method of claim 1, further comprising: while displaying the user interface, receiving user input at a first location on the user interface; and in response to receiving the user input at the first location on the user interface, displaying, via the display generation component, a second user interface that includes a selectable user interface object that, when selected via one or more input devices that is in communication with the computer system, causes one or more external media playback devices that are playing back media to modify a media playback operation. ### Claim 10 (depends on 1) The method of claim 1, wherein, while displaying the user interface, displaying the third selectable user interface object includes, prior to displaying the third selectable user interface object, displaying a single user interface object that includes: a first portion of the first selectable user interface object; and a first portion of the second selectable user interface object. ### Claim 11 (depends on 1) The method of claim 1, further comprising: after receiving the user input corresponding to the selection of the second selectable user interface object, displaying a selectable user interface object for confirming the third selectable user interface object. ### Claim 12 A computer system that is configured to communication with a display generation component, comprising: one or more processors; and memory storing one or more programs configured to be executed by the one or more processors, the one or more programs including instructions for: displaying, via the display generation component, a user interface that includes: a first selectable user interface object for controlling a first set of one or more media playback devices; a second selectable user interface object for controlling a second set of one or more media playback devices that are different from the first set of one or more media playback devices; and an indication that the first selectable user interface object is selected; while the user interface includes the indication, receiving user input corresponding to a selection of the second selectable user interface object; and in response to receiving the user input corresponding to the selection of the second selectable user interface object and in accordance with a determination that a set of criteria is met, wherein the set of criteria includes a criterion that is met when the indication that the first selectable user interface object is selected and an indication that the second selectable user interface object is selected are concurrently displayed for a predetermined period of time: displaying, in the user interface, a third selectable user interface object for controlling a third set of one or more media playback devices that includes the first set of one or more media playback devices and the second set of one or more media playback devices; and ceasing to display the first selectable user interface object and the second selectable user interface object. ### Claim 13 (depends on 12) The computer system of claim 12, wherein the one or more programs further include instructions for: while displaying the third selectable user interface object, receiving a first user input corresponding to a selection of the third selectable user interface object; and in response to receiving the first user input corresponding to the selection of the third selectable user interface object: in accordance with the user interface including an indication that the third selectable user interface object is selected, ceasing to display, in the user interface the indication that the third selectable user interface object is selected; and in accordance with the user interface not including the indication that the third selectable user interface object is selected, displaying, in the user interface, the indication that the third selectable user interface object is selected. ### Claim 14 (depends on 12) The computer system of claim 12, wherein: the first selectable user interface object includes an identifier associated with the first set of one or more media playback devices; the second selectable user interface object includes an identifier associated with the second set of one or more media playback devices; and displaying the third selectable user interface object includes displaying an identifier associated with the third set of one or more media playback devices, wherein the identifier associated with the third set of one or more media playback devices includes a portion that is representative of the identifier associated with the first set of one or more media playback devices and a portion that is representative of the identifier associated with the second set of one or more media playback devices. ### Claim 15 (depends on 12) The computer system of claim 12, wherein the one or more programs further include instructions for: while displaying the third selectable user interface object, receiving a second user input corresponding to a selection of the third selectable user interface object; and in response to receiving the second user input corresponding to the selection of the third selectable user interface object: displaying the first selectable user interface object for controlling the first set of one or more media playback devices and the indication that the first selectable user interface object is selected; and displaying the second selectable user interface object for controlling the second set of one or more media playback devices and the indication that the second selectable user interface object is selected. ### Claim 16 (depends on 15) The computer system of claim 15, wherein the one or more programs further include instructions for: while displaying the first selectable user interface object for controlling the first set of one or more media playback devices and the indication that the first selectable user interface object is selected and displaying the second selectable user interface object for controlling the second set of one or more media playback devices and the indication that the second selectable user interface object is selected, receiving one or more user inputs corresponding to one or more selections of the first selectable user interface object; and in response to receiving the one or more user inputs: in accordance with a determination that the last of the one or more selections of the first selectable user interface object caused the computer system to cease to display the indication that that the first selectable user interface object is selected, continuing to display the first selectable user interface object and the second selectable user interface object without displaying the third selectable user interface object. ### Claim 17 (depends on 12) The computer system of claim 12, wherein: the user interface includes a fourth selectable user interface object for controlling a fourth set of one or more media playback devices; and the one or more programs further include instructions for: while displaying the third selectable user interface object and the fourth selectable user interface object, receiving a user input corresponding to selection of the fourth selectable user interface object; and in response to receiving the user input corresponding to selection of the fourth selectable user interface object: displaying, in the user interface, a fifth selectable user interface object for controlling the first set of one or more media playback devices, the second set of one or more media playback devices, and the fourth set of one or more media playback devices; and ceasing to display the third selectable user interface object and the fourth selectable user interface object. ### Claim 18 (depends on 12) The computer system of claim 12, wherein the one or more programs further include instructions for: in response to receiving the user input corresponding to the selection of the second selectable user interface object, transmitting, to a second computer system different from the computer system, grouping information that indicates a grouping of the first selectable user interface object and the second selectable user interface object. ### Claim 19 (depends on 12) The computer system of claim 12, wherein the one or more programs further include instructions for: after displaying, in the user interface, the third selectable user interface object, receiving data corresponding to an indication that the first set of one or more media playback devices are connected to the computer system; and in response to receiving the data, displaying a notification that indicates that the first set of one or more media playback devices and the second set of one or more media playback devices are connected to the computer system. ### Claim 20 (depends on 12) The computer system of claim 12, wherein the one or more programs further include instructions for: while displaying the user interface, receiving user input at a first location on the user interface; and in response to receiving the user input at the first location on the user interface, displaying, via the display generation component, a second user interface that includes a selectable user interface object that, when selected via one or more input devices that is in communication with the computer system, causes one or more external media playback devices that are playing back media to modify a media playback operation. ### Claim 21 (depends on 12) The computer system of claim 12, wherein, while displaying the user interface, displaying the third selectable user interface object includes, prior to displaying the third selectable user interface object, displaying a single user interface object that includes: a first portion of the first selectable user interface object; and a first portion of the second selectable user interface object. ### Claim 22 (depends on 12) The computer system of claim 12, wherein the one or more programs further include instructions for: after receiving the user input corresponding to the selection of the second selectable user interface object, displaying a selectable user interface object for confirming the third selectable user interface object. ### Claim 23 A non-transitory computer-readable storage medium storing one or more programs configured to be executed by one or more processors of a computer system that is in communication with a display generation component, the one or more programs including instructions for: displaying, via the display generation component, a user interface that includes: a first selectable user interface object for controlling a first set of one or more media playback devices; a second selectable user interface object for controlling a second set of one or more media playback devices that are different from the first set of one or more media playback devices; and an indication that the first selectable user interface object is selected; while the user interface includes the indication, receiving user input corresponding to a selection of the second selectable user interface object; and in response to receiving the user input corresponding to the selection of the second selectable user interface object and in accordance with a determination that a set of criteria is met, wherein the set of criteria includes a criterion that is met when the indication that the first selectable user interface object is selected and an indication that the second selectable user interface object is selected are concurrently displayed for a predetermined period of time: displaying, in the user interface, a third selectable user interface object for controlling a third set of one or more media playback devices that includes the first set of one or more media playback devices and the second set of one or more media playback devices; and ceasing to display the first selectable user interface object and the second selectable user interface object. ### Claim 24 (depends on 23) The non-transitory computer-readable storage medium of claim 23, wherein the one or more programs further include instructions for: while displaying the third selectable user interface object, receiving a first user input corresponding to a selection of the third selectable user interface object; and in response to receiving the first user input corresponding to the selection of the third selectable user interface object: in accordance with the user interface including an indication that the third selectable user interface object is selected, ceasing to display, in the user interface the indication that the third selectable user interface object is selected; and in accordance with the user interface not including the indication that the third selectable user interface object is selected, displaying, in the user interface, the indication that the third selectable user interface object is selected. ### Claim 25 (depends on 23) The non-transitory computer-readable storage medium of claim 23, wherein: the first selectable user interface object includes an identifier associated with the first set of one or more media playback devices; the second selectable user interface object includes an identifier associated with the second set of one or more media playback devices; and displaying the third selectable user interface object includes displaying an identifier associated with the third set of one or more media playback devices, wherein the identifier associated with the third set of one or more media playback devices includes a portion that is representative of the identifier associated with the first set of one or more media playback devices and a portion that is representative of the identifier associated with the second set of one or more media playback devices. ### Claim 26 (depends on 23) The non-transitory computer-readable storage medium of claim 23, wherein the one or more programs further include instructions for: while displaying the third selectable user interface object, receiving a second user input corresponding to a selection of the third selectable user interface object; and in response to receiving the second user input corresponding to the selection of the third selectable user interface object: displaying the first selectable user interface object for controlling the first set of one or more media playback devices and the indication that the first selectable user interface object is selected; and displaying the second selectable user interface object for controlling the second set of one or more media playback devices and the indication that the second selectable user interface object is selected. ### Claim 27 (depends on 26) The non-transitory computer-readable storage medium of claim 26, wherein the one or more programs further include instructions for: while displaying the first selectable user interface object for controlling the first set of one or more media playback devices and the indication that the first selectable user interface object is selected and displaying the second selectable user interface object for controlling the second set of one or more media playback devices and the indication that the second selectable user interface object is selected, receiving one or more user inputs corresponding to one or more selections of the first selectable user interface object; and in response to receiving the one or more user inputs: in accordance with a determination that the last of the one or more selections of the first selectable user interface object caused the computer system to cease to display the indication that that the first selectable user interface object is selected, continuing to display the first selectable user interface object and the second selectable user interface object without displaying the third selectable user interface object. ### Claim 28 (depends on 23) The non-transitory computer-readable storage medium of claim 23, wherein: the user interface includes a fourth selectable user interface object for controlling a fourth set of one or more media playback devices; and the one or more programs further include instructions for: while displaying the third selectable user interface object and the fourth selectable user interface object, receiving a user input corresponding to selection of the fourth selectable user interface object; and in response to receiving the user input corresponding to selection of the fourth selectable user interface object: displaying, in the user interface, a fifth selectable user interface object for controlling the first set of one or more media playback devices, the second set of one or more media playback devices, and the fourth set of one or more media playback devices; and ceasing to display the third selectable user interface object and the fourth selectable user interface object. ### Claim 29 (depends on 23) The non-transitory computer-readable storage medium of claim 23, wherein the one or more programs further include instructions for: in response to receiving the user input corresponding to the selection of the second selectable user interface object, transmitting, to a second computer system different from the computer system, grouping information that indicates a grouping of the first selectable user interface object and the second selectable user interface object. ### Claim 30 (depends on 23) The non-transitory computer-readable storage medium of claim 23, wherein the one or more programs further include instructions for: after displaying, in the user interface, the third selectable user interface object, receiving data corresponding to an indication that the first set of one or more media playback devices are connected to the computer system; and in response to receiving the data, displaying a notification that indicates that the first set of one or more media playback devices and the second set of one or more media playback devices are connected to the computer system. ### Claim 31 (depends on 23) The non-transitory computer-readable storage medium of claim 23, wherein the one or more programs further include instructions for: while displaying the user interface, receiving user input at a first location on the user interface; and in response to receiving the user input at the first location on the user interface, displaying, via the display generation component, a second user interface that includes a selectable user interface object that, when selected via one or more input devices that is in communication with the computer system, causes one or more external media playback devices that are playing back media to modify a media playback operation. ### Claim 32 (depends on 23) The non-transitory computer-readable storage medium of claim 23, wherein, while displaying the user interface, displaying the third selectable user interface object includes, prior to displaying the third selectable user interface object, displaying a single user interface object that includes: a first portion of the first selectable user interface object; and a first portion of the second selectable user interface object. ### Claim 33 (depends on 23) The non-transitory computer-readable storage medium of claim 23, wherein the one or more programs further include instructions for: after receiving the user input corresponding to the selection of the second selectable user interface object, displaying a selectable user interface object for confirming the third selectable user interface object.