Contextual user interface element detection

A media playback system comprising:

• one or more processors;
• data storage having instructions thereon that, when executed by the one or more processors, cause the media playback system to perform operations comprising:
• receiving, at a computing device, a media data stream from a media content provider;
• transmitting the media data stream from the computing device to a display device for playback;
• determining that the media data stream comprises a contextual display user interface element, wherein determining that the media data stream comprises the contextual display user interface element comprises utilizing an accessibility service to identify the contextual display user interface element;
• after the determination, causing a controller device to present a contextual controller user interface element corresponding to the contextual display user interface element;
• receiving, at the computing device, an indication of user input via the contextual controller user interface element;
• and
• after receiving the indication, transmitting, via the computing device, a signal corresponding to the received indication of user input to the media content provider.

A method performed by a media playback system, the method comprising:

• receiving, at a computing device, a media data stream from a media content provider;
• transmitting the media data stream from the computing device to a display device for playback;
• determining that the media data stream comprises a contextual display user interface element, wherein determining that the media data stream comprises the contextual display user interface element comprises utilizing an accessibility service to identify the contextual display user interface element;
• after the determination, causing a controller device to present a contextual controller user interface element corresponding to the contextual display user interface element;
• receiving, at the computing device, an indication of a user input via the contextual controller user interface element;
• and
• after receiving the indication, transmitting, via the computing device, a signal corresponding to the received indication of user input to the media content provider.

A method performed by a media playback system, the method comprising:

• receiving, at a computing device, a media data stream from a media content provider;
• transmitting the media data stream from the computing device to a display device for playback;
• determining that the media data stream comprises a contextual display user interface element, wherein determining that the media data stream comprises the contextual display user interface element comprises analyzing pixels displayed via the display device to identify the contextual display user interface element;
• after the determination, causing a controller device to present a contextual controller user interface element corresponding to the contextual display user interface element;
• receiving, at the computing device, an indication of a user input via the contextual controller user interface element;
• and
• after receiving the indication, transmitting, via the computing device, a signal corresponding to the received indication of user input to the media content provider.

One or more tangible, non-transitory computer-readable media storing instructions that, when executed by one or more processors of a media playback system, cause the media playback system to perform operations comprising:

• receiving, at a computing device, a media data stream from a media content provider;
• transmitting the media data stream from the computing device to a display device for playback;
• determining that the media data stream comprises a contextual display user interface element, wherein determining that the media data stream comprises the contextual display user interface element comprises analyzing pixels displayed via the display device to identify the contextual display user interface element;
• after the determination, causing a controller device to present a contextual controller user interface element corresponding to the contextual display user interface element;
• receiving, at the computing device, an indication of a user input via the contextual controller user interface element;
• and
• after receiving the indication, transmitting, via the computing device, a signal corresponding to the received indication of user input to the media content provider.

One or more tangible, non-transitory computer-readable media storing instructions that, when executed by one or more processors of a media playback system, cause the media playback system to perform operations comprising:

• receiving, at a computing device, a media data stream from a media content provider;
• transmitting the media data stream from the computing device to a display device for playback;
• determining that the media data stream comprises a contextual display user interface element, wherein determining that the media data stream comprises the contextual display user interface element comprises utilizing an accessibility service to identify the contextual display user interface element;
• after the determination, causing a controller device to present a contextual controller user interface element corresponding to the contextual display user interface element;
• receiving, at the computing device, an indication of a user input via the contextual controller user interface element;
• and
• after receiving the indication, transmitting, via the computing device, a signal corresponding to the received indication of user input to the media content provider.