Display device and method for operating same

A method for controlling a display device, the method comprising:

• displaying content in a first display mode;
• receiving a user utterance during playback of the content;
• identifying a context of the user utterance to determine a target category;
• retrieving metadata associated with a current playback time of the content, the metadata containing a plurality of objects;
• generating a customized list of objects by filtering the metadata to select objects that match the target category;
• automatically switching the display device from the first display mode to a second display mode that includes a content window and a side panel;
• and
• displaying the content in the content window and the customized list of objects in the side panel,
• wherein the content continues to play in the content window without pausing during the switching and displaying steps.

A system for content experience optimization, the system comprising:

• a metadata server configured to provide time-synchronized information for content;
• and
• a display apparatus comprising a display and a processor, the processor configured to:
• detect a voice query while playing the content;
• request metadata from the metadata server based on a timestamp of the content at a time of the voice query;
• analyze the voice query to determine a user interest;
• dynamically reconfigure a layout of the display to present a main viewing zone and an auxiliary information zone;
• populate the auxiliary information zone with items from the metadata that are relevant to the user interest, while excluding irrelevant items;
• and
• preserve a playback state of the content in the main viewing zone while the user interacts with the auxiliary information zone.

A non-transitory computer-readable medium containing instructions that, when executed by a processor of a display device, cause the display device to perform operations comprising:

• outputting video content on a display;
• receiving a voice input;
• determining a playback time of the video content when the voice input is received;
• obtaining metadata corresponding to the playback time;
• parsing the voice input to identify a filtering constraint specified by a user;
• applying the filtering constraint to the metadata to isolate a subset of items matching the filtering constraint;
• and
• rendering a multi-window interface comprising a first window for the video content and a second window for the subset of items,
• wherein the video content is continuously output in the first window without interruption.