Video display preference filtering
AssigneeApple Inc.
InventorsAlexandros Tourapis, Athanasios Leontaris
Life of this patent
Estimating the term…
Abstract
Video coding schemes may include one or more filters to reduce coding artifacts and improve video quality. These filters may be applied to decode video data in a predetermined sequence. The output from one or more of these filters may be selected for different images, blocks, or sets of video data and then copied and/or routed to a display or a buffer storing reference data that is used to decode other video data in a data stream. Providing the ability to select which filter output is used for display and as a reference may result in better video quality for multiple types of video data. The filters that are selected for display and for reference may be different and may vary for different images, blocks, and data sets.
Claims27
A computer-implemented method comprising, by an encoder:
- in a coding system employing a plurality of sequential filters for filtering decoded video data, where the output of each previous filter feeds into the input of each subsequent filter:
- selecting, for a new frame of video data to be coded, which of the plurality of filters is to be applied to produce a first filtered output during decoding of the new frame of video data before outputting the decoded new frame of video data to a display, and independently selecting, for the same new frame of video data to be coded, which of the plurality of filters is to be applied to produce a second filtered output during decoding of the new frame of video data for generating a reference picture,
- coding the new frame of video data according to predictive coding techniques;
- and
- transmitting the coded new frame of video data along with metadata for the coded new frame of video data, the metadata for the coded new frame of video data including first metadata representing the first filter selections for outputting the frame to the display and second metadata, separate from the first metadata, representing the second filter selections for generating the reference picture of the frame.
A computer-implemented method comprising, at a decoder having a plurality of filters for filtering decoded video data, the filters arranged in a predetermined sequence where the output of each previous filter feeds into the input of each subsequent filter:
- decoding an frame of video data received via a channel;
- responsive to first metadata received in the channel that identifies which of the plurality of filters is to be applied during decoding of the frame of video data for outputting to a display:
- first filtering the decoded frame of video data by the identified filter(s) and
- storing the first filtered video data in a display buffer;
- and
- responsive to second metadata, separate from the first metadata, received in the channel that identifies which of the plurality of filters is to be applied during decoding of the frame of video data for generating reference pictures:
- second filtering the decoded frame of video data by the identified filter(s) and
- storing the second filtered video data in a reference picture buffer.
A computer-implemented method comprising, by an encoder in a coding system employing a plurality of filters for filtering decoded video data arranged in a predetermined sequence where the output of each previous filter feeds into the input of each subsequent filter:
- selecting, for a new frame of video data to be coded, which of the plurality of filters has a first output for the new frame of video data that is optimized for use as a reference during subsequent decoding based on a type of video data to be encoded, and separately selecting, for the new frame of video data to be coded, which of the plurality of filters has a second output for the new frame of video data that is optimized for output to a display during subsequent decoding based on a type of video data to be encoded;
- and
- encoding the first and second filter selections for the new frame of video data in respective first and second metadata items transmitted with the encoded video data to a decoder.
A computer-implemented method comprising, at a decoder having a plurality of filters for filtering decoded video data arranged in a predetermined sequence where the output of each previous filter feeds into the input of each subsequent filter:
- decoding reference frame of video data received via a channel;
- identifying, from first metadata received in the channel, a first filter within the plurality of filters that is to be applied during decoding of the reference frame of video data for use as a reference for decoding subsequent video data from the channel, the identified first filter varying for different received frames of video data using a processing device;
- saving a first output from the identified filter for the reference frame of received video data in a reference buffer storing reference data used during decoding;
- identifying, from second metadata, separate from the first metadata, received in the channel, a second filter within the plurality of filters that is to be applied during decoding of the reference frame of video data for use in displaying the decoded reference frame of video data, the filter varying for different received frames of video data using a processing device;
- saving a second output from the identified filter for the reference frame of received video data in a display buffer storing decoded video data to be displayed;
- outputting to a display the saved second output;
- and
- decoding subsequently received frames of video data using the saved first output as reference data.
A processor comprising:
- a plurality of sequential filters, where the output of each previous filter feeds into the input of each subsequent filter;
- a decision unit for:
- selecting, for a new frame of video data to be coded, which of the plurality of filters is to be applied during decoding of the new frame of video data before outputting the decoded new frame of video data to a display, and
- selecting, for the new frame of video data to be coded, which of the plurality of filters is to be applied during decoding of the new frame of video data for generating reference pictures,
- a processing device for coding the new frame of video data according to predictive coding techniques;
- and
- a transmitter for transmitting the coded new frame of video data along with metadata for the coded new frame of video data, the metadata including a first metadata representing the first filter selections for outputting to the display and a second metadata, separate from the first metadata, representing the second filter selections for generating the reference picture.
A processor comprising:
- a plurality of sequential filters, where the output of each previous filter feeds into the input of each subsequent filter;
- a decision unit for
- identifying, for an frame of video data to be decoded, a filter from the plurality of filters having a first output used as a reference for decoding subsequent items of video data in an encoded video stream from a first metadata included in the encoded video stream, the identified filter varying for different frames of video data in the stream;
- and
- identifying, for the frame of video data to be decoded, a filter from the plurality of filters having a second output for outputting video data to a display and being identified from a second metadata, separate from the first metadata, included in the encoded video stream;
- a reference buffer for storing reference data;
- a processing device for routing the first output to the reference buffer and routing the second output to a display;
- and
- a decoder for decoding the subsequent items of video data based on the reference data stored in the reference buffer.
A computer-implemented method comprising, at a decoder having a plurality of filters arranged in a predetermined sequence:
- decoding a reference frame of video data received via a channel;
- selecting, based on first metadata received in the channel, a first filter within the plurality of filters that is to be applied during decoding of the reference frame for use as a reference for decoding subsequent video data from the channel;
- saving a first output from the first filter in a reference buffer storing reference data used during decoding;
- selecting, based on second metadata, separate from the first metadata, received in the channel, a second filter within the plurality of filters that is to be applied during decoding of the reference frame for use in displaying the decoded reference frame;
- saving a second output from the second filter in a display buffer storing decoded video data to be displayed;
- outputting the saved second output to a display;
- and
- decoding subsequently received video data using the saved first output as reference data.
Dependencies are inferred from the claim text.