# US 2016/0366408 A1 · Video processing system > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 2016/0366408 A1 - Document ID: US20160366408A1 - Status: Published application - Kind: A1 - Application number: 15/177,685 - Filed: 2016-06-09 - Published: 2016-12-15 - Earliest claimed: 2015-06-11 - Assignees: Arm Limited - Applicants: ARM Limited - Inventors: Tomas Edsö; Ola Hugosson - Web page: https://patentgrep.com/patent/US20160366408A1 ## CPC - H04N 19/115: Selection of the code volume for a coding unit prior to coding - H04N 19/172: the region being a picture, frame or field - H04N 19/182: the unit being a pixel - H04N 19/423: characterised by memory arrangements (H04N19/433 takes precedence) - H04N 19/53: Multi-resolution motion estimation; Hierarchical motion estimation - H04N 19/59: involving spatial sub-sampling or interpolation, e.g. alteration of picture size or resolution - H04N 19/44: Decoders specially adapted therefor, e.g. video decoders which are asymmetric with respect to the encoder - H04N 19/63: using sub-band based transform, e.g. wavelets - H04N 19/85: using pre-processing or post-processing specially adapted for video compression - H04N 19/96: Tree coding, e.g. quad-tree coding - H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION ## Abstract A video processing system includes a video processing unit (VPU) and one or more display processing units, all having access to external memory. Video data representing frames to be displayed is generated. The VPU generates pixel data representing the frames and stores it in memory. The display processing units then read the pixel data to display the frames. The VPU is configured to generate and store in memory pixel data representing reference frames for the sequence of video frames at the full resolution of the reference frame and also at at least one lower resolution to the full resolution. ## Claims ### Claim 1 A method of processing frames of video data in a video processing system, in which data for one or more frames in a sequence of video frames is defined with respect to a reference frame, the method comprising: when all or part of a reference frame for a sequence of video frames is to be used when decoding the sequence of video frames: generating and storing in memory pixel data representing all or part of the reference frame at a first resolution; and generating and storing in memory pixel data representing all or part of the reference frame at at least one different resolution to the first resolution. ### Claim 2 (depends on 1) The method of claim 1, further comprising: configuring the reference frame pixel data generation based on how the pixel data for the reference frame is to be used by the video processing system. ### Claim 3 (depends on 1) The method of claim 1, further comprising: configuring the reference frame pixel data generation based on how the pixel data for the reference frame is to be used by the video processing system such that: the one or more or all of the at least one different resolution to the first resolution that the reference frame is generated at is selected based on the resolution of an electronic display or displays on which the video sequence is to be displayed; and/or the number of different resolutions at which the reference frame is to be represented is selected based on the number of different displays in the overall video processing system on which the video sequence is to be displayed. ### Claim 4 (depends on 1) The method of claim 1, further comprising: processing at least one frame in the sequence of video frames using the pixel data representing the reference frame at at least one different resolution to the first resolution. ### Claim 5 (depends on 4) The method of claim 4, wherein the step of processing at least one frame in the sequence of video frames comprises: generating pixel data representing an output frame for display using the pixel data representing the reference frame at at least one lower resolution than the first resolution. ### Claim 6 (depends on 1) The method of claim 1, wherein, if the reference frame is to be displayed: the pixel data for a lower resolution reference frame is used as the pixel data for the displayed frame. ### Claim 7 (depends on 1) The method of claim 1, further comprising, when a current frame in the sequence of video frames is encoded with respect to the reference frame: decoding the current frame using the encoded pixel data for the current frame and the pixel data representing the reference frame at the first resolution; and generating and storing in memory pixel data representing the current frame at at least one different resolution to the first resolution. ### Claim 8 (depends on 1) The method of claim 1, further comprising: reading in from the memory the pixel data representing all or part of the reference frame at the at least one different resolution to the first resolution; using the pixel data when generating an output frame to be displayed; and outputting the generated output frame to a display. ### Claim 9 (depends on 1) The method of claim 1, further comprising: reading in from memory the pixel data representing all or part of the reference frame at the at least one different resolution to the first resolution; using the pixel data when performing a processing operation on a frame in the sequence of video frames to generate pixel data representing a processed frame; and storing the pixel data representing the processed frame in memory. ### Claim 10 An apparatus for processing frames of video data in a video processing system, in which data for one or more frames in a sequence of video frames is defined with respect to a reference frame, the apparatus comprising: reference frame generation circuitry configured to, when all or part of a reference frame for a sequence of video frames is to be used when decoding the sequence of video frames: generate and store in a memory of the video processing system, pixel data representing all or part of the reference frame at a first resolution; and generate and store in the memory pixel data representing all or part of the reference frame at at least one different resolution to the first resolution. ### Claim 11 (depends on 10) The apparatus of claim 10, wherein the reference frame pixel data generation is configured based on how the pixel data for the reference frame is to be used by the video processing system. ### Claim 12 (depends on 10) The apparatus of claim 10, wherein: the reference frame pixel data generation is configured based on how the pixel data for the reference frame is to be used by the video processing system such that: one or more or all of the at least one different resolution to the first resolution that the reference frame is generated at is selected based on the resolution of an electronic display or displays on which the video sequence is to be displayed; and/or the number of different resolutions at which the reference frame is to be represented is selected based on the number of different displays in the overall video processing system on which the video sequence is to be displayed. ### Claim 13 (depends on 10) The apparatus of claim 10, further comprising: frame processing circuitry configured to process at least one frame in the sequence of video frames using the pixel data representing the reference frame at at least one different resolution to the first resolution. ### Claim 14 (depends on 13) The apparatus of claim 13, wherein the frame processing circuitry is configured to: generate pixel data representing an output frame for display using the pixel data representing the reference frame at at least one lower resolution than the first resolution. ### Claim 15 (depends on 10) The apparatus of claim 10, wherein, if the reference frame is to be displayed: the pixel data for a lower resolution reference frame is used as the pixel data for the displayed frame. ### Claim 16 (depends on 10) The apparatus of claim 10, wherein the apparatus is configured to, when the current frame in the sequence of video frames is encoded with respect to the reference frame: decode the current frame using the encoded pixel data for the current frame and the pixel data representing the reference frame at the first resolution; and generate and store in memory pixel data representing the current frame at at least one different resolution to the first resolution. ### Claim 17 A video processing system, in which data for one or more frames in a sequence of video frames is defined with respect to a reference frame, the video processing system comprising: a memory; reference frame generation circuitry configured to, when all or part of a reference frame for a sequence of video frames is to be used when decoding the sequence of video frames, generate and store in the memory pixel data representing all or part of the reference frame at a first resolution, and generate and store in the memory pixel data representing all or part of the reference frame at at least one different resolution to the first resolution; and a processing unit configured to read in from memory the pixel data representing all or part of the reference frame at the at least one different resolution to the first resolution, use the pixel data when performing a processing operation on a frame in the sequence of video frames to generate pixel data representing a processed frame and store the pixel data representing the processed frame in memory. ### Claim 18 (depends on 17) The system of claim 17, wherein: the processing unit is a display controller of the video processing system; using the pixel data when performing a processing operation on a frame in the sequence of video frames to generate pixel data representing a processed frame comprises generating pixel data representing an output frame to be displayed, using the pixel data representing all or part of the reference frame at the at least one different resolution to the first resolution; and the display controller is configured to output the generated output frame data to a display. ### Claim 19 A computer readable storage medium storing computer software code which when executing on a processor performs a method of processing frames of video data in a video processing system, in which data for one or more frames in a sequence of video frames is defined with respect to a reference frame, the method comprising: when all or part of a reference frame for a sequence of video frames is to be used when decoding the sequence of video frames: generating and storing in memory pixel data representing all or part of the reference frame at a first resolution; and generating and storing in memory pixel data representing all or part of the reference frame at at least one different resolution to the first resolution.