# US 8,542,732 B1 · Video encoder using GPU > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 8,542,732 B1 - Document ID: US8542732B1 - Status: Granted - Kind: B1 - Application number: 12/342,145 - Filed: 2008-12-23 - Granted: 2013-09-24 - Earliest claimed: 2008-12-23 - Assignees: Elemental Technologies, Inc. - Applicants: Brian G. Lewis; Jesse J. Rosenzweig - Inventors: Brian G. Lewis; Jesse J. Rosenzweig - Web page: https://patentgrep.com/patent/US8542732B1 ## CPC - G06T 1/60: Memory management - G06T 1/20: Processor architectures; Processor configuration, e.g. pipelining - H04N 19/423: characterised by memory arrangements (H04N19/433 takes precedence) - H04N 19/433: characterised by techniques for memory access - H04N 19/436: using parallelised computational arrangements - H04N 19/513: Processing of motion vectors - G09G 2360/125: using unified memory architecture [UMA] - G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL - H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION - G09G: ARRANGEMENTS OR CIRCUITS FOR CONTROL OF INDICATING DEVICES USING STATIC MEANS TO PRESENT VARIABLE INFORMATION (arrangements for transferring data between digital computers and displays G06F3/14; static indicating arrangements comprising an association of a number of separate sources or light control cells G09F9/00; static indicating arrangements comprising integral associations of a number of light sources H01J, H01K, H10H, H05B33/12; scanning, transmission or reproduction of documents or the like, e.g. facsimile transmission, details thereof H04N1/00) ## Abstract An apparatus comprising a graphics processing unit, a processor and a memory. The memory stores computer executable instructions. The computer executable instructions use (i) the graphics processing unit to perform prediction, decimation, transformation and quantization, and inverse-quantization and inverse-transformation and (ii) the processor to perform entropy encoding and rate control. ## Claims ### Claim 1 An apparatus comprising: a graphics processing unit comprising an array of software-configurable general purpose parallel processors and a globally shared memory, wherein each of the software-configurable general purpose parallel processors in said array of software-configurable general purpose parallel processors has access to said globally shared memory; a central processing unit; a first memory accessible by said graphics processing unit, said first memory storing a first set of computer executable instructions, said first set of computer executable instructions using said software-configurable general purpose parallel processors in said array of software-configurable general purpose parallel processors to perform video compression steps including prediction, decimation, transformation and quantization, and inverse-quantization and inverse-transformation involving a current frame and one or more reference frames stored in said globally shared memory; and a second memory accessible by said central processing unit, said second memory storing a second set of computer executable instructions, said second set of computer executable instructions using said central processing unit to perform rate control on said current frame by choosing at least one target parameter value for transformation and quantization as said video compression steps are performed by said graphics processing unit and to perform entropy encoding on said current frame in response to a signal indicating the current frame is ready for encoding. ### Claim 2 (depends on 1) The apparatus according to claim 1, wherein said first set of computer executable instructions use one or more of said software-configurable general purpose parallel processors of said graphics processing unit as a deblocking filter. ### Claim 3 (depends on 1) The apparatus according to claim 1, wherein said first set of computer executable instructions comprise a plurality of subroutines, each of said subroutines comprising computer executable instructions for performing one or more of the video compression steps including said prediction, said decimation, said transformation and quantization, and said inverse-quantization and inverse-transformation using the processor array of said graphics processing unit. ### Claim 4 (depends on 3) The apparatus according to claim 3, wherein said plurality of subroutines include a prediction subroutine, a motion estimation subroutine and a transform subroutine. ### Claim 5 (depends on 4) The apparatus according to claim 4, wherein said transform subroutine comprises a first transform subroutine for intra blocks and a second transform subroutine for inter predicted blocks. ### Claim 6 (depends on 4) The apparatus according to claim 4, wherein said plurality of subroutines further comprises a decimation subroutine. ### Claim 7 (depends on 1) The apparatus according to claim 1, wherein said first and said second sets of computer executable instructions are configured to encode a picture using one graphics processing unit thread and two central processing unit threads. ### Claim 8 (depends on 1) The apparatus according to claim 1, wherein said graphic processing unit further comprises a plurality of local registers. ### Claim 9 (depends on 8) The apparatus according to claim 8, wherein each software-configurable General purpose parallel processor has access to the globally shared memory, a portion of the first memory and a number of the local registers. ### Claim 10 (depends on 9) The apparatus according to claim 9, wherein a subset of the plurality of software-configurable general purpose parallel processors share a respective portion of the first memory. ### Claim 11 (depends on 8) The apparatus according to claim 8, further comprising: a texture cache providing consecutive two-dimensional reads from the globally shared memory; and interpolation hardware configured to generate decimation images and estimate sub-pixel values through bi-linear interpolation. ### Claim 12 A method for encoding a frame of video comprising the steps of: using a first set of computer executable instructions stored in a first memory that is accessible by a graphics processing unit, said first set of computer executable instructions, when executed, using an array of software-configurable general purpose parallel processors and a globally shared memory of said graphics processing unit to perform video compression steps including prediction, decimation, transformation and quantization, and inverse-quantization and inverse-transformation, wherein each of the software-configurable general purpose parallel processors in said array of software-configurable general purpose parallel processors has access to said globally shared memory and said frame and one or more reference frames are stored in said globally shared memory; and using a second set of computer executable instructions stored in a second memory that is accessible by a central processing unit, said second set of computer executable instructions, when executed, using said central processing unit to concurrently perform entropy encoding, rate control and management of said graphics processing unit, wherein said central processing unit performs rate control by choosing at least one target parameter value for transformation and quantization as said video compression steps are performed by said graphics processing unit and performs entropy encoding on the frame in response to a signal indicating the frame is ready for encoding. ### Claim 13 (depends on 12) The method according to claim 12, further comprising: using one or more of said software-configurable general purpose parallel processors of said graphics processing unit as a deblocking filter. ### Claim 14 (depends on 12) The method according to claim 12, further comprising: providing one or more subroutines stored in a computer readable medium, said one or more subroutines comprising computer executable instructions for performing said prediction, decimation, transformation and quantization, and inverse-quantization and inverse-transformation using the array of software-configurable general purpose parallel processors of said graphics processing unit. ### Claim 15 (depends on 14) The method according to claim 14, wherein said subroutines further comprise computer executable instructions for performing entropy encoding, rate control and management of said graphics processing unit using said central processing unit. ### Claim 16 (depends on 14) The method according to claim 14, wherein said subroutines comprise a first transform subroutine for intra blocks and a second transform subroutine for inter predicted blocks. ### Claim 17 (depends on 14) The method according to claim 14, wherein said first and said second sets of computer executable instructions are configured to encode a picture using multiple graphics processing unit threads and two central processing unit threads. ### Claim 18 (depends on 14) The method according to claim 14, further comprising: using a texture cache and interpolation hardware to provide two-dimensional reads from said globally shared memory; using a decimation subroutine to prepare frames for a parallel pyramid search; and keeping frames in the globally shared memory of the graphics processing unit to avoid copying the frames back and forth to the central processing unit. ### Claim 19 (depends on 12) The method according to claim 12, wherein said encoding of said frame of video is H.264 compliant. ### Claim 20 A non-transitory computer readable medium storing computer executable instructions comprising: a first set of computer executable instructions, which, when loaded into a first memory that is accessible by a graphics processing unit and executed, perform video compression steps including prediction, decimation, transformation and quantization, and inverse-quantization and inverse-transformation on a slice using an array of software-configurable general purpose parallel processors and a globally shared memory of said graphics processing unit; and a second set of computer executable instructions, which, when loaded into a second memory that is accessible by a central processing unit and executed, perform entropy encoding, rate control and management of said graphics processing unit using said central processing unit, wherein said rate control is performed by said central processing unit choosing at least one target parameter value for transformation and quantization as said video compression steps are performed by said graphics processing unit and said entropy encoding is performed on the slice by said central processing unit in response to a signal indicating the slice is ready for encoding.