# US 10,873,754 B2 · Low latency video codec and transmission with parallel processing > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 10,873,754 B2 - Document ID: US10873754B2 - Status: Granted - Kind: B2 - Application number: 16/216,967 - Filed: 2018-12-11 - Published: 2019-06-13 - Granted: 2020-12-22 - Earliest claimed: 2017-12-12 - Assignees: Coherent Logix, Incorporated - Applicants: Coherent Logix, Incorporated - Inventors: Michael W. Bruns; Martin A. Hunt; Manjunath H. Siddaiah; John C. Sievers - Web page: https://patentgrep.com/patent/US10873754B2 ## CPC - H04N 19/436: using parallelised computational arrangements - G06F 9/3877: {using a secondary processor, e.g. coprocessor (peripheral processor G06F13/12)} - H04N 19/107: between spatial and temporal predictive coding, e.g. picture refresh - H04N 19/146: Data rate or code amount at the encoder output - H04N 19/147: according to rate distortion criteria (rate-distortion as a criterion for motion estimation H04N19/567) - H04N 19/176: the region being a block, e.g. a macroblock - H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) ## Abstract Methods and devices are described for a parallel multi-processor encoder system for encoding video data, wherein the video data comprises a sequence of frames, wherein each frame comprises a plurality of blocks of pixels in sequential rows. For each frame, the system may divide the plurality of blocks into a plurality of subsets of blocks, wherein each subset of blocks is allocated to a respective processor of the parallel multi-processor system. Each respective processor of the parallel multi-processor system may sequentially encode rows of the subset of blocks allocated to the respective processor and sequentially transmit each encoded row of blocks as a bit stream to a decoder on a channel. For each row, the respective encoded row of blocks may be transmitted to the decoder for each processor prior to transmission of the next sequential respective encoded row of blocks for any processor. Additionally, a similar parallel multi-processor decoder system is described. ## Claims ### Claim 1 A method for encoding video data, wherein the video data comprises a sequence of frames, wherein each frame comprises a plurality of blocks of pixels in sequential rows, the method comprising: for each frame: dividing the plurality of blocks into a plurality of subsets of blocks, wherein each subset of blocks is allocated to a respective processor of a parallel multi-processor system; by each respective processor of the parallel multi-processor system: sequentially encoding rows of the subset of blocks allocated to the respective processor, wherein said encoding comprises encoding non-intra-refresh blocks with or without reference to a previous encoded frame, and wherein intra-refresh blocks are encoded without reference to the previous encoded frame; and sequentially transmitting each encoded row of blocks as a bit stream to a decoder on a channel; and implementing rate control for each row of blocks of the frame, independent of the division of the frame into subsets of blocks; and storing the encoded rows of the subsets of blocks as a respective encoded frame in a memory buffer accessible to the processors of the parallel multiprocessor system, wherein, for each row, the encoded row of blocks of the respective row is transmitted to the decoder for each processor prior to transmission of the encoded row of blocks of a row sequential to the respective row for any processor. ### Claim 2 (depends on 1) The method of claim 1, wherein a second subset of the plurality of blocks comprise intra-refresh blocks, wherein the remaining blocks of the plurality of blocks outside the second subset comprise non-intra-refresh blocks, and wherein the intra-refresh blocks are equitably distributed between the rows of the frame. ### Claim 3 (depends on 2) The method of claim 2, wherein, for each sequential frame, the blocks comprising the second subset of the plurality of blocks are allocated to a different column or columns of the sequential frame. ### Claim 4 (depends on 1) The method of claim 1, wherein said rate control serves to promote comparable coded picture buffer occupancy for each of the rows of the frame. ### Claim 5 (depends on 1) The method of claim 1, wherein said independent rate control is used to maintain a similar bit rate for each of the rows of the frame. ### Claim 6 (depends on 1) The method of claim 1, the method further comprising: performing video analysis to determine encoding hints for each row of blocks; and communicating the respective encoding hints for each row of blocks to each processor of the parallel multi-processor system. ### Claim 7 (depends on 1) The method of claim 1, wherein each subset of the plurality of blocks comprises one or more columns of blocks of their respective frame. ### Claim 8 (depends on 1) The method of claim 1, wherein sequentially transmitting each encoded row of blocks as the bit stream to the decoder comprises tagging the encoded row of blocks with an identifier of a column location of the encoded row of blocks within the frame and transmitting the encoded row of blocks to an encoded row buffer for transmission to the decoder. ### Claim 9 A parallel multi-processor video encoder system, comprising: a plurality of parallel processors coupled to respective non-transitory computer readable memory media; wherein the parallel multi-processor video encoder system is configured to encode video data comprising a sequence of frames, wherein each frame comprises a plurality of blocks of pixels in sequential columns, wherein, in encoding the video data, the parallel multi-processor video encoder system is configured to: for each frame: divide the plurality of blocks into a plurality of subsets of blocks, wherein each subset of blocks is allocated to a respective processor of the plurality of parallel processors; and implement rate control for each column of blocks of the frame, independent of the division of the frame into subsets of blocks; by each processor of the plurality of parallel processors: sequentially encode columns of the subset of blocks allocated to the respective processor, wherein non-intra-refresh blocks are encoded based on an encoded frame corresponding to a previous frame, and wherein intra-refresh blocks are encoded without reference to the encoded frame corresponding to the previous frame; and sequentially transmit each encoded column of blocks as a bit stream to a decoder on a channel; and store the encoded columns of the subsets of blocks as a respective encoded frame in a memory buffer accessible to plurality of parallel processors; wherein, for each column, the encoded column of blocks of the respective column is transmitted to the decoder for each processor prior to transmission of the encoded column of blocks of a column sequential to the respective column for any processor. ### Claim 10 (depends on 9) The parallel multi-processor video encoder system of claim 9, wherein said rate control serves to promote comparable coded picture buffer occupancy for each of the columns of the frame. ### Claim 11 (depends on 9) The parallel multi-processor video encoder system of claim 9, wherein said rate control serves to maintain a similar bit rate for each of the columns of the frame. ### Claim 12 (depends on 9) The parallel multi-processor video encoder system of claim 9, wherein a second subset of the plurality of blocks comprise intra-refresh blocks, wherein the remaining blocks of the plurality of blocks outside the second subset comprise non-intra-refresh blocks, and wherein the intra-refresh blocks are equitably distributed between the columns of the frame. ### Claim 13 (depends on 9) The parallel multi-processor video encoder system of claim 9, wherein the parallel multi-processor video encoder system is further configured to: perform video analysis to determine encoding hints for each column of blocks; and for each column of blocks, communicate the respective encoding hints for each column to each of the plurality of parallel processors. ### Claim 14 An encoder device, comprising: a non-transitory computer-readable memory medium; a plurality of processors of a parallel multi-processor system operably coupled to the memory medium; and a radio, wherein the encoder device is configured to: receive video data comprising a sequence of frames, wherein each frame comprises a plurality of blocks of pixels in sequential rows; for each frame: divide the plurality of blocks into a plurality of subsets of blocks, wherein each subset of blocks is allocated to a respective processor of the parallel multi-processor system; by each respective processor of the parallel multi-processor system: sequentially encode rows of the subset of blocks allocated to the respective processor, wherein said encoding comprises encoding non-intra-refresh blocks with or without reference to a previous encoded frame, and wherein intra-refresh blocks are encoded without reference to the previous encoded frame; and using the radio, sequentially transmit each encoded row of blocks as a bit stream to a decoder on a channel; and implement rate control for each row of blocks of the frame, independent of the division of the frame into subsets of blocks; and store the encoded rows of the subsets of blocks as a respective encoded frame in a memory buffer accessible to the processors of the parallel multiprocessor system, wherein, for each row, the encoded row of blocks of the respective row is transmitted to the decoder for each processor prior to transmission of the encoded row of blocks of a row sequential to the respective row for any processor. ### Claim 15 (depends on 14) The encoder device of claim 14, wherein a second subset of the plurality of blocks comprise intra-refresh blocks, wherein the remaining blocks of the plurality of blocks outside the second subset comprise non-intra-refresh blocks, and wherein the intra-refresh blocks are equitably distributed between the rows of the frame. ### Claim 16 (depends on 15) The encoder device of claim 15, wherein, for each sequential frame, the blocks comprising the second subset of the plurality of blocks are allocated to a different column or columns of the sequential frame. ### Claim 17 (depends on 14) The encoder device of claim 14, wherein said rate control serves to promote comparable coded picture buffer occupancy for each of the rows of the frame. ### Claim 18 (depends on 14) The encoder device of claim 14, wherein said independent rate control is used to maintain a similar bit rate for each of the rows of the frame. ### Claim 19 (depends on 14) The encoder device of claim 14, wherein the encoder device is further configured to: perform video analysis to determine encoding hints for each row of blocks; and communicate the respective encoding hints for each row of blocks to each processor of the parallel multi-processor system. ### Claim 20 (depends on 14) The encoder device of claim 14, wherein sequentially transmitting each encoded row of blocks as the bit stream to the decoder comprises tagging the encoded row of blocks with an identifier of a column location of the encoded row of blocks within the frame and transmitting the encoded row of blocks to an encoded row buffer for transmission to the decoder.