# US 10,375,399 B2 · Methods and systems of generating a background picture for video coding > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 10,375,399 B2 - Document ID: US10375399B2 - Status: Granted - Kind: B2 - Application number: 15/134,183 - Filed: 2016-04-20 - Published: 2017-10-26 - Granted: 2019-08-06 - Earliest claimed: 2016-04-20 - Assignees: QUALCOMM Incorporated - Applicants: QUALCOMM Incorporated - Inventors: Lei Wang; Ying Chen; Ning Bi; Xuerui Zhang; Lei Ma; Chinchuan Chiu; Jianfeng Ren - Web page: https://patentgrep.com/patent/US10375399B2 ## CPC - H04N 19/159: Prediction type, e.g. intra-frame, inter-frame or bidirectional frame prediction - G06T 7/254: involving subtraction of images - H04N 19/105: Selection of the reference unit for prediction within a chosen coding or prediction mode, e.g. adaptive choice of position and number of pixels used for prediction - H04N 19/172: the region being a picture, frame or field - H04N 19/182: the unit being a pixel - H04N 19/23: with coding of regions that are present throughout a whole video segment, e.g. sprites, background or mosaic - H04N 19/58: Motion compensation with long-term prediction, i.e. the reference frame for a current frame not being the temporally closest one (H04N19/23 takes precedence) - H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION - G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL ## Abstract Techniques and systems are provided for generating a background picture. The background picture can be used for coding one or more pictures. For example, a method of generating a background picture includes generating a long-term background model for one or more pixels of a background picture. The long-term background model includes a statistical model for detecting long-term motion of the one or more pixels in a sequence of pictures. The method further includes generating a short-term background model for the one or more pixels of the background picture. The short-term background model detects short-term motion of the one or more pixels between two or more pictures. The method further includes determining a value for the one or more pixels of the background picture using the long-term background model and the short-term background model. ## Claims ### Claim 1 A method of generating a background picture for coding one or more pictures, the method comprising: generating a long-term background model for one or more pixels of a background picture, the long-term background model including a statistical model for modeling a background portion of a scene captured by a sequence of pictures; generating a short-term background model for the one or more pixels of the background picture, the short-term background model detecting short-term motion of the one or more pixels between two or more pictures, wherein short-term motion of a pixel at a pixel location in the two or more pictures is detected by determining an absolute difference between pixel values of the pixel at the pixel location in the two or more pictures; obtaining a picture, the picture including one or more pixels at one or more pixel locations corresponding to the one or more pixels of the background picture; and updating one or more values for the one or more pixels of the background picture using the long-term background model and the short-term background model, wherein a value of a pixel at a pixel location in the background picture is updated to a value of the pixel at the pixel location in the obtained picture when the long-term background model detects a long-term change in intensity within a long-term range and the short-term background model detects a short-term change in intensity within a short-term range, and wherein the background picture is used for coding at least one picture. ### Claim 2 (depends on 1) The method of claim 1, wherein the long-term background model is a single statistical model. ### Claim 3 (depends on 1) The method of claim 1, wherein the long-term range is based on a mean value of the long-term background for the pixel location and a variation from the mean value; and wherein the short-term change in intensity of the pixel at the pixel location ranges from no intensity change up to a threshold intensity change. ### Claim 4 (depends on 1) The method of claim 1, wherein the long-term background model and the short-term background model are generated to represent a single pixel. ### Claim 5 (depends on 1) The method of claim 1, wherein the long-term background model and the short-term background model are generated to represent a plurality of pixels. ### Claim 6 (depends on 1) The method of claim 1, wherein the one or more values for the one or more pixels of the background picture include one or more pixel intensity values. ### Claim 7 (depends on 1) The method of claim 1, further comprising: generating a foreground model, wherein the foreground model includes a mixture of statistical models representing one or more pixels of the sequence of pictures; determining long-term motion of the one or more pixels of the sequence of pictures using the long-term background model; determining a static state of the one or more pixels of the sequence of pictures using the short-term background model; and updating the long-term background model using a statistical model of the foreground model. ### Claim 8 (depends on 1) The method of claim 1, wherein the background picture is used for coding at least one random access picture. ### Claim 9 (depends on 1) The method of claim 1, further comprising providing the background picture to a video encoder. ### Claim 10 (depends on 9) The method of claim 9, further comprising: detecting an expiration of a period of time; and providing the background picture to the video encoder upon detecting expiration of the period of time. ### Claim 11 (depends on 9) The method of claim 9, further comprising: receiving a request for the background picture from the video encoder; and providing the background picture to the video encoder upon receiving the request for the background picture. ### Claim 12 (depends on 1) The method of claim 1, further comprising predicting at least a portion of a random access picture using the background picture as a reference picture. ### Claim 13 (depends on 1) The method of claim 1, wherein the long-term background model includes a Gaussian model. ### Claim 14 (depends on 13) The method of claim 13, wherein the short-term background model does not include a Gaussian model. ### Claim 15 An apparatus for generating a background picture for coding one or more pictures comprising: a memory configured to store video data; and a processor configured to: generate a long-term background model for one or more pixels of a background picture, the long-term background model including a statistical model for modeling a background portion of a scene captured by a sequence of pictures; generate a short-term background model for the one or more pixels of the background picture, the short-term background model detecting short-term motion of the one or more pixels between two or more pictures, wherein short-term motion of a pixel at a pixel location in the two or more pictures is detected by determining an absolute difference between pixel values of the pixel at the pixel location in the two or more pictures; obtain a picture, the picture including one or more pixels at one or more pixel locations corresponding to the one or more pixels of the background picture; and update one or more values for the one or more pixels of the background picture using the long-term background model and the short-term background model, wherein a value of a pixel at a pixel location in the background picture is updated to a value of the pixel at the pixel location in the obtained picture when the long-term background model detects a long-term change in intensity within a long-term range and the short-term background model detects a short-term change in intensity within a short-term range, and wherein the background picture is used for coding at least one picture. ### Claim 16 (depends on 15) The apparatus of claim 15, wherein the long-term background model is a single statistical model. ### Claim 17 (depends on 16) The apparatus of claim 16, wherein the long-term range is based on a mean value of the long-term background for the pixel location and a variation from the mean value; and wherein the short-term change in intensity of the pixel at the pixel location ranges from no intensity change up to a threshold intensity change. ### Claim 18 (depends on 15) The apparatus of claim 15, wherein the long-term background model and the short-term background model are generated to represent a single pixel. ### Claim 19 (depends on 15) The apparatus of claim 15, wherein the long-term background model and the short-term background model are generated to represent a plurality of pixels. ### Claim 20 (depends on 15) The apparatus of claim 15, further comprising: generating a foreground model, wherein the foreground model includes a mixture of statistical models representing one or more pixels of the sequence of pictures; determining long-term motion of the one or more pixels of the sequence of pictures using the long-term background model; determining a static state of the one or more pixels of the sequence of pictures using the short-term background model; and updating the long-term background model using a statistical model of the foreground model. ### Claim 21 (depends on 15) The apparatus of claim 15, wherein the background picture is used for coding at least one random access picture. ### Claim 22 (depends on 15) The apparatus of claim 15, further comprising providing the background picture to a video encoder. ### Claim 23 (depends on 22) The apparatus of claim 22, further comprising: detecting an expiration of a period of time; and providing the background picture to the video encoder upon detecting expiration of the period of time. ### Claim 24 (depends on 22) The apparatus of claim 22, further comprising: receiving a request for the background picture from the video encoder; and providing the background picture to the video encoder upon receiving the request for the background picture. ### Claim 25 (depends on 15) The apparatus of claim 15, further comprising predicting at least a portion of a random access picture using the background picture as a reference picture. ### Claim 26 A non-transitory computer readable medium having stored thereon instructions that, when executed by one or more processors, cause the one or more processor to: generate a long-term background model for one or more pixels of a background picture, the long-term background model including a statistical model for modeling a background portion of a scene captured by a sequence of pictures; generate a short-term background model for the one or more pixels of the background picture, the short-term background model detecting short-term motion of the one or more pixels between two or more pictures, wherein short-term motion of a pixel at a pixel location in the two or more pictures is detected by determining an absolute difference between pixel values of the pixel at the pixel location in the two or more pictures; obtain a picture, the picture including one or more pixels at one or more pixel locations corresponding to the one or more pixels of the background picture; and update one or more values for the one or more pixels of the background picture using the long-term background model and the short-term background model, wherein a value of a pixel at a pixel location in the background picture is updated to a value of the pixel at the pixel location in the obtained picture when the long-term background model detects a long-term change in intensity within a long-term range and the short-term background model detects a short-term change in intensity within a short-term range, and wherein the background picture is used for coding at least one picture. ### Claim 27 (depends on 26) The non-transitory computer readable medium of claim 26, wherein the long-term background model is a single statistical model. ### Claim 28 (depends on 27) The non-transitory computer readable medium of claim 27, wherein the long-term range is based on a mean value of the long-term background for the pixel location and a variation from the mean value; and wherein the short-term change in intensity of the pixel at the pixel location ranges from no intensity change up to a threshold intensity change. ### Claim 29 (depends on 26) The non-transitory computer readable medium of claim 26, further comprising instructions that, when executed by the one or more processors, cause the one or more processors to: generate a foreground model, wherein the foreground model includes a mixture of statistical models representing one or more pixels of the sequence of pictures; determine long-term motion of the one or more pixels of the sequence of pictures using the long-term background model; determine a static state of the one or more pixels of the sequence of pictures using the short-term background model; and update the long-term background model using a statistical model of the foreground model. ### Claim 30 (depends on 26) The non-transitory computer readable medium of claim 26, further comprising instructions that, when executed by the one or more processors, cause the one or more processors to predict at least a portion of a random access picture using the background picture as a reference picture.