# US 10,484,710 B2 · Video encoding apparatus and method > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 10,484,710 B2 - Document ID: US10484710B2 - Status: Granted - Kind: B2 - Application number: 15/927,113 - Filed: 2018-03-21 - Published: 2019-09-26 - Granted: 2019-11-19 - Earliest claimed: 2018-03-21 - Assignees: Novatek Microelectronics Corp. - Applicants: Novatek Microelectronics Corp. - Inventors: Fan-Di Jou - Web page: https://patentgrep.com/patent/US10484710B2 ## CPC - H04N 19/573: Motion compensation with multiple frame prediction using two or more reference frames in a given prediction direction - H04N 19/147: according to rate distortion criteria (rate-distortion as a criterion for motion estimation H04N19/567) - H04N 19/159: Prediction type, e.g. intra-frame, inter-frame or bidirectional frame prediction - H04N 19/172: the region being a picture, frame or field - H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION ## Abstract A video encoding apparatus and method are provided. The video encoding apparatus includes a video encoding circuit and a control circuit. The video encoding circuit performs a video encoding operation on a video stream to produce an encoded stream. The control circuit controls the video coding circuit to perform the video encoding operation. The control circuit dynamically sets a current frame in the video stream to at least one of a long-term reference frame, a short-term reference frame, and a non-reference frame according to at least one of a playback latency control condition and an inter-frame correlation condition. The long-term reference frame and the short-term reference frame are used as the decoding reference frame in a video decoding operation. ## Claims ### Claim 1 A video encoding apparatus, comprising: a video encoding circuit, performing a video encoding operation on a video stream to generate an encoded stream; and a control circuit, controlling the video encoding circuit to perform the video encoding operation, and dynamically setting a current frame in the video stream to at least one of a long-term reference frame, a short-term reference frame and a non-reference frame according to at least one of a playback latency control condition and an inter-frame correlation condition, wherein the long-term reference frame and the short-term reference frame are used as a decoding reference frame in a video decoding operation, wherein the playback latency control condition comprises: forecasting a playback latency of a reference frame of the current frame in the video decoding operation, wherein the playback latency represents a time for decoding the reference frame of the current frame; determining the playback latency control condition as being satisfied when the playback latency is less than a latency threshold; and determining the playback latency control condition as being not satisfied when the playback latency is not less than the latency threshold. ### Claim 2 (depends on 1) The video encoding apparatus according to claim 1, wherein the inter-frame correlation condition comprises: estimating a similarity between the current frame and the reference frame; determining the inter-frame correlation condition as being satisfied when the similarity is less than a similarity threshold; and determining the inter-frame correlation condition as being not satisfied when the similarity is not less than the similarity threshold. ### Claim 3 (depends on 2) The video encoding apparatus according to claim 2, wherein the similarity is a sum of absolute difference between the current frame and the reference frame. ### Claim 4 (depends on 2) The video encoding apparatus according to claim 2, wherein the similarity is a total rate-distortion cost for encoding of the current frame. ### Claim 5 (depends on 2) The video encoding apparatus according to claim 2, wherein the similarity is a total bit-stream size of the current frame after being encoded. ### Claim 6 (depends on 1) The video encoding apparatus according to claim 1, wherein the current frame is a key P frame, and the control circuit sets the current frame to the long-term reference frame when both the playback latency control condition and the inter-frame correlation condition are satisfied or otherwise, does not set the current frame to the long-term reference frame. ### Claim 7 (depends on 1) The video encoding apparatus according to claim 1, wherein the current frame is a P frame, and the control circuit sets the current frame to the short-term reference frame when both the playback latency control condition and the inter-frame correlation condition are satisfied or otherwise, sets the current frame to the non-reference frame. ### Claim 8 A video encoding method, comprising: performing, by a video encoding circuit, a video encoding operation on a video stream to generate an encoded stream; controlling, by a control circuit, the video encoding circuit to perform the video encoding operation; and dynamically setting, by the control circuit, a current frame in the video stream to at least one of a long-term reference frame, a short-term reference frame and a non-reference frame according to at least one of a playback latency control condition and an inter-frame correlation condition, wherein the long-term reference frame and the short-term reference frame are used as a decoding reference frame in a video decoding operation, wherein the playback latency control condition comprises: forecasting a playback latency of a reference frame of the current frame in the video decoding operation, wherein the playback latency represents a time for decoding the reference frame of the current frame; determining the playback latency control condition as being satisfied when the playback latency is less than a latency threshold; and determining the playback latency control condition as being not satisfied when the playback latency is not less than the latency threshold. ### Claim 9 (depends on 8) The video encoding method according to claim 8, wherein the inter-frame correlation condition comprises: estimating a similarity between the current frame and the reference frame; determining the inter-frame correlation condition as being satisfied when the similarity is less than a similarity threshold; and determining the inter-frame correlation condition as being not satisfied when the similarity is not less than the similarity threshold. ### Claim 10 (depends on 9) The video encoding method according to claim 9, wherein the similarity is a sum of absolute difference between the current frame and the reference frame. ### Claim 11 (depends on 9) The video encoding method according to claim 9, wherein the similarity is a total rate-distortion cost for encoding of the current frame. ### Claim 12 (depends on 9) The video encoding method according to claim 9, wherein the similarity is a total bit-stream size of the current frame after being encoded. ### Claim 13 (depends on 8) The video encoding method according to claim 8, wherein the current frame is a key P frame, and the step of setting the current frame to the at least one of the long-term reference frame, the short-term reference frame and the non-reference frame comprises: setting, by the control circuit, the current frame to the long-term reference frame when both the playback latency control condition and the inter-frame correlation condition are satisfied or otherwise, not setting the current frame to the long-term reference frame. ### Claim 14 (depends on 8) The video encoding method according to claim 8, wherein the current frame is a P frame, and the step of setting the current frame to the at least one of the long-term reference frame, the short-term reference frame and the non-reference frame comprises: setting, by the control circuit, the current frame to the short-term reference frame when both the playback latency control condition and the inter-frame correlation condition are satisfied or otherwise, setting the current frame to the non-reference frame.