# US 8,542,734 B2 · Multi-view video encoding apparatus and method > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 8,542,734 B2 - Document ID: US8542734B2 - Status: Granted - Kind: B2 - Application number: 11/653,903 - Filed: 2007-01-17 - Published: 2007-08-09 - Granted: 2013-09-24 - Earliest claimed: 2006-02-07 - Assignees: Samsung Electronics Co., Ltd. - Applicants: Tae-hee Kim - Inventors: Tae-hee Kim - Web page: https://patentgrep.com/patent/US8542734B2 ## CPC - H04N 19/107: between spatial and temporal predictive coding, e.g. picture refresh - H04N 19/152: by measuring the fullness of the transmission buffer - H04N 19/172: the region being a picture, frame or field - H04N 19/597: specially adapted for multi-view video sequence encoding - H04N 19/61: in combination with predictive coding - H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION ## Abstract A multi-view video encoding apparatus and a method thereof are provided. The apparatus includes: a multi-view video input unit which receives a multi-view video; an I frame location determination unit which determines a location of an I frame which is a basis for encoding the multi-view video according to characteristics of multi-view frames at a predetermined time; and an encoding unit which encodes the multi-view video based on the determined location of the I frame. When the apparatus and the method are used, the location of the I frame, which is a basis for compression-encoding a multi-view video, can be adaptively determined and not fixed, thereby enhancing prediction efficiency and compression rates. ## Claims ### Claim 1 A multi-view video encoding apparatus comprising: a multi-view video input unit which receives a multi-view video; an I frame location determination unit which designates a location of an I reference frame among I frames included in multi-view frames, which is a basis for encoding the multi-view video, according to characteristics of multi-view frames at an initial time; and an encoding unit which encodes the multi-view video based on the designated location of the I reference frame, wherein the I frame location determination unit and the encoding unit are implemented in a processor, and wherein the I frame location determination unit designates the location of the I reference frame by designating a first I frame, among the I frames at the initial time as the reference I frame and prediction-encoding remaining I frames at the initial time as P reference frames to determine a first compression efficiency, and designates a second I frame, among the I frames at the initial time as the reference I frame and prediction-encoding remaining I frames at the initial time as P reference frames to determine a second compression efficiency, and compares the first compression efficiency and the second compression efficiency in order to determine a highest compression efficiency. ### Claim 2 (depends on 1) The apparatus of claim 1, wherein the multi-view video input unit transmits the multi-view frames which include I frames at the initial time to the I frame location determination unit, and transmits a multi-view video input which include the multi-view frames and P frames predicted at each view point based on the multi-view frames for each viewpoint of a multi-view image to the encoding unit. ### Claim 3 (depends on 1) The apparatus of claim 1, wherein the I frame location determination unit designates the location of the I reference frame, which has the highest compression efficiency for predicting P reference frames when one I frame of the multi-view frames which include I frames at the initial time is designated as the I reference frame and encoded, and the remaining I frames of the multi-view frames obtained at the same time as the I reference frame are prediction-encoded as the P reference frames with reference to the I reference frame. ### Claim 4 (depends on 1) The apparatus of claim 1, wherein the I frame location determination unit selects one I frame from the multi-view frames which include I frames of each viewpoint at the initial time and encodes the selected I frame as the I reference frame, prediction-encodes remaining I frames of the multi-view frames of each viewpoint obtained at the same time as the I reference frame as P reference frames with reference to the I reference frame, adds numbers of bits used to encode the I reference frame and the P reference frames, stores respectively the added numbers of bits generated at each viewpoint, and designates a location of a frame that, when encoded as the I reference frame, results in the smallest number of bits used as the location of the I reference frame. ### Claim 5 (depends on 4) The apparatus of claim 4, wherein, when selecting one I frame from the multi-view frames which include I frames of each viewpoint at the initial time and encoding the selected I frame as the I reference frame, the I frame location determination unit sequentially selects each I frame of the multi-view frames which include I frames of each viewpoint at the initial time as the I reference frame and prediction-encodes the remaining multi-view frames based on the I reference frame, wherein, the I frame location determination unit selects a first I frame at a first viewpoint at the initial time as the I reference frame and encodes and prediction-encode frames at remaining viewpoints as the P reference frames with reference to the first I frame, adds numbers of bits used to encode, and stores a total number of bits, then selects an a second I frame at a second viewpoint at the initial time as the I reference frame and encodes and prediction-encode frames at remaining viewpoints as the P reference frames with reference to the second I frame, adds numbers of bits used to encode, and stores a total number of bits. ### Claim 6 (depends on 1) The apparatus of claim 1, wherein the I frame location determination unit prediction-encodes multi-view frames located before the designated location of the I reference frame as P reference frames in a reverse direction from the I reference frame and prediction-encodes multi-view frames located after the designated location of the I reference frame as the P reference frames in a forward direction from the I reference frame. ### Claim 7 (depends on 1) The apparatus of claim 1, wherein the encoding unit comprises a multi-view video arrangement unit which adds at least one multi-view frame for prediction encoding to which another multi-view frame at a viewpoint before or after the designated location of the I reference frame refers and arranges a bitstream for each viewpoint required to encode a video of each viewpoint. ### Claim 8 (depends on 7) The apparatus of claim 7, wherein the encoding unit further comprises an H. 264 encoding unit which performs an H. 264 encoding process using the arranged bitstream, wherein the H. 264 encoding unit encodes sequentially input bitstream of each viewpoint. ### Claim 9 (depends on 8) The apparatus of claim 8, wherein the encoding unit further comprises a multi-view video bitstream output unit which removes the added at least one multi-view frame from the encoded bitstream of each viewpoint. ### Claim 10 A multi-view video encoding method comprising: receiving a multi-view video; designating a location of an I reference frame among I frames included in multi-view frames, which is a basis for encoding the multi-view video, according to characteristics of multi-view frames at an initial time; and encoding the multi-view video based on the designated location of the I reference frame, wherein the designating the location of the I reference frame comprises designating a first I frame, among the I frames at the initial time as the reference frame and prediction-encoding remaining I frames at the initial time as P reference frames to determine a first compression efficiency, and designating a second I frame, among the I frames at the initial time as the reference I frame and prediction-encoding remaining I frames at the initial time as P reference frames to determine a second compression efficiency, and comparing the first compression efficiency and the second compression efficiency in order o determine a highest compression efficiency. ### Claim 11 (depends on 10) The method of claim 10, wherein the d designation of the location of the I reference frame comprises designating the location of the I reference frame, which has the highest compression efficiency for predicting P reference frames when one I frame of the multi-view frames which include I frames at the initial time is encoded as the I reference frame, and the remaining I frames of the multi-view frames obtained at the same time as the I reference frame are prediction-encoded as the P reference frames with reference to the I reference frame. ### Claim 12 (depends on 11) The method of claim 11, wherein the prediction-encoding the remaining multi-view frames placed at the same time as the I reference frame as the P reference frames comprises prediction-encoding multi-view frames located before the designated location of the I reference frame as the P reference frames in a reverse direction from the I reference frame and prediction-encoding multi-view frames located after the designated location of the I reference frame as the P reference frames in a forward direction from the I reference frame. ### Claim 13 (depends on 10) The method of claim 10, wherein the designation of the location of the I reference frame comprises: selecting one I frame from the multi-view frames which include I frames of each viewpoint at the initial time and encoding the selected I frame as the I reference frame, and prediction-encoding remaining I frames of the multi-view frames obtained at the same time as the I reference frame as P reference frames with reference to the I reference frame; adding numbers of bits used to encode the I reference frame and the P reference frames; storing the added numbers of bits generated at each viewpoint respectively; and designating a location of a frame that, when encoded as the I reference frame, results in the smallest number of bits used as the location of the I reference frame. ### Claim 14 (depends on 13) The method of claim 13, wherein the selecting one I frame from the multi-view frames which include I frames of each viewpoint at the initial time and encoding the selected I frame as the I reference frame comprises sequentially selecting each I frame of the multi-view frames which include I frames of each viewpoint at the initial time as the I reference frame and prediction-encoding the remaining multi-view frames based on the I reference frame, wherein, sequentially selecting comprises selecting a first I frame at a first viewpoint at the initial time as the I reference frame and encoding and prediction-encoding frames at remaining viewpoints as the P reference frames with reference to the first I frame, adding numbers of bits used to encode, and storing a total number of bits, then selecting an a second I frame at a second viewpoint at the initial time as the I reference frame and encoding and prediction-encoding frames at remaining viewpoints as the P reference frames with reference to the second I frame, adding numbers of bits used to encode, and storing a total number of bits. ### Claim 15 (depends on 10) The method of claim 10, wherein the encoding of the multi-view video comprises: adding at least one multi-view frame for prediction encoding to which another multi-view frame at a viewpoint before or after the designated location of the I reference frame refers; and arranging a bitstream for each viewpoint required to encode a video of each viewpoint. ### Claim 16 (depends on 15) The method of claim 15, wherein the encoding of the multi-view video further comprises performing H. 264 encoding using the arranged bitstream, wherein sequentially input bitstream of each viewpoint is encoded in the H. 264 encoding process. ### Claim 17 (depends on 16) The method of claim 16, wherein the encoding of the multi-view video further comprises removing the added at least one multi-view frame from the encoded bitstream of each viewpoint. ### Claim 18 A non-transitory computer-readable recording medium on which a program for executing a multi-view video encoding method comprising: receiving a multi-view video; designating a location of an I reference frame among I frames included in multi-view frames, which is a basis for encoding the multi-view video, according to characteristics of multi-view frames at an initial time; and encoding the multi-view video based on the designated location of the I reference frame, wherein the designating the location of the I reference frame comprises designating a first I frame, among the I frames at the initial time as the reference I frame and prediction-encoding remaining I frames at the initial time as P reference frames to determine a first compression efficiency, and designating a second I frame, among the I frames at the initial time as the reference I frame and prediction-encoding remaining I frames at the initial time as P reference frames to determine a second compression efficiency, and comparing the first compression efficiency and the second compression efficiency in order determine a highest compression efficiency. ### Claim 19 (depends on 18) The non-transitory computer-readable recording medium of claim 18, wherein the designation of the location of the I reference frame comprises designating the location of the I reference frame, which has the highest compression efficiency for predicting P reference frames when one I frame of the multi-view frames which include I frames at the initial time is encoded as the I reference frame, and the remaining I frames of the multi-view frames obtained at the same time as the I reference frame are prediction-encoded as the P reference frames with reference to the I reference frame. ### Claim 20 (depends on 19) The non-transitory computer-readable recording medium of claim 19, wherein the prediction-encoding the remaining multi-view frames placed at the same time as the I reference frame as the P reference frames comprises prediction-encoding multi-view frames located before the designated location of the I reference frame as the P reference frames in a reverse direction from the I reference frame and prediction-encoding multi-view frames located after the designated location of the I reference frame as the P reference frames in a forward direction from the I reference frame. ### Claim 21 (depends on 18) The non-transitory computer-readable recording medium of claim 18, wherein the designation of the location of the I frame comprises: selecting one I frame from the multi-view frames which include I frames of each viewpoint at the initial time and encoding the selected I frame as the I reference frame, and prediction-encoding remaining I frames of the multi-view frames obtained at the same time as the I reference frame as P reference frames with reference to the I reference frame; adding numbers of bits used to encode the I reference frame and the P reference frames; storing the added numbers of bits generated at each viewpoint respectively; and designating a location of a frame that, when encoded as the I reference frame, results in the smallest number of bits used as the location of the I reference frame. ### Claim 22 (depends on 21) The non-transitory computer-readable recording medium of claim 21, wherein the selecting one I frame from the multi-view frames which include I frames of each viewpoint at the initial time and encoding the selected I frame as the I reference frame comprises sequentially selecting each I frame of the multi-view frames which include I frames of each viewpoint at the predetermined time as the I reference frame and encoding the remaining multi-view frames based on the I reference frame, wherein, sequentially selecting comprises selecting a first I frame at a first viewpoint at the initial time as the I reference frame and encoding and prediction-encoding frames at remaining viewpoints as the P reference frames with reference to the first I frame, adding numbers of bits used to encode, and storing a total number of bits, then selecting an a second I frame at a second viewpoint at the initial time as the I reference frame and encoding and prediction-encoding frames at remaining viewpoints as the P reference frames with reference to the second I frame, adding numbers of bits used to encode, and storing a total number of bits. ### Claim 23 (depends on 18) The non-transitory computer-readable recording medium of claim 18, wherein the encoding of the multi-view video comprises: adding at least one multi-view frame for prediction encoding to which another multi-view frame at a viewpoint before or after the designated location of the I reference frame refers; and arranging a bitstream for each viewpoint required to encode a video of each viewpoint. ### Claim 24 (depends on 23) The non-transitory computer-readable recording medium of claim 23, wherein the encoding of the multi-view video further comprises performing H. 264 encoding using the arranged bitstream, wherein sequentially input bitstream of each viewpoint is encoded in the H. 264 encoding process. ### Claim 25 (depends on 24) The non-transitory computer-readable recording medium of claim 24, wherein the encoding of the multi-view video further comprises removing the added at least one multi-view frame from the encoded bitstream of each viewpoint.