# US 2018/0324355 A1 · Video encoding method and apparatus, device, and storage medium

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2018/0324355 A1
- Document ID: US20180324355A1
- Status: Published application
- Kind: A1
- Application number: 15/775,069
- Filed: 2016-11-23
- Published: 2018-11-08
- Earliest claimed: 2016-03-22
- Assignees: Tencent Technology (Shenzhen) Company Limited
- Applicants: Tencent Technology (Shenzhen) Company Limited
- Inventors: Yao WANG
- Web page: https://patentgrep.com/patent/US20180324355A1

## CPC

- H04N 23/698: for achieving an enlarged field of view, e.g. panoramic image capture
- H04N 19/176: the region being a block, e.g. a macroblock
- H04N 19/46: Embedding additional information in the video signal during the compression process (H04N19/517, H04N19/68, H04N19/70 take precedence)
- H04N 19/597: specially adapted for multi-view video sequence encoding
- H04N 21/21805: {enabling multiple viewpoints, e.g. using a plurality of cameras}
- H04N 21/2187: Live feed
- H04N 21/2343: involving reformatting operations of video signals for distribution or compliance with end-user requests or end-user device requirements {(video transcoding H04N19/40; media packet handling at the source H04L65/762)}
- H04N 21/4402: involving reformatting operations of video signals for household redistribution, storage or real-time display {(details of conversion of video standards at pixel level H04N7/01; video transcoding H04N19/40; adapting incoming signals to the display format of the display terminal G09G5/005; media handling at the source in data packet switching networks H04L65/764)}
- H04N 21/6587: Control parameters, e.g. trick play commands, viewpoint selection
- H04N 21/816: {involving special video data, e.g 3D video}
- H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION

## Abstract

A method and apparatus are provided. The method includes obtaining a panoramic video formed by plural panoramic picture frames, each being formed to record a scenery of a viewing angle. The panoramic video is divided into N viewing angle video sequences, each sequence being formed by areas that are in the panoramic picture frames, that do not overlap with each other, and that have a same horizontal viewing angle. The sum of the viewing angles of the N video sequences is a total horizontal viewing angle of the panoramic picture frame, N being a positive integer. The N video sequences are respectively encoded to obtain N respective groups of encoded data, and a viewing angle is determined. A viewing angle video sequence whose viewing angle comprises the determined viewing angle is selected, and encoded data of the selected viewing angle video sequence is transmitted.

## Claims

### Claim at position 1

1-26. (canceled)

### Claim 27

A method comprising:
obtaining, by at least one central processing unit (CPU), a panoramic video, the panoramic video being formed by a plurality of panoramic picture frames, each of the plurality of panoramic picture frames being a picture frame formed to record a scenery of a viewing angle;
dividing, by the at least one CPU, the panoramic video into N viewing angle video sequences, each viewing angle video sequence being formed by areas that are in the plurality of panoramic picture frames, that do not overlap with each other, and that have a same horizontal viewing angle, the sum of the horizontal viewing angles of the N viewing angle video sequences being a total horizontal viewing angle of the panoramic picture frame, N being a positive integer;
encoding, respectively by the at least one CPU, the N viewing angle video sequences to obtain N respective groups of encoded data;
determining, by the at least one CPU, a viewing angle; and
selecting, by the at least one CPU, a viewing angle video sequence, from among the N viewing angle video sequences, whose horizontal viewing angle comprises the determined viewing angle, and transmitting encoded data of the selected viewing angle video sequence.

### Claim 28 (depends on 27)

The method according to claim 27, wherein the dividing the panoramic video to N viewing angle video sequences comprises:
dividing, by the at least one CPU, each panoramic picture frame of the panoramic video to N areas according to a viewing angle division rule and the horizontal viewing angle; and
obtaining, by the at least one CPU, the N viewing angle video sequences by, for each video angle video sequence, combining all areas of the plurality of panoramic picture frames that have the same horizontal viewing angle to form the video angle video sequence.

### Claim 29 (depends on 27)

The method according to claim 27, wherein the method further comprises:
in response to determining, at a moment Ti, that the viewing angle changes, skipping, by the at least one CPU, transmitting encoded data of an ith area in the viewing angle video sequence, where i is a positive integer;
selecting, by the at least one CPU, an updated viewing angle video sequence, and transmitting encoded data of an ith area in the updated viewing angle video sequence to the user, wherein the updated viewing angle video sequence is a viewing angle video sequence whose horizontal viewing angle matches the changed viewing angle; and
updating, by the at least one CPU, i to i+1, and cyclically performing the step of transmitting encoded data of an ith area in the updated viewing angle video sequence to the user at the moment Ti.

### Claim 30 (depends on 27)

The method according to claim 27, wherein the method further comprises, after the selecting the viewing angle video sequence:
obtaining, by the at least one CPU, viewing angle video sequences corresponding to n consecutive horizontal viewing angles that are adjacent to the horizontal viewing angle and less than the horizontal viewing angle; and obtaining viewing angle video sequences corresponding to m consecutive horizontal viewing angles that are adjacent to the horizontal viewing angle and greater than the horizontal viewing angle, wherein n and m are both positive integers, and n+m≤N−1; and
sequentially transmitting, by the at least one CPU, encoded data of each n+m areas that are in the n+m viewing angle video sequences and that belong to a same panoramic picture frame.

### Claim 31 (depends on 27)

The method according to claim 27, wherein
in each viewing angle video sequence, encoded data of a j area is generated according to encoded data of a (j−1)th area; or encoded data of a jth area is generated according to encoded data of a (j−1)th area and encoded data of an lth area, where j and l are both positive integers, and l>j≥2; and
encoded data of any two areas of different viewing angle video sequences are irrelevant in a time domain.

### Claim 32 (depends on 27)

The method according to claim 27, wherein, for each area of the areas that comprises at least two macroblocks,
encoded data of a first macroblock of the at least two macroblocks is generated according to encoded data of a second macroblock of the at least two macroblocks that is adjacent to the first macroblock in a space domain; and
encoded data of two macroblocks that are of different areas and that are adjacent in a space domain is irrelevant.

### Claim 33

A method comprising:
transmitting, by at least one central processing unit (CPU), a viewing angle to an encoder;
receiving, by the at least one CPU, encoded data from the encoder, the encoded data being of a viewing angle video sequence selected from among N viewing angle video sequences according to the viewing angle that was transmitted, each of the N viewing angle video sequences formed by areas that are in a plurality of panoramic picture frames that comprise a panoramic video, that do not overlap with each other, and that have a same horizontal viewing angle, the sum of the horizontal viewing angles of the N viewing angle video sequences being a total horizontal viewing angle of the panoramic picture frame, N being a positive integer; and
decoding, by the at least one CPU, the encoded data that is received.

### Claim 34 (depends on 33)

The method according to claim 33, wherein the method further comprises:
sequentially receiving, by the at least one CPU, encoded data of each of n+m areas that are in n+m viewing angle video sequences and that belong to a same panoramic picture frame, wherein the n+m viewing angle video sequences are viewing angle video sequences corresponding to n consecutive horizontal viewing angles that are adjacent to the horizontal viewing angle and less than the horizontal viewing angle, and viewing angle video sequences corresponding to m consecutive horizontal viewing angles that are adjacent to the horizontal viewing angle and greater than the horizontal viewing angle, n and m are both positive integers, and n+m≤N−1.

### Claim 35 (depends on 34)

The method according to claim 34, wherein the decoding the encoded data comprises:
decoding, by the at least one CPU, encoded data that is received at a current moment and that is of an area in a viewing angle video sequence whose horizontal viewing angle matches the viewing angle, and playing the obtained area; and
buffering, by the at least one CPU, encoded data of n+m areas in the n+m viewing angle video sequences that is received at the current moment.

### Claim 36 (depends on 35)

The method according to claim 35, wherein, the method further comprises, in response to determining that the viewing angle changes at a moment Ti:
selecting, by the at least one CPU, encoded data of an ith area in an updated viewing angle video sequence from the buffered encoded data of the n+m areas, decoding the encoded data that is selected, and playing the obtained ith area,
wherein the updated viewing angle video sequence is a viewing angle video sequence whose horizontal viewing angle matches the changed viewing angle.

### Claim 37 (depends on 34)

The method according to claim 34, wherein the decoding the encoded data comprises:
decoding, by the at least one CPU, encoded data that is received at a current moment and that is of an area in a viewing angle video sequence whose horizontal viewing angle matches the viewing angle, and playing the obtained area; and
decoding, by the at least one CPU, encoded data of n+m areas in the n+m viewing angle video sequences that is received at the current moment, and buffering the obtained n+m areas.

### Claim 38 (depends on 37)

The method according to claim 37, wherein the method further comprises, in response to determining that the viewing angle changes at a moment Ti:
selecting, by the at least one CPU, an ith area in an updated viewing angle video sequence from the buffered n+m areas, and playing the ith area, wherein
the updated viewing angle video sequence is a viewing angle video sequence whose horizontal viewing angle matches the changed viewing angle.

### Claim 39

An apparatus comprising:
at least one memory configure to store computer program code; and
at least one processor configured to access the at least one memory and operate according to the computer program code, the program code comprising:
video obtaining code configured to cause at least one of said at least one processor to obtain a panoramic video, the panoramic video being formed by a plurality of panoramic picture frames, each of the plurality of panoramic picture frames being a picture frame formed to record a scenery of a viewing angle;
video division code configured to cause at least one of said at least one processor to divide the panoramic video into N viewing angle video sequences, each viewing angle video sequence being formed by areas that are in the plurality of panoramic picture frames, that do not overlap with each other, and that have a same horizontal viewing angle, the sum of the horizontal viewing angles of the N viewing angle video sequences being a total horizontal viewing angle of the panoramic picture frame, N being a positive integer;
video encoding code configured to cause at least one of said at least one processor to encode, respectively, the N viewing angle video sequences to obtain N respective groups of encoded data;
viewing angle determining code configured to cause at least one of said at least one processor to determine a viewing angle; and
first provision code configured to cause at least one of said at least one processor to select a viewing angle video sequence, from among the N viewing angle video sequences, whose horizontal viewing angle comprises the determined viewing angle, and transmit encoded data of the selected viewing angle video sequence.

### Claim 40 (depends on 39)

The apparatus according to claim 39, wherein the video division code comprises:
area division code configured to cause at least one of said at least one processor to divide each panoramic picture frame of the panoramic video to N areas according to a viewing angle division rule and the horizontal viewing angle; and
sequence generation code configured to cause at least one of said at least one processor to obtain the N viewing angle video sequences by, for each video angle video sequence, combining all areas of the plurality of panoramic picture frames that have the same horizontal viewing angle to form the video angle video sequence.

### Claim 41 (depends on 39)

The apparatus according to claim 39, wherein the first provision code is further configured to:
in response to determining, at a moment Ti, that the viewing angle changes, skip transmitting encoded data of an ith area in the viewing angle video sequence, where i is a positive integer;
select an updated viewing angle video sequence, and transmit encoded data of an ith area in the updated viewing angle video sequence to the user, wherein the updated viewing angle video sequence is a viewing angle video sequence whose horizontal viewing angle matches the changed viewing angle; and
update i to i+1, and cyclically transmit encoded data of an ith area in the updated viewing angle video sequence to the user at the moment Ti.

### Claim 42 (depends on 39)

The apparatus according to claim 39, wherein the apparatus further comprises:
sequence selection code configured to cause at least one of said at least one processor to, after the selecting the viewing angle video sequence:
obtain viewing angle video sequences corresponding to n consecutive horizontal viewing angles that are adjacent to the horizontal viewing angle and less than the horizontal viewing angle; and obtain viewing angle video sequences corresponding to m consecutive horizontal viewing angles that are adjacent to the horizontal viewing angle and greater than the horizontal viewing angle, wherein n and m are both positive integers, and n+m≤N−1; and
second provision code configured to cause at least one of said at least one processor to sequentially transmit encoded data of each n+m areas that are in the n+m viewing angle video sequences and that belong to a same panoramic picture frame.

### Claim 43 (depends on 39)

The apparatus according to claim 39, wherein
in each viewing angle video sequence, encoded data of a j area is generated according to encoded data of a (j−1)th area; or encoded data of a jth area is generated according to encoded data of a (j−1)th area and encoded data of an lth area, wherein j and l are both positive integers, and l>j≥2; and
encoded data of any two areas of different viewing angle video sequences are irrelevant in a time domain.

### Claim 44 (depends on 39)

The apparatus according to claim 39, wherein, for each area of the areas that comprises at least two macroblocks,
encoded data of a first macroblock of the at least two macroblocks is generated according to encoded data of a second macroblock of the at least two macroblocks that is adjacent to the first macroblock in a space domain; and
encoded data of two macroblocks that are of different areas and that are adjacent in a space domain is irrelevant.

### Claim 45

An apparatus comprising:
at least one memory configure to store computer program code; and
at least one processor configured to access the at least one memory and operate according to the computer program code, the program code comprising:
viewing angle provision code configured to cause at least one of said at least one processor to transmit a viewing angle to an encoder;
first receiving code configured to cause at least one of said at least one processor to receive encoded data from the encoder, the encoded data being of a viewing angle video sequence selected from among N viewing angle video sequences according to the viewing angle that was transmitted, each of the N viewing angle video sequences formed by areas that are in a plurality of panoramic picture frames that comprise a panoramic video, that do not overlap with each other, and that have a same horizontal viewing angle, the sum of the horizontal viewing angles of the N viewing angle video sequences being a total horizontal viewing angle of the panoramic picture frame, N being a positive integer; and
data decoding code configured to cause at least one of said at least one processor to decode the encoded data that is received.

### Claim 46 (depends on 45)

The apparatus according to claim 45, wherein the program code further comprises:
second receiving code configured to cause at least one of said at least one processor to sequentially receive encoded data of each of n+m areas that are in n+m viewing angle video sequences and that belong to a same panoramic picture frame, wherein the n+m viewing angle video sequences are viewing angle video sequences corresponding to n consecutive horizontal viewing angles that are adjacent to the horizontal viewing angle and less than the horizontal viewing angle, and viewing angle video sequences corresponding to m consecutive horizontal viewing angles that are adjacent to the horizontal viewing angle and greater than the horizontal viewing angle, n and m are both positive integers, and n+m≤N−1.

### Claim 47 (depends on 46)

The apparatus according to claim 46, wherein the data decoding code comprises:
first processing code receiving code configured to cause at least one of said at least one processor to decode encoded data that is received at a current moment and that is of an area in a viewing angle video sequence whose horizontal viewing angle matches the viewing angle, and play the obtained area; and
buffer encoded data of n+m areas in the n+m viewing angle video sequences that is received at the current moment.

### Claim 48 (depends on 47)

The apparatus according to claim 47, wherein, the data decoding code further comprises:
second processing code receiving code configured to cause at least one of said at least one processor to, in response to determining that the viewing angle changes at a moment Ti:
select encoded data of an ith area in an updated viewing angle video sequence from the buffered encoded data of the n+m areas, decode the encoded data that is selected, and play the obtained ith area,
wherein the updated viewing angle video sequence is a viewing angle video sequence whose horizontal viewing angle matches the changed viewing angle.

### Claim 49 (depends on 46)

The apparatus according to claim 46, wherein the data decoding code comprises:
third processing code receiving code configured to cause at least one of said at least one processor to:
decode encoded data that is received at a current moment and that is of an area in a viewing angle video sequence whose horizontal viewing angle matches the viewing angle, and
play the obtained area; and
decode encoded data of n+m areas in the n+m viewing angle video sequences that is received at the current moment, and buffering the obtained n+m areas.

### Claim 50 (depends on 49)

The apparatus according to claim 49, wherein the data decoding code further comprises:
fourth processing code receiving code configured to cause at least one of said at least one processor to, in response to determining that the viewing angle changes at a moment Ti:
select an ith area in an updated viewing angle video sequence from the buffered n+m areas, and playing the ith area,
wherein the updated viewing angle video sequence is a viewing angle video sequence whose horizontal viewing angle matches the changed viewing angle.

### Claim 51

A non-transitory computer readable storage medium storing computer program code that, when executed by a computer, causes the computer to perform:
transmitting a viewing angle to an encoder;
receiving encoded data from the encoder, the encoded data being of a viewing angle video sequence selected from among N viewing angle video sequences according to the viewing angle that was transmitted, each of the N viewing angle video sequences formed by areas that are in a plurality of panoramic picture frames that comprise a panoramic video, that do not overlap with each other, and that have a same horizontal viewing angle, the sum of the horizontal viewing angles of the N viewing angle video sequences being a total horizontal viewing angle of the panoramic picture frame, N being a positive integer; and
decoding the encoded data that is received.

### Claim 52

An encoder comprising:
at least one memory configure to store computer program code; and
at least one processor configured to access the at least one memory and operate according to the computer program code, the program code comprising:
video obtaining code configured to cause at least one of said at least one processor to obtain a panoramic video, the panoramic video being formed by a plurality of panoramic picture frames, each of the plurality of panoramic picture frames being a picture frame formed to record a scenery of a viewing angle;
video division code configured to cause at least one of said at least one processor to divide the panoramic video into N viewing angle video sequences, each viewing angle video sequence being formed by areas that are in the plurality of panoramic picture frames, that do not overlap with each other, and that have a same horizontal viewing angle, the sum of the horizontal viewing angles of the N viewing angle video sequences being a total horizontal viewing angle of the panoramic picture frame, N being a positive integer;
video encoding code configured to cause at least one of said at least one processor to encode, respectively, the N viewing angle video sequences to obtain N respective groups of encoded data;
viewing angle determining code configured to cause at least one of said at least one processor to determine a viewing angle; and
first provision code configured to cause at least one of said at least one processor to select a viewing angle video sequence, from among the N viewing angle video sequences, whose horizontal viewing angle comprises the determined viewing angle, and transmit encoded data of the selected viewing angle video sequence.

### Claim 53

A method comprising:
obtaining, by at least one central processing unit (CPU), a panoramic video formed by a plurality of panoramic picture frames, each of the plurality of panoramic picture frames formed to record a scenery of a panoramic viewing angle;
dividing, by the at least one CPU, the panoramic video into N viewing angle video sequences, each of the N viewing angle video sequences having a viewing angle that is a non-overlapping subset of the panoramic viewing angle, wherein a sum of the viewing angles of the N viewing angle video sequences totals the panoramic viewing angle, N being a positive integer;
encoding, by the at least one CPU, the N viewing angle video sequences;
determining, by the at least one CPU, a viewing angle;
selecting, by the at least one CPU, a viewing angle video sequence, from among the N viewing angle video sequences, whose horizontal viewing angle comprises the determined viewing angle; and
transmitting, by the at least one CPU, only encoded data of the selected viewing angle video sequence.
