Re-projecting flat projections of pictures of panoramic video for rendering by application

A computer system comprising one or more processing units and memory, wherein the computer system implements a panoramic video playback system that includes:

• a view-dependent operation controller configured to:
• receive an indication of a view direction for an application;
• based at least in part on the view direction, identify a section of a picture of panoramic video in an input flat projection, the picture of panoramic video in the input flat projection being based on sample values of different camera views recorded at a given time in multiple directions around a central position, wherein the identified section includes a portion of the picture of panoramic video in the input flat projection;
• and
• define a bounding box, for the identified section, around the portion of the picture of panoramic video in the input flat projection, the bounding box including a buffer area outside the portion of the picture of panoramic video in the input flat projection;
• a streaming controller configured to request encoded data for one or more spatial regions, wherein the bounding box aligns with boundaries of the one or more spatial regions or fits within the one or more spatial regions, and wherein the one or more spatial regions are slices or tiles;
• an input buffer configured to store the encoded data;
• a video decoder configured to decode at least some of the encoded data, thereby producing sample values of the identified section, the sample values of the identified section including sample values of the portion of the picture of panoramic video in the input flat projection and sample values of the buffer area;
• and
• a mapper configured to map at least some of the sample values of the identified section in the input flat projection to an output flat projection, wherein the mapper is configured to re-project the at least some of the sample values of the identified section in the input flat projection towards a center location for the view direction.

In a computer system, a method comprising:

• receiving an indication of a view direction for an application;
• based at least in part on the view direction, identifying a section of a picture of panoramic video in an input flat projection, the picture of panoramic video in the input flat projection being based on sample values of different camera views recorded at a given time in multiple directions around a central position, wherein the identified section includes a portion of the picture of panoramic video in the input flat projection;
• defining a bounding box, for the identified section, around the portion of the picture of panoramic video in the input flat projection, the bounding box including a buffer area outside the portion of the picture of panoramic video in the input flat projection;
• receiving encoded data for one or more spatial regions, wherein the bounding box aligns with boundaries of the one or more spatial regions or fits within the one or more spatial regions, and wherein the one or more spatial regions are slices or tiles;
• decoding at least some of the encoded data, thereby producing sample values of the identified section, the sample values of the identified section including sample values of the portion of the picture of panoramic video in the input flat projection and sample values of the buffer area;
• and
• mapping at least some of the sample values of the identified section in the input flat projection to an output flat projection, including re-projecting the at least some of the sample values of the identified section in the input flat projection towards a center location for the view direction.

One or more computer-readable media storing computer-executable instructions for causing a computer system, when programmed thereby, to perform operations comprising:

• receiving an indication of a view direction for an application;
• based at least in part on the view direction, identifying a section of a picture of panoramic video in an input flat projection, the picture of panoramic video in the input flat projection being based on sample values of different camera views recorded at a given time in multiple directions around a central position, wherein the identified section includes a portion of the picture of panoramic video in the input flat projection;
• defining a bounding box, for the identified section, around the portion of the picture of panoramic video in the input flat projection, the bounding box including a buffer area outside the portion of the picture of panoramic video in the input flat projection;
• receiving encoded data for one or more spatial regions, wherein the bounding box aligns with boundaries of the one or more spatial regions or fits within the one or more spatial regions, and wherein the one or more spatial regions are slices or tiles;
• decoding at least some of the encoded data, thereby producing sample values of the identified section, the sample values of the identified section including sample values of the portion of the picture of panoramic video in the input flat projection and sample values of the buffer area;
• and
• mapping at least some of the sample values of the identified section in the input flat projection to an output flat projection, including re-projecting the at least some of the sample values of the identified section in the input flat projection towards a center location for the view direction.