Scalable FOV+ for VR 360 video delivery to remote end users

An encoding apparatus for generating a plurality of extended viewport streams of virtual reality (VR) data for delivery to a plurality of client devices, comprising:

• a processor connected to receive a VR video file, the processor configured for:
• segmenting a sphere defined in the VR video file projected in an equirectangular projection (ERP) format into at least first and second overlapping segments that overlap one another;
• and
• creating at least first and second extended viewport streams for the VR video file, the first extended viewport stream including respective encoded extended field of view (EFOV) frames corresponding to the first overlapping segment, the second extended viewport stream including respective encoded extended field of view (EFOV) frames corresponding to the second overlapping segment, by iteratively performing operations comprising:
• applying a rotation to the first overlapping segment of the sphere,
• cropping from the first rotated overlapping segment of the sphere,
• converting the cropped first overlapping segment to a respective first extended field of view (EFOV) frame,
• applying a rotation to the second overlapping segment of the sphere,
• cropping from the second rotated overlapping segment of the sphere, and
• converting the cropped second overlapping segment to a respective second extended field of view (EFOV) frame, and
• instructing a network to transmit the encoded extended field of view (EFOV) frames of the respective first and second extended viewport streams to one or more distribution devices,
• wherein the processor is further configured to create a plurality of instantaneous decoding refresh, IDR, shifted viewport streams for each of the at least first and second extended viewport streams by repeating each of the plurality of iterated operations starting in a consecutive frame of a group of pictures, GOP, of the VR video file, wherein the number of the plurality of IDR shifted viewport streams created for each of the at least first and second extended viewport streams equals a number of frames in the GOP such that each of the plurality of IDR shifted viewport streams of each of the at least first and second extended viewport streams includes an IDR frame at a different frame position in the GOP.

A method of generating a plurality of extended viewport streams of virtual reality (VR) data for delivery to a plurality of client devices, comprising:

• operating a processor of an encoding apparatus to perform operations comprising:
• reading from a VR video file;
• segmenting a sphere defined in the VR video file projected in an equirectangular projection, ERP format into at least first and second overlapping segments that overlap one another;
• creating a first extended viewport stream for the VR video file by iteratively performing operations comprising the following:
• applying a rotation to the first overlapping segment of the sphere,
• cropping from the first rotated overlapping segment of the sphere, and
• converting the cropped first overlapping segment to a respective first encoded extended field of view, EFOV, frame,
• creating a second extended viewport stream for the VR video file by iteratively performing operations comprising the following:
• applying a rotation to the second overlapping segment of the sphere,
• cropping from the second rotated overlapping segment of the sphere, and
• converting the cropped second overlapping segment to a respective second encoded extended field of view, EFOV, frame,
• and
• instructing the network to transmit the encoded extended field of view (EFOV) frames of the respective first and second extended viewport streams to one or more distribution devices,
• wherein the processor is further configured to segment the sphere into the at least first and second overlapping segments according to an EFOV frame area calculated according to a maximum angular velocity of a display used to present one of the first and second extended viewport streams.

A distribution device for delivering a selected one of a plurality of viewport streams of virtual reality (VR) data to each of a plurality of client devices, comprising:

• a processor configured to perform operations comprising:
• receiving, via a network, at least first and second extended viewport streams of a VR video file, the at least first and second extended viewport streams each comprising a sequence of encoded extended field of view (EFOV) frames created for a respective one of a plurality of rotated and cropped overlapping segments constituting a sphere defined in the VR video file, wherein the plurality of overlapping segments constituting the sphere overlap one another;
• and
• delivering the at least first and second extended viewport streams to a plurality of client devices connected to the distribution device via one or more other networks by performing the following for each of the plurality of client devices in each of a plurality of iterations:
• receiving a current orientation data of the respective client device,
• selecting one of the at least first and second extended viewport streams according to the received current orientation data, and
• transmitting the selected extended viewport stream to the respective client device,
• wherein for each of the plurality of client devices, the processor is further configured to select one of a plurality of instantaneous decoding refresh, IDR, shifted viewport streams received for each of the at least first and second extended viewport streams, each of the plurality of IDR shifted viewport streams received for each of the at least first and second extended viewport streams includes an IDR frame at a different frame position in the GOP, of the respective extended viewport stream,
• the processor being further configured to select the selected IDR shifted viewport stream according to the indication of the currently presented frame received from the respective client device in case, based on the current orientation data received from the respective client device, the processor being further configured to switch from a current extended viewport stream of the at least first and second extended viewport streams currently transmitted to the respective client device to an adjacent viewport stream of the at least first and second extended viewport streams,
• the adjacent extended viewport stream comprising EFOV frames created for an overlapping segment of the at least first and second overlapping segments which is adjacent to a current overlapping segment of the at least first and second overlapping segments encoded in the current extended viewport stream.