# US 9,438,658 B2 · Streaming with coordination of video orientation (CVO)

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 9,438,658 B2
- Document ID: US9438658B2
- Status: Granted
- Kind: B2
- Application number: 14/956,130
- Filed: 2015-12-01
- Published: 2016-03-24
- Granted: 2016-09-06
- Earliest claimed: 2012-10-26
- Assignees: Intel Corporation
- Applicants: Intel Corporation
- Inventors: Ozgur Oyman
- Web page: https://patentgrep.com/patent/US9438658B2

## CPC

- H04L 65/65: Network streaming protocols, e.g. real-time transport protocol [RTP] or real-time control protocol [RTCP]
- H04L 67/02: based on web technology, e.g. hypertext transfer protocol [HTTP]
- H04L 67/04: specially adapted for terminals or networks with limited capabilities; specially adapted for terminal portability
- H04N 21/2343: involving reformatting operations of video signals for distribution or compliance with end-user requests or end-user device requirements {(video transcoding H04N19/40; media packet handling at the source H04L65/762)}
- H04N 21/23439: {for generating different versions}
- H04N 21/25808: {Management of client data (terminal profiles in network data switching protocols H04L67/303)}
- H04N 21/41407: {embedded in a portable device, e.g. video client on a mobile phone, PDA, laptop (constructional details of equipment or arrangements specially adapted for portable computer application G06F1/1626; arrangements specially adapted for mobile receivers in broadcast systems H04H20/57)}
- H04N 21/422: Input-only peripherals {, i.e. input devices connected to specially adapted client devices}, e.g. global positioning system [GPS] {(input devices also receiving signals from specially adapted client devices H04N21/4104)}
- H04N 21/64322: {IP}
- H04N 21/6582: {Data stored in the client, e.g. viewing habits, hardware capabilities, credit card number (arrangements where receivers interact with the broadcast H04H20/38)}
- H04N 21/8456: {by decomposing the content in the time domain, e.g. in time segments}
- H04N 21/85406: {involving a specific file format, e.g. MP4 format}
- H04W 88/02: Terminal devices
- Y02D 30/70: in wireless communication networks
- H04L: TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION (arrangements common to telegraphic and telephonic communication H04M)
- H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04W: WIRELESS COMMUNICATION NETWORKS (broadcast communication H04H; communication systems using wireless links for non-selective communication, e.g. wireless extensions H04M1/72)
- Y02D: CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE

## Abstract

Technology to provide streaming with coordination of video orientation (CVO) is disclosed. In an example, a server can include computer circuitry configured to: receive a device capability for a client; and modify streamed content to the client based on an inclusion of a CVO attribute in the device capability.

## Claims

### Claim 1

An apparatus of a server operable to provide streaming with coordination of video orientation (CVO), the apparatus comprising one or more processors and memory configured to:
process, at the server, a device capability exchange message received from a client, wherein the device capability exchange message includes a CVO attribute; and
modify, at the server, content to be streamed to the client based on the CVO attribute received in the device capability exchange message, wherein the content is modified by correcting a rendering orientation of a real-time transport protocol (RTP) stream or streamed dynamic adaptive streaming over hypertext transfer protocol (HTTP) (DASH) content for misalignment prior to delivery to the client when the CVO attribute included in the device capability exchange message indicates that the client is not an orientation-aware terminal.

### Claim 2 (depends on 1)

The apparatus of claim 1, wherein the one or more processors and memory are further configured to:
process, at the server for transmission to the client, a media presentation description (MPD) metadata file for a streamed dynamic adaptive streaming over hypertext transfer protocol (HTTP) (DASH) content with a CVO indication attribute provided in a codecs common attribute or element for an AdaptationSet, Representation, or SubRepresentation, wherein the codecs attribute is set to ‘3gvo’ indicating a presence of an oriented video component and associated CVO information in the streamed DASH content; and
process, at the server for transmission to the client, a session description protocol (SDP) file for a real-time transport protocol (RTP) stream with a CVO indication via an a=extmap attribute with a CVO uniform resource name (URN) “urn:3gpp:video-orientation” representing a 2-bit granularity for a CVO or “urn:3gpp:video-orientation:6” representing a 6-bit granularity for a Higher Granularity CVO on the CVO information contained in an RTP extension header.

### Claim 3 (depends on 1)

The apparatus of claim 1, wherein the one or more processors and memory are further configured to:
store CVO data in a third generation partnership project (3GPP) file format (3GP) file using a CVOSampleEntry for a CVO timed metadata track in a sample description box of International Organization for Standardization (ISO) file format box structure, wherein CVOSampleEntry fields include a BoxHeader size or type, a Data-reference-index, or a Granularity, and wherein the BoxHeader type is configured for a ‘3gvo’ value; or
store CVO data for a real-time transport protocol (RTP) stream in an RTP extension header.

### Claim 4 (depends on 1)

The apparatus of claim 1, wherein the one or more processors and memory are further configured to:
exchange a packet switched streaming service (PSS) capability, wherein the CVO attribute includes:
a StreamingCVOCapable attribute to indicate whether the client is a CVO capable receiver of real-time transport protocol (RTP) streams,
a StreamingHighGranularityCVOCapable attribute to indicate whether the client is a Higher Granularity CVO capable receiver of the RTP streams,
a ThreeGPCVOCapable attribute to indicate whether the client is a CVO capable receiver of third generation partnership project (3GPP) file format (3GP) files, or
a ThreeGPHighGranularityCVOCapable attribute to indicate whether the client is a Higher Granularity CVO capable receiver of the 3GP files.

### Claim 5 (depends on 1)

The apparatus of claim 1, wherein the one or more processors and memory are further configured to modify the streamed content by performing orientation-aware content adaptation, orientation-aware content selection, orientation-aware transcoding, or orientation-aware format conversion to correct a video orientation misalignment and ensure a content playback at the client with a correct video orientation.

### Claim 6 (depends on 1)

The apparatus of claim 1, wherein the server includes a third generation partnership project (3GPP) long term evolution (LTE) packet switched streaming service (PSS) server, a dynamic adaptive streaming over hypertext transfer protocol (HTTP) (DASH) server, or an integrated multimedia subsystem (IMS) based PSS and multimedia broadcast and multicast service (MBMS) (IMS_PSS_MBMS) server.

### Claim 7

An apparatus of a mobile terminal (MT) operable to provide coordination of video orientation (CVO) capability, the apparatus comprising one or more processors and memory configured to:
determine, at the MT, the CVO capability of the MT; and
process, at the MT, a streaming component attribute that includes the CVO capability of the MT for transmission to a server, wherein the server is configured to modify content to be streamed to the MT based on the CVO capability of the MT, wherein the server is configured to modify the content by correcting a rendering orientation of the content for misalignment when the CVO capability of the MT indicates that the MT is not an orientation-aware terminal.

### Claim 8 (depends on 7)

The apparatus of claim 7, further comprising:
a transceiver configured to receive a real-time transport protocol (RTP) extension header for an RTP stream or a third generation partnership project (3GPP) file format (3GP) file for a hypertext transfer protocol (HTTP) stream or dynamic adaptive streaming over HTTP (DASH),
wherein the one or more processors and memory are further configured to: parse a media presentation description (MPD) metadata file for the 3GP file for a CVO indication attribute or parse the 3GP file for embedded CVO information,
determine a orientation correction term based on the parsed CVO information and a current orientation of the MT, and
correct a rendering orientation of the HTTP stream or DASH for misalignment based on the determined orientation correction term when the MPD metadata file includes the CVO indication attribute and the MT is an orientation-aware terminal; or
parse a session description protocol (SDP) file for the RTP stream for the CVO indication attribute or parse the RTP extension header for the RTP stream for the embedded CVO information,
determine a orientation correction term based on the parsed CVO information and current orientation of the client device, and
correct a rendering orientation of the RTP stream for misalignment based on the determined orientation correction term when the SDP file includes the CVO indication attribute and the MT is the orientation-aware terminal, wherein correcting the rendering orientation compensates for rotation or flip of an orientation.

### Claim 9 (depends on 7)

The apparatus of claim 7, further comprising:
a transceiver configured to receive a media presentation description (MPD) metadata file from the server for streamed dynamic adaptive streaming over hypertext transfer protocol (HTTP) (DASH) content with a CVO indication attribute provided in a codecs common attribute or element for an AdaptationSet, Representation, or SubRepresentation, wherein the codecs attribute is set to ‘3gvo’ indicating a presence of an oriented video component and an associated CVO information in the streamed DASH content,
wherein the one or more processors and memory are further configured to:
parse the MPD metadata file for the CVO indication attribute; and modify a rendering orientation of a HTTP stream or the streamed DASH content for misalignment when the MPD metadata file includes the CVO indication attribute and the MT is an orientation-aware terminal.

### Claim 10 (depends on 7)

The apparatus of claim 7, further comprising:
a transceiver configured to receive a third generation partnership project (3GPP) file format (3GP) file including a CVOSampleEntry for a CVO timed metadata track in a sample description box of International Organization for Standardization (ISO) file format box structure, wherein CVOSampleEntry fields include a BoxHeader size or type, a Data-reference-index, or a Granularity, and wherein the BoxHeader type is configured for a 3gvo value, wherein the one or more processors and memory are further configured to: parse the 3GP file for the CVOSampleEntry; and
modify a rendering orientation of a hypertext transfer protocol (HTTP) stream or streamed dynamic adaptive streaming over HTTP (DASH) content for misalignment when the 3GP file includes the CVOSampleEntry and the MT is an orientation-aware terminal.

### Claim 11 (depends on 7)

The apparatus of claim 7, further comprising a transceiver configured to:
exchange a packet switched streaming service (PSS) client capability with the CVO capability of the MT, wherein the CVO capability of the MT includes:
a StreamingCVOCapable attribute to indicate whether the client is a CVO capable receiver of real-time transport protocol (RTP) streams,
a StreamingHighGranularityCVOCapable attribute to indicate whether the client is a Higher Granularity CVO capable receiver of RTP streams,
a ThreeGPCVOCapable attribute to indicate whether the client is a CVO capable receiver of third generation partnership project (3GPP) file format (3GP) files, or
a ThreeGPHighGranularityCVOCapable attribute to indicate whether the client is a Higher Granularity CVO capable receiver of 3GP files.

### Claim 12 (depends on 7)

The apparatus of claim 7, wherein the one or more processors and memory are further configured to:
capture user-generated content (UGC) video with a specified orientation; and embed CVO information for the UGC video in a third generation partnership project (3GPP) file format (3GP) file; and
process for transmission a 3GP file for a hypertext transfer protocol (HTTP) stream or dynamic adaptive streaming over HTTP (DASH).

### Claim 13 (depends on 7)

The apparatus of claim 7, wherein the one or more processors and memory are further configured to:
capture CVO data with a specified orientation; and
store the CVO data in a third generation partnership project (3GPP) file format (3GP) file using CVOSampleEntry for a CVO timed metadata track in a sample description box of International Organization for Standardization (ISO) file format box structure, wherein CVOSampleEntry fields include a BoxHeader size or type, a Data-reference-index, or a Granularity, and wherein the BoxHeader type is configured for a 3gvo value; and
process for transmission a 3GP file for a hypertext transfer protocol (HTTP) stream or dynamic adaptive streaming over HTTP (DASH).

### Claim 14 (depends on 7)

The apparatus of claim 7, wherein the CVO capability of the MT is provided in a third generation partnership project (3GPP) long term evolution (LTE) packet switched streaming service (PSS) session, a dynamic adaptive streaming over hypertext transfer protocol (HTTP) (DASH) session, or an integrated multimedia subsystem (IMS) based PSS and multimedia broadcast and multicast service (MBMS) (IMS_PSS_MBMS) session.

### Claim 15 (depends on 7)

The apparatus of claim 7, further comprising an orientation sensor to determine an orientation of the MT.

### Claim 16 (depends on 7)

The apparatus of claim 7, wherein the MT includes a user equipment (UE) or a mobile station (MS), and the mobile terminal includes an antenna, a camera, a touch sensitive display screen, a speaker, a microphone, a graphics processor, an application processor, internal memory, or a non-volatile memory port.

### Claim 17

At least one non-transitory machine readable storage medium having instructions embodied thereon for signaling coordination of video orientation (CVO) capability of a mobile terminal (MT) at a server, the instructions when executed perform the following:
processing, using one or more processors of the server, a device capability exchange message received from a client, wherein the device capability exchange message includes a CVO attribute; and
adapting, using the one or more processors of the server, content to be streamed to the client based on the CVO attribute received in the device capability exchange message, wherein the content is modified by correcting a rendering orientation of a real-time transport protocol (RTP) stream or streamed dynamic adaptive streaming over hypertext transfer protocol (HTTP) (DASH) content for misalignment prior to delivery to the client when the CVO attribute included in the device capability exchange message indicates that the client is not an orientation-aware terminal.

### Claim 18 (depends on 17)

The at least one non-transitory machine readable storage medium of claim 17, wherein adapting the streamed content further comprises:
embedding a CVO indication attribute in a media presentation description (MPD) metadata file or a session description protocol (SDP) file when the device capability for the client includes the CVO attribute indicating that the client is an orientation-aware terminal to modify the display orientation of a stream.

### Claim 19 (depends on 17)

The at least one non-transitory machine readable storage medium of claim 17, wherein adapting the streamed content further comprises: processing, at the server for transmission to the client, a media presentation description (MPD) metadata file for a streamed dynamic adaptive streaming over hypertext transfer protocol (HTTP) (DASH) content with a CVO indication attribute provided in a codecs common attribute or element for an AdaptationSet, Representation, or SubRepresentation, wherein the codecs attribute is set to ‘3gvo’ indicating a presence of an oriented video component and associated CVO information in the stream DASH content; and processing, at the server for transmission to the client, a session description protocol (SDP) file for a real-time transport protocol (RTP) stream with a CVO indication via an a=extmap attribute with a CVO uniform resource name (URN) urn:3gpp:video-orientation” representing a 2-bit granularity for a CVO or “urn:3gpp:video-orientation:6” representing a 6-bit granularity for a Higher Granularity CVO on the CVO information contained in a RTP extension header.

### Claim 20 (depends on 17)

The at least one non-transitory machine readable storage medium of claim 17, wherein adapting the streamed content further comprises:
processing, at the server, user-generated content (UGC) video received from the client, wherein the UGC video includes embedded CVO information for the UGC video in a third generation partnership project (3GPP) file format (3GP) file, wherein the 3GP file uses:
a CVOSampleEntry for a CVO timed metadata track in a sample description box of International Organization for Standardization (ISO) file format box structure, wherein CVOSampleEntry fields include a BoxHeader size or type, a Data-reference-index, or a Granularity, and wherein the BoxHeader type is configured for a 3gvo value; or
a codecs common attribute or element for an AdaptationSet, Representation, or SubRepresentation of a media presentation description (MPD), wherein the codecs attribute is set to ‘3gvo’ indicating a presence of an oriented video component and associated CVO information in a streamed dynamic adaptive streaming over hypertext transfer protocol (HTTP) (DASH) content.

### Claim 21 (depends on 17)

The at least one non-transitory machine readable storage medium of claim 17, wherein adapting the streamed content further comprises:
storing CVO data in a third generation partnership project (3GPP) file format (3GP) file using a CVOSampleEntry for a CVO timed metadata track in a sample description box of International Organization for Standardization (ISO) file format box structure, wherein CVOSampleEntry fields include a BoxHeader size or type, a Data-reference-index, or a Granularity, and wherein the BoxHeader type is configured for a 3gvo value; or
storing CVO data for a real-time transport protocol (RTP) stream in a RTP extension header.

### Claim 22 (depends on 17)

The at least one non-transitory machine readable storage medium of claim 17, wherein processing the device capability exchange message received from a client further comprises:
processing a packet switched streaming service (PSS) client capability received from the client, wherein the CVO capability includes:
a StreamingCVOCapable attribute to indicate whether the client is a CVO capable receiver of real-time transport protocol (RTP) streams, a StreamingHighGranularityCVOCapable attribute to indicate whether the client is a Higher Granularity CVO capable receiver of RTP streams, a ThreeGPCVOCapable attribute to indicate whether the client is a CVO capable receiver of third generation partnership project (3GPP) file format (3GP) files, or a ThreeGPHighGranularityCVOCapable attribute to indicate whether the client is a Higher Granularity CVO capable receiver of 3GP files.