# US 9,438,818 B2 · Device and method for multimedia communications with picture orientation information > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 9,438,818 B2 - Document ID: US9438818B2 - Status: Granted - Kind: B2 - Application number: 13/918,477 - Filed: 2013-06-14 - Published: 2013-12-26 - Granted: 2016-09-06 - Earliest claimed: 2012-06-20 - Assignees: QUALCOMM Incorporated - Applicants: QUALCOMM Incorporated - Inventors: Ye-Kui Wang; Nikolai Konrad Leung - Web page: https://patentgrep.com/patent/US9438818B2 ## CPC - H04N 5/2628: {Alteration of picture size, shape, position or orientation, e.g. zooming, rotation, rolling, perspective, translation} - H04N 19/46: Embedding additional information in the video signal during the compression process (H04N19/517, H04N19/68, H04N19/70 take precedence) - H04N 19/527: Global motion vector estimation - H04N 19/70: characterised by syntax aspects related to video coding, e.g. related to compression standards - H04N 19/90: using coding techniques not provided for in groups H04N19/10-H04N19/85, e.g. fractals - H04N 21/23605: {Creation or processing of packetized elementary streams [PES]} - H04N 21/25825: {involving client display capabilities, e.g. screen resolution of a mobile phone (optimising the visualisation of content during browsing in the Internet G06F16/9577; processing of terminal status or physical abilities in wireless networks H04W8/22; authentication in wireless network security H04W12/06)} - H04N 21/41407: {embedded in a portable device, e.g. video client on a mobile phone, PDA, laptop (constructional details of equipment or arrangements specially adapted for portable computer application G06F1/1626; arrangements specially adapted for mobile receivers in broadcast systems H04H20/57)} - H04N 21/4223: Cameras (H04N23/00 takes precedence) - H04N 1/3877: {Image rotation} - H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION ## Abstract Systems, devices, and methods for capturing and displaying picture data including picture orientation information are described. In one innovative aspect, a method for transmitting media information is provided. The method includes obtaining picture or video information, said picture or video information including image data and orientation information of a media capture unit when the picture or video information is obtained. The method further includes encoding said picture or video information, wherein the orientation information is included in a first portion and the image data is included in a second portion, the second portion being encoded and the first portion being distinct from the second portion. The method also includes transmitting the first portion and the second portion. ## Claims ### Claim 1 A method for displaying media information, the method comprising: obtaining picture or video information, said picture or video information including at least one output picture and rotation information for the at least one output picture, the rotation information included in a header portion of the picture or video information and the at least one output picture included in a picture field portion of the picture or video information, the picture field portion being encoded and the header portion being distinct from the picture field portion; decoding at least one output picture included in the picture field portion of the picture or video information; identifying rotation data and a period for the rotation data based on the rotation information included in the header portion of the picture or video information, the period for the rotation data being identified based on a start time stamp of a picture to which the rotation applies, the start time stamp included in a start field, and an end time stamp of a picture after the end of the rotation period, the end time stamp included in an end field; and rotating the decoded at least one output picture in accordance with the identified rotation data and the identified period. ### Claim 2 (depends on 1) The method of claim 1, wherein the identified period includes at least one of a rotation start point and a rotation end point. ### Claim 3 (depends on 1) The method of claim 1, wherein the period identifies a packet sequence number of a packet including a picture to be rotated. ### Claim 4 (depends on 1) The method of claim 1, wherein the period identifies a presentation time of a picture to be rotated. ### Claim 5 (depends on 1) The method of claim 1, wherein the period is identified in a picture orientation information element included in said information, said information comprising a real time transport control protocol message. ### Claim 6 (depends on 1) The method of claim 1, wherein the period is identified in one or more extension headers included in said information, said information comprising a real time transport protocol message. ### Claim 7 (depends on 1) The method of claim 1, wherein the period is identified in one or more attributes included in a session description protocol file, said information comprising said session description protocol file. ### Claim 8 (depends on 7) The method of claim 7, wherein the session description protocol file is received via one or more of session initiation protocol, real time streaming protocol, and hypertext transfer protocol. ### Claim 9 (depends on 1) The method of claim 1, wherein the period is identified in a media presentation description, said information comprising said media presentation description. ### Claim 10 (depends on 9) The method of claim 9, wherein the period is identified for all or a sub-grouping of video streams of a media presentation associated with the media presentation description. ### Claim 11 (depends on 10) The method of claim 10, wherein the sub-grouping comprises at least one of a media presentation level grouping, a period level grouping, an adaptation set level grouping, a representation level grouping, and a sub-representation level grouping. ### Claim 12 (depends on 1) The method of claim 1, wherein the period is identified in one or more entries in an ISO base media file, said information comprising the ISO base media file. ### Claim 13 (depends on 12) The method of claim 12, wherein the one or more entries includes a sample description entry in a track associated with the picture or video information. ### Claim 14 (depends on 12) The method of claim 12, wherein the one or more entries includes a picture orientation information sample group entry. ### Claim 15 (depends on 12) The method of claim 12, wherein the one or more entries include a picture orientation information box entry, the box entry associated with at least a portion of the picture or video information. ### Claim 16 (depends on 1) The method of claim 1, further comprising receiving said picture or video information from a sender device. ### Claim 17 (depends on 16) The method of claim 16, wherein the sender device comprises at least one of a camera, a video source, a fixed picture capturing device, and a mobile device. ### Claim 18 (depends on 16) The method of claim 16, wherein rotating the at least one output picture occurs after decoding said picture or video information. ### Claim 19 (depends on 16) The method of claim 16, further comprising displaying the at least one output picture with substantially the same orientation as the orientation of said picture or video as originally captured by the sender device. ### Claim 20 (depends on 1) The method of claim 1, further comprising rotating the at least one output picture based on information for a target display. ### Claim 21 (depends on 20) The method of claim 20, wherein the information for a target display includes one or more of display dimension information, display orientation information, display resolution information, and display identification information. ### Claim 22 (depends on 21) The method of claim 21, further comprising obtaining information for a target display based at least in part on the display identification information. ### Claim 23 (depends on 1) The method of claim 1, wherein the processor is further configured to modify the at least one output picture based on the rotation data and period. ### Claim 24 (depends on 23) The method of claim 23, wherein the modification includes at least one of color correction, time correction, aspect correction, scaling, and cropping. ### Claim 25 An apparatus for displaying media information, the apparatus comprising: a memory unit configured to store picture or video information, said picture or video information including at least one output picture and rotation information for the at least one output picture, the rotation information included in a header portion of the picture or video information and the at least one output picture included in a picture field portion of the picture or video information, the picture field portion being encoded and the header portion being distinct from the picture field portion; a transport layer processor configured to: extract rotation data and a period for the rotation data from the rotation information included in the header portion, the period for the rotation data being identified based on a start time stamp of a picture to which the rotation applies, the start time stamp included in a start field, and an end time stamp of a picture after the end of the rotation period, the end time stamp included in an end field; and extract the at least one output picture from the picture field portion; a decoder configured to decode the extracted at least one output picture; and a presentation processor configured to generate a version of the decoded at least one output picture for display based on the identified rotation data and the identified period. ### Claim 26 (depends on 25) The apparatus of claim 25, wherein the identified period includes at least one of a rotation start point and a rotation end point. ### Claim 27 (depends on 25) The apparatus of claim 25, wherein the period identifies a packet sequence number of a packet including a picture to be rotated. ### Claim 28 (depends on 25) The apparatus of claim 25, wherein the period identifies a presentation time of a picture to be rotated. ### Claim 29 (depends on 25) The apparatus of claim 25, wherein the period is identified in a picture orientation information element included in said information, said information comprising a real time transport control protocol message. ### Claim 30 (depends on 25) The apparatus of claim 25, wherein the period is identified in one or more extension headers included in said information, said information comprising a real time transport protocol message. ### Claim 31 (depends on 25) The apparatus of claim 25, wherein the period is identified in one or more attributes included in a session description protocol file, said information comprising said session description protocol file. ### Claim 32 (depends on 31) The apparatus of claim 31, wherein the session description protocol file is received via one or more of session initiation protocol, real time streaming protocol, and hypertext transfer protocol. ### Claim 33 (depends on 25) The apparatus of claim 25, wherein the period is identified in a media presentation description, said information comprising said media presentation description. ### Claim 34 (depends on 33) The apparatus of claim 33, wherein the period is identified for all or a sub-grouping of video streams of a media presentation associated with the media presentation description. ### Claim 35 (depends on 34) The apparatus of claim 34, wherein the sub-grouping comprises at least one of a media presentation level grouping, a period level grouping, an adaptation set level grouping, a representation level grouping, and a sub-representation level grouping. ### Claim 36 (depends on 25) The apparatus of claim 25, wherein the period is identified in one or more entries in an ISO base media file, said information comprising the ISO base media file. ### Claim 37 (depends on 36) The apparatus of claim 36, wherein the one or more entries includes a sample description entry in a track associated with the video information. ### Claim 38 (depends on 36) The apparatus of claim 36, wherein the one or more entries includes a picture orientation information sample group entry. ### Claim 39 (depends on 36) The apparatus of claim 36, wherein the one or more entries includes a picture orientation information box entry, the box entry associated with at least a portion of the video information. ### Claim 40 (depends on 25) The apparatus of claim 25, further comprising a receiver configured to receive said picture or video information from a sender device. ### Claim 41 (depends on 40) The apparatus of claim 40, wherein the sender device comprises at least one of a camera, a video source, a fixed picture capturing device, and a mobile device. ### Claim 42 (depends on 40) The apparatus of claim 40, wherein the version of the decoded at least one output picture includes a substantially similar orientation as said picture or video as originally captured by the sender device. ### Claim 43 (depends on 25) The apparatus of claim 25, wherein the presentation processor is further configured to generate a version of the decoded at least one output picture based on the decoded at least one output picture. ### Claim 44 (depends on 25) The apparatus of claim 25, wherein the presentation processor is further configured to generate a rotated version of the decoded at least one output picture based on information for a target display. ### Claim 45 (depends on 44) The apparatus of claim 44, wherein the information for a target display includes one or more of display dimension information, display orientation information, display resolution information, and display identification information. ### Claim 46 (depends on 45) The apparatus of claim 45, wherein the presentation processor is configured to obtain the information for the target display based at least in part on the display identification information. ### Claim 47 (depends on 25) The apparatus of claim 25, wherein the presentation processor is further configured to modify the at least one output picture based on the rotation data and period. ### Claim 48 (depends on 47) The apparatus of claim 47, wherein the modification includes at least one of color correction, time correction, aspect correction, scaling, and cropping. ### Claim 49 A non-transitory computer readable storage medium comprising instructions executable by a processor of an apparatus, the instructions causing the apparatus to: obtain picture or video information, said picture or video information including at least one output picture and rotation information for the at least one output picture, the rotation information included in a header portion of the picture or video information and the at least one output picture included in a picture field portion of the picture or video information, the picture field portion being encoded and the header portion being distinct from the picture field portion; decode at least one output picture included in the picture field portion of the picture or video information; identify rotation data and a period for the rotation data based on the rotation information included in the header portion of the picture or video information, the period for the rotation data being identified based on a start time stamp of a picture to which the rotation applies, the start time stamp included in a start field, and an end time stamp of a picture after the end of the rotation period, the end time stamp included in an end field; and rotate the decoded at least one output picture in accordance with the identified rotation data and the identified period. ### Claim 50 An apparatus for displaying media information, the apparatus comprising: means for obtaining picture or video information, said picture or video information including at least one output picture and rotation information for the at least one output picture, the rotation information included in a header portion of the picture or video information and the at least one output picture included in a picture field portion of the picture or video information, the picture field portion being encoded and the header portion being distinct from the picture field portion; means for processing transport layer information configured to: extract rotation data and a period for the rotation data from the rotation information included in the header portion, the period for the rotation data being identified based on a start time stamp of a picture to which the rotation applies, the start time stamp included in a start field, and an end time stamp of a picture after the end of the rotation period, the end time stamp included in an end field; and extract the at least one output picture from the picture field portion; means for decoding configured to decode the extracted at least one output picture; and means for generating a display configured to generate a version of the decoded at least one output picture for display based on the identified rotation data and the identified period.