# US 8,782,713 B2 · Systems and methods for encoding multimedia content > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 8,782,713 B2 - Document ID: US8782713B2 - Status: Granted - Kind: B2 - Application number: 13/742,523 - Filed: 2013-01-16 - Published: 2013-08-29 - Granted: 2014-07-15 - Earliest claimed: 2008-09-16 - Assignees: Intel Corporation - Applicants: Intel Corporation - Inventors: Hari Kalva; Atul Puri - Web page: https://patentgrep.com/patent/US8782713B2 ## CPC - H04N 21/23412: {for generating or manipulating the scene composition of objects, e.g. MPEG-4 objects} - G06F 3/0481: based on specific properties of the displayed interaction object or a metaphor-based environment, e.g. interaction with desktop elements like windows or icons, or assisted by a cursor's changing behaviour or appearance - G06T 5/50: using two or more images, e.g. averaging or subtraction - G06T 11/60: Creating or editing images; Combining images with text - G11B 27/3081: {used signal is a video-frame or a video-field (P.I.P)} - H04N 5/45: Picture in picture {, e.g. displaying simultaneously another television channel in a region of the screen} - H04N 19/115: Selection of the code volume for a coding unit prior to coding - H04N 19/167: Position within a video image, e.g. region of interest [ROI] - H04N 19/61: in combination with predictive coding - H04N 21/234318: {by decomposing into objects, e.g. MPEG-4 objects} - H04N 21/234345: {the reformatting operation being performed only on part of the stream, e.g. a region of the image or a time segment} - H04N 21/44012: {involving rendering scenes according to scene graphs, e.g. MPEG-4 scene graphs} - H04N 21/440245: {the reformatting operation being performed only on part of the stream, e.g. a region of the image or a time segment} - H04N 21/47205: {for manipulating displayed content, e.g. interacting with MPEG-4 objects, editing locally} - H04N 21/4722: for requesting additional data associated with the content - H04N 21/4725: using interactive regions of the image, e.g. hot spots {(details of information retrieval from the Internet by using URLs G06F16/955; processing chained hypermedia data for information retrieval G06F16/94)} - H04N 21/4728: for selecting a Region Of Interest [ROI], e.g. for requesting a higher resolution version of a selected region - H04N 21/8543: using a description language, e.g. Multimedia and Hypermedia information coding Expert Group [MHEG], eXtensible Markup Language [XML] {(information retrieval of semistructured data, the underlying structure being taken into account, e.g. mark-up language structure data G06F16/80)} - H04N 21/8545: for generating interactive applications - G06T 2200/24: involving graphical user interfaces [GUIs] - G06T 2200/32: involving image mosaicing - H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) - G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL - G11B: INFORMATION STORAGE BASED ON RELATIVE MOVEMENT BETWEEN RECORD CARRIER AND TRANSDUCER (recording measured values in a way that does not require playback through a transducer G01D9/00; recording or playback apparatus using mechanically marked tape, e.g. punched paper tape, or using unit records, e.g. punched or magnetically marked cards G06K; transferring data from one type of record carrier to another G06K1/18; circuits for coupling output of reproducer to radio receiver H04B1/20; gramophone pick-ups or like acoustic electromechanical transducers or circuits therefor H04R) ## Abstract An interactive video/multimedia application (IVM application) may specify one or more media assets for playback. The IVM application may define the rendering, composition, and interactivity of one or more the assets, such as video. Video multimedia application data (IVMA data may) be used to define the behavior of the IVM application. The IVMA data may be embodied as a standalone file in a text or binary, compressed format. Alternatively, the IVMA data may be embedded within other media content. A video asset used in the IVM application may include embedded, content-aware metadata that is tightly coupled to the asset. The IVM application may reference the content-aware metadata embedded within the asset to define the rendering and composition of application display elements and user-interactivity features. The interactive video/multimedia application (defined by the video and multimedia application data) may be presented to a viewer in a player application. ## Claims ### Claim 1 An apparatus, comprising: processing logic to determine metadata associated with a region of interest (ROI) of at least one image frame of video content, wherein the ROI corresponds to at least one of a shape, object, motion vector, or scene identified in at least a portion of the image frame; encode the image frame to generate an encoded image frame, wherein encoding comprises encoding the ROI using a higher bitrate than a bitrate of another region of the image frame; and embed the encoded image frame and the metadata in a bitstream, wherein the bitstream includes syntax, and wherein the metadata is embedded according to the bitstream syntax to tightly couple the metadata to the encoded image frame to provide for metadata-to-content synchronization that is independent of playback performance, playback timing, and user interaction; memory coupled to the processing logic, the memory to store the at least one image frame. ### Claim 2 (depends on 1) The apparatus of claim 1, wherein the shape or object identified in the at least a portion of the image frame comprises a human face automatically detected by the processing logic in the at least a portion of the image frame. ### Claim 3 (depends on 2) The apparatus of claim 2, further comprising a display device coupled to the processing logic, the display device to display a multimedia object overlaid on the ROI based on the metadata. ### Claim 4 (depends on 1) The apparatus of claim 1, wherein the processing logic to associate a multimedia element with the metadata, wherein the multimedia element is to provide for user interactivity based on the metadata. ### Claim 5 (depends on 1) The apparatus of claim 1, wherein the metadata specifies a location of the ROI within the image frame. ### Claim 6 (depends on 1) The apparatus of claim 1, wherein the processing logic comprises at least one of a processor, a preprocessor, a computing device, a mobile computing device, a cell phone, a smart phone, a portable media player, a personal computer, a set top box, or a display device. ### Claim 7 A method comprising, at a processor: determining metadata associated with a region of interest (ROI) of at least one image frame of video content, wherein the ROI corresponds to at least one of a shape, object, motion vector, or scene identified in at least a portion of the image frame; encoding the image frame to generate an encoded image frame, wherein encoding comprises encoding the ROI using a higher bitrate than a bitrate of another region of the image frame; and embedding the metadata in a bitstream including the encoded image frame, wherein the bitstream includes syntax, and wherein the metadata is embedded according to the bitstream syntax to tightly couple the metadata to the encoded image frame to provide for metadata-to-content synchronization that is independent of playback performance, playback timing, and user interaction. ### Claim 8 (depends on 7) The method of claim 7, wherein the shape or object identified in the at least the portion of the image frame comprises a human face automatically detected in at least the portion of the image frame at the processor. ### Claim 9 (depends on 7) The method of claim 7, wherein a multimedia element is associated with the metadata. ### Claim 10 (depends on 9) The method of claim 9, wherein the multimedia element is to provide for user interactivity based on the metadata. ### Claim 11 (depends on 10) The method of claim 10, wherein to provide for user interactivity the multimedia element is to provide a user interface element. ### Claim 12 (depends on 9) The method of claim 9, wherein the multimedia element comprises at least one of video content, audio content, or graphical content; wherein the graphical content comprises at least one of an image, text, a rendered two-dimensional graphic, or a rendered three-dimensional graphic. ### Claim 13 (depends on 7) The method of claim 7, wherein the metadata comprises at least one of text data, formatted data, or compressed binary data. ### Claim 14 (depends on 7) The method of claim 7, wherein the metadata specifies a location of the ROI within the image frame. ### Claim 15 One or more non-transitory computer readable media comprising instructions to cause an apparatus, upon execution of the instructions by the apparatus, to: determine metadata associated with a region of interest (ROI) of at least one image frame of video content, wherein the ROI corresponds to at least one of a shape, object, motion vector, or scene identified in at least a portion of the image frame; encode the image frame to generate an encoded image frame, wherein encoding comprises encoding the ROI using a higher bitrate than a bitrate of another region of the image frame; and embed the encoded image frame and the metadata in a bitstream, wherein the bitstream includes syntax, and wherein the metadata is embedded according to the bitstream syntax to tightly couple the metadata to the encoded image frame to provide for metadata-to-content synchronization that is independent of playback performance, playback timing, and user interaction. ### Claim 16 (depends on 15) The one or more computer readable media of claim 15, wherein the shape or object identified in at least the portion of the image frame comprises a human face automatically detected in at least the portion of the image frame at the processor. ### Claim 17 (depends on 15) The one or more computer readable media of claim 15, wherein a multimedia element is associated with the metadata. ### Claim 18 (depends on 15) The one or more computer readable media of claim 15, wherein the multimedia element is to provide for user interactivity based on the metadata. ### Claim 19 (depends on 15) The one or more computer readable media of claim 15, wherein the multimedia element comprises at least one of an image, text, a rendered two-dimensional graphic, or a rendered three-dimensional graphic; wherein the graphical content comprises at least one of an image, text, a rendered two-dimensional graphic, or a rendered three-dimensional graphic. ### Claim 20 (depends on 15) The one or more computer readable media of claim 15, wherein the metadata specifies a location of the ROI within the image frame.