# US 9,883,213 B2 · Embedding encoded audio into transport stream for perfect splicing > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 9,883,213 B2 - Document ID: US9883213B2 - Status: Granted - Kind: B2 - Application number: 15/111,988 - Filed: 2015-02-10 - Published: 2016-12-15 - Granted: 2018-01-30 - Earliest claimed: 2014-02-10 - Assignees: Dolby International AB - Applicants: Dolby International AB - Inventors: Marc Roessler; Malte Schmidt - Web page: https://patentgrep.com/patent/US9883213B2 ## CPC - H04N 21/23424: {involving splicing one content stream with another content stream, e.g. for inserting or substituting an advertisement} - H04N 21/2335: {involving reformatting operations of audio signals, e.g. by converting from one coding standard to another (details of audio signal transcoding G10L19/173)} - H04N 21/23605: {Creation or processing of packetized elementary streams [PES]} - H04N 21/2362: Generation or processing of Service Information [SI] - H04N 21/6112: {involving terrestrial transmission, e.g. DVB-T} - H04N 21/6143: {involving transmission via a satellite} - H04N 21/6168: {involving cable transmission, e.g. using a cable modem} - H04N 21/6175: {involving transmission via Internet (broadcast-related systems characterised by the transmission system being the Internet H04H60/82)} - H04N 21/6181: {involving transmission via a mobile phone network (arrangements for providing broadcast or conference services to substation in data switching networks in combination with wireless systems H04L12/189; wireless uplink channel access H04W74/004)} - H04N 21/6187: {involving transmission via a telephone network, e.g. POTS} - H04N 21/6473: {Monitoring network processes errors (for recovering from a failure of a protocol instance or entity H04L69/40)} - H04N 21/64776: {for requesting retransmission, e.g. of data packets lost or corrupted during transmission from server (ARQ protocols H04L1/18; implementation or adaptation of Internet protocol [IP], of transmission control protocol [TCP] or of user datagram protocol [UDP] H04L69/16)} - H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION ## Abstract Methods for generating a transport stream (e.g., an MPEG-2 transport stream) such that the transport stream has at least one perfect splicing property, and/or such that the transport stream is indicative of at least one audio/video program and includes metadata indicative of whether the program has a perfect splicing property. Other aspects are methods for splicing such a transport stream, audio/video processing units (e.g., splicers) configured to perform any embodiment of the inventive method, and audio/video processing units which include a buffer memory which stores at least one segment of transport stream generated in accordance with any embodiment of the inventive method. ## Claims ### Claim 1 A method for generating a transport stream, including steps of: (a) providing, using a transport stream generator implemented by a processor, encoded audio data and video data indicative of at least one program, wherein for each program of the at least one program including a given program, the encoded audio data and video data indicative of the given program includes frames of video data which determine at least one video elementary stream and frames of encoded audio data which determine at least one audio elementary stream, the frames of video data include I-frames of video data, and the frames of encoded audio data include I-frames of encoded audio data; and (b) including the encoded audio data and the video data in the transport stream such that said transport stream has an essential perfect splicing property, where the essential perfect splicing property is that: for each program of the at least one program including the given program indicated by the transport stream, each video elementary stream of the given program has a frame rate which is the same as the frame rate of each encoded audio elementary stream of the given program, and for each video frame of the given program there is at least one corresponding encoded audio frame of the given program; and for each program of the at least one program including the given program indicated by the transport stream, for each video I-frame in each video elementary stream of the given program, there is at least one matching audio I-frame in an audio elementary stream of the given program, wherein the transport stream includes metadata that indicates that the transport stream has the essential perfect splicing property, and wherein the metadata does not identify splice points in the transport stream, wherein a splicer device uses the metadata to generate a warning when the transport stream has less than a full set of perfect splicing properties, and wherein including the encoded audio data and the video data in the transport stream includes: determining, by the transport stream generator, that the frame rate of each video elementary stream of the program is not the same as the frame rate of each encoded audio elementary stream of the program; and in response to the determining, re-encoding at least one elementary stream of the program when the frame rate for another elementary stream of the program has changed, so that the frame rate of each video elementary stream of the program is the same as the frame rate of each encoded audio elementary stream of the program after re-encoding. ### Claim 2 (depends on 1) The method of claim 1, wherein the transport stream is an MPEG-2 transport stream. ### Claim 3 (depends on 2) The method of claim 2, wherein the encoded audio data are compressed audio data having AC-4 format. ### Claim 4 (depends on 1) The method of claim 1, wherein step (b) is performed such that said transport stream also has at least one additional perfect splicing property. ### Claim 5 (depends on 4) The method of claim 4, wherein each said video elementary stream and each said audio elementary stream is packetized in packetized elementary stream (“PES”) packets, and the at least one additional perfect splicing property is that: for each program of the transport stream, each video I-frame in a video elementary stream of the program is PES packet aligned with an encoded audio I-frame of the program. ### Claim 6 (depends on 5) The method of claim 5, wherein the transport stream is an MPEG-2 transport stream. ### Claim 7 (depends on 4) The method of claim 4, wherein the at least one additional perfect splicing property is that: for each program of the transport stream, for each video I-frame of the program there is at least one matching audio I-frame, where each pair of I-frames comprising one said video I-frame and an audio I-frame which matches said video I-frame is an A/V I-frame pair, and where the I-frames of each said A/V I-frame pair have matching presentation time stamp (PTS) values. ### Claim 8 (depends on 7) The method of claim 7, wherein the transport stream is an MPEG-2 transport stream. ### Claim 9 (depends on 7) The method of claim 7, wherein the at least one additional perfect splicing property is that: for each program of the transport stream, the average bit rate for every hyperframe of the program is constant, where each said hyperframe is a sequence of audio frames starting with an audio I-frame of an A/V-I-frame pair of the program, and ending with the audio frame just previous to the next A/V I-Frame pair of the program. ### Claim 10 (depends on 9) The method of claim 9, wherein the transport stream is an MPEG-2 transport stream. ### Claim 11 (depends on 7) The method of claim 7, wherein the at least one additional perfect splicing property is that: for each program of the transport stream, each sequence of A/V frames that comprises an A/V I-frame pair of the program and the A/V frames that follow up to the next A/V I-frame pair of the program, is time-multiplexed to occur in a single, continuous segment of the transport stream. ### Claim 12 (depends on 7) The method of claim 7, wherein the at least one additional perfect splicing property is that: for each program of the transport stream, for each A/V I-frame pair of the program, sending of transport stream packets carrying the encoded audio data of said A/V I-frame pair does not start before the sending of transport stream packets carrying the video data of said A/V I-frame pair has started, and for an audio frame that occurs just before each A/V I-frame pair of the program, all audio data transport stream packets for said audio frame are sent before the sending of the last video data transport stream packet that precedes the A/V I-frame pair. ### Claim 13 (depends on 7) The method of claim 7, also including a step of: including in the transport stream additional metadata which explicitly indicates suitable splice points in the transport stream, and the additional metadata indicates only splice points which occur just before suitable A/V I-Frame pairs. ### Claim 14 (depends on 1) The method of claim 1, wherein the at least one program includes at least a first program and a second program, wherein the metadata indicates that the first program has the essential perfect splicing property, and wherein the metadata indicates that the second program does not have the essential perfect splicing property. ### Claim 15 (depends on 1) The method of claim 1, wherein the metadata indicates that the transport stream has at least one additional perfect splicing property. ### Claim 16 A method for transport stream splicing, including steps of: (a) providing, using a transport stream generator implemented by a processor, a transport stream having an essential perfect splicing property, where the transport stream is indicative of encoded audio data and video data, the encoded audio data and the video data are indicative of at least one program, for each program of the at least one program including a given program the encoded audio data and video data indicative of the given program includes frames of video data which determine at least one video elementary stream and frames of encoded audio data which determine at least one audio elementary stream, the frames of video data include I-frames of video data, and the frames of encoded audio data include I-frames of encoded audio data, and wherein the essential perfect splicing property is that: for each program of the at least one program including the given program indicated by the transport stream, each video elementary stream of the given program has a frame rate which is the same as the frame rate of each encoded audio elementary stream of the given program, and for each video frame of the given program there is at least one corresponding encoded audio frame of the given program; and for each program of the at least one program including the given program indicated by the transport stream, for each video I-frame in each video elementary stream of the given program, there is at least one matching audio I-frame in an audio elementary stream of the program; and (b) splicing the transport stream, using a splicer device, thereby generating a spliced transport stream, without modifying any encoded audio elementary stream of the transport stream, wherein the transport stream includes metadata that indicates that the transport stream has the essential perfect splicing property, and wherein the metadata does not identify splice points in the transport stream, wherein the splicer device uses the metadata to generate a warning when the transport stream has less than a full set of perfect splicing properties, and wherein providing the transport stream includes: determining, by the transport stream generator, that the frame rate of each video elementary stream of the program is not the same as the frame rate of each encoded audio elementary stream of the program; and in response to the determining, re-encoding at least one elementary stream of the program when the frame rate for another elementary stream of the program has changed, so that the frame rate of each video elementary stream of the program is the same as the frame rate of each encoded audio elementary stream of the program after re-encoding. ### Claim 17 (depends on 16) The method of claim 16, wherein the transport stream is an MPEG-2 transport stream. ### Claim 18 (depends on 17) The method of claim 17, wherein the encoded audio data are compressed audio data having AC-4 format. ### Claim 19 (depends on 16) The method of claim 16, wherein the transport stream also has at least one additional perfect splicing property. ### Claim 20 A system for generating a transport stream, said system including: an encoder, implemented by a processor, that encodes audio data and generates encoded audio data; and a transport stream generator, implemented by the processor that generates the transport stream in response to video data and the encoded audio data, such that the video data and the encoded audio data are included in said transport stream in such a manner that said transport stream has an essential perfect splicing property, the video data and the encoded audio data are indicative of at least one program, wherein for each program of the at least one program including a given program, the encoded audio data and video data indicative of the given program includes frames of video data which determine at least one video elementary stream and frames of encoded audio data which determine at least one audio elementary stream, the frames of video data include I-frames of video data, and the frames of encoded audio data include I-frames of encoded audio data, and where the essential perfect splicing property is that: for each program of the at least one program including the given program indicated by the transport stream, each video elementary stream of the given program has a frame rate which is the same as the frame rate of each encoded audio elementary stream of the given program, and for each video frame of the given program there is at least one corresponding encoded audio frame of the given program; and for each program of the at least one program including the given program indicated by the transport stream, for each video I-frame in each video elementary stream of the given program, there is at least one matching audio I-frame in an audio elementary stream of the given program, wherein the transport stream includes metadata that indicates that the transport stream has the essential perfect splicing property, and wherein the metadata does not identify splice points in the transport stream, wherein a splicer device uses the metadata to generate a warning when the transport stream has less than a full set of perfect splicing properties, and wherein including the encoded audio data and the video data in the transport stream includes: determining that the frame rate of each video elementary stream of the program is not the same as the frame rate of each encoded audio elementary stream of the program; and in response to the determining, re-encoding at least one elementary stream of the program when the frame rate for another elementary stream of the program has changed, so that the frame rate of each video elementary stream of the program is the same as the frame rate of each encoded audio elementary stream of the program after re-encoding. ### Claim 21 (depends on 20) The system of claim 20, wherein the transport stream is an MPEG-2 transport stream. ### Claim 22 (depends on 20) The system of claim 20, wherein the encoder encodes the audio data so that the encoded audio data are compressed audio data having AC-4 format. ### Claim 23 (depends on 20) The system of claim 20, wherein the transport stream generator generates the transport stream such that said transport stream also has at least one additional perfect splicing property.