# US 7,020,195 B1 · Layered coding and decoding of image data > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 7,020,195 B1 - Document ID: US7020195B1 - Status: Granted - Kind: B1 - Application number: 09/458,300 - Filed: 1999-12-10 - Granted: 2006-03-28 - Earliest claimed: 1999-12-10 - Assignees: Microsoft Corporation - Inventors: Thomas L. McMahon - Web page: https://patentgrep.com/patent/US7020195B1 ## CPC - H04N 7/0122: {the input and the output signals having different aspect ratios} - H04N 19/33: in the spatial domain - H04N 21/234327: {by decomposing into layers, e.g. base layer and one or more enhancement layers} - H04N 21/234372: {for performing aspect ratio conversion} - H04N 21/4122: {additional display device, e.g. video projector (digital output for controlling a plurality of local displays G06F3/1423)} - H04N 21/4143: {embedded in a} Personal Computer [PC] - H04N 21/426: Internal components of the client {; Characteristics thereof} (H04N5/44 takes precedence) - H04N 21/440227: {by decomposing into layers, e.g. base layer and one or more enhancement layers} - H04N 21/440272: {for performing aspect ratio conversion} - H04N 21/4621: {Controlling the complexity of the content stream or additional data, e.g. lowering the resolution or bit-rate of the video stream for a mobile client with a small screen (arrangements for using the results of monitoring on user's side in broadcast systems H04H60/65; flow control in packet networks H04L47/10)} - H04N 21/43632: {involving a wired protocol, e.g. IEEE 1394 (high-speed IEEE 1394 serial bus H04L12/40052)} - H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION ## Abstract An image processing system generates a base layer that represents a low-resolution portion of a source image. The image processing system also generates an enhancement layer that represents a high-resolution portion of the source image. The base layer has an associated aspect ratio and the enhancement layer has an associated aspect ratio that differs from the aspect ratio associated with the base layer. The aspect ratio associated with the base layer corresponds to an aspect ratio associated with low-resolution televisions. The aspect ratio associated with the enhancement layer corresponds to an aspect ratio associated with high-resolution televisions. The base layer can be generated by low-pass filtering the source image and the enhancement layer can be generated by high-pass filtering the source image. The base layer can also be generated by subtracting a portion of the base layer from a corresponding portion of the source image. An image decoding system is used to identify the base layer and the enhancement layer from a transport stream and recreate the source image. ## Claims ### Claim 1 A method of encoding a high-resolution source image having a first aspect ratio, the method comprising: generating a base layer representing a low-resolution portion of the source image, wherein the base layer has the first aspect ratio; and generating an enhancement layer representing a high-resolution portion of the source image, wherein the enhancement layer has a second aspect ratio, and wherein the second aspect ratio differs from the first aspect ratio. ### Claim 2 (depends on 1) A method as recited in claim 1 wherein the first aspect ratio corresponds to an aspect ratio associated with low-resolution televisions. ### Claim 3 (depends on 1) A method as recited in claim 1 wherein the second aspect ratio corresponds to an aspect ratio associated with high-resolution televisions. ### Claim 4 (depends on 1) A method as recited in claim 1 wherein the first aspect ratio is 4:3. ### Claim 5 (depends on 1) A method as recited in claim 1 wherein the second aspect ratio is 16:9. ### Claim 6 (depends on 1) A method as recited in claim 1 wherein the step of generating a base layer includes low-pass filtering the source image. ### Claim 7 (depends on 1) A method as recited in claim 1 wherein the step of generating an enhancement layer includes subtracting a portion of the base layer from a corresponding portion of the source image. ### Claim 8 (depends on 1) A method as recited in claim 1 wherein the step of generating an enhancement layer comprises: determining an offset value; extracting a first image from the base layer based on the offset value, wherein the first image has the second aspect ratio, and wherein the first image is less than the entire image represented by the base layer; extracting a second image from the source image based on the offset value, wherein the second image also has the second aspect ratio, and wherein the second image is less than the entire image represented by the source image; and subtracting the first image from the second image. ### Claim 9 (depends on 8) A method as recited in claim 8, wherein the offset value indicates a vertical location from which the first image is extracted from the base layer. ### Claim 10 (depends on 8) A method as recited in claim 8, wherein the offset value indicates a horizontal location from which the first image is extracted from the base layer. ### Claim 11 (depends on 1) A method as recited in claim 1 wherein the step of generating an enhancement layer includes high-pass filtering the source image. ### Claim 12 (depends on 1) A method as recited in claim 1 further including combining the base layer and the enhancement layer into a single transport stream. ### Claim 13 (depends on 1) A method as recited in claim 1 further including transmitting the base layer and the enhancement layer to an image decoding system. ### Claim 14 (depends on 1) A method as recited in claim 1 further including transmitting only the base layer to an image decoding system. ### Claim 15 (depends on 1) A method as recited in claim 1 further including transmitting the base layer to an image decoding system using a first transmission medium and transmitting the enhancement layer to the image decoding system using a second transmission medium. ### Claim 16 (depends on 1) A method as recited in claim 1 further including transmitting the base layer to an image decoding system using a first transmission format and transmitting the enhancement layer to the image decoding system using a second transmission format. ### Claim 17 (depends on 1) A method as recited in claim 1 further including storing the base layer and the enhancement layer on a storage medium. ### Claim 18 (depends on 1) A method as recited in claim 1 further including storing the base layer on a first storage medium and storing the enhancement layer on a second storage medium. ### Claim 19 (depends on 1) One or more computer-readable memories containing a computer program that is executable by a processor to perform the method recited in claim 1. ### Claim 20 (depends on 1) A method as recited in claim 1 wherein both the base layer and the enhancement layer are used to generate a high-resolution image. ### Claim 21 (depends on 1) A method as recited in claim 1 wherein the enhancement layer contains only a high-resolution portion of the source image. ### Claim 22 A method comprising: decoding a base layer representing a low-resolution portion of an encoded source image, wherein the base layer and the source image have a first aspect ratio; and decoding an enhancement layer representing a high-resolution portion of the encoded source image, wherein the second layer has a second aspect ratio, and wherein the second aspect ratio differs from the first aspect ratio. ### Claim 23 (depends on 22) A method as recited in claim 22 wherein the aspect ratio associated with the base layer corresponds to an aspect ratio associated with low-resolution televisions. ### Claim 24 (depends on 22) A method as recited in claim 22 wherein the aspect ratio associated with the enhancement layer corresponds to an aspect ratio associated with high-resolution televisions. ### Claim 25 (depends on 22) A method as recited in claim 22 wherein the aspect ratio associated with the base layer is 4:3. ### Claim 26 (depends on 22) A method as recited in claim 22 wherein the aspect ratio associated with the enhancement layer is 16:9. ### Claim 27 (depends on 22) A method as recited in claim 22 further including communicating the base layer to a low-resolution television. ### Claim 28 (depends on 22) A method as recited in claim 22 further including combining the enhancement layer and a portion of the base layer to generate high-resolution image data. ### Claim 29 (depends on 22) A method as recited in claim 22 wherein the method is executed by a television. ### Claim 30 (depends on 22) A method as recited in claim 22 wherein the base layer is decoded from a physical medium and the enhancement layer is decoded from a received data stream. ### Claim 31 (depends on 22) A method as recited in claim 22 further including correcting an anamorphic squeeze in the base layer. ### Claim 32 (depends on 22) A method as recited in claim 22 wherein the base layer is received at a first time and the enhancement layer is received at a second time. ### Claim 33 (depends on 22) A method as recited in claim 22 wherein the base layer is received from a first media and the enhancement layer is received from a second media. ### Claim 34 (depends on 22) One or more computer-readable memories containing a computer program that is executable by a processor to perform the method recited in claim 22. ### Claim 35 (depends on 22) A method as recited in claim 22 wherein both the base layer and the enhancement layer are used to generate high-resolution image data. ### Claim 36 (depends on 22) A method as recited in claim 22 wherein the enhancement layer contains only high-resolution image data. ### Claim 37 A method comprising: transmitting a base layer representing a low-resolution portion of an image, wherein the base layer and the image have a first aspect ratio; and transmitting an enhancement layer representing a high-resolution portion of the image, wherein the enhancement layer has a second aspect ratio, and wherein the second aspect ratio differs from the first aspect ratio. ### Claim 38 (depends on 37) A method as recited in claim 37 wherein the first aspect ratio corresponds to an aspect ratio associated with low-resolution televisions. ### Claim 39 (depends on 37) A method as recited in claim 37 wherein the second aspect ratio corresponds to an aspect ratio associated with high-resolution televisions. ### Claim 40 (depends on 37) A method as recited in claim 37 wherein the base layer is transmitted using a first transmission medium and the enhancement layer is transmitted using a second transmission medium. ### Claim 41 (depends on 37) A method as recited in claim 37 wherein the base layer is transmitted using a first transmission format and the enhancement layer is transmitted using a second transmission format. ### Claim 42 (depends on 37) One or more computer-readable memories containing a computer program that is executable by a processor to perform the method recited in claim 37. ### Claim 43 (depends on 37) A method as recited in claim 37 wherein both the base layer and the enhancement layer are used to generate high-resolution image data. ### Claim 44 An apparatus comprising; a base layer generator to generate a base layer having a first aspect ratio, wherein the base layer represents a low-resolution portion of an image, and wherein the image also has the first aspect ratio; and an enhancement layer generator coupled to the base layer generator to generate an enhancement layer having a second aspect ratio, wherein the enhancement layer represents a high-resolution portion of the image, and wherein the first aspect ratio differs from the second aspect ratio. ### Claim 45 (depends on 44) An apparatus as recited in claim 44 wherein the base layer generator is a low-pass filter. ### Claim 46 (depends on 44) An apparatus as recited in claim 44 wherein the enhancement layer generator is a high-pass filter. ### Claim 47 (depends on 44) An apparatus as recited in claim 44 wherein the enhancement layer generator includes an image extractor to extract a portion of the base layer and a differencing module to subtract the extracted portion of the base layer from a corresponding portion of the image. ### Claim 48 (depends on 44) An apparatus as recited in claim 44 further including a transmitter coupled to the base layer generator and the enhancement layer generator, wherein the transmitter combines the base layer and the enhancement layer into a single transport stream. ### Claim 49 (depends on 44) An apparatus as recited in claim 44 further including a transmitter coupled to the base layer generator and the enhancement layer generator, wherein the transmitter transmits the base layer and the enhancement layer to an image decoding system. ### Claim 50 (depends on 44) An apparatus as recited in claim 44 further including a transmitter coupled to the base layer generator, wherein the transmitter transmits only the base layer to an image decoding system. ### Claim 51 (depends on 44) An apparatus as recited in claim 44 further including a storage medium coupled to the base layer generator and the enhancement layer generator, wherein the storage medium stores the base layer and the enhancement layer. ### Claim 52 An apparatus comprising: a base layer decoder to decode a base layer representing a low-resolution portion of an image, wherein the base layer and the image have a first aspect ratio; and an enhancement layer decoder coupled to the base layer decoder to decode an enhancement layer representing a high-resolution portion of the image, wherein the enhancement layer has a second aspect ratio that differs from the first aspect ratio. ### Claim 53 (depends on 52) An apparatus as recited in claim 52 wherein the first aspect ratio corresponds to an aspect ratio associated with low-resolution televisions. ### Claim 54 (depends on 52) An apparatus as recited in claim 52 wherein the second aspect ratio corresponds to an aspect ratio associated with high-resolution televisions. ### Claim 55 (depends on 52) An apparatus as recited in claim 52 wherein the apparatus is a television. ### Claim 56 (depends on 52) An apparatus as recited in claim 52 further including a receiver coupled to the base layer decoder and the enhancement layer decoder, wherein the receiver is to receive a transport stream containing the base layer and the enhancement layer. ### Claim 57 (depends on 52) A method as recited in claim 52 wherein both the base layer and the enhancement layer are used to generate high-resolution image data. ### Claim 58 One or more computer-readable media having stored thereon a computer program comprising the following steps: generating a base layer representing a low-resolution portion of a source image, wherein the layer and the source image have a first aspect ratio; and generating an enhancement layer representing a high-resolution portion of the source image, wherein the enhancement layer has a second aspect ratio, and wherein the second aspect ratio is different from the first aspect ratio. ### Claim 59 (depends on 58) One or more computer-readable media as recited in claim 58 wherein the first aspect ratio is 4:3. ### Claim 60 (depends on 58) One or more computer-readable media as recited in claim 58 wherein the second aspect ratio is 16:9. ### Claim 61 (depends on 58) One or more computer-readable media as recited in claim 58 wherein the step of generating an enhancements layer includes comparing a portion of the base layer with a corresponding portion of the source image. ### Claim 62 (depends on 58) One or more computer-readable media as recited in claim 58 further including transmitting the base layer and the enhancement layer to an image decoding system. ### Claim 63 (depends on 58) One or more computer-readable media as recited in claim 58 further including transmitting only the base layer to an image decoding system. ### Claim 64 (depends on 58) One or more computer-readable media as recited in claim 58 further including storing the base layer on a storage medium. ### Claim 65 (depends on 58) One or more computer-readable media as recited in claim 58 further including storing the enhancement layer on a storage medium. ### Claim 66 One or more computer-readable media having stored thereon a computer program comprising the following steps: decoding a base layer representing a low-resolution portion of an encoded source image, wherein the base layer and the source image have a first aspect ratio; and decoding an enhancement layer representing a high-resolution portion of the encoded source image, wherein the enhancement layer has a second aspect ratio, and wherein the first aspect ratio is different from the second aspect ratio. ### Claim 67 (depends on 66) One or more computer-readable media as recited in claim 66 wherein the first aspect ratio is 4:3. ### Claim 68 (depends on 66) One or more computer-readable media as recited in claim 66 wherein the second aspect ratio Is 16:9. ### Claim 69 (depends on 66) One or more computer-readable media as recited in claim 66 further including communicating the base layer to a low-resolution television. ### Claim 70 (depends on 66) One or more computer-readable media as recited in claim 66 further including communicating the base layer and the enhancement layer to a high-resolution television. ### Claim 71 A method of encoding a source image having a first aspect ratio, the method comprising: generating a base layer representing a low-resolution portion of the source image, wherein the base layer has the first aspect ratio; and generating an enhancement layer representing a high-resolution portion of the source image, wherein the enhancement layer has second aspect ratio, wherein the second aspect ratio differs from the first aspect ratio, and wherein both the base layer and the enhancement layer are used to generate a high-resolution image. ### Claim 72 (depends on 71) A method as recited in claim 71 wherein the first aspect ratio corresponds to an aspect ratio associated with low-resolution televisions. ### Claim 73 (depends on 71) A method as recited in claim 71 wherein the second aspect ratio corresponds to an aspect ratio associated with high-resolution televisions. ### Claim 74 A method comprising: decoding a first layer representing a low-resolution portion of an encoded source image, wherein the first layer and the source image have associated first aspect ratio; decoding a second layer representing a high-resolution portion of the encoded source image, wherein the second layer has an associated second aspect ratio, and wherein the second aspect ratio differs from the first aspect ratio; and combining the second layer and the first layer to generate high-resolution image data. ### Claim 75 (depends on 74) A method as recited in claim 74 further comprising communicating the first layer to a low-resolution television. ### Claim 76 (depends on 74) A method as recited in claim 74 further comprising communicating the high-resolution image data to a high-resolution television.