# US 2014/0152872 A1 · On-chip image sensor data compression > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 2014/0152872 A1 - Document ID: US20140152872A1 - Status: Published application - Kind: A1 - Application number: 14/174,814 - Filed: 2014-02-06 - Published: 2014-06-05 - Earliest claimed: 2012-03-30 - Assignees: GoPro, Inc. - Applicants: GoPro, Inc. - Inventors: Scott Patrick Campbell; David A. Newman; Brian G. Schunck - Web page: https://patentgrep.com/patent/US20140152872A1 ## CPC - H04N 1/2137: {with temporary storage before final recording, e.g. in a frame buffer} - H04N 1/00209: {Transmitting or receiving image data, e.g. facsimile data, via a computer, e.g. using e-mail, a computer network, the internet, I-fax} - H04N 19/137: Motion inside a coding unit, e.g. average field, frame or block difference - H04N 19/156: Availability of hardware or computational resources, e.g. encoding based on power-saving criteria - H04N 19/172: the region being a picture, frame or field - H04N 19/423: characterised by memory arrangements (H04N19/433 takes precedence) - H04N 19/426: using memory downsizing methods - H04N 19/625: using discrete cosine transform [DCT] - H04N 19/63: using sub-band based transform, e.g. wavelets - H04N 19/91: Entropy coding, e.g. variable length coding [VLC] or arithmetic coding - H04N 23/661: Transmitting camera control signals through networks, e.g. control via the Internet - H04N 25/00: Circuitry of solid-state image sensors [SSIS]; Control thereof - H04N 25/134: {based on three different wavelength filter elements} - H04N 25/40: Extracting pixel data from image sensors by controlling scanning circuits, e.g. by modifying the number of pixels sampled or to be sampled - H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION ## Abstract An image sensor compresses image data prior to transmitting the image data to a DSP. The image sensor captures light representing an image, for instance via a camera's aperture. A focal plane array converts the captured light into pixel data. The pixel data is sorted into categories, and is compressed in parallel by a compression engine. The compressed pixel data is then sent to a DSP, which may be located off-chip. The DSP then decompresses the compressed pixel data, performs image signal processing operations on the compressed pixel data, and then compresses the processed pixel data into a digital image format. The image sensor may buffer the pixel data for one or more images to accommodate for slowdown by the compression engine. The pixel data may be sorted by row and column of a pixel array. Alternatively, the pixel data may be sorted by color from a Bayer color filter. ## Claims ### Claim 1 A method for compressing image data, the method comprising: receiving pixel data representative of an image captured by an image sensor chip; determining an amount of power available to the image sensor chip; sorting the received pixel data into a plurality of categories; selecting a data compression factor based on the determined amount of available power; compressing, by the image sensor chip, the categorized pixel data at a magnitude of compression selected based on the selected data compression factor; transmitting, by the image sensor chip, the compressed pixel data to a digital signal processor (DSP); decompressing, by the DSP, the compressed pixel data; and compressing, by the DSP, the decompressed pixel data into a digital image format. ### Claim 2 (depends on 1) The method of claim 1, further comprising: capturing light by the image sensor chip; converting the captured light into the received pixel data by a focal plane array (FPA); and performing one or more image signal processing operations on the decompressed pixel data before compressing the decompressed pixel data into a digital image format. ### Claim 3 (depends on 1) The method of claim 1, wherein sorting the received pixel data into a plurality of categories comprises sorting the received pixel data into odd row pixel data and even row pixel data, and further sorting the received pixel data into odd column and even column odd row pixel data and odd column and even column even row pixel data. ### Claim 4 (depends on 1) The method of claim 1, wherein sorting the received pixel data into a plurality of categories comprises sorting the received pixel data into color categories with a Bayer color filter, the color categories comprising red pixel data, blue pixel data, a first portion of green pixel data, and a second portion of green pixel data. ### Claim 5 (depends on 1) The method of claim 1, further comprising: downscaling the decompressed pixel data into a resolution lower than the resolution of the captured image. ### Claim 6 A system for compressing image data, the system comprising: an image sensor chip configured to: receive pixel data representative of an image; determine an amount of available power; sort the received pixel data into a plurality of categories; select a data compression factor based on the determined amount of available power; compress the categorized pixel data at a magnitude of compression selected based on the selected data compression factor; and output the compressed pixel data; and a digital signal processor (DSP) configured to: decompress the compressed pixel data; and compress the decompressed pixel data into a digital image format. ### Claim 7 (depends on 6) The system of claim 6, wherein the image sensor chip is further configured to capture light and convert the captured light into the received pixel data using a focal plane array (FPA), and wherein the DSP is further configured to perform one or more image signal processing operations on the decompressed pixel data before compressing the decompressed pixel data into a digital image format. ### Claim 8 (depends on 6) The system of claim 6, wherein sorting the received pixel data into a plurality of categories comprises sorting the received pixel data into odd row pixel data and even row pixel data, and further sorting the received pixel data into odd column and even column odd row pixel data and odd column and even column even row pixel data. ### Claim 9 (depends on 6) The system of claim 6, wherein sorting the received pixel data into a plurality of categories comprises sorting the received pixel data into color categories with a Bayer color filter, the color categories comprising red pixel data, blue pixel data, a first portion of green pixel data, and a second portion of green pixel data. ### Claim 10 (depends on 6) The system of claim 6, wherein the DSP is further configured to downscale the decompressed pixel data into a resolution lower than the resolution of the captured image. ### Claim 11 A method for compressing image data, the method comprising: receiving pixel data representative of an image captured by an image sensor chip; determining an amount of power available to the image sensor chip; sorting the received pixel data into a plurality of categories; selecting a data compression factor based on the determined amount of available power; and compressing, by the image sensor chip, the categorized pixel data at a magnitude of compression selected based on the selected data compression factor prior to outputting the categorized pixel data to a digital signal processor (DSP). ### Claim 12 (depends on 11) The method of claim 11, further comprising: capturing light by the image sensor chip; and converting the captured light into the received pixel data by a focal plane array (FPA). ### Claim 13 (depends on 11) The method of claim 11, wherein sorting the received pixel data into a plurality of categories comprises sorting the received pixel data into odd row pixel data and even row pixel data, and further sorting the received pixel data into odd column and even column odd row pixel data and odd column and even column even row pixel data. ### Claim 14 (depends on 11) The method of claim 11, wherein sorting the received pixel data into a plurality of categories comprises sorting the received pixel data into color categories with a Bayer color filter, the color categories comprising red pixel data, blue pixel data, a first portion of green pixel data, and a second portion of green pixel data. ### Claim 15 (depends on 11) The method of claim 11, wherein determining an amount of power available to the image sensor chip comprises determining an amount of power based on a user-selected power setting. ### Claim 16 A system for compressing image data, the system comprising: an image sensor chip configured to: receive pixel data representative of an image; determine an amount of available power; sort the received pixel data into a plurality of categories; select a data compression factor based on the determined amount of available power; and compress the categorized pixel data at a magnitude of compression selected based on the selected data compression factor before outputting the categorized pixel data to a digital signal processor (DSP). ### Claim 17 (depends on 16) The system of claim 16, wherein the image sensor chip is further configured to capture light and convert the captured light into the received pixel data using a focal plane array (FPA), and wherein the DSP is further configured to perform one or more image signal processing operations on the decompressed pixel data before compressing the decompressed pixel data into a digital image format. ### Claim 18 (depends on 16) The system of claim 16, wherein sorting the received pixel data into a plurality of categories comprises sorting the received pixel data into odd row pixel data and even row pixel data, and further sorting the received pixel data into odd column and even column odd row pixel data and odd column and even column even row pixel data. ### Claim 19 (depends on 16) The system of claim 16, wherein sorting the received pixel data into a plurality of categories comprises sorting the received pixel data into color categories with a Bayer color filter, the color categories comprising red pixel data, blue pixel data, a first portion of green pixel data, and a second portion of green pixel data. ### Claim 20 (depends on 16) The system of claim 16, wherein determining an amount of available power comprises determining an amount of power based on a user-selected power setting.