# US 2014/0152871 A1 · Image sensor data compression and dsp decompression

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2014/0152871 A1
- Document ID: US20140152871A1
- Status: Published application
- Kind: A1
- Application number: 14/174,812
- Filed: 2014-02-06
- Published: 2014-06-05
- Earliest claimed: 2012-03-30
- Assignees: GoPro, Inc.
- Applicants: GoPro, Inc.
- Inventors: Scott Patrick Campbell; David A. Newman; Brian G. Schunck
- Web page: https://patentgrep.com/patent/US20140152871A1

## CPC

- H04N 1/2137: {with temporary storage before final recording, e.g. in a frame buffer}
- H04N 1/00209: {Transmitting or receiving image data, e.g. facsimile data, via a computer, e.g. using e-mail, a computer network, the internet, I-fax}
- H04N 19/137: Motion inside a coding unit, e.g. average field, frame or block difference
- H04N 19/156: Availability of hardware or computational resources, e.g. encoding based on power-saving criteria
- H04N 19/172: the region being a picture, frame or field
- H04N 19/423: characterised by memory arrangements (H04N19/433 takes precedence)
- H04N 19/426: using memory downsizing methods
- H04N 19/625: using discrete cosine transform [DCT]
- H04N 19/63: using sub-band based transform, e.g. wavelets
- H04N 19/91: Entropy coding, e.g. variable length coding [VLC] or arithmetic coding
- H04N 23/661: Transmitting camera control signals through networks, e.g. control via the Internet
- H04N 25/00: Circuitry of solid-state image sensors [SSIS]; Control thereof
- H04N 25/134: {based on three different wavelength filter elements}
- H04N 25/40: Extracting pixel data from image sensors by controlling scanning circuits, e.g. by modifying the number of pixels sampled or to be sampled
- H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION

## Abstract

An image sensor compresses image data prior to transmitting the image data to a DSP. The image sensor captures light representing an image, for instance via a camera's aperture. A focal plane array converts the captured light into pixel data. The pixel data is sorted into categories, and is compressed in parallel by a compression engine. The compressed pixel data is then sent to a DSP, which may be located off-chip. The DSP then decompresses the compressed pixel data, performs image signal processing operations on the compressed pixel data, and then compresses the processed pixel data into a digital image format. The image sensor may buffer the pixel data for one or more images to accommodate for slowdown by the compression engine. The pixel data may be sorted by row and column of a pixel array. Alternatively, the pixel data may be sorted by color from a Bayer color filter.

## Claims

### Claim 1

A method for decompressing image data, the method comprising:
receiving, from an image sensor chip, compressed pixel data, the compressed pixel data comprising pixel data organized into a plurality categories and compressed in parallel by the image sensor chip at a magnitude of compression selected based on one or more of a rate at which the pixel data is captured and an amount of power available to the image sensor chip;
decompressing, by a digital signal processor (DSP), the compressed pixel data to produce decompressed pixel data organized into the plurality of categories;
combining the categories of decompressed pixel data to produce combined pixel data;
compressing the combined pixel data to produce a digital image comprising a digital image format; and
outputting the digital image.

### Claim 2 (depends on 1)

The method of claim 1, wherein each pixel of the decompressed pixel data is organized into a plurality of categories comprising a) either odd column pixel data or even column pixel data, and b) either odd row pixel data or even row pixel data.

### Claim 3 (depends on 2)

The method of claim 2, wherein combining the categories of decompressed pixel data comprises interleaving pixel data comprising odd column pixel data and pixel data comprising even column pixel data, and interleaving pixel data comprising odd row pixel data and pixel data comprising even row pixel data.

### Claim 4 (depends on 1)

The method of claim 1, wherein the decompressed pixel data is organized into a plurality of categories comprising red pixel data, blue pixel data, a first portion of green pixel data, and a second portion of green pixel data.

### Claim 5 (depends on 4)

The method of claim 4, wherein combining the categories of decompressed pixel data comprises forming a combined pixel, the combined pixel comprising a pixel of the red pixel data, a pixel of the blue pixel data, a pixel of the first portion of green pixel data, and a pixel of the second portion of the green pixel data.

### Claim 6

A digital signal processor (DSP) comprising an instruction set that, when executed, causes the DSP to perform steps comprising:
receiving, from an image sensor chip, compressed pixel data, the compressed pixel data comprising pixel data organized into a plurality categories and compressed in parallel by the image sensor chip at a magnitude of compression selected based on one or more of a rate at which the pixel data is captured and an amount of power available to the image sensor chip;
decompressing the compressed pixel data to produce decompressed pixel data organized into the plurality of categories;
combining the categories of decompressed pixel data to produce combined pixel data;
compressing the combined pixel data to produce a digital image comprising a digital image format; and
outputting the digital image.

### Claim 7 (depends on 6)

The DSP of claim 6, wherein each pixel of the decompressed pixel data is organized into a plurality of categories comprising a) either odd column pixel data or even column pixel data, and b) either odd row pixel data or even row pixel data.

### Claim 8 (depends on 7)

The DSP of claim 7, wherein combining the categories of decompressed pixel data comprises interleaving pixel data comprising odd column pixel data and pixel data comprising even column pixel data, and interleaving pixel data comprising odd row pixel data and pixel data comprising even row pixel data.

### Claim 9 (depends on 6)

The DSP of claim 6, wherein the decompressed pixel data is organized into a plurality of categories comprising red pixel data, blue pixel data, a first portion of green pixel data, and a second portion of green pixel data.

### Claim 10 (depends on 9)

The DSP of claim 9, wherein combining the categories of decompressed pixel data comprises forming a combined pixel, the combined pixel comprising a pixel of the red pixel data, a pixel of the blue pixel data, a pixel of the first portion of green pixel data, and a pixel of the second portion of the green pixel data.

### Claim 11

A method for decompressing image data, the method comprising:
receiving, from an image sensor chip, compressed pixel data, the compressed pixel data comprising pixel data organized into a plurality categories and compressed;
decompressing, by a digital signal processor (DSP), the compressed pixel data to produce decompressed pixel data organized into the plurality of categories;
combining the categories of decompressed pixel data to produce combined pixel data; and
processing the combined pixel data to produce a digital image.

### Claim 12 (depends on 11)

The method of claim 11, wherein each pixel of the decompressed pixel data is organized into a plurality of categories comprising a) either odd column pixel data or even column pixel data, and b) either odd row pixel data or even row pixel data.

### Claim 13 (depends on 12)

The method of claim 12, wherein combining the categories of decompressed pixel data comprises interleaving pixel data comprising odd column pixel data and pixel data comprising even column pixel data, and interleaving pixel data comprising odd row pixel data and pixel data comprising even row pixel data.

### Claim 14 (depends on 11)

The method of claim 11, wherein the decompressed pixel data is organized into a plurality of categories comprising red pixel data, blue pixel data, a first portion of green pixel data, and a second portion of green pixel data.

### Claim 15 (depends on 14)

The method of claim 14, wherein combining the categories of decompressed pixel data comprises forming a combined pixel, the combined pixel comprising a pixel of the red pixel data, a pixel of the blue pixel data, a pixel of the first portion of green pixel data, and a pixel of the second portion of the green pixel data.

### Claim 16

A digital signal processor (DSP) comprising an instruction set that, when executed, causes the DSP to perform steps comprising:
receiving, from an image sensor chip, compressed pixel data, the compressed pixel data comprising pixel data organized into a plurality categories and compressed;
decompressing the compressed pixel data to produce decompressed pixel data organized into the plurality of categories;
combining the categories of decompressed pixel data to produce combined pixel data; and
processing the combined pixel data to produce a digital image.

### Claim 17 (depends on 16)

The DSP of claim 16, wherein each pixel of the decompressed pixel data is organized into a plurality of categories comprising a) either odd column pixel data or even column pixel data, and b) either odd row pixel data or even row pixel data.

### Claim 18 (depends on 17)

The DSP of claim 17, wherein combining the categories of decompressed pixel data comprises interleaving pixel data comprising odd column pixel data and pixel data comprising even column pixel data, and interleaving pixel data comprising odd row pixel data and pixel data comprising even row pixel data.

### Claim 19 (depends on 16)

The DSP of claim 16, wherein the decompressed pixel data is organized into a plurality of categories comprising red pixel data, blue pixel data, a first portion of green pixel data, and a second portion of green pixel data.

### Claim 20 (depends on 19)

The DSP of claim 19, wherein combining the categories of decompressed pixel data comprises forming a combined pixel, the combined pixel comprising a pixel of the red pixel data, a pixel of the blue pixel data, a pixel of the first portion of green pixel data, and a pixel of the second portion of the green pixel data.