# US 2019/0320202 A1 · Method and circuit of assigning selected depth values to rgb subpixels and recovering selected depth values from rgb subpixels for colored depth frame packing and depacking

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2019/0320202 A1
- Document ID: US20190320202A1
- Status: Published application
- Kind: A1
- Application number: 16/380,767
- Filed: 2019-04-10
- Published: 2019-10-17
- Earliest claimed: 2018-04-12
- Assignees: National Cheng Kung University
- Applicants: National Cheng Kung University
- Inventors: Jar-Ferr YANG
- Web page: https://patentgrep.com/patent/US20190320202A1

## CPC

- H04N 19/597: specially adapted for multi-view video sequence encoding
- H04N 19/88: involving rearrangement of data among different coding units, e.g. shuffling, interleaving, scrambling or permutation of pixel data or permutation of transform coefficient data among different blocks
- H04N 19/132: Sampling, masking or truncation of coding units, e.g. adaptive resampling, frame skipping, frame interpolation or high-frequency transform coefficient masking
- H04N 19/182: the unit being a pixel
- H04N 19/186: the unit being a colour or a chrominance component
- H04N 19/46: Embedding additional information in the video signal during the compression process (H04N19/517, H04N19/68, H04N19/70 take precedence)
- H04N 2213/005: Aspects relating to the "3D+depth" image format
- H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION

## Abstract

A method comprises: obtaining two depth values from each of a first pixel depth value and a fourth pixel depth value, and obtaining one depth value from each of a second pixel depth value and a third pixel depth value; and assigning the two depth values obtained from the first pixel depth value to the R-subpixel and B-subpixel values of the first pixel, assigning the depth value obtained from the second pixel depth value to the R-subpixel, G-subpixel and B-subpixel values of the second pixel, assigning the depth value obtained from the third pixel depth value to the R-subpixel, G-subpixel and B-subpixel values of the third pixel, and assigning the two depth values obtained from the fourth pixel depth value to the G-subpixel value of the first pixel and the R-subpixel, G-subpixel and B-subpixel values of the fourth pixel.

## Claims

### Claim 1

A method of assigning selected depth values to RGB subpixels for a colored depth frame packing and depacking, wherein an RGB format comprises an R subpixel, a G subpixel and a B subpixel in order, a RGB depth frame comprises a group of four pixels, the R subpixels, G subpixels and B subpixels of the group of four pixels are vertically or horizontally arranged, the group of four pixels comprises a first pixel, a second pixel, a third pixel and a fourth pixel, the first pixel is located neighboring the second pixel and the third pixel, the fourth pixel is located neighboring the second pixel and the third pixel, and geometrical positions of the first pixel, the second pixel, the third pixel and the fourth pixel correspondingly have a first pixel depth value, a second pixel depth value, a third pixel depth value and a fourth pixel depth value, respectively, the method comprising:
obtaining two depth values from each of the first pixel depth value and the fourth pixel depth value, and obtaining one depth value from each of the second pixel depth value and the third pixel depth value, wherein the six depth values obtained from the first pixel depth value, the second pixel depth value, the third pixel depth value and the fourth pixel depth value have interleaved positions; and
assigning one of the two depth values obtained from the first pixel depth value to an R-subpixel value of the first pixel in a converted RGB format, assigning the other one of the two depth values obtained from the first pixel depth value to a B-subpixel value of the first pixel in the converted RGB format, assigning the depth value obtained from the second pixel depth value to an R-subpixel value, a G-subpixel value and a B-subpixel value of the second pixel in the converted RGB format, assigning the depth value obtained from the third pixel depth value to an R-subpixel value, a G-subpixel value and a B-subpixel value of the third pixel in the converted RGB format, assigning one of the two depth values obtained from the fourth pixel depth value to a G-subpixel value of the first pixel in the converted RGB format, and assigning the other one of the two depth values obtained from the fourth pixel depth value to an R-subpixel value, a G-subpixel value and a B-subpixel value of the fourth pixel in the converted RGB format.

### Claim 2 (depends on 1)

The method according to claim 1, wherein the step of obtaining the depth values is to obtain depth values of geometrical positions corresponding to the R subpixel and the B subpixel of the first pixel from the first pixel depth value, to obtain a depth value of a geometrical position corresponding to the G subpixel of the second pixel from the second pixel depth value, to obtain a depth value of a geometrical position corresponding to the G subpixel of the third pixel from the third pixel depth value, and to obtain depth values of geometrical positions corresponding to the R subpixel and the B subpixel of the fourth pixel from the fourth pixel depth value.

### Claim 3 (depends on 2)

The method according to claim 2, wherein the step of assigning to the pixels in the converted RGB format is to assign the depth value of the geometrical position corresponding to the R subpixel of the first pixel obtained from the first pixel depth value to the R-subpixel value of the first pixel in the converted RGB format, to assign the depth value of the geometrical position corresponding to the B subpixel of the first pixel obtained from the first pixel depth value to the B-subpixel value of the first pixel in the converted RGB format, to assign the depth value of the geometrical position corresponding to the G subpixel of the second pixel obtained from the second pixel depth value to the R-subpixel value, the G-subpixel value and the B-subpixel value of the second pixel in the converted RGB format, to assign the depth value of the geometrical position corresponding to the G subpixel of the third pixel obtained from the third pixel depth value to the R-subpixel value, the G-subpixel value and the B-subpixel value of the third pixel in the converted RGB format, to assign the depth value of the geometrical position corresponding to the R subpixel of the fourth pixel obtained from the fourth pixel depth value to the G-subpixel value of the first pixel in the converted RGB format, and to assign the depth value of the geometrical position corresponding to the B subpixel of the fourth pixel obtained from the fourth pixel depth value to the R-subpixel value, the G-subpixel value and the B-subpixel value of the fourth pixel in the converted RGB format.

### Claim 4

A method of assigning selected depth values to RGB subpixels for a colored depth frame packing and depacking, wherein an RGB format comprises an R subpixel, a G subpixel and a B subpixel in order, a RGB depth frame comprises a group of four pixels, the R subpixels, G subpixels and B subpixels of the group of four pixels are vertically or horizontally arranged, the group of four pixels comprises a first pixel, a second pixel, a third pixel and a fourth pixel, the first pixel is located neighboring the second pixel and the third pixel, the fourth pixel is located neighboring the second pixel and the third pixel, and geometrical positions of the first pixel, the second pixel, the third pixel and the fourth pixel correspondingly have a first pixel depth value, a second pixel depth value, a third pixel depth value and a fourth pixel depth value, respectively, the method comprising:
obtaining two depth values from each of the first pixel depth value, the second pixel depth value, the third pixel depth value and the fourth pixel depth value; and
assigning one of the two depth values obtained from the first pixel depth value to an R-subpixel value of the first pixel in a converted RGB format, assigning the other one of the two depth values obtained from the first pixel depth value to a B-subpixel value of the first pixel in the converted RGB format, assigning one of the two depth values obtained from the second pixel depth value to a G-subpixel value of the second pixel in the converted RGB format, assigning the other one of the two depth values obtained from the second pixel depth value to a B-subpixel value of the second pixel in the converted RGB format, assigning one of the two depth values obtained from the third pixel depth value to an R-subpixel value, a G-subpixel value and a B-subpixel value of the third pixel in the converted RGB format, assigning the other one of the two depth values obtained from the third pixel depth value to a G-subpixel value of the first pixel in the converted RGB format, assigning one of the two depth values obtained from the fourth pixel depth value to an R-subpixel value of the second pixel in the converted RGB format, and assigning the other one of the two depth values obtained from the fourth pixel depth value to an R-subpixel value, a G-subpixel value and a B-subpixel value of the fourth pixel in the converted RGB format.

### Claim 5 (depends on 4)

The method according to claim 4, wherein the step of obtaining the depth values is to obtain depth values of geometrical positions corresponding to the R subpixel and the B subpixel of the first pixel from the first pixel depth value, to obtain depth values of geometrical positions corresponding to the G subpixel and the B subpixel of the second pixel from the second pixel depth value, to obtain depth values of geometrical positions corresponding to the R subpixel and the G subpixel of the third pixel from the third pixel depth value, and to obtain depth values of geometrical positions corresponding to the R subpixel and the B subpixel of the fourth pixel from the fourth pixel depth value.

### Claim 6 (depends on 5)

The method according to claim 5, wherein the step of assigning to the pixels in the converted RGB format is to assign the depth value of the geometrical position corresponding to the R subpixel of the first pixel obtained from the first pixel depth value to the R-subpixel value of the first pixel in the converted RGB format, to assign the depth value of the geometrical position corresponding to the B subpixel of the first pixel obtained from the first pixel depth value to the B-subpixel value of the first pixel in the converted RGB format, to assign the depth value of the geometrical position corresponding to the G subpixel of the second pixel obtained from the second pixel depth value to the G-subpixel value of the second pixel in the converted RGB format, to assign the depth value of the geometrical position corresponding to the B subpixel of the second pixel obtained from the second pixel depth value to the B-subpixel value of the second pixel in the converted RGB format, to assign the depth value of the geometrical position corresponding to the R subpixel of the third pixel obtained from the third pixel depth value to the R-subpixel value, the G-subpixel value and the B-subpixel value of the third pixel in the converted RGB format, to assign the depth value of the geometrical position corresponding to the G subpixel of the third pixel obtained from the third pixel depth value to the G-subpixel value of the first pixel in the converted RGB format, to assign the depth value of the geometrical position corresponding to the R subpixel of the fourth pixel obtained from the fourth pixel depth value to the R-subpixel value of the second pixel in the converted RGB format, and to assign the depth value of the geometrical position corresponding to the B subpixel of the fourth pixel obtained from the fourth pixel depth value to the R-subpixel value, the G-subpixel value and the B-subpixel value of the fourth pixel in the converted RGB format.

### Claim 7

A method of assigning selected depth values to RGB subpixels for a colored depth frame packing and depacking, wherein an RGB format comprises an R subpixel, a G subpixel and a B subpixel in order, a RGB depth frame comprises a group of four pixels, the R subpixels, G subpixels and B subpixels of the group of four pixels are vertically or horizontally arranged, the group of four pixels comprises a first pixel, a second pixel, a third pixel and a fourth pixel, the first pixel is located neighboring the second pixel and the third pixel, the fourth pixel is located neighboring the second pixel and the third pixel, and geometrical positions of the first pixel, the second pixel, the third pixel and the fourth pixel correspondingly have a first pixel depth value, a second pixel depth value, a third pixel depth value and a fourth pixel depth value, respectively, the method comprising:
obtaining two depth values from each of the first pixel depth value, the second pixel depth value, the third pixel depth value and the fourth pixel depth value; and
assigning one of the two depth values obtained from the first pixel depth value to an R-subpixel value of the first pixel in a converted RGB format, assigning the other one of the two depth values obtained from the first pixel depth value to a G-subpixel value of the first pixel in the converted RGB format, assigning one of the two depth values obtained from the second pixel depth value to a B-subpixel value of the first pixel in the converted RGB format, assigning the other one of the two depth values obtained from the second pixel depth value to an R-subpixel value, a G-subpixel value and a B-subpixel value of the second pixel in the converted RGB format, assigning one of the two depth values obtained from the third pixel depth value to an R-subpixel value of the third pixel in the converted RGB format, assigning the other one of the two depth values obtained from the third pixel depth value to a G-subpixel value of the third pixel in the converted RGB format, assigning one of the two depth values obtained from the fourth pixel depth value to a B-subpixel value of the third pixel in the converted RGB format, and assigning the other one of the two depth values obtained from the fourth pixel depth value to an R-subpixel value, a G-subpixel value and a B-subpixel value of the fourth pixel in the converted RGB format.

### Claim 8 (depends on 7)

The method according to claim 7, wherein the step of obtaining the depth values is to obtain depth values of geometrical positions corresponding to the R subpixel and the G subpixel of the first pixel from the first pixel depth value, to obtain depth values of geometrical positions corresponding to the R subpixel and the B subpixel of the second pixel from the second pixel depth value, to obtain depth values of geometrical positions corresponding to the R subpixel and the G subpixel of the third pixel from the third pixel depth value, and to obtain depth values of geometrical positions corresponding to the R subpixel and the B subpixel of the fourth pixel from the fourth pixel depth value.

### Claim 9 (depends on 8)

The method according to claim 8, wherein the step of assigning to the pixels in the converted RGB format is to assign the depth value of the geometrical position corresponding to the R subpixel of the first pixel obtained from the first pixel depth value to the R-subpixel value of the first pixel in the converted RGB format, to assign the depth value of the geometrical position corresponding to the G subpixel of the first pixel obtained from the first pixel depth value to the G-subpixel value of the first pixel in the converted RGB format, to assign the depth value of the geometrical position corresponding to the R subpixel of the second pixel obtained from the second pixel depth value to the B-subpixel value of the first pixel in the converted RGB format, to assign the depth value of the geometrical position corresponding to the B subpixel of the second pixel obtained from the second pixel depth value to the R-subpixel value, the G-subpixel value and the B-subpixel value of the second pixel in the converted RGB format, to assign the depth value of the geometrical position corresponding to the R subpixel of the third pixel obtained from the third pixel depth value to the R-subpixel value of the third pixel in the converted RGB format, to assign the depth value of the geometrical position corresponding to the G subpixel of the third pixel obtained from the third pixel depth value to the G-subpixel value of the third pixel in the converted RGB format, to assign the depth value of the geometrical position corresponding to the R subpixel of the fourth pixel obtained from the fourth pixel depth value to the B-subpixel value of the third pixel in the converted RGB format, and to assign the depth value of the geometrical position corresponding to the B subpixel of the fourth pixel obtained from the fourth pixel depth value to the R-subpixel value, the G-subpixel value and the B-subpixel value of the fourth pixel in the converted RGB format.

### Claim 10

A method of assigning selected depth values to RGB subpixels for a colored depth frame packing and depacking, wherein an RGB format comprises an R subpixel, a G subpixel and a B subpixel in order, a RGB depth frame comprises a group of four pixels, the R subpixels, G subpixels and B subpixels of the group of four pixels are vertically or horizontally arranged, the group of four pixels comprises a first pixel, a second pixel, a third pixel and a fourth pixel, the first pixel is located neighboring the second pixel and the third pixel, the fourth pixel is located neighboring the second pixel and the third pixel, and geometrical positions of the first pixel, the second pixel, the third pixel and the fourth pixel correspondingly have a first pixel depth value, a second pixel depth value, a third pixel depth value and a fourth pixel depth value, respectively, the method comprising:
obtaining one depth value from each of the first pixel depth value, the second pixel depth value, the third pixel depth value and the fourth pixel depth value, wherein the two depth values obtained from the first pixel depth value and the third pixel depth value have interleaved positions, and the two depth values obtained from the second pixel depth value and the fourth pixel depth value have interleaved positions; and
assigning the depth value obtained from the first pixel depth value to an R-subpixel value, a G-subpixel value and a B-subpixel value of the first pixel in a converted RGB format, assigning the depth value obtained from the second pixel depth value to an R-subpixel value, a G-subpixel value and a B-subpixel value of the second pixel in the converted RGB format, assigning the depth value obtained from the third pixel depth value to an R-subpixel value, a G-subpixel value and a B-subpixel value of the third pixel in the converted RGB format, assigning the depth value obtained from the fourth pixel depth value to an R-subpixel value, a G-subpixel value and a B-subpixel value of the fourth pixel in the converted RGB format.

### Claim 11 (depends on 10)

The method according to claim 10, wherein the step of obtaining the depth values is to obtain a depth value of a geometrical position corresponding to the R subpixel of the first pixel from the first pixel depth value, to obtain a depth value of a geometrical position corresponding to the G subpixel of the second pixel from the second pixel depth value, to obtain a depth value of a geometrical position corresponding to the G subpixel of the third pixel from the third pixel depth value, and to obtain a depth value of a geometrical position corresponding to the B subpixel of the fourth pixel from the fourth pixel depth value.
