# US 11,122,294 B2 · Methods, devices and stream for encoding and decoding volumetric video > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 11,122,294 B2 - Document ID: US11122294B2 - Status: Granted - Kind: B2 - Application number: 16/632,480 - Filed: 2018-07-16 - Published: 2020-05-14 - Granted: 2021-09-14 - Earliest claimed: 2017-07-21 - Assignees: InterDigital CE Patent Holdings, SAS - Applicants: InterDigital CE Patent Holdings, SAS - Inventors: Julien Fleureau; Thierry Tapie; Franck Thudor - Web page: https://patentgrep.com/patent/US11122294B2 ## CPC - H04N 19/597: specially adapted for multi-view video sequence encoding - G06T 9/00: Image coding (bandwidth or redundancy reduction for static pictures H04N1/41; coding or decoding of static colour picture signals H04N1/64; methods or arrangements for coding, decoding, compressing or decompressing digital video signals H04N19/00) - G06T 15/20: Perspective computation - H04N 19/136: Incoming video signal characteristics or properties - H04N 19/172: the region being a picture, frame or field - H04N 19/177: the unit being a group of pictures [GOP] - H04N 19/182: the unit being a pixel - H04N 19/186: the unit being a colour or a chrominance component - H04N 19/20: using video object coding - H04N 19/85: using pre-processing or post-processing specially adapted for video compression - G06T 2210/56: Particle system, point based geometry or rendering - H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION - G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL ## Abstract A sequence of point clouds is encoded as a video by an encoder and transmitted to a decoder which retrieves the sequence of point clouds. Visible points of a point cloud are iteratively projected on projection maps according to at least two centers of projection, to determine a patch data item lists. One of the centers of projection is selected and corresponding image patches are generated and packed into a picture. Pictures and associated patch data item list are encoded in a stream. The decoding method decodes pictures and associated patch data item lists. Pixels of image patches comprised in pictures are un-projected according to data stored in associated patches. The methods have the advantage of encoding every point of point clouds in a manner avoiding artifacts and allowing decoding at video frame rate. ## Claims ### Claim 1 A method of encoding a point cloud into a stream, the method comprising: obtaining a group of centers of projection determined according to a viewing point relative to the point cloud; determining, for said point cloud, a list of patch data items by projecting on a projection map, points of said point cloud visible from one center of projection of said group of centers of projection and clustering adjacent pixels as image patches, a patch data item being associated with an image patch and comprising: information identifying said one center of projection; a description of a region of the space, said region being defined according to said one center of projection, an angular range and a depth range; and, information identifying an area for the associated image patch within a picture; generating said picture comprising a set of image patches; and encoding, in said stream, the picture and the list of patch data items. ### Claim 2 (depends on 1) The method of claim 1, wherein determining, for said point cloud, a list of data items comprises: for a first center of projection and a different second centers of projection among the group of centers of projection: determining a first list of patch data items by projecting on a first projection map, points of the point cloud visible from the first center of projection and by clustering adjacent pixels of the first projection map according to depth information; determining a second list of patch data items by projecting on a second projection map, points of the point cloud visible from the second center of projection and by clustering adjacent pixels of the second projection map according to depth information; selecting one of the first and second list of patch data items and adding the selected list of patch data items to the list of patch data items; and removing the points of the point cloud visible from the center of projection of the selected list; and iterating the above steps until the point cloud is empty or the patch data item list is full. ### Claim 3 (depends on 2) The method of claim 2, wherein said point cloud comprises points of a group of point clouds of a sequence of point clouds, a list of patch data items being determined for a group of pictures, each picture of said group of pictures being generated for a point cloud of said group of point clouds, and said encoding comprising encoding, in said stream, the group of pictures and the list of patch data items. ### Claim 4 (depends on 2) The method of claim 2, wherein a pixel of an image patch comprises a depth value, said depth value being determined according to depth range of the patch data item associated with said image patch. ### Claim 5 (depends on 4) The method of claim 4, wherein said a pixel of the image patches further comprises a color value. ### Claim 6 (depends on 5) The method of claim 5, wherein depth values are stored by pixels of a first picture and color values are stored by pixels of a second picture, first and second pictures being paired and encoded into the stream. ### Claim 7 A device for encoding a point cloud into a stream, the device comprising a memory associated with at least one processor configured to: obtain a group of centers of projection determined according to a viewing point; determine, for said point cloud, a list of patch data items by projecting on a projection map, points of said point cloud visible from one center of projection of said group of centers of projection and clustering adjacent pixels as image patches, a patch data item being associated with an image patch and comprising: information identifying said one center of projection; a description of a region of the space, said region being defined according to said one center of projection, an angular range and a depth range; and, information identifying an area for the associated image patch within a picture; generate said picture comprising a set of image patches; and encode, in said stream, the picture and the list of patch data items. ### Claim 8 (depends on 7) The device of claim 7, wherein said at least one processor is configured to determine, for said point cloud, a list of data items by: for a first and a different second center of projection among the group of centers of projection: determining a first list of patch data items by projecting on a first projection map, points of the point cloud visible from the first center of projection and by clustering adjacent pixels of the first projection map according to depth information; determining a second list of patch data items by projecting on a second projection map, points of the point cloud visible from the second center of projection and by clustering adjacent pixels of the second projection map according to depth information; selecting one of the first and second list of patch data items and adding the selected list of patch data items to the list of patch data items; and removing the points of the point cloud visible from the center of projection of the selected list; and iterating the above steps until the point cloud is empty or the patch data item list is full. ### Claim 9 (depends on 8) The device of claim 8, wherein said point cloud comprises points of a group of point clouds of a sequence of point clouds, a list of patch data item list being determined for a group of pictures, each picture of said group of pictures being generated for a point cloud of said group of point clouds, said at least one processor being configured to encode, in said stream, the group of pictures and the list of patch data items. ### Claim 10 (depends on 8) The device of claim 8, wherein a pixel of an image patch comprises a depth value, said depth value being determined according to depth range of the patch data item associated with said image patch. ### Claim 11 A method of decoding a point cloud from a stream, the method comprising: obtaining a group of centers of projection determined according to a viewing point relative to the point cloud; decoding a picture packing image patches and a list of patch data items from the stream, a patch data item comprising: information identifying one center of projection within the group of centers of projection; a description of a region of the space, said region being defined according to the one center of projection, an angular range and a depth range; and, information identifying an area of an image patch within the picture; decoding points of said point cloud by un-projecting pixels of each image patch according to the associated patch data item. ### Claim 12 (depends on 11) The method of claim 11, wherein coordinates of the centers of projection of the group of centers of projection are obtained from a memory. ### Claim 13 A device for decoding a point cloud from a stream, the device comprising a memory associated with at least a processor configured to: obtain a group of centers of projection determined according to a viewing point relative to the point cloud; decode a picture packing image patches and a list of patch data items from the stream, a patch data item comprising: information identifying one center of projection within the group of centers of projection; a description of a region of the space, said region being defined according to the one center of projection, an angular range and a depth range; and, information identifying an area of an image patch within the picture; decode points of said point cloud by un-projecting pixels of each image patch according to the associated patch data item. ### Claim 14 (depends on 13) The device of claim 13, wherein coordinates of the centers of projection of the group of centers of projection are obtained from the stream.