# US 2018/0025467 A1 · Mapping of spherical image data into rectangular faces for transport and decoding across networks > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 2018/0025467 A1 - Document ID: US20180025467A1 - Status: Published application - Kind: A1 - Application number: 15/357,968 - Filed: 2016-11-21 - Published: 2018-01-25 - Earliest claimed: 2016-07-19 - Assignees: GoPro, Inc. - Applicants: GoPro, Inc. - Inventors: Timothy Macmillan; Adeel Abbas - Web page: https://patentgrep.com/patent/US20180025467A1 ## CPC - G06T 3/073: Transforming surfaces of revolution to planar images, e.g. cylindrical surfaces to planar images - G06F 18/22: Matching criteria, e.g. proximity measures - G06T 3/14: Transformations for image registration, e.g. adjusting or mapping for alignment of images - G06T 3/20: Linear translation of whole images or parts thereof, e.g. panning - G06T 7/0002: {Inspection of images, e.g. flaw detection} - G06T 7/38: Registration of image sequences - G06T 7/55: from multiple images - G06T 7/593: from stereo images - G06T 7/73: using feature-based methods - G06T 11/60: Creating or editing images; Combining images with text - G06T 15/20: Perspective computation - H04N 5/265: Mixing - H04N 13/10: Processing, recording or transmission of stereoscopic or multi-view image signals - H04N 13/243: using three or more two-dimensional [2D] image sensors - H04N 13/282: for generating image signals corresponding to three or more geometrical viewpoints, e.g. multi-view systems - H04N 13/302: for viewing without the aid of special glasses, i.e. using autostereoscopic displays - H04N 13/334: using spectral multiplexing - H04N 13/344: with head-mounted left-right displays - H04N 19/136: Incoming video signal characteristics or properties - H04N 19/44: Decoders specially adapted therefor, e.g. video decoders which are asymmetric with respect to the encoder - H04N 23/698: for achieving an enlarged field of view, e.g. panoramic image capture - G06T 2200/04: involving 3D image data - G06T 2207/20221: Image fusion; Image merging - H04N 13/161: Encoding, multiplexing or demultiplexing different image signal components (for multi-view video sequence encoding H04N19/597) - G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) - H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION ## Abstract A system receives an encoded image representative of the 2D projection of a cubic image, the encoded image generated from two overlapping hemispherical images separated along a longitudinal plane of a sphere. The system decodes the encoded image to produce a decoded 2D projection of the cubic image, and perform a stitching operation to portions of the decoded 2D projection representative of overlapping portions of the hemispherical images to produce stitched overlapping portions. The system combine the stitched overlapping portions with portions of the decoded 2D projection representative of the non-overlapping portions of the hemispherical images to produce a stitched 2D projection of the cubic image, and encode the stitched 2D projection of the cubic image to produce an encoded cubic projection of the stitched hemispherical images. ## Claims ### Claim 1 A method for stitching overlapping portions of a 2D projection of a cubic representation of a spherical field of view (FOV) comprising: receiving an encoded image representative of the 2D projection of a cubic image, the encoded image generated from two overlapping hemispherical images separated along a longitudinal plane of a sphere; decoding the encoded image to produce a decoded 2D projection of the cubic image; performing a stitching operation to portions of the decoded 2D projection representative of overlapping portions of the hemispherical images to produce stitched overlapping portions; combining the stitched overlapping portions with portions of the decoded 2D projection representative of the non-overlapping portions of the hemispherical images to produce a stitched 2D projection of the cubic image; and encoding the stitched 2D projection of the cubic image to produce an encoded cubic projection of the stitched hemispherical images. ### Claim 2 (depends on 1) The method of claim 1, wherein performing a stitching operation further comprises: identifying the overlapping portions of the decoded 2D projection of the cubic image; and for each identified overlapping portion, performing an image stitching operation on the overlapping portion to stitch the overlapping portion with a complementary overlapping portion, the overlapping portion and the complementary overlapping representative of an overlapping FOV. ### Claim 3 (depends on 2) The method of claim 2, further comprising: extracting depth information from the overlapping portion and the complementary overlapping portion , each of the overlapping portion and the complementary overlapping portion having a stereo separation; and stitching the overlapping portion and the complementary overlapping portion together based on features within portion the overlapping portion and the complementary overlapping portion identified using the depth information. ### Claim 4 (depends on 2) The method of claim 2, further comprising: identifying visual features in the overlapping portion and the complementary overlapping portion using a feature detector; matching the visual features in the overlapping portion with the visual features in the overlapping portion using an image registration process; aligning the overlapping portion and the corresponding overlapping portion to reduce lens artifacts; and blending the adjusted overlapping portion and the adjusted corresponding overlapping portion using the matched visual features to generate the stitched overlapping portion. ### Claim 5 (depends on 1) The method of claim 1, wherein each face of a set of four faces of the stitched 2D projection of the cubic image includes a stitched overlapping portion, and wherein each of a fifth and a sixth face of the stitched 2D projection of the cubic image does not include stitched overlapping portions. ### Claim 6 (depends on 5) The method of claim 5, wherein the encoding the stitched 2D projection of the cubic image further comprises: encoding the set of four faces of the stitched 2D projection of the cubic image horizontally to each other; and encoding each of the fifth and sixth faces of the stitched 2D projection of the cubic image with one of the set of four faces to create to generate a visually continuous pair of faces. ### Claim 7 (depends on 5) The method of claim 5, wherein the encoding the stitched 2D projection of the cubic image further comprises: encoding the set of four faces and the fifth and sixth face of the stitched 2D projection of the cubic image adjacent to each other such that all six faces of the stitched 2D projection of the cubic image fills a rectangular boundary box. ### Claim 8 (depends on 7) The method of claim 7, wherein at least one of the faces of the stitched 2D projection of the cubic image is visually continuous with another face of the stitched 2D projection of the cubic image. ### Claim 9 A specially-configured hardware system for stitching overlapping portions of a projection of a cubic representation of a spherical field of view (FOV) configured to: receive an encoded image representative of the 2D projection of a cubic image, the encoded image generated from two overlapping hemispherical images separated along a longitudinal plane of a sphere; decode the encoded image to produce a decoded 2D projection of the cubic image; perform a stitching operation to portions of the decoded 2D projection representative of overlapping portions of the hemispherical images to produce stitched overlapping portions; combine the stitched overlapping portions with portions of the decoded 2D projection representative of the non-overlapping portions of the hemispherical images to produce a stitched 2D projection of the cubic image; and encode the stitched 2D projection of the cubic image to produce an encoded cubic projection of the stitched hemispherical images. ### Claim 10 (depends on 9) The system of claim 9, further configured to: identify the overlapping portions of the decoded 2D projection of the cubic image; and for each identified overlapping portion, perform an image stitching operation on the overlapping portion to stitch the overlapping portion with a complementary overlapping portion, the overlapping portion and the complementary overlapping representative of an overlapping FOV. ### Claim 11 (depends on 10) The system of claim 10, further configured to: extract depth information from the overlapping portion and the complementary overlapping portion , each of the overlapping portion and the complementary overlapping portion having a stereo separation; and stitch the overlapping portion and the complementary overlapping portion together based on features within portion the overlapping portion and the complementary overlapping portion identified using the depth information. ### Claim 12 (depends on 10) The system of claim 10, further configured to: identify visual features in the overlapping portion and the complementary overlapping portion using a feature detector; match the visual features in the overlapping portion with the visual features in the overlapping portion using an image registration process; align the overlapping portion and the corresponding overlapping portion to reduce lens artifacts; and blend the adjusted overlapping portion and the adjusted corresponding overlapping portion using the matched visual features to generate the stitched overlapping portion. ### Claim 13 (depends on 9) The system of claim 9, wherein each face of a set of four faces of the stitched 2D projection of the cubic image includes a stitched overlapping portion, and wherein each of a fifth and a sixth face of the stitched 2D projection of the cubic image does not include stitched overlapping portions. ### Claim 14 (depends on 13) The system of claim 13, further configured to: encode the set of four faces of the stitched 2D projection of the cubic image horizontally to each other; and encode each of the fifth and sixth faces of the stitched 2D projection of the cubic image with one of the set of four faces to create to generate a visually continuous pair of faces. ### Claim 15 (depends on 13) The system of claim 13, further configured to: encode the set of four faces and the fifth and sixth face of the stitched 2D projection of the cubic image adjacent to each other such that all six faces of the stitched 2D projection of the cubic image fills a rectangular boundary box. ### Claim 16 (depends on 15) The system of claim 15, wherein at least one of the faces of the stitched 2D projection of the cubic image is visually continuous with another face of the stitched 2D projection of the cubic image.