# US 12,603,979 B2 · Processing images using neural style transfer network > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 12,603,979 B2 - Document ID: US12603979B2 - Status: Granted - Kind: B2 - Application number: 18/467,850 - Filed: 2023-09-15 - Published: 2025-03-20 - Granted: 2026-04-14 - Earliest claimed: 2023-09-15 - Assignees: Varjo Technologies Oy - Applicants: Varjo Technologies Oy - Inventors: Mikko Ollila; Mikko Strandborg - Web page: https://patentgrep.com/patent/US12603979B2 ## CPC - H04N 13/156: Mixing image signals - H04N 23/951: by using two or more images to influence resolution, frame rate or aspect ratio - H04N 13/383: for tracking with gaze detection, i.e. detecting the lines of sight of the viewer's eyes - H04N 23/90: Arrangement of cameras or camera modules, e.g. multiple cameras in TV studios or sports stadiums - H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION ## Abstract Disclosed is a system with server(s) configured to: obtain a plurality of input images captured by a plurality of input cameras comprising at least a first input camera per eye and a second input camera per eye, wherein a first field of view of the first input camera is narrower than a second field of view of the second input camera and is overlapping with an overlapping portion of the second field of view, the plurality of input images comprising at least a first input image and a second input image captured by the first input camera and the second input camera, respectively; and process the plurality of input images, by using neural style transfer network(s), to generate a single output image per eye. ## Claims ### Claim 1 A system comprising at least one server configured to: obtain a plurality of input images captured by a plurality of input cameras comprising at least a first input camera per eye and a second input camera per eye, wherein a first field of view of the first input camera is narrower than a second field of view of the second input camera and is overlapping with an overlapping portion of the second field of view, the plurality of input images comprising at least a first input image and a second input image captured by the first input camera and the second input camera, respectively; and process the plurality of input images, by using at least one neural style transfer network, to generate a single output image per eye; wherein when processing the plurality of input images, the at least one server is configured to: identify respective gaze regions in the first input image and the second input image, based on a given gaze direction; detect when a gaze region of the second input image is in a proximity of a boundary between the overlapping portion of the second field of view and a non-overlapping portion of the second field of view; when it is detected that the gaze region is in the proximity of the boundary, identify respective surrounding regions in the first input image and the second input image that include and surround the respective gaze regions; and use the at least one neural style transfer network to combine at least the first input image and the second input image, whilst blending the respective surrounding regions of the first input image and the second input image. ### Claim 2 (depends on 1) The system of claim 1, wherein when processing the plurality of input images, the at least one server is configured to use the at least one neural style transfer network to combine at least the first input image and the second input image, whilst blending the first input image with a corresponding portion of the second input image that pertains to the overlapping portion of the second field of view. ### Claim 3 (depends on 1) The system of claim 1, wherein when processing the plurality of input images, the at least one server is configured to: identify respective gaze regions in the first input image and the second input image, based on a given gaze direction; and use the at least one neural style transfer network to combine at least the first input image and the second input image, whilst blending the respective gaze regions of the first input image and the second input image. ### Claim 4 (depends on 2) The system of claim 2, wherein said blending is performed using different blending weights for the first input image and the second input image. ### Claim 5 (depends on 1) The system of claim 1, wherein the at least one server is configured to train the at least one neural style transfer network to generate output images in a style of any one of: a set of images captured by the first input camera, a set of reference images. ### Claim 6 (depends on 1) The system of claim 1, wherein when processing the plurality of input images, the at least one server is configured to use the at least one neural style transfer network to generate the single output image in a style of any one of: a set of images captured by the first input camera, a set of reference images. ### Claim 7 A system comprising at least one server configured to: obtain a plurality of input images captured by a plurality of input cameras comprising at least a first input camera per eye and a second input camera per eye, wherein a first field of view of the first input camera is narrower than a second field of view of the second input camera and is overlapping with an overlapping portion of the second field of view, the plurality of input images comprising at least a first input image and a second input image captured by the first input camera and the second input camera, respectively; and process the plurality of input images, by using at least one neural style transfer network, to generate a single output image per eye; wherein when processing the plurality of input images, the at least one server is configured to use the at least one neural style transfer network to combine at least the first input image and the second input image, whilst blending a transition region of the first input image with a transition region of the second input image, wherein an outer boundary of the transition region of the first input image and an outer boundary of the transition region of the second input image represent a boundary between the overlapping portion of the second field of view and a non-overlapping portion of the second field of view. ### Claim 8 A system comprising at least one server configured to: obtain a plurality of input images captured by a plurality of input cameras comprising at least a first input camera per eye and a second input camera per eye, wherein a first field of view of the first input camera is narrower than a second field of view of the second input camera and is overlapping with an overlapping portion of the second field of view, the plurality of input images comprising at least a first input image and a second input image captured by the first input camera and the second input camera, respectively; and process the plurality of input images, by using at least one neural style transfer network, to generate a single output image per eye; wherein when processing the plurality of input images, the at least one server is configured to use the at least one neural style transfer network to combine at least the first input image and the second input image, whilst blending the first input image with a corresponding portion of the second input image that pertains to the overlapping portion of the second field of view; wherein when processing the plurality of input images, the at least one server is configured to: use the at least one neural style transfer network to perform a style transfer on at least the first input image and the second input image to generate a first style-transferred image and a second style-transferred image, respectively; and use the at least one neural style transfer network to combine the first input image, the second input image, the first style-transferred image and the second style-transferred image, to generate the single output image. ### Claim 9 A method comprising: obtaining a plurality of input images captured by a plurality of input cameras comprising at least a first input camera per eye and a second input camera per eye, wherein a first field of view of the first input camera is narrower than a second field of view of the second input camera and is overlapping with an overlapping portion of the second field of view, the plurality of input images comprising at least a first input image and a second input image captured by the first input camera and the second input camera, respectively; and processing the plurality of input images by using at least one neural style transfer network, to generate a single output image per eye; wherein the step of processing the plurality of input images comprises: identifying respective gaze regions in the first input image and the second input image, based on a given gaze direction; detecting when a gaze region of the second input image is in a proximity of a boundary between the overlapping portion of the second field of view and a non-overlapping portion of the second field of view; when it is detected that the gaze region is in the proximity of the boundary, identifying respective surrounding regions in the first input image and the second input image that include and surround the respective gaze regions; and using the at least one neural style transfer network to combine at least the first input image and the second input image, whilst blending the respective surrounding regions of the first input image and the second input image. ### Claim 10 (depends on 9) The method of claim 9, wherein the step of processing the plurality of input images comprises using the at least one neural style transfer network to combine at least the first input image and the second input image, whilst blending the first input image with a corresponding portion of the second input image that pertains to the overlapping portion of the second field of view. ### Claim 11 (depends on 9) The method of claim 9, wherein the step of processing the plurality of input images comprises: identifying respective gaze regions in the first input image and the second input image, based on a given gaze direction; and using the at least one neural style transfer network to combine at least the first input image and the second input image, whilst blending the respective gaze regions of the first input image and the second input image. ### Claim 12 A method comprising: obtaining a plurality of input images captured by a plurality of input cameras comprising at least a first input camera per eye and a second input camera per eye, wherein a first field of view of the first input camera is narrower than a second field of view of the second input camera and is overlapping with an overlapping portion of the second field of view, the plurality of input images comprising at least a first input image and a second input image captured by the first input camera and the second input camera, respectively; and processing the plurality of input images by using at least one neural style transfer network, to generate a single output image per eye; wherein the step of processing the plurality of input images comprises using the at least one neural style transfer network to combine at least the first input image and the second input image, whilst blending a transition region of the first input image with a transition region of the second input image, wherein an outer boundary of the transition region of the first input image and an outer boundary of the transition region of the second input image represent a boundary between the overlapping portion of the second field of view and a non-overlapping portion of the second field of view. ### Claim 13 A method comprising: obtaining a plurality of input images captured by a plurality of input cameras comprising at least a first input camera per eye and a second input camera per eye, wherein a first field of view of the first input camera is narrower than a second field of view of the second input camera and is overlapping with an overlapping portion of the second field of view, the plurality of input images comprising at least a first input image and a second input image captured by the first input camera and the second input camera, respectively; and processing the plurality of input images by using at least one neural style transfer network, to generate a single output image per eye; wherein the step of processing the plurality of input images comprises using the at least one neural style transfer network to combine at least the first input image and the second input image, whilst blending the first input image with a corresponding portion of the second input image that pertains to the overlapping portion of the second field of view; wherein the step of processing the plurality of input images comprises: using the at least one neural style transfer network to perform a style transfer on at least the first input image and the second input image to generate a first style-transferred image and a second style-transferred image, respectively; and using the at least one neural style transfer network to combine the first input image, the second input image, the first style-transferred image and the second style-transferred image, to generate the single output image.