# US 11,445,099 B2 · Multi-camera device > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 11,445,099 B2 - Document ID: US11445099B2 - Status: Granted - Kind: B2 - Application number: 16/769,706 - Filed: 2018-12-17 - Published: 2020-11-26 - Granted: 2022-09-13 - Earliest claimed: 2017-12-20 - Assignees: Nokia Technologies Oy - Applicants: Nokia Technologies Oy - Inventors: Sujeet Shyamsundar Mate; Arto Lehtiniemi; Antti Eronen; Jussi Leppänen - Web page: https://patentgrep.com/patent/US11445099B2 ## CPC - H04N 7/147: {Communication arrangements, e.g. identifying the communication as a video-communication, intermediate storage of the signals (selecting H04Q)} - G06T 3/00: Geometric image transformations in the plane of the image - G06T 7/292: Multi-camera tracking - G06T 7/33: using feature-based methods - H04N 5/2624: {for obtaining an image which is composed of whole input images, e.g. splitscreen} - H04N 23/45: for generating image signals from two or more image sensors being of different type or operating in different modes, e.g. with a CMOS sensor for moving images in combination with a charge-coupled device [CCD] for still images - H04N 23/62: Control of parameters via user interfaces - H04N 23/63: by using electronic viewfinders - H04N 23/632: {for displaying or modifying preview images prior to image capturing, e.g. variety of image resolutions or capturing parameters} - H04N 23/635: {Region indicators; Field of view indicators} - H04N 23/69: Control of means for changing angle of the field of view, e.g. optical zoom objectives or electronic zooming - H04N 23/695: Control of camera direction for changing a field of view, e.g. pan, tilt or based on tracking of objects - H04N 23/90: Arrangement of cameras or camera modules, e.g. multiple cameras in TV studios or sports stadiums - G06T 2200/32: involving image mosaicing - G06T 2207/10012: Stereo images - G06T 2207/10024: Color image - G06T 2207/20221: Image fusion; Image merging - H04N 2007/145: {Handheld terminals} - H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION - G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL ## Abstract A method and apparatus is described comprising: obtaining first input data from a first camera of a multi-camera device; obtaining second input data from a second camera of the multi-camera device (wherein the second camera is orientated in a different direction to the first camera); defining a first region or object of interest for the first camera and/or a second region or object of interest for the second camera; tracking the first region or object of interest and/or the second region or object of interest; generating a first output data based on the first input data; generating a second output data based on the second input data; and generating a multi-camera output that includes the first and second output data. ## Claims ### Claim 1 An apparatus, comprising: at least one processor; and at least one memory including computer program code; the at least one memory and the computer program code configured to, with the at least one processor, cause the apparatus at least to perform: obtaining first input data from a first camera of a multi-camera device, wherein the first input data is data from a field of view of the first camera; obtaining second input data from a second camera of the multi-camera device, wherein the second input data is data from a field of view of the second camera and wherein the second camera is orientated in a different direction to the first camera; defining a first region or object of interest for the first camera and a second region or object of interest for the second camera; tracking the first region or object of interest while also tracking the second region or object of interest; generating a first output data based on the first input data, wherein the first output data is data from a viewport of the first camera, wherein the viewport of the first camera depends on the tracking the first region or object of interest; generating a second output data based on the second input data, wherein the second output data is data from a viewport of the second camera, wherein the viewport of the second camera depends on the tracking the second region or object of interest; and generating a multi-camera output that includes the first and second output data, wherein: simultaneously, the viewport of the first camera is narrower than the field of view of the first camera and the viewport of the second camera is narrower than the field of view of the second camera; and tracking the first region or object of interest comprises keeping the first region or object of interest within the first output data and tracking the second region or object of interest comprises keeping the second region or object of interest within the second output data. ### Claim 2 (depends on 1) The apparatus as claimed in claim 1, wherein the first region or object of interest or the second region or object of interest is defined by a user indication. ### Claim 3 (depends on 1) The apparatus as claimed in claim 1, wherein the memory and computer program code, with the at least one processor, are further configured to cause the apparatus to cease to track the first region or object of interest or the second region or object of interest when the said region or object of interest moves outside a field of view of the relevant camera. ### Claim 4 (depends on 3) The apparatus as claimed in claim 3, wherein the memory and computer program code, with the at least one processor, are further configured to cause the apparatus to resume tracking of the first region or object of interest or the second region or object of interest when the said region or object of interest moves back within a field of view of the relevant camera. ### Claim 5 (depends on 1) The apparatus as claimed in claim 1, wherein the memory and computer program code, with the at least one processor, are further configured to cause the apparatus to provide an alert when the first region or object of interest or the second region or object of interest moves outside a field of view of the relevant camera. ### Claim 6 (depends on 1) The apparatus as claimed in claim 1, wherein the memory and computer program code, with the at least one processor, are further configured to cause the apparatus to deselect the first region or object of interest for the first camera or the second region or object of interest for the second camera and ceasing to track the deselected region or object of interest. ### Claim 7 (depends on 1) The apparatus as claimed in claim 1, wherein the multi-camera output is configured to present the first and second outputs data side-by-side. ### Claim 8 (depends on 1) The apparatus as claimed in claim 1, wherein the multi-camera output is configured to present the first and second outputs data one on top of the other. ### Claim 9 (depends on 1) The apparatus as claimed in claim 1, wherein the first camera is a front camera and the second camera is a rear camera. ### Claim 10 A method, comprising: obtaining first input data from a first camera of a multi-camera device, wherein the first input data is data from a field of view of the first camera; obtaining second input data from a second camera of the multi-camera device, wherein the second input data is data from a field of view of the second camera and wherein the second camera is orientated in a different direction to the first camera; defining a first region or object of interest for the first camera and a second region or object of interest for the second camera; tracking the first region or object of interest while also tracking the second region or object of interest; generating a first output data based on the first input data, wherein the first output data is data from a viewport of the first camera, wherein the viewport of the first camera depends on the tracking the first region or object of interest; generating a second output data based on the second input data, wherein the second output data is data from a viewport of the second camera, wherein the viewport of the second camera depends on the tracking the second region or object of interest; and generating a multi-camera output that includes the first and second output data, wherein: simultaneously, the viewport of the first camera is narrower than the field of view of the first camera and the viewport of the second camera is narrower than the field of view of the second camera; and tracking the first region or object of interest comprises keeping the first region or object of interest within the first output data and tracking the second region or object of interest comprises keeping the second region or object of interest within the second output data. ### Claim 11 (depends on 10) The method as claimed in claim 10, wherein the first region or object of interest or the second region or object of interest is defined by a user indication. ### Claim 12 (depends on 10) The method as claimed in claim 10, further comprising ceasing to track the first region or object of interest or the second region or object of interest when the said region or object of interest moves outside a field of view of the relevant camera. ### Claim 13 (depends on 12) The method as claimed in claim 12, further comprising resuming tracking of the first region or object of interest or the second region or object of interest when the said region or object of interest moves back within a field of view of the relevant camera. ### Claim 14 (depends on 10) The method as claimed in claim 10, further comprising providing an alert when the first region or object of interest or the second region or object of interest moves outside a field of view of the relevant camera. ### Claim 15 (depends on 10) The method as claimed in claim 10, further comprising deselecting the first region or object of interest for the first camera or the second region or object of interest for the second camera and ceasing to track the deselected region or object of interest. ### Claim 16 (depends on 10) The method as claimed in claim 10, wherein the multi-camera output presents the first and second outputs data either side-by-side or one on top of the other. ### Claim 17 A computer program embodied on a non-transitory computer-readable medium, said computer program comprising computer-readable code which, when executed by at least one processor, causing performance of: obtaining first input data from a first camera of a multi-camera device, wherein the first input data is data from a field of view of the first camera; obtaining second input data from a second camera of the multi-camera device, wherein the second input data is data from a field of view of the second camera and wherein the second camera is orientated in a different direction to the first camera; defining a first region or object of interest for the first camera and a second region or object of interest for the second camera; tracking the first region or object of interest while also tracking the second region or object of interest; generating a first output data based on the first input data, wherein the first output data is data from a viewport of the first camera, wherein the viewport of the first camera depends on the tracking the first region or object of interest; generating a second output data based on the second input data, wherein the second output data is data from a viewport of the second camera, wherein the viewport of the second camera depends on the tracking the second region or object of interest; and generating a multi-camera output that includes the first and second output data, wherein: simultaneously, the viewport of the first camera is narrower than the field of view of the first camera and the viewport of the second camera is narrower than the field of view of the second camera; and tracking the first region or object of interest comprises keeping the first region or object of interest within the first output data and tracking the second region or object of interest comprises keeping the second region or object of interest within the second output data. ### Claim 18 (depends on 17) The computer program as claimed in claim 17, wherein the computer program, when executed by at least one processor, further causes ceasing to track the first region or object of interest or the second region or object of interest when the said region or object of interest moves outside a field of view of the relevant camera. ### Claim 19 (depends on 18) The computer program as claimed in claim 18, wherein the computer program, when executed by at least one processor, further causes resuming tracking of the first region or object of interest or the second region or object of interest when the said region or object of interest moves back within a field of view of the relevant camera.