# US 12,192,619 B2 · Electronic device and controlling method of electronic device > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 12,192,619 B2 - Document ID: US12192619B2 - Status: Granted - Kind: B2 - Application number: 18/378,390 - Filed: 2023-10-10 - Published: 2024-02-01 - Granted: 2025-01-07 - Earliest claimed: 2020-06-10 - Assignees: Samsung Electronics Co., Ltd. - Applicants: Samsung Electronics Co., Ltd. - Inventors: Jaeyeop Lee; Dongchan Kim; Jongmyeong Ban; Hyechan Bae; Dongnam Byun; Seongwo Oh; Haedong Yeo; Jinyoung Hwang - Web page: https://patentgrep.com/patent/US12192619B2 ## CPC - H04N 23/64: {Computer-aided capture of images, e.g. transfer from script file into camera, check of taken image quality, advice or proposal for image composition or decision on when to take image} - H04N 1/2129: {Recording in, or reproducing from, a specific memory area or areas, or recording or reproducing at a specific moment} - H04N 5/2621: {Cameras specially adapted for the electronic generation of special effects during image pickup, e.g. digital cameras, camcorders, video cameras having integrated special effects capability} - H04N 5/2624: {for obtaining an image which is composed of whole input images, e.g. splitscreen} - H04N 5/2628: {Alteration of picture size, shape, position or orientation, e.g. zooming, rotation, rolling, perspective, translation} - H04N 23/45: for generating image signals from two or more image sensors being of different type or operating in different modes, e.g. with a CMOS sensor for moving images in combination with a charge-coupled device [CCD] for still images - H04N 23/62: Control of parameters via user interfaces - H04N 23/631: {Graphical user interfaces [GUI] specially adapted for controlling image capture or setting capture parameters} - H04N 23/667: Camera operation mode switching, e.g. between still and video, sport and normal or high- and low-resolution modes - H04N 23/951: by using two or more images to influence resolution, frame rate or aspect ratio - H04N 23/57: Mechanical or electrical details of cameras or camera modules specially adapted for being embedded in other devices - H04N 23/80: Camera processing pipelines; Components thereof - H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION ## Abstract An electronic device and a controlling method of the electronic device are provided. The controlling method of an electronic device according to the disclosure includes the steps of, based on a first user input for or related to acquiring a live view image through a camera including a plurality of lenses different from one another being received, acquiring a plurality of image frames for each of the plurality of lenses and storing the image frames in a first memory, inputting the plurality of image frames for each lens stored in the first memory into a neural network model, by a predetermined time interval, and acquiring score information including composition preference information of each of the input image frames, selecting at least one lens among the plurality of lenses based on the score information, storing image frames acquired through the selected at least one lens in a second memory during the predetermined time interval, and based on a second user input for initiating recording of the live view image being received, storing an image related to the image frames stored in the second memory in a third memory until a time point when a third user input for ending the recording is received. ## Claims ### Claim 1 An electronic device comprising: a camera system comprising a plurality of lenses that are different; memory; and a processor configured to: based on receiving a first user input relating to acquiring a live view image, control the camera system to acquire a plurality of image frames through each of the plurality of lenses, input image frame of the plurality of image frames into at least one neural network model comprising a first neural network model based on a location of an object in the input image frame, and acquire score information by using the at least one neural network model, the score information indicating an aesthetic of each of the input image frames, select at least one lens among the plurality of lenses based on the score information, store in the memory, image frames acquired through the selected at least one lens, and based on receiving a second user input for starting recording of the live view image, store an image related to the image frames stored in the memory until a third user input for ending the recording is received. ### Claim 2 (depends on 1) The electronic device of claim 1, wherein the at least one neural network model further comprises a second neural network model based on how much similar an input image frame of the input frames, is to learning data, and a third neural network model based on a type of the object and the location of the object in the input image frame. ### Claim 3 (depends on 1) The electronic device of claim 1, wherein the plurality of lenses comprises a first lens and a second lens, and the at least one processor is configured to: based on score information of a first image frame acquired through the first lens being higher than or identical to score information of a second image frame acquired through the second lens at the time point when the first image frame was acquired, select the first lens, and based on the score information of the first image frame being lower than the score information of the second image frame, select the second lens. ### Claim 4 (depends on 1) The electronic device of claim 1, wherein the score information comprises information regarding whether a determined object is included in the plurality of image frames. ### Claim 5 (depends on 4) The electronic device of claim 4, wherein the first lens and the second lens are disposed at a first surface of the electronic device, a field of view of the first lens is less than the field of view of the second lens, and based on the predetermined object not being included in the first image frame and being included in the second image frame, the score information of the first image frame is lower than the score information of the second image frame. ### Claim 6 (depends on 5) The electronic device of claim 5, wherein the predetermined object is an object selected by the user of the electronic device among a plurality of objects included in the first image frame or the second image frame. ### Claim 7 (depends on 3) The electronic device of claim 3, wherein the first lens is at a rear of the electronic device, the second lens is at a front of the electronic device, and based on a predetermined operation of a user being included in the second image frame, the score information of the first image frame is lower than the score information of the second image frame. ### Claim 8 (depends on 3) The electronic device of claim 3, wherein the memory comprises: a first memory configured to store the plurality of image frames acquired through each of the plurality of lenses; a second memory configured to store image frames for a live view among the plurality of image frames stored in the first memory; and a third memory configured to store an image related to the image frames for the live view stored in the second, wherein the plurality of image frames acquired through the plurality of lenses according to the first user input is stored in the first memory, wherein the image frames acquired through the selected at least one lens is stored in the second memory during a predetermined time interval at which the image frames are input to the neural network mode; and wherein the image related to the image frames are stored in the third memory. ### Claim 9 (depends on 8) The electronic device of claim 8, wherein the image related to the image frames stored in the second memory is an image generated by merging at least one of the image frames stored in the second memory according to a time order, or the image related to the image frames stored in the second memory is an image wherein a scene transition effect was applied to at least one of the image frames acquired through the first lens and the image frames acquired through the second lens. ### Claim 10 (depends on 9) The electronic device of claim 9, wherein the scene transition effect is applied to the image related to the image frames stored in the second memory by inserting a prestored image into at least one of image frames before and after a boundary time point between the image frames acquired through the first lens and the image frames acquired through the second lens, or applying a predetermined editing technique based on at least one of the image frames before and after the boundary time point. ### Claim 11 A controlling method of controlling an electronic device, the method comprising: based on receiving a first user input relating to acquiring a live view image through a camera system including a plurality of lenses that are different, acquiring a plurality of image frames through each of the plurality of lenses; inputting image frame of the plurality of image frames into at least one neural network model comprising a first neural network model based on a location of an object in the input image frame, and acquiring score information by using the at least one neural network model, the score information indicating an aesthetic of each of the input image frames; selecting at least one lens among the plurality of lenses based on the score information; storing in a memory, image frames acquired through the selected at least one lens; and based on receiving a user input for starting recording of the live view image, storing an image related to the image frames stored in the memory until a third user input for ending the recording is received. ### Claim 12 (depends on 11) The method of claim 11, wherein the at least one neural network model further comprises a second neural network model based on how much similar an input image frame of the input frames, is to learning data, and a third neural network model based on a type of the object and the location of the object in the input image frame. ### Claim 13 (depends on 11) The method of claim 11, wherein the plurality of lenses includes a first lens and a second lens, and the selecting at least one lens comprises: based on score information of a first image frame acquired through the first lens being higher than or identical to score information of a second image frame acquired through the second lens at the time point when the first image frame was acquired, selecting the first lens; and based on the score information of the first image frame being lower than the score information of the second image frame, selecting the second lens. ### Claim 14 (depends on 11) The method of claim 11, wherein the score information comprises information regarding whether a determined object is included in the plurality of image frames. ### Claim 15 (depends on 14) The method of claim 14, wherein the first lens and the second lens are disposed at a first surface of the electronic device, a field of view of the first lens is less than the field of view of the second lens, and based on a predetermined object not being included in the first image frame and being included in the second image frame, the score information of the first image frame is lower than the score information of the second image frame. ### Claim 16 (depends on 15) The method of claim 15, wherein the predetermined object is an object selected by the user of the electronic device among a plurality of objects included in the first image frame or the second image frame. ### Claim 17 (depends on 13) The method of claim 13, wherein the first lens is arranged at the rear surface of the electronic device, the second lens is arranged at the front surface of the electronic device, and based on a predetermined operation of a user being included in the second image frame, the score information of the first image frame is lower than the score information of the second image frame. ### Claim 18 (depends on 13) The method of claim 13, wherein the memory includes: a first memory configured to store the plurality of image frames acquired through each of the plurality of lenses; a second memory configured to store image frames for a live view among the plurality of image frames stored in the first memory; and a third memory configured to store an image related to the image frames for the live view stored in the second, wherein the plurality of image frames acquired through the plurality of lenses according to the first user input is stored in the first memory, wherein the image frames acquired through the selected at least one lens is stored in the second memory during a predetermined time interval at which the image frames are input to the neural network mode; and wherein the image related to the image frames are stored in the third memory. ### Claim 19 (depends on 18) The method of claim 18, wherein the image related to the image frames stored in the second memory is an image generated by merging at least one of the image frames stored in the second memory according to a time order, or the image related to the image frames stored in the second memory is an image wherein a scene transition effect was applied to at least one of the image frames acquired through the first lens and the image frames acquired through the second lens. ### Claim 20 A non-transitory computer readable recording medium including a program executing a method of controlling an electronic device, wherein the method comprises: based on receiving a first user input relating to acquiring a live view image through a camera system including a plurality of lenses that are different, acquiring a plurality of image frames through each of the plurality of lenses; inputting image frame of the plurality of image frames into at least one neural network model, comprising a first neural network model based on a location of an object in the input image frame, and acquiring score information by using the at least one neural network model, the score information indicating an aesthetic of each of the input image frames; selecting at least one lens among the plurality of lenses based on the score information; storing in a memory, image frames acquired through the selected at least one lens; and based on receiving a user input for starting recording of the live view image, storing an image related to the image frames stored in the memory until a third user input for ending the recording is received.