# US 10,872,259 B2 · Image production from video > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 10,872,259 B2 - Document ID: US10872259B2 - Status: Granted - Kind: B2 - Application number: 16/411,023 - Filed: 2019-05-13 - Published: 2019-08-29 - Granted: 2020-12-22 - Earliest claimed: 2015-07-16 - Assignees: Google Inc. - Applicants: Google LLC - Inventors: Jason Adam Karsh; Jason Robert Richard Sanio - Web page: https://patentgrep.com/patent/US10872259B2 ## CPC - G08B 13/19602: {Image analysis to detect motion of the intruder, e.g. by frame subtraction} - G06F 3/02: Input arrangements using manually operated switches, e.g. using keyboards or dials - G06T 7/20: Analysis of motion (motion estimation for coding, decoding, compressing or decompressing digital video signals H04N19/43, H04N19/51) - G06V 20/40: in video content (extracting overlay text G06V20/62; video retrieval G06F16/70; processing of video elementary streams in video servers H04N21/234; processing of video elementary streams in video clients H04N21/44) - G08B 13/19606: {Discriminating between target movement or movement in an area of interest and other non-signicative movements, e.g. target movements induced by camera shake or movements of pets, falling leaves, rotating fan} - H04N 5/144: {Movement detection (for video coding H04N19/503; analysis of motion in general G06T7/20)} - H04N 23/61: based on recognised objects - H04N 23/63: by using electronic viewfinders - H04N 23/6812: {based on additional sensors, e.g. acceleration sensors} - H04N 23/683: {performed by a processor, e.g. controlling the readout of an image memory} - G08B: SIGNALLING SYSTEMS, e.g. PERSONAL CALLING SYSTEMS; ORDER TELEGRAPHS; ALARM SYSTEMS - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) - G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL - G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING - H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION ## Abstract Implementations generally relate to producing a still image from a video or series of continuous frames. In some implementations, a method includes receiving the frames that a capture device shot while moving in at least two dimensions. The method further includes analyzing the frames to determine changes of positions of objects in at least two of the frames due to movement of the objects in the scene relative to changes of positions of objects due to the movement of the capture device during the shoot time. The method further includes determining, based at least in part on the variability of the objects, one or more target subjects which the capture device captures during the shoot time. One or more still images are generated from the plurality of frames having at least a portion of the target subject. ## Claims ### Claim 1 A computer-implemented method to generate a still image, the method comprising: receiving, at a computing device, a video that includes a plurality of sequential frames captured by a capture device during a shoot time, wherein at least two frames of the plurality of sequential frames each include one or more target subjects, and wherein the capture device is moved in at least two dimensions during the shoot time; identifying, by one or more processors of the computing device, a defective frame of the at least two frames of the plurality of sequential frames in which the one or more target subjects are detected as having an image defect, wherein the defective frame is a first frame in the plurality of sequential frames to depict at least a portion of the one or more target subjects with the image defect; identifying, by the one or more processors of the computing device, a reference frame of the at least two frames of the plurality of sequential frames, that is immediately prior to the defective frame in the plurality of the sequential frames, wherein the reference frame includes the one or more target subjects being free of the image defect, and wherein movement of the capture device during the shoot time includes changing an orientation of the capture device between capture of the defective frame and the reference frame; overlapping, by the one or more processors of the computing device, the reference frame with the defective frame to align the one or more target subjects of the reference frame with the one or more target subjects of the defective frame, wherein overlapping includes rotating at least one of the defective frame or the reference frame; and generating, by the one or more processors of the computing device, the still image based on the overlapped reference frame and defective frame, wherein the still image includes the one or more target subjects free of the image defect. ### Claim 2 (depends on 1) The computer-implemented method of claim 1, wherein the image defect includes at least one of the one or more target subjects being blurry. ### Claim 3 (depends on 1) The computer-implemented method of claim 1, wherein the movement of the capture device in the at least two dimensions during the shoot time includes a sweeping movement in a path that is at least a partial arch. ### Claim 4 (depends on 1) The computer-implemented method of claim 1, wherein changing the orientation of the capture device in the at least two dimensions during the shoot time includes rotating orientation between a landscape view and a portrait view. ### Claim 5 (depends on 1) The computer-implemented method of claim 1, further comprising identifying the one or more target subjects by user input including a user touching the one or more target subjects on a display screen, the user inputting a text description of the one or more target subjects, or the user voicing a description of the one or more target subjects. ### Claim 6 (depends on 1) The computer-implemented method of claim 1, wherein the defective frame excludes a portion of the one or more target subjects and the method further comprises filling in the excluded portion of the one or more target subjects in the still image with pixels of the excluded portion from the reference frame. ### Claim 7 A system comprising: one or more processors; and non-transitory computer-readable medium couple to the one or more processors and with logic encoded thereon that, when executed by the one or more processors cause the one or more processors to perform operations comprising: receiving a video that includes a plurality of sequential frames captured by a capture device during a shoot time, wherein at least two frames of the plurality of sequential frames each include one or more target subjects, and wherein the capture device is moved in at least two dimensions during the shoot time; identifying a defective frame of the at least two frames of the plurality of sequential frames in which the one or more target subjects is detected as having an image defect, wherein the defective frame is a first frame in the plurality of sequential frames to depict at least a portion of the one or more target subjects with the image defect; identifying a reference frame of the at least two frames of the plurality of sequential frames that is immediately prior to the defective frame in the plurality of the sequential frames, wherein the reference frame includes the one or more target subjects being free of the image defect, and wherein movement of the capture device during the shoot time includes changing an orientation of the capture device between capture of the defective frame and the reference frame; overlapping the reference frame with the defective frame to align the one or more target subjects of the reference frame with the one or more target subjects of the defective frame, wherein overlapping includes rotating at least one of the defective frame or the reference frame; and generating a still image based on the overlapped reference frame and defective frame, wherein the still image includes the one or more target subjects free of the image defect. ### Claim 8 (depends on 7) The system of claim 7, wherein the image defect includes at least one of the one or more target subjects being blurry. ### Claim 9 (depends on 7) The system of claim 7, wherein moving the capture device in the at least two dimensions during the shoot time includes a sweeping movement in a path that is at least a partial arch. ### Claim 10 (depends on 7) The system of claim 7, wherein changing the orientation of the capture device in the at least two dimensions during the shoot time includes rotating orientation between a landscape view and a portrait view. ### Claim 11 (depends on 7) The system of claim 7, wherein the operations further comprise identifying the one or more target subjects by user input including a user touching the one or more target subjects on a display screen, the user inputting a text description of the one or more target subjects, or the user voicing a description of the one or more target subjects. ### Claim 12 (depends on 7) The system of claim 7, wherein the defective frame excludes a portion of the one or more target subjects and the operations further comprise filling in the excluded portion of the one or more target subjects in the still image with pixels of the excluded portion from the reference frame. ### Claim 13 A non-transitory computer-readable medium storing instructions that, when executed by one or more processors, cause the one or more processors to perform operations comprising: receiving a video that includes a plurality of sequential frames captured by a capture device during a shoot time, wherein at least two frames of the plurality of sequential frames each include one or more target subjects, and wherein the capture device is moved in at least two dimensions during the shoot time; identifying a defective frame of the at least two frames of the plurality of sequential frames in which the one or more target subjects is detected as having an image defect, wherein the defective frame is a first frame in the plurality of sequential frames to depict at least a portion of the one or more target subjects with the image defect; identifying a reference frame of the at least two frames of the plurality of sequential frames that is immediately prior to the defective frame in the plurality of the sequential frames, wherein the reference frame includes the one or more target subjects being free of the image defect, and wherein movement of the capture device during the shoot time includes changing an orientation of the capture device between capture of the defective frame and the reference frame; overlapping the reference frame with the defective frame to align the one or more target subjects of the reference frame with the one or more target subjects of the defective frame, wherein overlapping includes rotating at least one of the defective frame or the reference frame; and generating a still image based on the overlapped reference frame and defective frame, wherein the still image includes the one or more target subjects free of the image defect. ### Claim 14 (depends on 13) The non-transitory computer-readable medium of claim 13, wherein the image defect includes at least one of the one or more target subjects being blurry. ### Claim 15 (depends on 13) The non-transitory computer-readable medium of claim 13, wherein moving the capture device in the at least two dimensions during the shoot time includes a sweeping movement in a path that is at least a partial arch. ### Claim 16 (depends on 13) The non-transitory computer-readable medium of claim 13, wherein changing the orientation of the capture device in the at least two dimensions during the shoot time includes rotating orientation between a landscape view and a portrait view. ### Claim 17 (depends on 13) The non-transitory computer-readable medium of claim 13, operations further comprise identifying the one or more target subjects by user input including a user touching the one or more target subjects on a display screen, the user inputting a text description of the one or more target subjects, or the user voicing a description of the one or more target subjects.