# US 2018/0025498 A1 · Subject Tracking Systems for a Movable Imaging System > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 2018/0025498 A1 - Document ID: US20180025498A1 - Status: Published application - Kind: A1 - Application number: 15/656,559 - Filed: 2017-07-21 - Published: 2018-01-25 - Earliest claimed: 2016-07-21 - Assignees: GoPro, Inc. - Applicants: GoPro, Inc. - Inventors: Sammy Omari; Joseph A. Enke - Web page: https://patentgrep.com/patent/US20180025498A1 ## CPC - G06V 20/13: Satellite images - G01C 21/1656: {with passive imaging devices, e.g. cameras} - G06T 7/248: {involving reference images or patches} - G06T 7/74: {involving reference images or patches} - G06V 20/17: taken from planes or by drones - G06V 20/40: in video content (extracting overlay text G06V20/62; video retrieval G06F16/70; processing of video elementary streams in video servers H04N21/234; processing of video elementary streams in video clients H04N21/44) - G06V 20/52: Surveillance or monitoring of activities, e.g. for recognising suspicious objects (recognising microscopic objects G06V20/69) - B64U 2101/30: for imaging, photography or videography - B64U 2201/20: Remote controls - G01C 19/00: Gyroscopes; Turn-sensitive devices using vibrating masses; Turn-sensitive devices without moving masses; Measuring angular rate using gyroscopic effects - G06T 2207/10032: Satellite or aerial image; Remote sensing - G06T 2207/30196: Human being; Person - G06T 2207/30232: Surveillance - G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING - G01C: MEASURING DISTANCES, LEVELS OR BEARINGS; SURVEYING; NAVIGATION; GYROSCOPIC INSTRUMENTS; PHOTOGRAMMETRY OR VIDEOGRAMMETRY (measuring liquid level G01F; radio navigation, determining distance or velocity by use of propagation effects, e.g. Doppler effects, propagation time, of radio waves, analogous arrangements using other waves G01S) - G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL - B64U: UNMANNED AERIAL VEHICLES [UAV]; EQUIPMENT THEREFOR ## Abstract A method for tracking a subject in successive image frames includes obtaining previous image frames with an imaging device, processing the previous image frames, obtaining motion information of the imaging device and a subject, determining a region of interest, obtaining a subsequent image frame, and processing the region of interest. The processing includes determining previous frame positions of the subject therein. The motion information is obtained with sensors physically associated with one o more of the imaging device and the subject. The region of interest is located in a predetermined spatial relationship relative to a predicted frame position of the subject. ## Claims ### Claim 1 A method for tracking a subject in successive image frames comprising: obtaining previous image frames with an imaging device at previous times; processing the previous image frames to determine previous frame positions of a subject therein; obtaining motion information from one or more sensors physically associated with one or more of the imaging device and the subject; determining a region of interest for a subsequent image frame to be obtained at a subsequent time after the previous times, including determining a predicted frame position of the subject in the subsequent image frame from at least one of the motion information or the previous frame positions, and locating the region of interest in a predetermined spatial relationship relative to the predicted frame position; obtaining the subsequent image frame at the subsequent time; and processing the region of interest of the subsequent image frame to determine a frame position of the subject. ### Claim 2 (depends on 1) The method of claim 1, wherein the determining the predicted frame position is determined from the previous frame positions, motion information of the imaging device, and motion information of the subject. ### Claim 3 (depends on 1) The method of claim 1, further comprising determining another region of interest for another subsequent image frame to be obtained at another subsequent time after the subsequent time, including determining another predicted frame position of the subject in the other subsequent image frame from at least one of the motion information or a combination of the subsequent frame position and one or more of the previous frame positions; obtaining the other subsequent image frame at the other subsequent time; and processing the other region of interest of the other subsequent image frame to determine another frame position of the subject. ### Claim 4 A method for tracking a subject in successive image frames comprising: determining a predicted frame location at which a subject is estimated to appear in a subsequent image frame to be obtained at a subsequent time; determining a region of interest corresponding to the subsequent image frame, including determining a location of the region of interest to be in a predetermined spatial relationship relative to the predicted frame location; obtaining the subsequent image frame at the subsequent time with an imaging device; and processing the region of interest of the subsequent image frame to locate the subject. ### Claim 5 (depends on 4) The method of claim 4, wherein the predetermined spatial relationship is the predicted frame location centered in the region of interest. ### Claim 6 (depends on 4) The method of claim 4, wherein the determining the region of interest includes determining a size of the region of interest. ### Claim 7 (depends on 6) The method of claim 6, wherein the determining the size of the region of interest includes increasing or decreasing the size of the region of interest relative to a previous region of interest corresponding to a previous image frame. ### Claim 8 (depends on 6) The method of claim 6, wherein the size of the region of interest is determined according to at least one of a predicted size of the subject or a predicted distance between the subject and the imaging device in real space. ### Claim 9 (depends on 4) The method of claim 4, wherein the predicted frame location is determined according to at least one of visual information from previous image frames, motion information of the imaging device, and motion information of the subject. ### Claim 10 (depends on 9) The method of claim 9, wherein the method further comprises obtaining the previous image frames with the imaging device, wherein the predicted frame location is determined according to the visual information derived from the previous image frames, the previous image frames were obtained by the imaging device, and the visual information includes previous frame positions of the subject within the previous image frames. ### Claim 11 (depends on 10) The method of claim 10, wherein predicted frame location is determined according to a visual motion estimate of the subject, the visual motion estimate being determined by applying a motion model to the previous frame positions. ### Claim 12 (depends on 9) The method of claim 9, wherein the predicted frame location is determined according to the motion information of the imaging device obtained from one or more sensors physically associated with the imaging device. ### Claim 13 (depends on 12) The method of claim 12, wherein the motion information of the imaging device includes one or more of position information and orientation information of the imaging device relative to real space. ### Claim 14 (depends on 13) The method of claim 13, wherein predicted frame location is determined according to a imaging device motion estimate of the imaging device, and the one or more sensors include a gyroscope. ### Claim 15 (depends on 9) The method of claim 9, wherein the predicted frame location is determined according to the motion information of the subject obtained from one or more sensors physically associated with the subject. ### Claim 16 (depends on 15) The method of claim 15, wherein the motion information of the subject includes position information of the subject relative to real space. ### Claim 17 (depends on 16) The method of claim 16, wherein the predicted frame location is determined according to a subject motion estimate of the subject. ### Claim 18 (depends on 9) The method of claim 9, wherein the predicted frame location is determined according to the image data from the previous image frames, the motion information of the imaging device, and the motion information of the subject. ### Claim 19 (depends on 18) The method of claim 18, wherein the predicted frame location is determined according to a visual motion estimate determined from frame positions of the subject in the previous image frames, an imaging device motion estimate determined from the motion information of the imaging device obtained from a sensor physically associated with the imaging device, and a subject motion estimate determined from the motion information of the subject obtained from another sensor physically associated with the subject. ### Claim 20 A movable imaging system comprising: a movable platform movable in real space; an imaging device for capturing successive image frames that form a video and being connected to the movable platform; and a tracking system for maintaining a subject in the successive image frames, wherein the tracking system locates a region of interest for a subsequent image frame at a predicted frame location of the subject in a future image frame based on previous frame positions of the subject in the successive images, motion information of the imaging device, and motion information of the subject, and thereafter processes the region of interest of the future image frame to locate the subject in the future image frame.