# US 2022/0044376 A1 · Systems and methods for generating panning images

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2022/0044376 A1
- Document ID: US20220044376A1
- Status: Published application
- Kind: A1
- Application number: 17/507,524
- Filed: 2021-10-21
- Published: 2022-02-10
- Earliest claimed: 2020-04-06
- Assignees: GoPro, Inc.
- Applicants: Gopro, Inc.
- Inventors: Gladys Jocelyne Marc; Balthazar Neveu; Nicolas Rahmouni; Maeva Escoulan
- Web page: https://patentgrep.com/patent/US20220044376A1

## CPC

- G06T 5/50: using two or more images, e.g. averaging or subtraction
- G06T 5/73: Deblurring; Sharpening
- G06T 5/70: Denoising; Smoothing
- G06T 7/20: Analysis of motion (motion estimation for coding, decoding, compressing or decompressing digital video signals H04N19/43, H04N19/51)
- G06T 2200/24: involving graphical user interfaces [GUIs]
- G06T 2207/10016: Video; Image sequence
- G06T 2207/20201: Motion blur correction
- G06T 2207/20221: Image fusion; Image merging
- G06T 2207/30196: Human being; Person
- G06T 2207/30221: Sports video; Sports image
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL

## Abstract

Images may be captured by a moving image capture device. A reference image and a background image may be selected from the images. The reference image may include depiction of an object, with the object blocking view of the background. The background image may include depiction of the background blocked by the object in the reference image. An object layer may be generated by segmenting the depiction of the object from the reference image. A background layer may be generated by combining the depiction of the background in the background image with the reference image. The background layer may be blurred and combined with the object layer to generate a panning image.

## Claims

### Claim 1

A system for generating panning images, the system comprising:
one or more physical processors configured by machine-readable instructions to:
obtain visual information, the visual information defining visual content of images captured by an image capture device while experiencing motion;
select a reference image from the images, the visual content of the reference image having a first field of view of a scene including an object of interest;
select a background image from the images, the background image different from the reference image, the visual content of the background image having a second field of view of the scene different from the first field of view based on the motion experienced by the image capture device between capture of the reference image and the background image;
generate an object layer including a depiction of the object of interest based on segmentation of the depiction of the object of interest from the visual content of the reference image;
generate a background layer based on combination of the visual content of the background image and the visual content of the reference image, wherein the combination of the visual content of the background image and the visual content of the reference image reduces the depiction of the object of interest in the background layer; and
generate a panning image based on combination of the object layer and the background layer.

### Claim 2 (depends on 1)

The system of claim 1, wherein the reference image and the background image are stabilized based on the motion experienced by the image capture device between capture of the reference image and the background image such that overlapping portions of the first field of view and the second field of view are aligned.

### Claim 3 (depends on 1)

The system of claim 1, wherein the background layer is blurred before the generation of the panning image.

### Claim 4 (depends on 3)

The system of claim 3, wherein the background layer is blurred based on the motion experienced by the image capture device between capture of the reference image and the background image.

### Claim 5 (depends on 4)

The system of claim 4, wherein blurring of the background layer based on the motion experienced by the image capture device between capture of the reference image and the background image includes:
computing a directional blur kernel that follows the motion experienced by the image capture device between capture of the reference image and the background image; and
applying the directional blur kernel to the background layer.

### Claim 6 (depends on 5)

The system of claim 5, wherein strength with which the background layer is blurred is determined based on the motion experienced by the image capture device between capture of the reference image and the background image.

### Claim 7 (depends on 3)

The system of claim 3, wherein strength and direction in which the background layer is blurred is adjusted based on user input.

### Claim 8 (depends on 1)

The system of claim 1, wherein the motion experienced by the image capture device includes rotation of the image capture device to follow motion of the object of interest.

### Claim 9 (depends on 1)

The system of claim 1, wherein the generation of the panning image based on the combination of the object layer and the background layer includes:
edge of the depiction of the object of interest within the object layer being blurred; and
the blurred edge of the depiction of the object of interest within the object layer being blended with the background layer.

### Claim 10 (depends on 1)

The system of claim 1, wherein the background image includes one of the images captured before or after the reference image.

### Claim 11

A method for generating panning images, the method performed by a computing system including one or more processors, the method comprising:
obtaining, by the computing system, visual information, the visual information defining visual content of images captured by an image capture device while experiencing motion;
selecting, by the computing system, a reference image from the images, the visual content of the reference image having a first field of view of a scene including an object of interest;
selecting, by the computing system, a background image from the images, the background image different from the reference image, the visual content of the background image having a second field of view of the scene different from the first field of view based on the motion experienced by the image capture device between capture of the reference image and the background image;
generating, by the computing system, an object layer including a depiction of the object of interest based on segmentation of the depiction of the object of interest from the visual content of the reference image;
generating, by the computing system, a background layer based on combination of the visual content of the background image and the visual content of the reference image, wherein the combination of the visual content of the background image and the visual content of the reference image reduces the depiction of the object of interest in the background layer; and
generating, by the computing system, a panning image based on combination of the object layer and the background layer.

### Claim 12 (depends on 11)

The method of claim 11, wherein the reference image and the background image are stabilized based on the motion experienced by the image capture device between capture of the reference image and the background image such that overlapping portions of the first field of view and the second field of view are aligned.

### Claim 13 (depends on 11)

The method of claim 11, wherein the background layer is blurred before generating the panning image.

### Claim 14 (depends on 13)

The method of claim 13, wherein the background layer is blurred based on the motion experienced by the image capture device between capture of the reference image and the background image.

### Claim 15 (depends on 14)

The method of claim 14, wherein blurring of the background layer based on the motion experienced by the image capture device between capture of the reference image and the background image includes:
computing a directional blur kernel that follows the motion experienced by the image capture device between capture of the reference image and the background image; and
applying the directional blur kernel to the background layer.

### Claim 16 (depends on 15)

The method of claim 15, wherein strength with which the background layer is blurred is determined based on the motion experienced by the image capture device between capture of the reference image and the background image.

### Claim 17 (depends on 13)

The method of claim 13, wherein strength and direction in which the background layer is blurred is adjusted based on user input.

### Claim 18 (depends on 11)

The method of claim 11, wherein the motion experienced by the image capture device includes rotation of the image capture device to follow motion of the object of interest.

### Claim 19 (depends on 11)

The method of claim 11, wherein generating the panning image based on the combination of the object layer and the background layer includes:
blurring edge of the depiction of the object of interest within the object layer; and
blending the blurred edge of the depiction of the object of interest within the object layer with the background layer.

### Claim 20 (depends on 11)

The method of claim 11, wherein the background image includes one of the images captured before or after the reference image.
