# US 2020/0404132 A1 · Background replacement system and methods

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2020/0404132 A1
- Document ID: US20200404132A1
- Status: Published application
- Kind: A1
- Application number: 16/920,211
- Filed: 2020-07-02
- Published: 2020-12-24
- Earliest claimed: 2015-04-24
- Assignees: Shutterfly, Inc.
- Applicants: Shutterfly, LLC
- Inventors: Keith A. Benson
- Web page: https://patentgrep.com/patent/US20200404132A1

## CPC

- G06T 7/11: Region-based segmentation
- G06T 7/194: involving foreground-background segmentation
- G06V 10/141: Control of illumination
- H04N 23/56: provided with illuminating means
- H04N 23/74: by influencing the scene brightness using illuminating means
- G06T 2207/10152: Varying illumination
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION

## Abstract

A method of illuminating a subject during photography is disclosed. The method includes illuminating a background lighting system during a first image capture, illuminating a subject lighting system during a second image capture, and illuminating the background lighting system during a third image capture. The time from the first image capture to the third image capture is less than about 100 milliseconds. Also disclosed is a method for generating a composite mask of a subject, including receiving the three images, determining any movements of the subject between the first and third images, estimating an intermediate position of the subject during the second image, and generating a composite mask at the estimated intermediate position.

## Claims

### Claim at position 1

1-20. (canceled)

### Claim 21

A method of illuminating a subject during a photography session, comprising:
receiving an image capture signal; and
in response to receiving the image capture signal, initiating an illumination sequence for a single image capture sequence, including:
initiating a first illumination condition while a first image is captured;
initiating a second illumination condition while a second image is captured; and
initiating the first illumination condition while a third image is captured,
wherein the first illumination condition is an illumination of a background using a background lighting system, and the second illumination condition is an illumination of the subject using a subject lighting system.

### Claim 22 (depends on 21)

The method of claim 21, wherein a new image is generated from the first image, the second image, and the third image.

### Claim 23 (depends on 21)

The method of claim 21, wherein initiating the illumination sequence for the single image capture sequence further includes initiating the first illumination condition while a fourth image is captured.

### Claim 24 (depends on 21)

The method of claim 21, wherein initiating the second illumination condition while the second image is captured comprises illuminating the subject using a main light of the subject lighting system.

### Claim 25 (depends on 24)

The method of claim 24, wherein initiating the illumination sequence for the single image capture sequence further includes initiating the second illumination condition while a fourth image is captured by illuminating the subject using a fill light of the subject lighting system.

### Claim 26 (depends on 21)

The method of claim 21, wherein initiating the illumination sequence for the single image capture sequence further includes initiating a third illumination condition while a fourth image is captured, wherein the third illumination condition is ambient lighting.

### Claim 27 (depends on 26)

The method of claim 26, wherein the ambient lighting is an illumination without using the subject lighting system or the background lighting system.

### Claim 28 (depends on 21)

The method of claim 21, wherein the background lighting system is not illuminated when the second image is captured.

### Claim 29 (depends on 21)

The method of claim 21, wherein the subject lighting system is not illuminated when the first image is captured and when the third image is captured.

### Claim 30 (depends on 21)

The method of claim 21, wherein the illumination sequence for the single image capture sequence is for a single pose of the subject.

### Claim 31

A photography system for capturing multiple images of a subject, the photography system comprising:
a controller;
a camera that captures at least a first image, a second image, and a third image; and
a lighting system synchronized by the controller, the lighting system including:
a background lighting system in communication with the controller; and
a subject lighting system in communication with the controller;
wherein the controller includes a non-transitory storage medium and one or more processors, the non-transitory storage medium storing instructions that, when executed by the one or more processors, cause the controller to:
receive an image capture signal;
initiate an illumination of a background using the background lighting system while the camera captures the first image;
initiate an illumination of the subject using the subject lighting system while the camera captures the second image; and
initiate an illumination of the background using the background lighting system while the camera captures the third image.

### Claim 32 (depends on 31)

The photography system of claim 31, further comprising a light block positioned relative to the background lighting system and the subject such that the light block prevents light emitted from the background lighting system from illuminating the subject while the camera captures the first image and the third image.

### Claim 33 (depends on 31)

The photography system of claim 31, wherein a time difference between the capture of the first image and the second image and a time difference between the capture of the second image and the third image are equal.

### Claim 34 (depends on 31)

The photography system of claim 31, further comprising an image processing system that generates a new image from the first image, the second image, and the third image.

### Claim 35 (depends on 34)

The photography system of claim 34, wherein, to generate the new image, the image processing system generates a composite mask using the first image and the third image, overlays the composite mask onto the second image to remove a background from the second image, and inserts a new background into the second image.

### Claim 36

A method of illuminating a subject during a photography session, comprising:
receiving an image capture signal; and
in response to receiving the image capture signal, initiating an illumination sequence for a single image capture sequence, including:
initiating a first illumination condition while a first image is captured;
initiating a second illumination condition while a second image is captured; and
initiating the first illumination condition while a third image is captured,
wherein the first illumination condition is an illumination of the subject using a subject lighting system, and the second illumination condition is an illumination of a background using a background lighting system.

### Claim 37 (depends on 36)

The method of claim 36, wherein the background lighting system is not illuminated when the first image is captured and when the third image is captured.

### Claim 38 (depends on 36)

The method of claim 36, wherein the subject lighting system is not illuminated when the second image is captured.

### Claim 39 (depends on 36)

The method of claim 36, wherein initiating the first illumination condition while the first image is captured comprises illuminating the subject using a main light of the subject lighting system, and initiating the first illumination condition while the third image is captured comprises illuminating the subject using a fill light of the subject lighting system.

### Claim 40 (depends on 36)

The method of claim 36, wherein initiating the illumination sequence for the single image capture sequence further includes initiating a further illumination condition while a fourth image is captured, wherein the further illumination condition is one of the second illumination condition, and a fourth illumination condition comprising ambient lighting.
