# US 6,731,825 B1 · Apparatus and method for producing images using digitally stored dynamic background sets

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 6,731,825 B1
- Document ID: US6731825B1
- Status: Granted
- Kind: B1
- Application number: 09/613,926
- Filed: 2000-07-11
- Granted: 2004-05-04
- Earliest claimed: 2000-07-11
- Assignees: Sarnoff Corporation
- Inventors: Alfonse Anthony Acampora; Nicola John Fedele; David Paul Bortfeld; Paul Hashfield
- Web page: https://patentgrep.com/patent/US6731825B1

## CPC

- H04N 5/2224: {related to virtual studio applications}
- H04N 5/272: Means for inserting a foreground image in a background image, i.e. inlay, outlay
- H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION

## Abstract

A method and apparatus for providing a composite of stored dynamic background activities with newly filmed or captured live foreground activities, thereby providing a real-time and on-site guide to actors and directors.

## Claims

### Claim 1

Method for providing a composite image sequence, said method comprising the steps of:
a) retrieving a background image sequence along with ancillary data representative of a scene condition when said background image sequence was captured, wherein said ancillary data for said background image sequence is generated when said background image sequence is captured;
b) retrieving a foreground image sequence along with ancillary data representative of the a scene condition when said foreground image sequence was captured, wherein said ancillary data for said foreground image sequence is generated when said foreground image sequence is captured; and
c) compositing said background image sequence with said foreground image sequence in accordance with said ancillary data of said background image sequence and said ancillary data of said foreground image sequence to form the composite image sequence.

### Claim 2 (depends on 1)

The method of claim 1, wherein said compositing step c) composites said background image sequence with said foreground image sequence in low resolution.

### Claim 3 (depends on 1)

The method of claim 1, further comprising the step of:
b1) performing an image processing operation on said retrieved background image sequence to produce a modified background image sequence, wherein said compositing step c) composites said modified background image sequence with said foreground image sequence.

### Claim 4 (depends on 3)

The method of claim 3, wherein said image processing operation is a mosaic operation that interpolates said retrieved background image sequence into said modified background image sequence having a different view point.

### Claim 5 (depends on 3)

The method of claim 3, wherein said image processing operation decimates said retrieved background image sequence into said modified background image sequence having a lower resolution.

### Claim 6 (depends on 1)

The method of claim 1, wherein said ancillary data comprises information associated with a camera parameter.

### Claim 7 (depends on 1)

The method of claim 1, wherein said ancillary data comprises information associated with a lighting condition.

### Claim 8 (depends on 1)

The method of claim 1, further comprising the step of:
d) distributing said composite image sequence to a viewfinder of a camera.

### Claim 9 (depends on 1)

The method of claim 1, further comprising the step of:
d) distributing said composite image sequence to a monitor.

### Claim 10 (depends on 1)

The method of claim 1, wherein said ancillary data is stored within a video field or frame of said background image sequence.

### Claim 11 (depends on 10)

The method of claim 10, wherein said ancillary data is stored within a vertical blanking interval of said video field or frame.

### Claim 12 (depends on 1)

The method of claim 1, wherein said ancillary data is stored within a video field or frame of said foreground image sequence.

### Claim 13 (depends on 12)

The method of claim 12, wherein said ancillary data is stored within a vertical blanking interval of said video field or frame.

### Claim 14

An apparatus for providing a composite image sequence, said apparatus comprising:
a first storage for storing a background image sequence along with ancillary data representative of a scene condition when said background image sequence was captured, wherein said ancillary data for said background image sequence is generated when said background image sequence is captured;
a second storage for storing a foreground image sequence along with ancillary data representative of then a scene condition when said foreground image sequence was captured, wherein said ancillary data for said foreground image sequence is generated when said foreground image sequence is captured; and
a controller, coupled to said first and second storages, for compositing said background image sequence with said foreground image sequence in accordance with said ancillary data of said background image sequence and said ancillary data of said foreground image sequence to form the composite image sequence.

### Claim 15 (depends on 14)

The apparatus of claim 14, wherein said controller composites said background image sequence with said foreground image sequence in low resolution.

### Claim 16 (depends on 14)

The apparatus of claim 14, further comprising:
a pyramid generator, coupled to said controller, for performing an image processing operation on said retrieved background image sequence to produce a modified background image sequence, wherein said controller composites said modified background image sequence with said foreground image sequence.

### Claim 17 (depends on 16)

The apparatus of claim 16, wherein said pyramid generator interpolates said retrieved background image sequence into said modified background image sequence having a different view point.

### Claim 18 (depends on 16)

The apparatus of claim 16, wherein said pyramid generator decimates said retrieved background image sequence into said modified background image sequence having a lower resolution.

### Claim 19 (depends on 14)

The apparatus of claim 14, further comprising:
a camera, coupled to said controller, wherein said composite image sequence is distributed to said viewfinder of said camera.

### Claim 20

A computer-readable medium having stored thereon a plurality of instructions, the plurality of instructions including instructions which, when executed by a processor, cause the processor to perform the steps comprising of:
a) retrieving a background image sequence along with ancillary data representative of a scene condition when said background image sequence was captured, wherein said ancillary data for said background image sequence is generated when said background image sequence is captured;
b) retrieving a foreground image sequence along with ancillary data representative of a scene condition when said foreground image sequence was captured, wherein said ancillary data for said foreground image sequence is generated when said foreground image sequence is captured; and
c) compositing said background image sequence with said foreground image sequence in accordance with said ancillary data of said background image sequence and said ancillary data of said foreground image sequence to form the composite image sequence.
