# US 2023/0066958 A1 · Method for generating a composite image

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2023/0066958 A1
- Document ID: US20230066958A1
- Status: Published application
- Kind: A1
- Application number: 17/411,887
- Filed: 2021-08-25
- Published: 2023-03-02
- Earliest claimed: 2021-08-25
- Assignees: FotoNation Limited
- Applicants: FotoNation Limited
- Inventors: Razvan CONDOROVICI; Andra STAN; Cosmin STAN
- Web page: https://patentgrep.com/patent/US20230066958A1

## CPC

- H04N 5/272: Means for inserting a foreground image in a background image, i.e. inlay, outlay
- H04N 9/73: Colour balance circuits, e.g. white balance circuits or colour temperature control (camera processing pipelines for colour balance H04N23/88)
- G06T 7/194: involving foreground-background segmentation
- H04N 9/76: for mixing of colour signals (H04N9/75 takes precedence)
- H04N 23/88: for colour balance, e.g. white-balance circuits or colour temperature control
- G06T 2207/30201: Face
- H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL

## Abstract

A method for generating a composite image comprises: detecting a color temperature of a background image; acquiring from a camera through an image signal processor, ISP, performing white balance correction of acquired image data, an image including a foreground region including face of a user; and detecting a color temperature of the foreground region. Responsive to the color temperature for the foreground region differing from that of the background image by more than a threshold amount, a color temperature for white balance correction of a subsequently acquired image is set which causes skin pixels within the foreground region of the subsequently acquired image to have a color temperature closer to the color temperature for the background image. Pixel values of the foreground region are combined with pixel values of the background image corresponding to a background region of the acquired image to provide the composite image.

## Claims

### Claim 1

A method for generating a composite image comprising:
a) selecting an image as a background for the composite image wherein the image comprises a static image or a video stream;
b) detecting a color temperature of at least a portion of said background image,
said color temperature for said background image corresponding to a triplet of red, green and blue values for an illuminant of said color temperature for said background image, wherein when said background image is the video stream, color temperature is detected at every Nth frame;
c) acquiring from a camera through an image signal processor “ISP” performing white balance correction of acquired image data, an image including a foreground region including face of a user;
d) detecting a color temperature of at least a portion of said foreground region, said color temperature for said foreground region corresponding to a triplet of red, green and blue values for skin under an illuminant of said color temperature for said foreground region;
e) responsive to said color temperature for said foreground region differing from said color temperature for said background image by more than a threshold amount, setting a color temperature for said white balance correction of a subsequently acquired image which causes skin pixels within said foreground region of said subsequently acquired image to have a color temperature closer to said color temperature for said background image;
f) combining pixel values of said foreground region with pixel values of said background image corresponding to a background region of said acquired image to provide said composite image; and
g) repeating steps c) to f) at least once.

### Claim 2 (depends on 1)

A method according to claim 1 further comprising:
responsive to said color temperature for said foreground region differing from said color temperature for said background image:
identifying a triplet for a foreground region temperature corresponding to a temperature of said background image; and
mapping values of said triplet for said foreground region to said identified triplet to determine an amplification factor for at least one of said red, green or blue channels of said acquired image; and
amplifying pixel values of said foreground region according to said one or more amplification factors; and
wherein said combining comprises:
combining amplified pixel values of said foreground region with pixel values of said background image corresponding to a background region of said acquired image to provide said composite image.

### Claim 3 (depends on 2)

The method of claim 2 wherein when said color temperature of said background image is greater than said color temperature for said foreground region, said amplification factor for each of said blue and green channels of said foreground region is greater than 1 and said amplification factor for said red channel of said foreground region is 1.

### Claim 4 (depends on 3)

The method of claim 3 comprising determining a gamma function for amplifying values of said blue and green channels without saturating said amplifying values of said blue and green channels.

### Claim 5 (depends on 1)

The method of claim 1 wherein said color temperature for said foreground region is one of a plurality of discrete foreground temperatures and wherein said step of detecting a color temperature of at least a portion of said foreground region comprises determining the one of the plurality of discrete foreground temperatures whose triplet is closest in value to a maximum number of pixels in at least a face region of said foreground region.

### Claim 6 (depends on 5)

The method of claim 5 wherein said color temperature for said background image is one of a plurality of discrete background temperatures and wherein said step of detecting a color temperature of at least a portion of said background image comprises determining the one of the plurality of discrete background temperatures whose triplet is closest in value to a maximum number of pixels in said at least a portion of said background image.

### Claim 7 (depends on 6)

The method of claim 6 wherein at least some of said triplets of values for temperatures of said foreground region do not correspond to said triplets of values for the same temperatures of said background image.

### Claim 8 (depends on 1)

The method of claim 1 performed by a conferencing system client application during a conference call and further comprising the step of transmitting said composite image to client applications of other users of said conference call.

### Claim 9 (depends on 1)

The method of claim 1 further comprising detecting a face region within said foreground region.

### Claim 10 (depends on 9)

The method of claim 9 further comprising determining whether said face region is over exposed or under exposed and responsive to said face region being over exposed or under exposed by more than a threshold amount, adjusting an exposure setting of said camera for said subsequently acquired image to reduce the extent of said over or under exposure.

### Claim 11 (depends on 10)

The method of claim 10 comprising determining whether said face region is over exposed or under exposed in accordance with whether an illumination value of said face region differs from an illumination value of said background image by more than a threshold amount.

### Claim 12 (depends on 11)

The method of claim 11 wherein said illumination value is an average illumination value.

### Claim 13 (depends on 1)

The method of claim 1 further comprising repeating steps a) and b) for successive images chosen as background images.

### Claim 14 (depends on 13)

The method of claim 13 wherein said successive images are images from either: a stored video sequence; or a live stream acquired from a source other than said camera.

### Claim 15 (depends on 1)

The method of claim 1 wherein steps a) and b) are performed after detecting said foreground region within said acquired image and wherein said at least a portion of said background image comprises a portion of said background image corresponding to a background region of said acquired image.

### Claim 16 (depends on 1)

The method of claim 1 further comprising: obtaining a color temperature for said acquired image determined by said ISP; and said setting a color temperature for said white balance correction comprises adjusting said color temperature for said acquired image according to the extent to which said color temperature for said foreground region differs from said color temperature for said background image.

### Claim 17 (depends on 1)

A conferencing system comprising memory and one or more processors configured to connect to a video camera; and
perform the steps of claim 1.

### Claim 18 (depends on 1)

A computer program product comprising a computer readable medium on which processor instructions are stored which when executed on a computing device causes the processor to perform a process of operably connecting to a video camera and performing the steps of claim 1.
