# US 2018/0322679 A1 · Systems and methods for appearance mapping for compositing overlay graphics

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2018/0322679 A1
- Document ID: US20180322679A1
- Status: Published application
- Kind: A1
- Application number: 16/037,576
- Filed: 2018-07-17
- Published: 2018-11-08
- Earliest claimed: 2013-02-21
- Assignees: Dolby Laboratories Licensing Corporation
- Applicants: Dolby Laboratories Licensing Corporation
- Inventors: Timo KUNKEL; Ning XU; Tao CHEN; Bongsun LEE; Samir N. HULYALKAR
- Web page: https://patentgrep.com/patent/US20180322679A1

## CPC

- H04N 5/2723: {Insertion of virtual advertisement; Replacing advertisements physical present in the scene by virtual advertisement (data processing systems or methods specially adapted for marketing G06Q30/02)}
- G06F 18/22: Matching criteria, e.g. proximity measures
- G06T 7/90: Determination of colour characteristics
- G06T 11/60: Creating or editing images; Combining images with text
- G09G 5/10: Intensity circuits
- H04N 5/445: for displaying additional information (H04N5/50 takes precedence)
- H04N 5/58: in dependence upon ambient light
- H04N 9/67: for matrixing (camera processing pipelines for matrixing of colour signals H04N23/85)
- H04N 21/4122: {additional display device, e.g. video projector (digital output for controlling a plurality of local displays G06F3/1423)}
- H04N 21/42202: {environmental sensors, e.g. for detecting temperature, luminosity, pressure, earthquakes}
- H04N 21/431: Generation of visual interfaces {for content selection or interaction}; Content or additional data rendering
- H04N 21/4318: {by altering the content in the rendering process, e.g. blanking, blurring or masking an image region (image enhancement or restoration in general G06T5/00)}
- H04N 21/4622: {Retrieving content or additional data from different sources, e.g. from a broadcast channel and the Internet (web site content organization and management for information retrieval from the Internet G06F16/958; transmission by internet of broadcast information H04H60/82; stock exchange data over packet-switching network H04L12/1804; push services including data channel over packet-switching network H04L12/1859)}
- H04N 21/812: {involving advertisement data (advertising per se G06Q30/02)}
- H04N 21/8146: {involving graphical data, e.g. 3D object, 2D graphics}
- G06F 3/1454: {involving copying of the display data of a local workstation or window to a remote workstation or window so that an actual copy of the data is displayed simultaneously on two or more displays, e.g. teledisplay}
- G09G 2320/08: Arrangements within a display terminal for setting, manually or automatically, display parameters of the display terminal
- G09G 2340/10: Mixing of images, i.e. displayed pixel being the result of an operation, e.g. adding, on the corresponding input pixels
- G09G 2340/12: Overlay of images, i.e. displayed pixel being the result of switching between the corresponding input pixels
- G09G 2360/144: the light being ambient light
- G09G 2370/04: Exchange of auxiliary data, i.e. other than image data, between monitor and graphics controller
- H04N 21/4884: {for displaying subtitles}
- H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G09G: ARRANGEMENTS OR CIRCUITS FOR CONTROL OF INDICATING DEVICES USING STATIC MEANS TO PRESENT VARIABLE INFORMATION (arrangements for transferring data between digital computers and displays G06F3/14; static indicating arrangements comprising an association of a number of separate sources or light control cells G09F9/00; static indicating arrangements comprising integral associations of a number of light sources H01J, H01K, H10H, H05B33/12; scanning, transmission or reproduction of documents or the like, e.g. facsimile transmission, details thereof H04N1/00)

## Abstract

Systems and methods for overlaying a second image/video data onto a first image/video data are described herein. The first image/video data may be intended to be rendered on a display with certain characteristics—e.g., HDR, EDR, VDR or UHD capabilities. The second image/video data may comprise graphics, closed captioning, text, advertisement—or any data that may be desired to be overlaid and/or composited onto the first image/video data. The second image/video data may be appearance mapped according to the image statistics and/or characteristics of the first image/video data. In addition, such appearance mapping may be made according to the characteristics of the display that the composite data is to be rendered. Such appearance mapping is desired to render a composite data that is visually pleasing to a viewer, rendered upon a desired display.

## Claims

### Claim 1

A method for overlaying a second image over a plurality of first images, the method comprising:
receiving said plurality of first images and said second image, said first images differing in dynamic range and size from said second image, the plurality of first images being temporally related and forming a first scene;
mapping a dynamic range of the first scene to a dynamic range of a display to generate adjusted first images, wherein said mapped dynamic range of the first scene is different from a maximum dynamic range of the display;
performing appearance mapping of said second image to determine an adjusted second image that differs in dynamic range from said second image, wherein performing appearance mapping of said second image (i) is performed according to mapping information about the mapped dynamic range of the first scene and image statistics regarding the second image, the image statistics including at least one of average luminance, minimum luminance, mean luminance, maximum luminance, dynamic range, reflective white, color temperature, whitepoint, color gamut and color palette, and (ii) involves selecting a dynamic range based on both the dynamic range of the first scene and the capabilities of the display; and
forming a composite image by overlaying said adjusted second image onto a portion of at least one of said adjusted first images.

### Claim 2 (depends on 1)

The method of claim 1 wherein said first images comprise one of a group, said group comprising: High Dynamic Range (HDR) images, Extended Dynamic Range (EDR) images and Visual Dynamic Range (VDR) images.

### Claim 3 (depends on 1)

The method according to claim 1, wherein all method steps are repeated for at least one further scene of first images, to implement a scene-by-scene overlay method.

### Claim 4 (depends on 1)

The method of claim 1 wherein said second image comprises one of a group, said group comprising: graphics, close captioning, text, video overlay, Graphical User Interface (GUI), television (TV) guides, signals from broadcasting company, signals from content creation, subtitles, signals from set top box and signals from Audio-Video (AV) receiver.

### Claim 5 (depends on 1)

The method of claim 1 wherein performing appearance mapping further comprises:
mapping the dynamic range of said second image in accordance with the dynamic range of said adjusted first images.

### Claim 6 (depends on 1)

The method of claim 1 wherein performing appearance mapping further comprises:
mapping the color gamut of said second image in accordance with the color gamut of said first adjusted images.

### Claim 7 (depends on 1)

The method of claim 1 wherein performing appearance mapping further comprises:
mapping the luminance of said second image in accordance with the luminance of said first adjusted images.

### Claim 8 (depends on 1)

The method of claim 1 wherein said method further comprises:
receiving metadata regarding said display characteristics.

### Claim 9 (depends on 8)

The method of claim 8 wherein said metadata regarding said display's characteristics comprises one of a group, said group comprising: average luminance, minimum luminance, mean luminance, maximum luminance, reflective white, color temperature, whitepoint and color gamut.

### Claim 10 (depends on 9)

The method of claim 9 wherein performing appearance mapping further comprises:
mapping said second image in accordance with said metadata regarding said display's characteristics.

### Claim 11 (depends on 1)

The method of claim 1 wherein said method further comprises:
performing dynamic mapping of a third image to composite said third image with said first images, said third image intended to replace a part of said second image previously composited with said first images.

### Claim 12

A system for overlaying a second image over a plurality of first images, said system comprising:
a display management module capable of receiving said plurality of first images, which are temporally related and form a first scene;
a compositor module capable of:
receiving said second image, the second image differing in size and dynamic range from each of the plurality of the first images;
mapping a dynamic range of the first scene to a dynamic range of a display to generate adjusted first images, wherein said mapped dynamic range of the first scene is different from a maximum dynamic range of the display; and
performing appearance mapping said second image to form an adjusted second image, said adjusted second image differing in dynamic range from said second image, wherein performing appearance mapping of said second image (i) is performed according to mapping information about the mapped dynamic range of the first scene and image statistics regarding the second image, the image statistics including at least one of average luminance, minimum luminance, mean luminance, maximum luminance, dynamic range, reflective white, color temperature, whitepoint, color gamut and color palette and (ii) involves selecting a dynamic range based on both the dynamic range of the first scene and the capabilities of the display; and
a mixing module capable of mixing said adjusted second image onto at least one of said adjusted first images to form composite images, said composite images intended to be rendered upon said display.

### Claim 13 (depends on 12)

The system of claim 12 wherein said compositor module is capable of performing appearance mapping wherein said appearance mapping further comprises one of a group, said group comprising: luminance mapping, dynamic range mapping and color gamut mapping.

### Claim 14 (depends on 12)

The system of claim 12 wherein said system further comprises:
a dynamic mapping module capable of replacing a part of said second image that has been composited with a first image, with a third image.