# US 12,118,916 B2 · Image display apparatus with AI-based image processing

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 12,118,916 B2
- Document ID: US12118916B2
- Status: Granted
- Kind: B2
- Application number: 17/599,385
- Filed: 2020-03-27
- Published: 2022-06-23
- Granted: 2024-10-15
- Earliest claimed: 2019-03-29
- Assignees: LG Electronics Inc.
- Applicants: LG Electronics Inc.
- Inventors: Jeonghyu Yang; Seoksoo Lee; Jingyeong Kim
- Web page: https://patentgrep.com/patent/US12118916B2

## CPC

- G09G 3/2096: {Details of the interface to the display terminal specific for a flat panel (suitable for both CRT and flat panel G09G5/006; specific for a CRT G09G1/167)}
- H04N 21/42204: {User interfaces specially adapted for controlling a client device through a remote control device; Remote control devices therefor (constructive details of casings for the remote control device H01H9/0235; user interfaces for controlling a tuning device of a television receiver through a remote control H03J9/00; remote control of peripheral devices connected to a television receiver through the remote control device of the television receiver H04B1/205)}
- G06F 3/005: {Input arrangements through a video camera}
- G06F 3/147: using display panels
- G06F 3/162: {Interface to dedicated audio devices, e.g. audio drivers, interface to CODECs}
- G06F 3/167: {Audio in a user interface, e.g. using voice commands for navigating, audio feedback}
- H04N 21/42203: {sound input device, e.g. microphone}
- H04N 21/4223: Cameras (H04N23/00 takes precedence)
- H04N 21/439: Processing of audio elementary streams
- H04N 21/44: Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs
- H04N 21/44008: {involving operations for analysing video streams, e.g. detecting features or characteristics in the video stream (arrangements characterised by components specially adapted for monitoring, identification or recognition of video in broadcast systems H04H60/59)}
- H04N 21/44213: {Monitoring of end-user related data (arrangements for monitoring the users' behaviour or opinions in broadcast systems H04H60/33)}
- H04N 21/44218: {Detecting physical presence or behaviour of the user, e.g. using sensors to detect if the user is leaving the room or changes his face expression during a TV programme (methods or arrangements for recognising human body or animal bodies or body parts G06V40/10; methods or arrangements for acquiring or recognising human faces, facial parts, facial sketches, facial expressions G06V40/16; methods or arrangements for recognising movements or behaviour G06V40/20; arrangements for identifying users in broadcast systems H04H60/45)}
- H04N 21/466: Learning process for intelligent management, e.g. learning user preferences for recommending movies {(services using the results of monitoring in broadcast systems H04H60/61)}
- H04N 21/4852: {for modifying audio parameters, e.g. switching between mono and stereo}
- H04N 21/4854: {for modifying image parameters, e.g. image brightness, contrast}
- G09G 2320/08: Arrangements within a display terminal for setting, manually or automatically, display parameters of the display terminal
- G09G 2370/12: Use of DVI or HDMI protocol in interfaces along the display data pipeline
- G09G: ARRANGEMENTS OR CIRCUITS FOR CONTROL OF INDICATING DEVICES USING STATIC MEANS TO PRESENT VARIABLE INFORMATION (arrangements for transferring data between digital computers and displays G06F3/14; static indicating arrangements comprising an association of a number of separate sources or light control cells G09F9/00; static indicating arrangements comprising integral associations of a number of light sources H01J, H01K, H10H, H05B33/12; scanning, transmission or reproduction of documents or the like, e.g. facsimile transmission, details thereof H04N1/00)
- H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)

## Abstract

The present disclosure relates to an image display apparatus. The image display apparatus according to an embodiment of the present disclosure comprises: a signal processor configured to perform image processing of an input image, and audio processing of audio corresponding to the input image; a virtual sensor device configured to receive image and audio information processed by the signal processor; a sensor device configured to collect externally captured image information or sound information; and an artificial intelligence (AI) processor configured to perform the image processing of the input image or the audio processing based on AI using the image and audio information from the virtual sensor device, and the image or sound information from the sensor device. Accordingly, the image display apparatus may operate based on AI.

## Claims

### Claim 1

An image display apparatus comprising:
a signal processor configured to perform image processing of an input image, and audio processing of audio corresponding to the input image;
a virtual sensor device configured to receive image and audio information processed by the signal processor;
a sensor device configured to collect externally captured image information or sound information; and
an artificial intelligence (AI) processor configured to:
perform the image processing of the input image or the audio processing based on AI using the image and audio information from the virtual sensor device, and the image or sound information from the sensor device, and
when recognition of an ambient illumination value, ambient noise, and speech is performed based on an activation of an illumination sensor and a microphone after the image display apparatus is turned on, perform a preferred picture quality setting corresponding to the ambient illuminance value and perform a preferred volume setting corresponding to the ambient noise and viewer identification based on the speech,
wherein the signal processor or the AI processor is configured to:
in response to a resolution of the input image being changed, calculate an original resolution and a compression level of the input image based on learning using a neural network, and perform image quality processing corresponding to the calculated original resolution of the input image,
as the calculated original resolution increases, increase an enhancement intensity for the input image,
as the calculated compression level increases, increase a blurring intensity for the input image, and
in response to the original resolution of the input image being changed while reproducing the input image, sequentially change an image quality setting of the input image from a first setting to a second setting.

### Claim 2 (depends on 1)

The image display apparatus of claim 1, wherein:
the signal processor is configured to process data associated with the input image;
the virtual sensor device is configured to receive data information processed by the signal processor; and
the AI processor is configured to perform the image processing of the input image or the audio processing based on AI using the data information, the image and audio information from the virtual sensor device, and the image or sound information from the sensor device.

### Claim 3 (depends on 1)

The image display apparatus of claim 1, wherein:
the signal processor receives a user input signal from a remote controller;
the sensor device is configured to collect information on the user input signal; and
the AI processor is configured to perform the image processing of the input image or the audio processing based on AI using the information on the user input signal, the image and audio information from the virtual sensor device, and the image or sound information from the sensor device.

### Claim 4 (depends on 1)

The image display apparatus of claim 1, wherein:
the sensor device is configured to collect viewing environment information based on illumination information from the illumination sensor and motion information from a motion sensor; and
the AI processor is configured to perform the image processing of the input image or the audio processing based on AI using the viewing environment information, the image and audio information from the virtual sensor device, and the image or sound information from the sensor device.

### Claim 5 (depends on 1)

The image display apparatus of claim 1, wherein the sensor device is configured to collect externally captured image information from a camera, and collects external sound information from the microphone.

### Claim 6 (depends on 1)

The image display apparatus of claim 1, wherein the viewing environment information comprises at least one of screen brightness information and viewing concentration information.

### Claim 7 (depends on 1)

The image display apparatus of claim 1, wherein the signal processor comprises an image quality processor configured to perform image quality processing,
wherein the image quality processor is configured to perform the image quality processing according to an image quality set by the AI processor, based on the image information from the virtual sensor device, and the image information from the sensor device.

### Claim 8 (depends on 7)

The image display apparatus of claim 7, wherein based on an image type of the input image and viewing environment information related to a viewer, the signal processor is configured to perform the image quality processing according to the image quality set by the AI processor.

### Claim 9 (depends on 1)

The image display apparatus of claim 1, wherein based on on-time information of the image display apparatus, channel information, video genre information, and image quality setting information according to illumination, the signal processor is configured to perform the image quality processing according to the image quality set by the AI processor.

### Claim 10 (depends on 1)

The image display apparatus of claim 1, wherein the signal processor comprises an audio processor configured to perform audio processing,
wherein based on the audio information from the virtual sensor device and the sound information from the sensor device, the audio processor is configured to perform the audio processing according to a sound quality set by the AI processor.

### Claim 11 (depends on 1)

The image display apparatus of claim 1, wherein based on an audio type of the audio corresponding to the input image, and the viewing environment information related to the viewer, the signal processor is configured to perform the audio processing according to the sound quality set by the AI processor.

### Claim 12 (depends on 1)

The image display apparatus of claim 1, wherein based on on-time information of the image display apparatus, channel information, video genre information, and volume setting information in response to the ambient noise, the signal processor is configured to perform the audio processing according to the sound quality set by the AI processor.

### Claim 13 (depends on 1)

The image display apparatus of claim 1, further comprising a remote controller including a volume control key, an image quality control key, and a channel control key,
wherein based on at least one of brightness, contrast, sharpness, and chroma information set by the image quality control key, the signal processor is configured to perform the image quality processing according to the image quality set by the AI processor.

### Claim 14 (depends on 1)

The image display apparatus of claim 1, wherein based on viewer position information which is calculated using image information captured by the sensor device, the signal processor is configured to perform the image quality processing according to the image quality set by the AI processor.

### Claim 15 (depends on 1)

The image display apparatus of claim 1, wherein based on the viewer position information which is calculated using the image information captured by the sensor device, the signal processor is configured to perform the audio processing according to the sound quality set by the AI processor.

### Claim 16 (depends on 1)

The image display apparatus of claim 1, wherein the signal processor comprises:
an object detector configured to detect an object in the input image;
an object recognition trainer configured to perform training for object recognition; and
an object recognizer configured to recognize an object based on information from the object recognition trainer and the detected object.

### Claim 17 (depends on 16)

The image display apparatus of claim 16, wherein the signal processor further comprises an object information processor configured to process object recognition information from an external source,
wherein the object recognizer is configured to recognize an object based on information from the object recognition trainer, the detected object, and the object recognition information received from the external source.

### Claim 18 (depends on 16)

The image display apparatus of claim 16, wherein the signal processor comprises:
a region extractor configured to extract a text region in the input image;
a text converter configured to convert the extracted text region into text;
an information extractor configured to extract information from the converted text; and
a genre analyzer configured to analyze a genre of the extracted information.

### Claim 19 (depends on 1)

The image display apparatus of claim 1, wherein the artificial intelligence (AI) processor is further configured to:
in response to operation of a volume control key of a remote controller and the ambient noise through the microphone, perform analysis of a volume control pattern and perform automatic volume control based on a result of the analysis.