# US 2023/0353814 A1 · Testing rendering of screen objects

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2023/0353814 A1
- Document ID: US20230353814A1
- Status: Published application
- Kind: A1
- Application number: 18/193,947
- Filed: 2023-03-31
- Published: 2023-11-02
- Earliest claimed: 2018-08-10
- Assignees: Nagravision S.A.
- Applicants: Nagravision S.A.
- Inventors: Douglas Gore; Ping Zou
- Web page: https://patentgrep.com/patent/US20230353814A1

## CPC

- H04N 21/43074: {of additional data with content streams on the same device, e.g. of EPG data or interactive icon with a TV programme}
- H04N 21/4884: {for displaying subtitles}
- H04N 17/04: for receivers
- H04N 21/2187: Live feed
- H04N 21/4348: {Demultiplexing of additional data and video streams}
- H04N 21/44008: {involving operations for analysing video streams, e.g. detecting features or characteristics in the video stream (arrangements characterised by components specially adapted for monitoring, identification or recognition of video in broadcast systems H04H60/59)}
- H04N 21/4425: Monitoring of client processing errors or hardware failure
- H04N 21/84: Generation or processing of descriptive data, e.g. content descriptors {(systems specially adapted for using meta-information in broadcast systems H04H60/73)}
- H04N 2017/008: {for television teletext}
- H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION

## Abstract

The present disclosure relates to methods and devices for testing video data being rendered at or using a media device. A plurality of video frames to be rendered is received, each frame comprising one or more primary screen objects and at least one further screen object. The received frames are rendered at or using the media device wherein the at least one further screen object is superimposed on the one or more primary screen objects of a given frame during rendering. The rendered frames are provided to a data model. Extracted metadata indicating the presence or absence of further screen objects in the rendered video frames is the output of the data model. The data model is also provided with original metadata associated with the video frames prior to rendering. The rendering of each further screen object is then tested based on the original metadata and extracted metadata relating to a given video frame. The disclosure also extends to associated methods and devices for generating training data for testing rendering of video frame and training a data model using the training data.

## Claims

### Claim 1

A method comprising:
accessing, by one or more processors, a rendered plurality of video frames rendered in accordance with first metadata that specifies rendering of one or more secondary screen objects superimposed over one or more primary screen objects;
generating, by the one or more processors, a validation result that indicates whether the rendered plurality of video frames is rendered correctly, the generating of the validation result comprising:
identifying a characteristic of a secondary screen object in a video frame among the rendered plurality of video frames;
comparing the identified characteristic to a corresponding characteristic represented in the first metadata in accordance with which the rendered plurality of video frames was rendered; and
detecting a variance in the identified characteristic based on the comparing of the identified characteristic to the corresponding characteristic; and
providing, by the one or more processors, the validation result that indicates whether the rendered plurality of video frames is rendered correctly.

### Claim 2 (depends on 1)

The method of claim 1, further comprising:
in response to the variance being detected, calculating art offset based on the detected variance; and
causing a media device to adjust a further rendering of the plurality of video frames based on the calculated offset.

### Claim 3 (depends on 1)

The method of claim 1, wherein:
the generating of the validation result further includes:
inputting the rendered plurality of video frames into a data model trained to indicate, for each video frame, whether any secondary screen objects are present in that video frame, the data model being trained based on a reference plurality of video frames and based on reference data items associated with the reference plurality of video frames, each of the reference data items indicating whether any secondary screen objects are present in a corresponding video frame among the reference plurality of video frames, the trained data model outputting second metadata that indicates, for each inputted video frame, whether any secondary screen objects are present in that inputted video frame.

### Claim 4 (depends on 3)

The method of claim 3, wherein:
the providing of the validation result that indicates whether the rendered plurality of video frames is rendered correctly is based on a comparison of the first metadata that specifies rendering of the one or more secondary screen objects to the second metadata that indicates, for each inputted video frame, whether any secondary screen objects are present in that inputted video frame.

### Claim 5 (depends on 3)

The method of claim 3, further comprising:
providing a media device with the plurality of video frames to be rendered in accordance with the first metadata, the media device outputting the rendered plurality of video frames rendered in accordance with the first metadata; and wherein:
the accessing of the rendered plurality of video frames includes accessing an output of the media device to which the plurality of video frames is provided.

### Claim 6 (depends on 3)

The method of claim 3, wherein:
the data model is trained based on extracted metadata that indicates, for each video frame in the reference plurality of video frames, whether any secondary screen object is present in that video frame.

### Claim 7 (depends on 3)

The method of claim 3, wherein:
the data model is trained based on a function that reduces errors between extracted metadata and the reference data items that each indicate whether any secondary screen objects are present in a corresponding video frame among the reference plurality of video frames, the extracted metadata indicating, for each video frame in the reference plurality of video frames, whether any secondary screen object is present in that video frame.

### Claim 8

A system comprising
one or more processors; and
a memory storing instructions that, when executed by at least one processor among the one or more processors, cause the system to perform operations comprising:
accessing a rendered plurality of video frames rendered in accordance with first metadata that specifies rendering of one or more secondary screen objects superimposed over one or more primary screen objects;
generating a validation result that indicates whether the rendered plurality of video frames is rendered correctly, the generating of the validation result comprising:
identifying a characteristic of a secondary screen object in a video frame among the rendered plurality of video frames;
comparing the identified characteristic to a corresponding characteristic represented in the first metadata in accordance with which the rendered plurality of video frames was rendered; and
detecting a variance in the identified characteristic based on the comparing of the identified characteristic to the corresponding characteristic; and
providing the validation result that indicates whether the rendered plurality of video frames is rendered correctly.

### Claim 9 (depends on 8)

The system of claim 8, wherein the operations further comprise:
in response to the variance being detected, calculating an offset based on the detected variance; and
causing a media device to adjust a further rendering of the plurality of video frames based on the calculated offset.

### Claim 10 (depends on 8)

The system of claim 8, wherein
the generating of the validation result further includes:
inputting the rendered plurality of video frames into a data model trained to indicate, for each video frame, whether any secondary screen objects are present in that video frame, the data model being trained based on a reference plurality of video frames and based on reference data items associated with the reference plurality of video frames, each of the reference data items indicating whether any secondary screen objects are present in a corresponding video frame among the reference plurality of video frames, the trained data model outputting second metadata that indicates, for each inputted video frame, whether any secondary screen objects are present in that inputted video frame.

### Claim 11 (depends on 10)

The system of claim 10, wherein:
the providing of the validation result that indicates whether the rendered plurality of video frames is rendered correctly is based on a comparison of the first metadata that specifies rendering of the one or more secondary screen objects to the second metadata that indicates, for each inputted video frame, whether any secondary screen objects are present in that inputted video frame.

### Claim 12 (depends on 10)

The system of claim 10, wherein the operations further comprise
providing a media device with the plurality of video frames to be rendered in accordance with the first metadata, the media device outputting the rendered plurality of video frames rendered in accordance with the first metadata; and wherein:
the accessing of the rendered plurality of video frames includes accessing an output of the media device to which the plurality of video frames is provided.

### Claim 13 (depends on 10)

The system of claim 10, wherein:
the data model is trained based on extracted metadata that indicates, for each video frame in the reference plurality of video frames, whether any secondary screen object is present in that video frame.

### Claim 14 (depends on 10)

The system of claim 10, wherein:
the data model is trained based on a function that reduces errors between extracted metadata and the reference data items that each indicate whether any secondary screen objects are present in a corresponding video frame among the reference plurality of video frames, the extracted metadata indicating, for each video frame in the reference plurality of video frames, whether any secondary screen object is present in that video frame.

### Claim 15

A non-transitory machine-readable medium comprising instructions that, when executed by one or more processors of a machine, cause the machine to perform operations comprising:
accessing a rendered plurality of video frames rendered in accordance with first metadata that specifies rendering of one or more secondary screen objects superimposed over one or more primary screen objects;
generating a validation result that indicates whether the rendered plurality of video frames is rendered correctly, the generating of the validation result comprising:
identifying a characteristic of a secondary screen object in a video frame among the rendered plurality of video frames;
comparing the identified characteristic to a corresponding characteristic represented in the first metadata in accordance with which the rendered plurality of video frames was rendered; and
detecting a variance in the identified characteristic based on the comparing of the identified characteristic to the corresponding characteristic; and
providing the validation result that indicates whether the rendered plurality of video frames is rendered correctly.

### Claim 16 (depends on 15)

The non-transitory machine-readable medium of claim 15, wherein the operations further comprise:
in response to the variance being detected, calculating an offset based on the detected variance; and
causing a media device to adjust a further rendering of the plurality of video frames based on the calculated offset.

### Claim 17 (depends on 15)

The non-transitory machine-readable medium of claim 15, wherein:
the generating of the validation result further includes:
inputting the rendered plurality of video frames into a data model trained to indicate, for each video frame, whether any secondary screen objects are present in that video frame, the data model being trained based on a reference plurality of video frames and based on reference data items associated with the reference plurality of video frames, each of the reference data items indicating whether any secondary screen objects are present in a corresponding video frame among the reference plurality of video frames, the trained data model outputting second metadata that indicates, for each inputted video frame, whether any secondary screen objects are present in that inputted video frame.

### Claim 18 (depends on 17)

The non-transitory machine-readable medium of claim 17, wherein:
the providing of the validation result that indicates whether the rendered plurality of video frames is rendered correctly is based on a comparison of the first metadata that specifies rendering of the one or more secondary screen objects to the second metadata that indicates, for each inputted video frame, whether any secondary screen objects are present in that inputted video frame.

### Claim 19 (depends on 17)

The non-transitory machine-readable medium of claim 17, wherein:
the data model is trained based on extracted metadata that indicates, for each video frame in the reference plurality of video frames, whether any secondary screen object is present in that video frame.

### Claim 20 (depends on 17)

The non-transitory machine-readable medium of claim 17, wherein:
the secondary screen object in the video frame among the rendered plurality of video frames is a timed text instruction;
the identifying of the characteristic of the timed text instruction identifies a time stamp relative to a primary screen object among the one or more primary screen objects; and
the detected variance indicates that timed text corresponding to the timed text instruction is out of sync.
