# US 2008/0304571 A1 · Content Receiving Apparatus, Method of Controlling Video-Audio Output Timing and Content Providing System

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2008/0304571 A1
- Document ID: US20080304571A1
- Status: Published application
- Kind: A1
- Application number: 11/661,347
- Filed: 2005-08-31
- Published: 2008-12-11
- Earliest claimed: 2004-09-02
- Assignees: Sony Corporation
- Inventors: Ikuo Tsukagoshi; Shinji Takada; Koichi Goto
- Web page: https://patentgrep.com/patent/US20080304571A1

## CPC

- H04N 21/8106: {involving special audio data, e.g. different tracks for different languages}
- H04N 21/2368: Multiplexing of audio and video streams
- H04N 21/43072: {of multiple content streams on the same device}
- H04N 21/4341: {Demultiplexing of audio and video streams}
- H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION

## Abstract

The present invention can reliably adjust the lip-sync between an video and audio at a decoder side, without making the viewer feel strangeness. In this invention, the encoded video frames to which video time-stamps VTS are attached and the encoded audio frames to which audio time-stamps ATS are attached, all received, are decoded, generating a plurality of video frames VF1 and a plurality of audio frames AF1. The video frames VF1 and audio frames AF1 are accumulated. Renderers 37 and 77 calculate the time difference resulting from a gap between the reference clock for the encoder side and the system time clock stc for the decoder side. In accordance with the time difference, the timing of outputting the plurality of video frames, one by one, on the basis of a timing of outputting the plurality of audio frames, one by one. Hence, the lip-sync can be achieved, while maintaining the continuity of the audio.

## Claims

### Claim 1

A content receiving apparatus characterized by comprising:
decoding means for receiving, from a content providing apparatus provided at an encoder side, a plurality of encoded video frames to which video time-stamps based on reference clock for the encoder side are attached and a plurality of encoded audio frames to which audio time-stamps based on a reference clock for the encoder side are attached, and for decoding the encoded video frames and the encoded audio frames;
storing means for storing a plurality of video frames that the decoding means has obtained by decoding the encoded video frames and a plurality of audio frames that the decoding means has obtained by decoding the encoded audio frames;
calculating means for calculating a time difference resulting from a gap between a clock frequency of a reference clock for the encoder side and a clock frequency of a system time clock for the decoder side; and
timing-adjusting means for adjusting a timing of outputting the plurality of video frames, one by one, in accordance with the time difference and on the basis of a timing of outputting the plurality of audio frames, one by one.

### Claim 2 (depends on 1)

The content receiving apparatus according to claim 1, characterized in that the timing-adjusting means outputs the video frames on the basis of the system time clock for the decoder side when the time difference is shorter than a predetermined time in accordance with the video time-stamps.

### Claim 3 (depends on 1)

The content receiving apparatus according to claim 1, characterized by further comprising receiving means which receives the reference clock for the encoder side transmitted by using UDP (User Datagram Protocol) which should work in real time from the content providing apparatus, and
the calculating means first synchronizes the system time clock for the decoder side with the reference clock for the encoder side and then calculates the time difference resulting from the gap between the clock frequency of the reference clock for the encoder side and the clock frequency of the system time clock for the decoder side.

### Claim 4 (depends on 1)

The content receiving apparatus according to claim 1, characterized by further comprising presetting means which presets the system time clock at the decoder side, by using the video time-stamp or the audio time-stamp, in accordance with a presetting sequence in which the video time-stamp and the audio time-stamp are applied in the order mentioned, and
the calculating means calculates the time difference resulting from the gap between the clock frequency of the reference clock for the encoder side and the clock frequency of the system time clock for the decoder side which has been preset.

### Claim 5 (depends on 1)

The content receiving apparatus according to claim 1, characterized by further comprising:
presetting means for presetting the system time clock for the decoder side by using the audio time-stamp, in accordance with a presetting sequence in which the video time-stamp and the audio time-stamp are applied in the order mentioned, when a content received is composed of audio data only; and
audio-data outputting means for outputting audio data of the audio frame on the basis of the system time clock, which has been preset, and the audio time-stamp.

### Claim 6 (depends on 1)

The content receiving apparatus according to claim 1, characterized by further comprising:
presetting means for presetting the system time clock for the decoder side by using the video time-stamp, in accordance with a presetting sequence in which the video time-stamp and the audio time-stamp are applied in the order mentioned, when a content received is composed of video data only; and
video-data outputting means for outputting video data of the video frame on the basis of the system time clock, which has been preset, and the video time-stamp.

### Claim 7

A method of controlling a video-audio output timing, characterized by comprising:
a decoding step of first receiving, from a content providing apparatus provided at an encoder side, a plurality of encoded video frames to which video time-stamps based on reference clock for the encoder side are attached and a plurality of encoded audio frames to which audio time-stamps based on a reference clock for the encoder side are attached, and then decoding the encoded video frames and the encoded audio frames in decoding means;
storing step of storing, in storing means, a plurality of video frames obtained by decoding the encoded video frames in the decoding means and a plurality of audio frames obtained by decoding the encoded audio frames in the decoding means;
difference calculating step of calculating, in calculating means, a time difference resulting from a gap between a clock frequency of a reference clock for the encoder side and a clock frequency of a system time clock for the decoder side; and
timing-adjusting step of adjusting, in adjusting means, a timing of outputting the plurality of video frames, one by one, in accordance with the time difference and on the basis of a timing of outputting the plurality of audio frames, one by one.

### Claim 8

A content providing system having a content providing apparatus and a content receiving apparatus, characterized in that the content providing apparatus comprises:
encoding means for generating a plurality of encoded video frames to which video time-stamps based on reference clock for an encoder side are attached and a plurality of encoded audio frames to which audio time-stamps based on a reference clock are attached; and
transmitting means for sequentially transmitting the plurality of encoded video frames and the plurality of encoded audio frames to the content receiving side, and
the content receiving apparatus comprises:
decoding means for receiving the plurality of encoded video frames to which video time-stamps are attached and the plurality of encoded audio frames to which audio time-stamps are attached from the content providing apparatus for an encoder side and for decoding the encoded video frames and the encoded audio frames;
storing means for storing the plurality of video frames that the decoding means has obtained by decoding the encoded video frames and the plurality of audio frames that the decoding means has obtained by decoding the encoded audio frames;
calculating means for calculating a time difference resulting from a gap between a clock frequency of a reference clock for the encoder side and a clock frequency of a system time clock for the decoder side; and
timing-adjusting means for adjusting a timing of outputting the plurality of video frames, one by one, in accordance with the time difference and on the basis of a timing of outputting the plurality of audio frames, one by one.
