# US 2022/0044675 A1 · Method for generating caption file through url of an av platform

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2022/0044675 A1
- Document ID: US20220044675A1
- Status: Published application
- Kind: A1
- Application number: 16/986,307
- Filed: 2020-08-06
- Published: 2022-02-10
- Earliest claimed: 2020-08-06
- Assignees: National Chiao Tung University
- Applicants: National Chiao Tung University
- Inventors: Sin Horng CHEN; Yuan Fu LIAO; Yih Ru WANG; Shaw Hwa HWANG; Bing Chih YAO; Cheng Yu YEH; You Shuo CHEN; Yao Hsing CHUNG; Yen Chun HUANG; Chi Jung HUANG; Li Te SHEN; Ning Yun KU
- Web page: https://patentgrep.com/patent/US20220044675A1

## CPC

- G10L 15/187: Phonemic context, e.g. pronunciation rules, phonotactical constraints or phoneme n-grams
- H04L 65/1089: by adding media; by removing media
- G10L 15/02: Feature extraction for speech recognition; Selection of recognition unit
- G10L 15/04: Segmentation; Word boundary detection
- G10L 15/16: using artificial neural networks
- G10L 15/18: using natural language modelling
- G10L 15/1822: {Parsing for meaning understanding}
- G10L 15/22: Procedures used during a speech recognition process, e.g. man-machine dialogue
- G10L 15/26: Speech to text systems (G10L15/08 takes precedence)
- G10L 15/30: Distributed recognition, e.g. in client-server systems, for mobile phones or network applications
- G10L 25/18: the extracted parameters being spectral information of each sub-band
- H04L 65/60: Network streaming of media packets
- H04L 65/612: for unicast
- H04L 67/02: based on web technology, e.g. hypertext transfer protocol [HTTP]
- G10L 2015/025: {Phonemes, fenemes or fenones being the recognition units}
- G10L: SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
- H04L: TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION (arrangements common to telegraphic and telephonic communication H04M)

## Abstract

The present invention provides a method for generating caption file through URL of an AV platform. By using various websites (such as YouTube, Instagram, Facebook, Twitter) for being inputted with the URL of a desired AV Platform and downloading a required AV file and inputting to an ASR (Automatic Speech Recognition) server according to the present invention. A speech recognition system in the ASR server can abstract an audio file from the AV file for a system operation to get a required caption file. Artificial Neural Networks are used in the present invention.

## Claims

### Claim 1

A method for generating caption file through URL of an AV platform, comprising steps as below:
(a) a server of an automatic speech recognition first parses a URL description given by a user and finds a relevant AV (audio-video) platform;
(b) sending an HTTP request to a web application interface provided, by a web server of the AV platform to obtain an HTTP reply of the web server;
(c) parsing a content in the HTTP reply to obtain a URL of an AV file, and download the AV file;
(d) abstracting an audio track in the AV file to obtain an audio sample, then send the audio sample to a speech recognition system for processing, and then generate a caption file.

### Claim 2 (depends on 1)

The method for generating caption file through URL of an AV platform according to claim 1, wherein the speech recognition system has a sentence breaking mechanism, firstly judging if a speech playing is ended. If the speech playing is not ended, detecting a beginning of a sentence, and then detecting a pause of the sentence, thereafter translating the sentence and recording a time interval, go back to judge if the speech playing is ended, if not ended, then repeat to translate, otherwise a processing is ended to form a caption file.

### Claim 3 (depends on 1)

The method for generating caption file through URL of an AV platform according to claim 1, wherein the speech recognition system includes a pre-processing step for audio, a step for extracting speech feature parameters, a phoneme recognition step, and a sentence decoding step.

### Claim 4 (depends on 3)

The method for generating caption file through URL of an AV platform according to claim 3, wherein the pre-processing step for audio includes a step for volume normalization and a step for noise reduction.

### Claim 5 (depends on 3)

The method for generating caption file through URL of an AV platform according to claim 3, wherein the step for extracting speech feature parameters uses a Short-Time Fourier Transform to obtain a Spectrogram.

### Claim 6 (depends on 5)

The method for generating caption file through URL of an AV platform according to claim 5, wherein the phoneme recognition step includes an acoustic model, the acoustic model is an artificial neural network for being inputted with the Spectrogram to obtain a pinyin sequence.

### Claim 7 (depends on 6)

The method for generating caption file through URL of an AV platform according to claim 6, wherein the inentence decoding step includes a language dictionary and a language model, the language model is an artificial neural network.

### Claim 8 (depends on 7)

The method for generating caption file through URL of an AV platform according to claim 7, wherein the language dictionary is used to spread the pinyin sequence into a two dimensional sequence.

### Claim 9 (depends on 8)

The method for generating caption file through URL of an AV platform according to claim 8, wherein the language model is used for interpreting the two dimensional sequence into the caption file.
