Diagnostic evaluation of machine translators
AssigneeMicrosoft Corporation
InventorMing Zhou
Life of this patent
Estimating the term…
Abstract
A system for evaluating translation quality of a machine translator is discussed. The system includes a bilingual data generator configured to intermittently access a wide area network and generate a bilingual corpus from data received from the wide area network. The method also includes an example extraction component configured to receive an ontology input indicative of a plurality of ontological categories of evaluation and to extract evaluation examples from the bilingual corpus based on the ontology input. The system further includes an evaluation component configured to evaluate translation results from translation by a machine translator of the evaluation examples and to score the translation results according to the ontological categories.
Claims20
A system for evaluating translation quality of a machine translator, comprising:
- a bilingual data generator configured to intermittently access a wide area network and generate a bilingual corpus from data received from the wide area network;
- an example extraction component configured to receive an ontology input indicative of a plurality of ontological categories of evaluation and to extract evaluation examples from the bilingual corpus based on the ontology input;
- and
- an evaluation component configured to evaluate translation results from translation by a machine translator of the evaluation examples and to score the translation results according to the ontological categories.
A method of evaluating translation quality of a machine translator, comprising:
- intermittently generating and updating a bilingual corpus, augmented with linguistic information generated from performing semantic analysis of the bilingual corpus;
- when the bilingual corpus is generated, extracting evaluation examples, each evaluation example being extracted to test the machine translator in one of a plurality of predefined categories of evaluation;
- receiving translation results from the machine translator, for the evaluation examples;
- and
- evaluating the translation results in each of the predefined categories of evaluation;
- and storing evaluation results for display to a user.
A method of creating an evaluation corpus to perform diagnostic evaluation of a machine translator that translates language from a source language to a target language, the method comprising:
- receiving from a data source a representation of textual data in the source language and a corresponding target language;
- extracting a plurality of translation pairs from the representation with each translation pair including data in the source language and a data corresponding to a translation in the target language;
- selecting a checkpoint from the plurality of translation pairs by assigning an ontological category to the source language portion of the translation pair;
- and
- storing the checkpoint in an evaluation corpus.
Dependencies are inferred from the claim text.