Access

You are not currently logged in.

Access your personal account or get JSTOR access through your library or other institution:

login

Log in to your personal account or through your institution.

The NIST 2008 Metrics for machine translation challenge—overview, methodology, metrics, and results

Mark Przybocki, Kay Peterson, Sébastien Bronsart and Gregory Sanders
Machine Translation
Vol. 23, No. 2/3, Automated Metrics for Machine Translation Evaluation (September 2009), pp. 71-103
Published by: Springer
Stable URL: http://www.jstor.org/stable/40783461
Page Count: 33
  • Download ($43.95)
  • Cite this Item
The NIST 2008 Metrics for machine translation challenge—overview, methodology, metrics, and results
Preview not available

Abstract

This paper discusses the evaluation of automated metrics developed for the purpose of evaluating machine translation (MT) technology. A general discussion of the usefulness of automated metrics is offered. The NIST MetricsMATR evaluation of MT metrology is described, including its objectives, protocols, participants, and test data. The methodology employed to evaluate the submitted metrics is reviewed. A summary is provided for the general classes of evaluated metrics. Overall results of this evaluation are presented, primarily by means of correlation statistics, showing the degree of agreement between the automated metric scores and the scores of human judgments. Metrics are analyzed at the sentence, document, and system level with results conditioned by various properties of the test data. This paper concludes with some perspective on the improvements that should be incorporated into future evaluations of metrics for MT evaluation.

Page Thumbnails

  • Thumbnail: Page 
[71]
    [71]
  • Thumbnail: Page 
72
    72
  • Thumbnail: Page 
73
    73
  • Thumbnail: Page 
74
    74
  • Thumbnail: Page 
75
    75
  • Thumbnail: Page 
76
    76
  • Thumbnail: Page 
77
    77
  • Thumbnail: Page 
78
    78
  • Thumbnail: Page 
79
    79
  • Thumbnail: Page 
80
    80
  • Thumbnail: Page 
81
    81
  • Thumbnail: Page 
82
    82
  • Thumbnail: Page 
83
    83
  • Thumbnail: Page 
84
    84
  • Thumbnail: Page 
85
    85
  • Thumbnail: Page 
86
    86
  • Thumbnail: Page 
87
    87
  • Thumbnail: Page 
88
    88
  • Thumbnail: Page 
89
    89
  • Thumbnail: Page 
90
    90
  • Thumbnail: Page 
91
    91
  • Thumbnail: Page 
92
    92
  • Thumbnail: Page 
93
    93
  • Thumbnail: Page 
94
    94
  • Thumbnail: Page 
95
    95
  • Thumbnail: Page 
96
    96
  • Thumbnail: Page 
97
    97
  • Thumbnail: Page 
98
    98
  • Thumbnail: Page 
99
    99
  • Thumbnail: Page 
100
    100
  • Thumbnail: Page 
101
    101
  • Thumbnail: Page 
102
    102
  • Thumbnail: Page 
103
    103