Skip to content

Commit

Permalink
comet
Browse files Browse the repository at this point in the history
  • Loading branch information
bhaddow committed Dec 22, 2021
1 parent 88a3c01 commit 20c714e
Show file tree
Hide file tree
Showing 29 changed files with 477 additions and 0 deletions.
9 changes: 9 additions & 0 deletions scores/COMET/bn-hi/COMET_florestest2021_ref-A.tsv
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
Online-A 0.4944866685156024
HuaweiTSC 0.528095833838697
MS-EgDC 0.5323735577781975
Online-B 0.6699410981867114
GTCOM 0.6924460230443338
Online-G 0.1159513361166948
UEdin 0.5844160843655378
Online-Y 0.5010408396808576
TRANSSION 0.6372559390569599
9 changes: 9 additions & 0 deletions scores/COMET/cs-en/COMET_newstest2021_ref-A.tsv
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
CUNI-Transformer2018 0.5099661294706166
Online-B 0.5572413276340813
Online-W 0.5945665929298848
Online-A 0.5338801001589746
CUNI-DocTransformer 0.5924746984299273
Facebook-AI 0.6278560407254845
Online-G 0.5167575676552951
Online-Y 0.4594358833935112
ref-B 0.3577755065951496
9 changes: 9 additions & 0 deletions scores/COMET/cs-en/COMET_newstest2021_ref-B.tsv
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
CUNI-Transformer2018 0.3910618550442159
Online-B 0.4310039976146072
Online-W 0.4862284462973476
Online-A 0.411068437056616
CUNI-DocTransformer 0.4655658185873181
Facebook-AI 0.5133688559848815
Online-G 0.38314993036352096
Online-Y 0.3216493892055005
ref-A 0.4144571543652564
20 changes: 20 additions & 0 deletions scores/COMET/de-en/COMET_newstest2021_ref-A.tsv
Original file line number Diff line number Diff line change
@@ -0,0 +1,20 @@
Facebook-AI 0.6376018260810524
Online-A 0.6164778842777013
Online-B 0.6170484947301448
P3AI 0.5999739068634808
ICL 0.617725057290867
Online-G 0.5929112797714771
Online-Y 0.5561339648626745
UEdin 0.6076933072395623
happypoet 0.4501871124599129
Borderline 0.6116312125474215
Manifold 0.5897817004397512
Online-W 0.6220677955076098
UF 0.6183068523537367
HuaweiTSC 0.6282702647410333
Nemo 0.6212955224458129
SMU 0.5886534917596727
VolcTrans-AT 0.6070798035375774
Watermelon 0.6081747666765004
VolcTrans-GLAT 0.6124866799023002
ref-B 0.5312660989519209
20 changes: 20 additions & 0 deletions scores/COMET/de-en/COMET_newstest2021_ref-B.tsv
Original file line number Diff line number Diff line change
@@ -0,0 +1,20 @@
Facebook-AI 0.6695082825347781
Online-A 0.6277283858060837
Online-B 0.6377259592767804
P3AI 0.6121555416211486
ICL 0.6415725985951721
Online-G 0.6089642052520067
Online-Y 0.5677008453514427
UEdin 0.6380528545752168
happypoet 0.4463956729080528
Borderline 0.6197115848641842
Manifold 0.6046005611792207
Online-W 0.6499519624691457
UF 0.6382243662495166
HuaweiTSC 0.6441734549235553
Nemo 0.640795193053782
SMU 0.5985516872890294
VolcTrans-AT 0.6258999608829617
Watermelon 0.6263321863040328
VolcTrans-GLAT 0.6241475880499929
ref-A 0.5639146824721247
9 changes: 9 additions & 0 deletions scores/COMET/de-fr/COMET_newstest2021_ref-A.tsv
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
talp_upc 0.465769455095753
Online-G 0.4488636138401926
Online-W 0.622209172744304
P3AI 0.5124466538224369
Online-Y 0.4631383177619428
Manifold 0.5862463528066874
Online-B 0.5443878766763955
LISN 0.4256647317744792
Online-A 0.5610158962849527
10 changes: 10 additions & 0 deletions scores/COMET/en-cs/COMET_newstest2021_ref-A.tsv
Original file line number Diff line number Diff line change
@@ -0,0 +1,10 @@
Online-A 0.5851360954281071
Online-W 0.7510519449221725
Online-B 0.5864381245235185
CUNI-Marian-Baselines 0.6209516635391408
Online-Y 0.4563133951942037
eTranslation 0.6248991691438619
Online-G 0.2932831645197795
CUNI-DocTransformer 0.6799525723192179
CUNI-Transformer2018 0.6707843583032578
Facebook-AI 0.7748924600380563
28 changes: 28 additions & 0 deletions scores/COMET/en-de/COMET_newstest2021_ref-A.tsv
Original file line number Diff line number Diff line change
@@ -0,0 +1,28 @@
ICL 0.1956987441768368
Online-W 0.5458583370438533
metricsystem2 0.4929363454567518
metricsystem4 0.4995031739824903
nuclear_trans 0.3863820813655735
UEdin 0.5168570387595428
eTranslation 0.5059861986936923
Online-G 0.4392613301615039
UF 0.5073472347218327
metricsystem1 0.5155317971367619
BUPT_rush 0.37146350975053577
Facebook-AI 0.5673316703019979
Manifold 0.4965566897973002
Online-A 0.5105716817549424
P3AI 0.49766468207078596
VolcTrans-AT 0.5184223625563575
VolcTrans-GLAT 0.5420599741370468
WeChat-AI 0.5484504846692413
happypoet 0.4515186086475492
metricsystem5 0.4949029334123709
Online-B 0.501960829060205
HuaweiTSC 0.5155721798356987
Online-Y 0.4653383203207733
Nemo 0.5313356626451671
metricsystem3 0.4813865591714274
ref-B 0.5590452246587285
ref-C 0.5395760091854457
ref-D 0.5293435405646106
28 changes: 28 additions & 0 deletions scores/COMET/en-de/COMET_newstest2021_ref-B.tsv
Original file line number Diff line number Diff line change
@@ -0,0 +1,28 @@
ICL 0.31474887313666994
Online-W 0.7074292110810856
metricsystem2 0.6132374596549662
metricsystem4 0.6236337630334728
nuclear_trans 0.511063304800399
UEdin 0.6406849555489039
eTranslation 0.6339538422712457
Online-G 0.5648905984525701
UF 0.6353962961268312
metricsystem1 0.6414016470733189
BUPT_rush 0.4951280417987806
Facebook-AI 0.6873179171178602
Manifold 0.6263645190314321
Online-A 0.6419464599530764
P3AI 0.6234122074976891
VolcTrans-AT 0.6500033939075089
VolcTrans-GLAT 0.7526446498201754
WeChat-AI 0.6747817264724812
happypoet 0.5811670235118377
metricsystem5 0.6303176909737185
Online-B 0.6297628360069887
HuaweiTSC 0.6469054195394148
Online-Y 0.5896903533161341
Nemo 0.6568873423038277
metricsystem3 0.5967578337380957
ref-A 0.5664161908256258
ref-C 0.7147549559286017
ref-D 0.610267076520491
28 changes: 28 additions & 0 deletions scores/COMET/en-de/COMET_newstest2021_ref-C.tsv
Original file line number Diff line number Diff line change
@@ -0,0 +1,28 @@
ICL 0.24610883473584094
Online-W 0.6157876445527026
metricsystem2 0.5554972406543658
metricsystem4 0.5634515462424643
nuclear_trans 0.4452522249702744
UEdin 0.5740190736034851
eTranslation 0.5676017299242422
Online-G 0.4971619933460674
UF 0.5732174082891551
metricsystem1 0.5805943624828449
BUPT_rush 0.4277295792389832
Facebook-AI 0.6302915668419884
Manifold 0.5572140785584787
Online-A 0.5734872240498811
P3AI 0.5597546708745931
VolcTrans-AT 0.5796454993230914
VolcTrans-GLAT 0.6164835528684353
WeChat-AI 0.6099020330432409
happypoet 0.5111197128858485
metricsystem5 0.5580669156073067
Online-B 0.5682665610726841
HuaweiTSC 0.5764747216754748
Online-Y 0.5216791827841374
Nemo 0.5920424303913604
metricsystem3 0.5427571161019945
ref-A 0.5537760779094285
ref-B 0.7182970782890619
ref-D 0.5897272019042911
28 changes: 28 additions & 0 deletions scores/COMET/en-de/COMET_newstest2021_ref-D.tsv
Original file line number Diff line number Diff line change
@@ -0,0 +1,28 @@
ICL 0.22628337546751764
Online-W 0.5991314984858036
metricsystem2 0.5427533149934873
metricsystem4 0.5420669498490894
nuclear_trans 0.4216100018333919
UEdin 0.5590610204435543
eTranslation 0.5527723688899935
Online-G 0.4800003196992084
UF 0.5532382804669901
metricsystem1 0.5612301842408057
BUPT_rush 0.4093077948406161
Facebook-AI 0.6131848477072523
Manifold 0.5420904118259123
Online-A 0.5555801667113177
P3AI 0.5448862853090087
VolcTrans-AT 0.5635845620397054
VolcTrans-GLAT 0.5968435833624707
WeChat-AI 0.5910758639643054
happypoet 0.4956510655931429
metricsystem5 0.5362741387897206
Online-B 0.549137338704082
HuaweiTSC 0.5610538104441233
Online-Y 0.5023422527560097
Nemo 0.576388213108712
metricsystem3 0.529433764446668
ref-A 0.5461690899434531
ref-B 0.6136073855211039
ref-C 0.5898134028799311
14 changes: 14 additions & 0 deletions scores/COMET/en-ha/COMET_newstest2021_ref-A.tsv
Original file line number Diff line number Diff line change
@@ -0,0 +1,14 @@
Facebook-AI 0.3294781222660095
Online-B 0.22350628213770687
AMU 0.09214720699191094
UEdin -0.06078274231404066
GTCOM 0.19688802845217288
P3AI 0.27277156932093205
HuaweiTSC 0.3072558329701424
ZMT 0.2299451541583985
MS-EgDC 0.08573734163492917
NiuTrans 0.3044902189578861
Online-Y 0.08302450661733747
TWB 0.0003465951066464186
Manifold 0.17527052303217353
TRANSSION 0.22807210091128946
11 changes: 11 additions & 0 deletions scores/COMET/en-is/COMET_newstest2021_ref-A.tsv
Original file line number Diff line number Diff line change
@@ -0,0 +1,11 @@
Manifold 0.6477012964654714
NiuTrans 0.6939217285122722
Facebook-AI 0.7763324126265944
HuaweiTSC 0.5599385729506612
Allegro 0.32282500026188793
Mideind 0.541518261205405
Online-Y 0.35137337222136555
Online-A 0.5497755182869732
Lan-Bridge-MT 0.5886455690190197
Online-B 0.5834363983329386
Online-G -0.3270440652705729
15 changes: 15 additions & 0 deletions scores/COMET/en-ja/COMET_newstest2021_ref-A.tsv
Original file line number Diff line number Diff line change
@@ -0,0 +1,15 @@
Facebook-AI 0.6520731575414538
Online-W 0.6020623920653015
movelikeajaguar 0.3786302324514836
capitalmarvel 0.4603433608766645
Online-A 0.42144396966695785
BUPT_rush 0.5487028277665377
Online-G 0.14255013136565686
Online-Y 0.3855081223044545
ephemeraler 0.4139937934912741
Illini 0.18901602489128708
MiSS 0.516909037142992
NiuTrans 0.6189761819057167
WeChat-AI 0.6154081705827266
HuaweiTSC 0.6136673338152469
Online-B 0.4880381527151912
15 changes: 15 additions & 0 deletions scores/COMET/en-ru/COMET_newstest2021_ref-A.tsv
Original file line number Diff line number Diff line change
@@ -0,0 +1,15 @@
Facebook-AI 0.6498479569691443
Nemo 0.581543805743168
NiuTrans 0.5123786361322937
metricsystem3 0.5090664816488644
Online-A 0.5757442427828848
metricsystem2 0.6147203550910105
metricsystem5 0.5445470151448708
Manifold 0.5662620577871978
metricsystem1 0.5525320110705025
metricsystem4 0.4905785658562225
Online-W 0.6639502621488181
Online-B 0.6040152996265424
Online-Y 0.4735368759243074
Online-G 0.5996611758605568
ref-B 0.6000894492811816
15 changes: 15 additions & 0 deletions scores/COMET/en-ru/COMET_newstest2021_ref-B.tsv
Original file line number Diff line number Diff line change
@@ -0,0 +1,15 @@
Facebook-AI 0.6441411290414438
Nemo 0.5776666861153886
NiuTrans 0.5096289294970755
metricsystem3 0.5018101900299182
Online-A 0.5585632701116259
metricsystem2 0.6108245407073143
metricsystem5 0.5310201503433094
Manifold 0.5661569007369365
metricsystem1 0.5349837624116572
metricsystem4 0.4753128160516213
Online-W 0.659637346830286
Online-B 0.6010218667714092
Online-Y 0.4696836285553352
Online-G 0.5945360950118084
ref-A 0.5994385696330411
30 changes: 30 additions & 0 deletions scores/COMET/en-zh/COMET_newstest2021_ref-A.tsv
Original file line number Diff line number Diff line change
@@ -0,0 +1,30 @@
BUPT_rush 0.42494449154204295
ICL 0.463328508691799
NiuTrans 0.4831108126172168
Online-A 0.3397288088762981
SMU 0.4743225043955558
ephemeraler 0.3818411107254986
bjtu_nmt 0.4740900631153774
HuaweiTSC 0.44652572718305794
P3AI 0.4357699391445119
nuclear_trans 0.4283947357833534
Online-G 0.3013025253617061
WeChat-AI 0.5007591267393348
ZengHuiMT 0.4477281910859926
MiSS 0.4676684611337538
windfall 0.3951706359843294
happypoet 0.3639958926723775
Online-W 0.4683297191134946
Online-Y 0.31661544779051626
Machine_Translation 0.4667270233140823
Online-B 0.458036420126994
Borderline 0.4732385658623721
Facebook-AI 0.4986881677808608
HappyNewYear 0.4683821282782657
Lan-Bridge-MT 0.4628541234810046
NJUSC_TSC 0.4389666647942153
capitalmarvel 0.37777913232406457
UF 0.4128415489851863
movelikeajaguar 0.3708227093604778
yyds 0.4742123289535681
ref-B 0.42688474641135055
30 changes: 30 additions & 0 deletions scores/COMET/en-zh/COMET_newstest2021_ref-B.tsv
Original file line number Diff line number Diff line change
@@ -0,0 +1,30 @@
BUPT_rush 0.36751673570806215
ICL 0.3963871961794124
NiuTrans 0.4110138603298369
Online-A 0.29208784049261594
SMU 0.40182740288320007
ephemeraler 0.3110875168614491
bjtu_nmt 0.4087831379209985
HuaweiTSC 0.37996399145714654
P3AI 0.3746396093657184
nuclear_trans 0.36125184402070537
Online-G 0.2375411669130692
WeChat-AI 0.4374620947696848
ZengHuiMT 0.3862222611640176
MiSS 0.4037502107401927
windfall 0.3126349671857205
happypoet 0.3072804024242266
Online-W 0.3906885549861068
Online-Y 0.2541714067870152
Machine_Translation 0.4030102517571071
Online-B 0.380537706583426
Borderline 0.4033812511816413
Facebook-AI 0.4246813686650076
HappyNewYear 0.4033711240654755
Lan-Bridge-MT 0.40561433518100937
NJUSC_TSC 0.38134789266279656
capitalmarvel 0.2994017788597537
UF 0.3608573151551589
movelikeajaguar 0.3092775781677958
yyds 0.4067020666192541
ref-A 0.4207317402880051
9 changes: 9 additions & 0 deletions scores/COMET/fr-de/COMET_newstest2021_ref-A.tsv
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
Online-B 0.5761245316054243
Online-Y 0.5026556400084763
Online-W 0.7139624191364582
eTranslation 0.5954281823495384
talp_upc 0.481338896490924
Online-A 0.5657009739287154
Online-G 0.4541989487485114
P3AI 0.582644519742876
LISN 0.5054005397806856
14 changes: 14 additions & 0 deletions scores/COMET/ha-en/COMET_newstest2021_ref-A.tsv
Original file line number Diff line number Diff line change
@@ -0,0 +1,14 @@
UEdin 0.07570173931770482
Facebook-AI 0.42211801921173453
ZMT 0.3436015019587492
HuaweiTSC 0.2532887554698537
MS-EgDC 0.14845099806254997
AMU 0.07012152738136836
Manifold 0.1270323553540134
NiuTrans 0.17429977441682978
Online-B 0.3350883402404868
P3AI 0.24505554680659275
GTCOM 0.34456383556299425
Online-Y 0.12443088880789616
TRANSSION 0.34516389485272986
TWB -0.04598524554099219
9 changes: 9 additions & 0 deletions scores/COMET/hi-bn/COMET_florestest2021_ref-A.tsv
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
Online-B 0.5345208266812188
TRANSSION 0.5364754929962936
GTCOM 0.5751743721858048
MS-EgDC 0.4107265196125845
Online-A 0.5515711451011521
Online-Y 0.3855171708723124
HuaweiTSC 0.5237357380025982
Online-G -0.2154955465661285
UEdin 0.5451431836720652
10 changes: 10 additions & 0 deletions scores/COMET/is-en/COMET_newstest2021_ref-A.tsv
Original file line number Diff line number Diff line change
@@ -0,0 +1,10 @@
Allegro 0.4673472764473409
Online-Y 0.42237371766753495
Manifold 0.628627554051578
NiuTrans 0.6193822158798575
Online-B 0.6450929906461388
HuaweiTSC 0.6042999076656997
Online-G -0.07124064044095575
Online-A 0.51209265893884
Mideind 0.5521592110078782
Facebook-AI 0.6829904889948666
Loading

0 comments on commit 20c714e

Please sign in to comment.