Reproduce results from the MOIRAI-MoE paper #185
Unanswered
kaushikb258
asked this question in
Q&A
Replies: 0 comments
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
-
Hello, I am trying to reproduce the results from Table 2 of the MOIRAI-MoE paper for ETTh1 & ETTh2.
This is my eval command line code using MOIRAI-S:
python -m cli.eval
run_name=lsf_eval
model=moirai_1.0_R_small
model.patch_size=32
model.context_length=512
data=lsf_test
data.dataset_name=ETTh1
data.mode=M
data.prediction_length=96
As in Table 2 of the paper, I am interested in CRPS and MASE metrics. However, with the above entries for context length, patch_size, etc. I get:
mean_weighted_sum_quantile_loss = 0.417 and MASE[0.5] = 0.95643 for ETTh1. However, the values reported in the paper are 0.275 and 1.701 for MOIRAI-S.
Could you please suggest how to reproduce the exact results from Table 2 of the MOIRAI-MoE paper? Also, could you confirm if MASE[0.5] is the same as the MASE reported in Table 2 of the paper?
Beta Was this translation helpful? Give feedback.
All reactions