How to start multi-variate pre-training?

[Leopold2333]

Seems that all pre-training Pytorch Dataset Classes select a single variable for fetching (including Dataset_ERA5_Pretrain)? Does Timer-XL allow for multivariate pretraining?

[WenWeiTHU]

Yes. I think it can be generally implemented under the supervised training for multivariate time series. However, the lack of a high-quality multivariate pretraining dataset can be the main obstacle.

[Leopold2333]

Get it. Has Timer-XL implemented the multivariate pre-training? Recently I came across some projects, take Moirai for example, it implemented a multivariate data loading process given varying datasets with different variate numbers. However, from Moirai's perspective, the amount of data in each dataset is determined by the number of the variables. This is different with Timer- models, where the amount of data is bound to the number of time points contained in a dataset. In this case, have any new multivariate data loading strategies been implemented?