Pre-training method differences between code and paper

[jakobamb]

Hi,

thank you for releasing your work.

In the paper you are writing that you use the SimMIM method with mean-image masks and spatial frequency masking. The same approach is described in the figure of this repository.

However, the code and description of this repo are describing pre-training based on BEiT, not SimMIM, and are not mentioning the frequency masking. Are you in the process of updating the repo?

Are the weights you published from BEiT pretraining, or from USFM pretraining described in MIA?

Thanks

Jakob