Samples seen for five pretrained models #472
-
|
Would you be able to provide the number of samples seen during training for the following models? The number of samples per epoch would be fine instead, although I'd then need the number of epochs for the RN50 trained on cc12m.
|
Beta Was this translation helpful? Give feedback.
Replies: 1 comment 1 reply
-
|
@isaacOnline roughly speaking, they are all 32 epochs except for the B-32 laion2b. So ~ dataset size * 32. cc12m dataset size is less than 12m though, roughly 10.8 I think at the time those were trained. Not sure what the exact sample count of YFCC is as I never worked with it... but 15 * 32 would be a decent estimate I believe. That |
Beta Was this translation helpful? Give feedback.
-
|
Perfect - Thank you! |
Beta Was this translation helpful? Give feedback.
@isaacOnline roughly speaking, they are all 32 epochs except for the B-32 laion2b. So ~ dataset size * 32. cc12m dataset size is less than 12m though, roughly 10.8 I think at the time those were trained. Not sure what the exact sample count of YFCC is as I never worked with it... but 15 * 32 would be a decent estimate I believe.
That
laion2b_e16B/32 was the first LAION-2B run, it was 16 full dataset epochs so 32B SS. This was before transitioning to using resampling w/ replacement by default.