Have you tested the codes with other models?

[shuxjweb]

Firstly, thanks for you great work.
I have done several experiments using your codes, and most experimental results are satisfied. For cifar10, the best acc is 0.8982 with 250 labels, 0.9438 with 4000labels. However, these results are based on the model of wideresnet, which is utilized in paper. The accuracy woud be much worse when I alternate it with resnet50. The best acc is 0.7384 with 250 labels, 0.8219 with 4000labels. I wonder why different models produce such different results ？

[blankWorld]

Hi, @shuxjweb. Did you trained on SVHN dataset? I have trained on SVHN dataset with same experiment setting as the paper, but there is always 1% point accuracy lower than paper's result.

[shuxjweb]

@blankWorld I only tested it on cifar10, and the accuracy in paper also has a fluctuate range, see B.1 and B.2.

[ClayZhang0403]

Firstly, thanks for you great work.
I have done several experiments using your codes, and most experimental results are satisfied. For cifar10, the best acc is 0.8982 with 250 labels, 0.9438 with 4000labels. However, these results are based on the model of wideresnet, which is utilized in paper. The accuracy woud be much worse when I alternate it with resnet50. The best acc is 0.7384 with 250 labels, 0.8219 with 4000labels. I wonder why different models produce such different results ？

have u try another models? like res2net? resnxet?

[shuxjweb]

@ClayZhang0403 I just tested the codes with wideresnet and resnet50, but resnet50 behaved much worse than wideresnet.

[sudalvxin]

@ClayZhang0403 I just tested the codes with wideresnet and resnet50, but resnet50 behaved much worse than wideresnet.

different network may need different parameters, such as batch size and lr

[dddzg]

@shuxjweb

Firstly, thanks for you great work.
I have done several experiments using your codes, and most experimental results are satisfied. For cifar10, the best acc is 0.8982 with 250 labels, 0.9438 with 4000labels. However, these results are based on the model of wideresnet, which is utilized in paper. The accuracy woud be much worse when I alternate it with resnet50. The best acc is 0.7384 with 250 labels, 0.8219 with 4000labels. I wonder why different models produce such different results ？

Could you share us with the hyper-parameters? I only get 86% acc with 250 labels by the default hyper-parameters.

[berzentine]

Has anybody tried replicating for CIFAR 100 using this code? I get 6-7% less accuracy than the reported numbers for n=10000 labelled samples

[sailist]

Has anybody tried replicating for CIFAR 100 using this code? I get 6-7% less accuracy than the reported numbers for n=10000 labelled samples

cifar100 need larger model, in this paper, they use 135 filters per layer in wideresnet when training cifar100...

[taehyeok-jang]

Hi, has anyone tried with larger model with 1 millions of params?

I found that there is a significant decrease in accuracy when I ran with larger models.

model: WRN_28_2
LR: 0.002000
Total params: 1.47M

n_labeled	accuracy (epoch: 200)	baseline (paper)
250 (0.5%)	78.100000	88.92 ± 0.87
500 (1%)	85.080000	90.35 ± 0.94
1000 (2.2%)	88.520000	92.25 ± 0.32
2000 (4.4%)	90.260000	92.97 ± 0.15
4000 (8.8%)	92.120000	93.76 ± 0.06
10000	94.200000

model: wide_resnet50_2
Total params: 66.85M
LR: 0.002000

n_labeled	accuracy (epoch: 100)	baseline (paper, WRN_28_2)
250 (0.5%)	60.05	88.92 ± 0.87
500 (1%)	67.62	90.35 ± 0.94
1000 (2.2%)	73.94	92.25 ± 0.32
2000 (4.4%)	73.67	92.97 ± 0.15
4000 (8.8%)	79.85	93.76 ± 0.06
10000 (22%)	84.96	N/A

model: vgg19
Total params: 139.61M
LR: 0.002

n_labeled	accuracy (epoch: 100)	baseline (paper)
250 (0.5%)	10.000000	88.92 ± 0.87
500 (1%)	10.000000	90.35 ± 0.94
1000 (2.2%)	10.000000	92.25 ± 0.32
2000 (4.4%)	10.000000	92.97 ± 0.15
4000 (8.8%)	10.000000	93.76 ± 0.06
10000	10.000000