Skip to content

Latest commit

 

History

History
227 lines (183 loc) · 12 KB

README.md

File metadata and controls

227 lines (183 loc) · 12 KB

Wasserstein distributional robustness of neural networks

This repository is the official implementation of Wasserstein distributional robustness of neural networks.

Requirements

  1. Install basic requirements.

    conda env create -f environment.yml

  2. Install RobustBench.

    pip install git+https://github.com/RobustBench/robustbench.git

Preparation of ImageNet dataset

We use the same ImageNet dataset as Robustbench which is a randomly selected subset of 5000 images from the validation set. Below, we reiterate the steps for preparing this dataset:

  1. Download the validation set of ILSVRC 2012 via here.

  2. Extract and preprocess the data into folder .src/data/imagenet.

    mkdir imagenet && mv ILSVRC2012_img_val.tar imagenet/ && cd imagenet && tar -xvf ILSVRC2012_img_val.tar
wget -qO- https://raw.githubusercontent.com/soumith/imagenetloader.torch/master/valprep.sh | bash

Evaluation on the clean dataset

Calculate clean accuracy $A$, clean loss $V(0)$, conditional loss $W(0)$ on the misclassified images, and sensitivity $\Upsilon$:

python main.py $1 $2 $3 $4 $5

Input:

  • $1 -- dataset, "cifar10", "cifar100" or "imagenet".

  • $2 -- network index, see the table below.

  • $3 -- $q=1, 2$, conjugate of Wasserstein distance index $p$.

  • $4 -- loss_fn, "CE", "DLR", or "ReDLR".

  • $5 -- attack type, "clean".

Output:

  • a dict with keys ["acc", "loss", "loss_cond", "Upsilon", "q", "s", "mname", "loss_fn"].

All output files are stored in ./network_stats.

Wasserstein distributionally adversarial attack

Do W-PGD or W-FGSM attack on given networks:

python main.py $1 $2 $3 $4 $5 $6

Input:

  • $1 -- dataset, "cifar10", "cifar100", or "imagenet".

  • $2 -- network index, see the table below.

  • $3 -- $q=1, 2$, conjugate of Wasserstein distance index $p$.

  • $4 -- loss_fn, "CE", "DLR", or "ReDLR".

  • $5 -- attack type, "FGSM" or "PGD".

  • $6 -- attack budget. $\delta$=$4/510 if $s=1$, $\delta$=$4/32 if $s=2$.

Output:

  • a dict with keys ["acc_min", "loss_max", "q", "s", "delta", "attack_type", "mname", "loss_fn"].

All output files are stored in ./network_stats.

Table of available networks

CIFAR-10

index $l_{\infty}$ networks index $l_2$ networks
0 Andriushchenko2020Understanding 69 Augustin2020Adversarial
1 Carmon2019Unlabeled 70 Engstrom2019Robustness
2 Sehwag2020Hydra 71 Rice2020Overfitting
3 Wang2020Improving 72 Rony2019Decoupling
4 Hendrycks2019Using 73 Standard
5 Rice2020Overfitting 74 Ding2020MMA
6 Zhang2019Theoretically 75 Wu2020Adversarial
7 Engstrom2019Robustness 76 Gowal2020Uncovering
8 Chen2020Adversarial 77 Gowal2020Uncovering_extra
9 Huang2020Self 78 Sehwag2021Proxy
10 Pang2020Boosting 79 Sehwag2021Proxy_R18
11 Wong2020Fast 80 Rebuffi2021Fixing_70_16_cutmix_ddpm
12 Ding2020MMA 81 Rebuffi2021Fixing_28_10_cutmix_ddpm
13 Zhang2019You 82 Rebuffi2021Fixing_70_16_cutmix_extra
14 Zhang2020Attacks 83 Augustin2020Adversarial_34_10
15 Wu2020Adversarial_extra 84 Augustin2020Adversarial_34_10_extra
16 Wu2020Adversarial 85 Rebuffi2021Fixing_R18_cutmix_ddpm
17 Gowal2020Uncovering_70_16 86 Rade2021Helper_R18_ddpm
18 Gowal2020Uncovering_70_16_extra 87 Wang2023Better_WRN-28-10
19 Gowal2020Uncovering_34_20 88 Wang2023Better_WRN-70-16
20 Gowal2020Uncovering_28_10_extra
21 Sehwag2021Proxy
22 Sehwag2021Proxy_R18
23 Sehwag2021Proxy_ResNest152
24 Sitawarin2020Improving
25 Chen2020Efficient
26 Cui2020Learnable_34_20
27 Cui2020Learnable_34_10
28 Zhang2020Geometry
29 Rebuffi2021Fixing_28_10_cutmix_ddpm
30 Rebuffi2021Fixing_106_16_cutmix_ddpm
31 Rebuffi2021Fixing_70_16_cutmix_ddpm
32 Rebuffi2021Fixing_70_16_cutmix_extra
33 Sridhar2021Robust
34 Sridhar2021Robust_34_15
35 Rebuffi2021Fixing_R18_ddpm
36 Rade2021Helper_R18_extra
37 Rade2021Helper_R18_ddpm
38 Rade2021Helper_extra
39 Rade2021Helper_ddpm
40 Huang2021Exploring
41 Huang2021Exploring_ema
42 Addepalli2021Towards_RN18
43 Addepalli2021Towards_WRN34
44 Gowal2021Improving_70_16_ddpm_100m
45 Dai2021Parameterizing
46 Gowal2021Improving_28_10_ddpm_100m
47 Gowal2021Improving_R18_ddpm_100m
48 Chen2021LTD_WRN34_10
49 Chen2021LTD_WRN34_20
50 Standard
51 Kang2021Stable
52 Jia2022LAS-AT_34_10
53 Jia2022LAS-AT_70_16
54 Pang2022Robustness_WRN28_10
55 Pang2022Robustness_WRN70_16
56 Addepalli2022Efficient_RN18
57 Addepalli2022Efficient_WRN_34_10
58 Debenedetti2022Light_XCiT-S12
59 Debenedetti2022Light_XCiT-M12
60 Debenedetti2022Light_XCiT-L12
61 Huang2022Revisiting_WRN-A4
62 Wang2023Better_WRN-28-10
63 Wang2023Better_WRN-70-16
64 Xu2023Exploring_WRN-28-10
65 Cui2023Decoupled_WRN-28-10
66 Cui2023Decoupled_WRN-34-10
67 Bai2023Improving_edm
68 Peng2023Robust

CIFAR-100

index $l_{\infty}$ networks
0 Gowal2020Uncovering
1 Gowal2020Uncovering_extra
2 Cui2020Learnable_34_20_LBGAT6
3 Cui2020Learnable_34_10_LBGAT0
4 Cui2020Learnable_34_10_LBGAT6
5 Chen2020Efficient
6 Wu2020Adversarial
7 Sehwag2021Proxy
8 Sitawarin2020Improving
9 Hendrycks2019Using
10 Rice2020Overfitting
11 Rebuffi2021Fixing_70_16_cutmix_ddpm
12 Rebuffi2021Fixing_28_10_cutmix_ddpm
13 Rebuffi2021Fixing_R18_ddpm
14 Rade2021Helper_R18_ddpm
15 Addepalli2021Towards_PARN18
16 Addepalli2021Towards_WRN34
17 Chen2021LTD_WRN34_10
18 Pang2022Robustness_WRN28_10
19 Pang2022Robustness_WRN70_16
20 Jia2022LAS-AT_34_10
21 Jia2022LAS-AT_34_20
22 Addepalli2022Efficient_RN18
23 Addepalli2022Efficient_WRN_34_10
24 Debenedetti2022Light_XCiT-S12
25 Debenedetti2022Light_XCiT-M12
26 Debenedetti2022Light_XCiT-L12
27 Cui2020Learnable_34_10_LBGAT9_eps_8_255
28 Wang2023Better_WRN-28-10
29 Wang2023Better_WRN-70-16
30 Bai2023Improving_edm
31 Bai2023Improving_trades
32 Cui2023Decoupled_WRN-28-10
33 Cui2023Decoupled_WRN-34-10
34 Cui2023Decoupled_WRN-34-10_autoaug

ImageNet

index $l_{\infty}$ networks
0 Wong2020Fast
1 Engstrom2019Robustness
2 Salman2020Do_R50
3 Salman2020Do_R18
4 Salman2020Do_50_2
5 Standard_R50
6 Debenedetti2022Light_XCiT-S12
7 Debenedetti2022Light_XCiT-M12
8 Debenedetti2022Light_XCiT-L12
9 Singh2023Revisiting_ViT-S-ConvStem
10 Singh2023Revisiting_ViT-B-ConvStem
11 Singh2023Revisiting_ConvNeXt-T-ConvStem
12 Singh2023Revisiting_ConvNeXt-S-ConvStem
13 Singh2023Revisiting_ConvNeXt-B-ConvStem
14 Singh2023Revisiting_ConvNeXt-L-ConvStem
15 Liu2023Comprehensive_ConvNeXt-B
16 Liu2023Comprehensive_ConvNeXt-L
17 Liu2023Comprehensive_Swin-B
18 Liu2023Comprehensive_Swin-L
19 Peng2023Robust