mnist_brelu&cifar_brelu models are missing

[chengwuxinlin]

Hello huanzhang,

I saw in Built-in Models, there is a brelu" (7-layer CNN with Bounded ReLU) option. but brelu is missing inside the train network codes(train_models.py; train_2layer.py; train_nlayer.py). Could you please offer pre-trained mnist_brelu&cifar_brelu networks? Thanks

[huanzhang12]

@chengwuxinlin Unfortunately I am not able to find these models now. But it should be easy to modify the training script a little bit to train them (just change activation function and model name).

Additionally, bounded ReLU is not a reliable way to enhance model robustness, so it is not recommended to use it anymore. For a more reliable way to improve robustness, you should look into adversarial training (e.g., Madry's work or TRADES) or certified defense (e.g., CROWN-IBP).

[chengwuxinlin]

Got it! Thanks

[chengwuxinlin]

@huanzhang12
Hi huanzhang, so in the compute clever score section, you used the average score among all chose data. But why not use the least score? the robustness level should be decided by the most vulnerable point right?

[huanzhang12]

@chengwuxinlin You are right that on a single image, the robustness level is decided by the most vulnerable point. But the average is done over the dataset level. Because for each image the model has different level of robustness, we have to use some metric like the average to report a statistic. Mean is also used in other papers like the CW attack paper, Table 3.

[chengwuxinlin]

Also I saw in collect_gradients.py, it says:
parser.add_argument("--ids", default = "", help = "use a filelist of image IDs in CSV file for attack (UNSUPPORTED)")

is this means I cannot use a file list of IDs for attack?