Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

40 transfer hpc #43

Merged
merged 18 commits into from
Apr 17, 2023
Merged

40 transfer hpc #43

merged 18 commits into from
Apr 17, 2023

Conversation

philswatton
Copy link
Contributor

@philswatton philswatton commented Apr 13, 2023

This PR resolves #40. It does this by implementing the following:

  • attack script generation now generates slurm instead of bash scripts
  • adversarial image generation now uses GPU if available
    • one exception worth noting: dependency chains means that MPS can't be used for the BoundaryAttack. There's an exception in the code for this, but this shouldn't be a problem on the HPC
  • measuring transfer success metrics now uses GPU if available
  • added a utility function for determining which GPU if any to use for adversarial image generation (as this uses pytorch and accelerator="auto" is a lightning feature)
  • README updated to reflect slurm use
  • updated values of epsilon to optimise over
  • matplotlib added for plotting of adversarial images. not used in source code or scripts, but was useful in designing the code and we'll likely need it later when writing the report
  • numpy version restricted to stop one of foolbox's dependencies breaking
  • some smaller fixes to previous code, removing superfluous lines

@philswatton philswatton marked this pull request as ready for review April 14, 2023 11:43
Copy link
Contributor

@lannelin lannelin left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good! Some minor comments.

haven't verified runs on HPC, let's pick this up next week.

@@ -1,5 +1,6 @@
num_attack_images: 1000
attack_names: ["L2FastGradientAttack","BoundaryAttack"]
epsilons: [0.0, 0.1, 0.3, 0.5, 1.0]
fga_epsilons: [0.1, 0.2, 0.3, 0.4, 0.5, 0.6, 0.7, 0.8, 0.9, 1]
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lots of vals to try, what's the rough expected cost implication of each additional val? (for each attack)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So in a test run on the HPC with the original five values (across 1000 images, 2x2=4 attacks) the script took 2 minutes 35 to run

Since once of these is 0, I'd assume about 40 seconds extra per (non-0) epsilon (while keeping the number of attack images and attacks constant)

Copy link
Contributor

@lannelin lannelin left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nice work!

Copy link
Contributor

@joannacknight joannacknight left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've run the tests locally, and they all passed. I can't see any issues in the code.

@philswatton philswatton merged commit a03b3ac into develop Apr 17, 2023
@philswatton philswatton deleted the 40-transfer-hpc branch April 17, 2023 13:19
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Add Transfer Attacks to HPC
3 participants