Defend against adversarial AI using Adversarial Robustness Toolbox
•Download as PPTX, PDF•
1 like•396 views
With great power comes great responsibility. Adversarial examples in AI pose an asymmetrical challenge with respect to attackers and defenders. AI developers must be empowered to defend deep neural networks against adversarial attacks and allow rapid crafting and analysis of attack and defense methods for machine learning models. Animesh Singh and Tommy Li explain how to implement state-of-the-art methods for attacking and defending classifiers using the open source Adversarial Robustness Toolbox. The library provides AI developers with interfaces that support the composition of comprehensive defense systems using individual methods as building blocks. Animesh and Tommy then demonstrate how to use a Jupyter notebook to leverage attack methods from the Adversarial Robustness Toolbox (ART) into a model training pipeline. This notebook trains a CNN model on the Fashion MNIST dataset, and the generated adversarial samples are used to evaluate the robustness of the trained model.
Recommended
Recommended
More Related Content
What's hot
What's hot (20)
Similar to Defend against adversarial AI using Adversarial Robustness Toolbox
Similar to Defend against adversarial AI using Adversarial Robustness Toolbox (20)
More from Animesh Singh
More from Animesh Singh (20)
Recently uploaded
Recently uploaded (20)
Defend against adversarial AI using Adversarial Robustness Toolbox
- 1. Animesh Singh & Tommy Li AI and Deep Learning Platform, IBM Defend against adversarial AI using Adversarial Robustness Toolbox ART CODAIT codait.org
- 2. Center for Open Source Data and AI Technologies (CODAIT) Code – Build and improve practical frameworks to enable more developers to realize immediate value. Content – Showcase solutions for complex and real-world AI problems. Community – Bring developers and data scientists to engage with IBM Improving Enterprise AI lifecycle in Open Source • Team contributes to over 10 open source projects • 17 committers and many contributors in Apache projects • Over 1100 JIRAs and 66,000 lines of code committed to Apache Spark itself; over 65,000 LoC into SystemML • Over 25 product lines within IBM leveraging Apache Spark • Speakers at over 100 conferences, meetups, unconferences and more CODAIT codait.org
- 3. 2011 IBM Watson Jeopardy 2017 AlphaGo Apple’s releases Siri 1997 … Facebook’s face recognition 2015 2016 Siri gets deep learning IBM Deep Blue chess AlexNet Progress in Deep Learning 2012 Introduced deep learning with GPUs 3 1997
- 4. 2011 IBM Watson Jeopardy 2017 AlphaGo Apple’s releases Siri 1997 … Facebook’s face recognition 2015 2016 Siri gets deep learning IBM Deep Blue chess AlexNet Progress in Deep Learning 2012 Introduced deep learning with GPUs 4 2011
- 5. 2011 IBM Watson Jeopardy 2017 AlphaGo Apple’s releases Siri 1997 … Facebook’s face recognition 2015 2016 Siri gets deep learning IBM Deep Blue chess AlexNet Progress in Deep Learning 2012 Introduced deep learning with GPUs 5 2017
- 6. 2011 IBM Watson Jeopardy 2017 AlphaGo Apple’s releases Siri 1997 … Facebook’s face recognition 2015 2016 Siri gets deep learning IBM Deep Blue chess AlexNet Progress in Deep Learning 2012 Introduced deep learning with GPUs 6 2017 2018
- 7. 2011 IBM Watson Jeopardy 2017 AlphaGo Apple’s releases Siri 1997 … Facebook’s face recognition 2015 2016 Siri gets deep learning IBM Deep Blue chess AlexNet Deep Leaning: Progress 2012 Introduced deep learning with GPUs IBM Cloud / Watson and Cloud Platform / © 2018 IBM Corporation 7
- 8. A human brain has: • 200 billion neurons • 32 trillion connections between them • 25 million “neurons” • 100 million connections (parameters) Deep Learning = Training Artificial Neural Networks IBM Cloud / Watson and Cloud Platform / © 2018 IBM Corporation 8
- 9. A human brain has: • 200 billion neurons • 32 trillion connections between them • 25 million “neurons” • 100 million connections (parameters) Deep Learning = Training Artificial Neural Networks IBM Cloud / Watson and Cloud Platform / © 2018 IBM Corporation 9
- 10. The Machine Learning Workflow 10
- 11. Perception
- 12. In reality the workflow spans teams …
- 13. Neural Network Design Workflow domain data design neural network HPO • neural network structure • hyperparameters NO Performance meets needs? Start another experiment optimal hyperparameters
- 14. Neural Network Design Workflow domain data HPO • neural network structure • hyperparameters NO yes Performance meets needs? Start another experiment trained model deployCloud optimal hyperparameters evaluat e BAD Still good! design neural network
- 15. AIOps Prepared and Analyzed Data AI PLATFORM Initial Model Trained Model Deployed Model AI Workflow
- 16. AIOps Prepared and Analyzed Data Trained Model Deployed Model Many tools available to build initial models Prepared and Analyzed Data Initial Model Deployed Model
- 17. AIOps Prepared and Analyzed Data Trained Model Deployed Model Many tools to train machine learning and deep learning models Prepared and Analyzed Data Initial Model Deployed Model FfDL
- 18. Fabric for Deep Learning https://github.com/IBM/FfDL FfDL Github Page https://github.com/IBM/FfDL FfDL dwOpen Page https://developer.ibm.com/code/open/projects/fabri c-for-deep-learning-ffdl/ FfDL Announcement Blog http://developer.ibm.com/code/2018/03/20/fabric- for-deep-learning FfDL Technical Architecture Blog http://developer.ibm.com/code/2018/03/20/democr atize-ai-with-fabric-for-deep-learning Deep Learning as a Service within Watson Studio https://www.ibm.com/cloud/deep-learning Research paper: “Scalable Multi-Framework Management of Deep Learning Training Jobs” http://learningsys.org/nips17/assets/papers/paper_ 29.pdf • Fabric for Deep Learning or FfDL (pronounced as ‘fiddle’ aims at making Deep Learning easily accessible to Data Scientists, and AI developers. • FfDL Provides a consistent way to train and visualize Deep Learning jobs across multiple frameworks like TensorFlow, Caffe, PyTorch, Keras etc. FfDL 18 Community Partners FfDL is one of InfoWorld’s 2018 Best of Open Source Software Award winners for machine learning and deep learning!
- 19. AIOps Trained Model Deployed Model And there are platforms to serve your models, create model catalogues etc. Prepared and Analyzed Data Initial Model Deployed Model FfDL kube-batch Jupyter Enterprise Gateway MAX Istio OpenWhisk
- 20. AIOps Prepared and Analyzed Data Trained Model Deployed Model But what about trust in AI? Prepared and Analyzed Data Initial Model Deployed Model Can the trained model be trusted? Can the dataset be trusted? Is the deployed model robust enough? Is the model vulnerable to adversarial attacks?
- 21. Is it fair? Is it easy to understand? Did anyone tamper with it? Is it accountable? #21, #32, #93 #21, #32, #93 What does it take to trust a decision made by a machine? (Other than that it is 99% accurate)?
- 22. FAIRNESS EXPLAINABILITY ROBUSTNESS ASSURANCE Our vision for Trusted AI Pillars of trust, woven into the lifecycle of an AI application
- 23. AIOps Prepared and Analyzed Data Trained Model Deployed Model Now how do we check for bias throughout AI lifecycle? Prepared and Analyzed Data Initial Model Deployed Model Are model weights biased? Are predictions biased? Is the dataset biased?
- 24. AIOps Prepared and Analyzed Data Trained Model Deployed Model Enter: AI Fairness 360 Prepared and Analyzed Data Initial Model Deployed Model AIF360
- 25. AI Fairness 360 https://github.com/IBM/AIF360 AIF360AIF360 toolkit is an open-source library to help detect and remove bias in machine learning models. The AI Fairness 360 Python package includes a comprehensive set of metrics for datasets and models to test for biases, explanations for these metrics, and algorithms to mitigate bias in datasets and models. Toolbox Fairness metrics (70+) Fairness metric explanations Bias mitigation algorithms (10) 25 Supported bias mitigation algorithms Optimized Preprocessing (Calmon et al., 2017) Disparate Impact Remover (Feldman et al., 2015) Equalized Odds Postprocessing (Hardt et al., 2016) Reweighing (Kamiran and Calders, 2012) Reject Option Classification (Kamiran et al., 2012) Prejudice Remover Regularizer (Kamishima et al., 2012) Calibrated Equalized Odds Postprocessing (Pleiss et al., 2017) Learning Fair Representations (Zemel et al., 2013) Adversarial Debiasing (Zhang et al., 2018) Supported fairness metrics Comprehensive set of group fairness metrics derived from selection rates and error rates Comprehensive set of sample distortion metrics Generalized Entropy Index (Speicher et al., 2018)
- 26. Metrics (70+)
- 27. (d’Alessandro et al., 2017) Algorithms (10)
- 28. © 2018 IBM Corporation IBM Confidential Demo Application: AI Fairness 360 Web Application http://aif360.mybluemix.net/
- 29. AIOps Prepared and Analyzed Data Trained Model Deployed Model Now let`s talk about Robustness Prepared and Analyzed Data Initial Model Deployed Model Is the model vulnerable to adversarial attacks? Is the dataset poisoned?
- 30. Deep learning and adversarial attacks 30 Deep Learning models are now used in many areas Can we trust them?
- 33. Data AttackerNeural Network poison train input perturb output result $$$ benefit Adversarial Threats to AI Evasion attacks Performed at test time Perturb inputs with crafted noise Model fails to predict correctly Undetectable by humans Poisoning attacks Performed at training time Insert poisoned sample in training data Use backdoor later
- 34. Exposure to poisoning • Could the attacker have created backdoors via poisoning of training data? Plausible deniability • How important is it for the adversary to use adversarial samples with strong resemblance to the original inputs? Type I vs type II errors • Is the attacker trying to bypass safeguards or aiming to cause false alarms? • What are the costs associated with such errors? Black vs white box • What knowledge does the attacker have about the AI model? • How does the attacker access the AI model? • Limitations to the number of queries? 34 Threat Models
- 35. Evasion attacks – an analysis 35 Why do adversarial examples exist? • Unless test error is 0%, there is always room for adversarial samples. • Attacks push inputs across the decision boundary. • Surprising: proximity of the nearest decision boundary! [Gilmer et al., 2018. Adversarial Spheres. https://arxiv.org/abs/1801.02774]
- 36. Evasion attacks – an analysis 36 Why do adversarial examples exist? Linearity hypothesis: • Neural Network outputs extrapolate linearly as a function of their inputs. • Adversarial examples push DNNs quickly outside their designated “operating range”. • Adversarial directions form a subspace. [Goodfellow et al., 2014. Explaining and Harnessing Adversarial Examples. https://arxiv.org/abs/1412.6572; Fawzi et al., 2016. Robustness of Classifiers: From Adversarial to Random Noise. Advances in Neural Information Processing Systems (NIPS)] Adversarial direction Modellogits(10classes)
- 37. Evasion attacks – an analysis 37 Why do adversarial examples exist? Fooling images: • DNNs don’t learn actually to recognize e.g. a schoolbus, but to discriminate it from any other object in the training set. [Nguyen et al., 2014. Deep Neural Networks are Easily Fooled: High Confidence Predictions for Unrecognizable Images. https://arxiv.org/abs/1412.1897]
- 38. Robustness Metrics DetectionModel Hardening Static Data Preprocessing Model Design Statistical Tests Detector Networks Bayesian Uncertainty Attack Independent Attack Specific Adversarial Training Dynamic GaussianData Augmentation FeatureSqueezing LabelSmoothing ShatteredGradients StochasticGradients Saddlepoint Optimization Dimensionality Reduction BReLUs CLEVER GlobalLipschitzBound LossSensitivity MinimalPerturbation Adversarial SuccessRates MMD KernelDensity Estimates LocalIntrinsic Dimensionality Magnet DetectorsonInputs DetectorsonInternal Representations DropoutUncertainty BayesianSVMs How to defend? Taxonomy of defenses
- 39. How to defend? 39 Adversarial training • Train DNNs solely on adversarial samples • Increase DNN capacity to maintain accuracy on clean data • Use specific algorithm for crafting the adversarial samples [Madry et al., 2017. Towards Deep Learning Models Resistant to Adversarial Attacks. https://arxiv.org/abs/1706.06083] Performance on CIFAR-10 data Data Model Accuracy Original A 87.3% PGD-20 A 45.8% PGD-7 A’ 64.2% FGSM Anat 85.6%
- 40. How to defend? 40 Preprocessing data • Process samples in order to remove adversarial noise • Input the cleaned samples to the classifier • Somewhat effective, however can be easily defeated by an adaptive adversary. Feature squeezing [W. Xu, D. Evans, and Y. Qi. Feature squeezing: Detecting adversarial examples in deep neural networks. CoRR, abs/1704.01155, 2017a.]
- 41. How to defend? CLEVER Estimate Lipschitz constant to construct an ϵ-ball within which all images are correctly classified. 41 https://bigcheck.mybluemix.net We fit the cross Lipschitz constant samples in S (see Algorithm 1) with reverse Weibull class dis- tribution to obtain the maximum likelihood estimate of the location parameter ˆaW , scale parameter ˆbW and shape parameter ˆcW , as introduced in Theorem 4.1. To validate that reverse Weibull distri- bution isagood fit to theempirical distribution of thecross Lipschitz constant samples, weconduct Kolmogorov-Smirnov goodness-of-fit test (a.k.a. K-S test) to calculate the K-S test statistics D and corresponding p-values. Thenull hypothesis isthat samples S follow areverseWeibull distribution. Figure 2 plots the probability distribution function of the cross Lipschitz constant samples and the fitted Reverse Weibull distribution for images from various data sets and network architectures. The estimated MLE parameters, p-values, and the K-S test statistics D are also shown. We also calculate thepercentage of exampleswhoseestimation havep-valuesgreater than 0.05, asillustrated in Figure 3. If the p-value is greater than 0.05, the null hypothesis cannot be rejected, meaning that the underlying data samples fit a reverse Weibull distribution well. Figure 3 shows that all numbers are close to 100%, validating the use of reverse Weibull distribution as an underlying distribution of gradient norm samples empirically. Therefore, the fitted location parameter of reverse Weibull distribution (i.e., the extreme value), ˆaW , can be used as a good estimation of local cross Lipschitz constant to calculate theCLEVER score. Theexact numbers areshown in Table 5 in Appendix E. (a) CIFAR-MLP (b) MNIST-CNN (c) ImageNet-MobileNet Figure2: ThecrossLipschitz constant samplesfor threeimagesfrom CIFAR, MNIST and ImageNet datasets, and their fitted Reverse Weibull distributions with the corresponding MLE estimates of location, scale and shape parameters (aW , bW , cW ) shown on the top of each plot. The D-statistics of K-Stest and p-valuesaredenoted asksand pval. With small ksand high p-value, thehypothesized reverse Weibull distribution fits the empirical distribution of cross Lipschitz constant samples well. 80 85 90 95 100 percentage (%) MobileNet Resnet Inception CIFAR-BReLU CIFAR-DD CIFAR-CNN CIFAR-MLP MNIST-BReLU MNIST-DD MNIST-CNN MNIST-MLP p = 1 p = 2 (a) Least likely target 80 85 90 95 100 percentage (%) MobileNet Resnet Inception CIFAR-BReLU CIFAR-DD CIFAR-CNN CIFAR-MLP MNIST-BReLU MNIST-DD MNIST-CNN MNIST-MLP p = 1 p = 2 (b) Random target 80 85 90 95 100 percentage (%) MobileNet Resnet Inception CIFAR-BReLU CIFAR-DD CIFAR-CNN CIFAR-MLP MNIST-BReLU MNIST-DD MNIST-CNN MNIST-MLP p = 1 p = 2 (c) Top 2 target Proposal Highlights & Preliminary Results Abstract: Although neural networks are becoming the core engine for driving Artificial Intelligence (AI) research and technology at an unprecedented speed, recent studies have highlighted their lack of model robustness to adversarial attacks, giving rise to new safety/security challenges in both the digital space and the physical world. In order to address the emerging AI-security issue, this proposal aims to provide a certified robustness evaluation framework that jointly takes into consideration an arbitrary neural network model and its underlying datasets. Specifically, we aim at developing an attack-agnostic robustness metric to evaluate the robustness of neural network classifiers. We further aim at providing efficient data-driven schemes to improve model robustness by pinpointing exemplary anchor points inferred from the underlying datasets. Introduction Neural network classifiers are easily fooled by adversarial perturbations Visual illustration of adversarial examples crafted by adversarial attack algorithms in [2]. The original example (a) is an ostrich image selected from the ImageNet dataset. The adversarial examples in (b) are classified as the target class labels (safe, shoe shop and vacuum respectively) by the Inception-v3 model Motivations How do we evaluate the robustness of a neural network? • Upper bounds: Current robustness measure of neural network models are mostly dependent on attack methods e.g. distortions found by FGSM, I-FGSM, DeepFool, C&W attacks, etc. • Lower bounds: Theoretical robustness guarantees are limited Our goal: Devise attack-agnostic robustness metric for neural networks We proved that the robustness of a network is related to its local Lipschitz constant, which can be evaluated numerically via extreme value theory. Our approach [1]: • Targeted attack • untargeted attack Our approach – Cross Lipschitz Extreme Value for nEtwork Robustness (more results in [1]): [3, Hein] [4, Bastani] MNIST: least likely target CIFAR: least likely target ImageNet: least likely target Comparison of L-inf distortion l (luca@mit.edu), Lily Weng (twweng@mit.edu), Pin-Yu Chen (Pin- models are mostly dependent on attack methods e.g. distortions found by FGSM, I-FGSM, DeepFool, C&W attacks, etc. • Lower bounds: Theoretical robustness guarantees are limited We proved that the robustness of a network is related to its local Lipschitz constant, which can be evaluated numerically via extreme value theory. Our approach [1]: • Targeted attack • untargeted attack Etwork Robustness (more results in [1]): of Neural Networks: An Extreme Value Theory Approach,“ ICLR 2018 s of a classifier against adversarial manipulation,“ NIPS 2017 s,“ NIPS 2016 [3, Hein] [4, Bastani] ia adversarial examples,“ AAAI 2018 likely target ImageNet: least likely target Comparison of the CLEVER score calculated with {50,100,250,500} samples and the L2 distortion by CW attack on ImageNet models nal of Global Optimization, 1996[Weng et al., 2018. Evaluating the Robustness of Neural Networks: An Extreme Value Theory Approach. ICLR 2018]
- 42. How to defend? 42 Poisoning detection Poisoned MNIST sample (will be classified as ‘1’ by poisoned model with high probability). Unsupervised clustering of training data based on DNN internal activations: Discovers partition of poisonous vs normal training samples.
- 43. AIOps Prepared and Analyzed Data Trained Model Deployed Model How to do all this? Enter Adversarial Robustness Toolbox Prepared and Analyzed Data Initial Model Deployed Model ART
- 44. IBM Adversarial Robustness Toolbox ART ART is a library dedicated to adversarial machine learning. Its purpose is to allow rapid crafting and analysis of attack and defense methods for machine learning models. The Adversarial Robustness Toolbox provides an implementation for many state-of-the-art methods for attacking and defending classifiers. 44 https://github.com/IBM/adversarial-robustness- toolbox The Adversarial Robustness Toolbox contains implementations of the following attacks: Deep Fool (Moosavi-Dezfooli et al., 2015) Fast Gradient Method (Goodfellow et al., 2014) Jacobian Saliency Map (Papernot et al., 2016) Universal Perturbation (Moosavi-Dezfooli et al., 2016) Virtual Adversarial Method (Moosavi-Dezfooli et al., 2015) C&W Attack (Carlini and Wagner, 2016) NewtonFool (Jang et al., 2017) The following defense methods are also supported: Feature squeezing (Xu et al., 2017) Spatial smoothing (Xu et al., 2017) Label smoothing (Warde-Farley and Goodfellow, 2016) Adversarial training (Szegedy et al., 2013) Virtual adversarial training (Miyato et al., 2017)
- 45. The Adversarial Robustness Toolbox (ART) 45 • Library for adversarial machine learning • Baseline implementation of attacks and defenses for classifiers • Dedicated to images • Python 2 & 3 • MIT license • Supported frameworks: Load classifier model (Keras, TF, PyTorch etc) Perform attack Load ART modules Evaluate robustness
- 46. The Adversarial Robustness Toolbox (ART) 46 • Library for adversarial machine learning • Baseline implementation of attacks and defenses for classifiers • Dedicated to images • Python 2 & 3 • MIT license • Supported frameworks:
- 47. Adversarial Robustness Toolbox (ART) Poisoning detection • Detection based on clustering activations • Proof of attack strategy Evasion detection • Detector based on inputs • Detector based on activations Robustness metrics • CLEVER • Empirical robustness • Loss sensitivity Unified model API • Training • Prediction • Access to loss and prediction gradients Evasion defenses • Feature squeezing • Spatial smoothing • Label smoothing • Adversarial training • Virtual adversarial training • Thermometer encoding • Gaussian augmentation • Total variance minimization Evasion attacks • FGSM • JSMA • BIM • PGD • Carlini & Wagner • DeepFool • NewtonFool • Elastic net attack • Universal perturbation • Spatial transformations 47
- 48. Conclusions 48 • Adversarial attacks pose a threat to the deployment of AI in security critical applications • There is ongoing work on practical defenses with strong guarantees • Future work: analyzing the adversarial threat on other types of data (text, speech, video, time series…) Bigger picture: Trusted AI Security ↔ Fairness ↔ Explainability ↔ Privacy https://www.research.ibm.com/artificial-intelligence/trusted-ai/
- 50. ADVERSARIAL ROBUSTNESS TOOLBOX (ART) ATTACKING ALGORITHMS DEFENDING ALGORITHMS FABRIC FOR DEEP LEARNING (FfDL) DISTRIBUTED DEEP LEARNING TRAINING CLI SDK BROWSER OBJECT STORAGE Model Definition Training Data Trained Models 1 2 3 4 Code Pattern: Adversarial Robustness Toolbox and FfDL integration for detecting Model vulnerabilities
- 51. Jupyter notebook with an example 51 https://nbviewer.jupyter.org/github/IBM/adversarial-robustness-toolbox/ blob/master/notebooks/attack_defense_imagenet.ipynb
- 52. 52
- 53. Load an ImageNet example image 53
- 54. 54
- 55. 55
- 56. Apply a defense method 56
- 57. AIOps Trained Model Deployed Model AI Lifecycle Prepared and Analyzed Data Initial Model Deployed Model FfDL kube-batch Jupyter Enterprise Gateway MAX AIF360 AIF360 Istio OpenWhisk ART