3. • AdversarialNAS: Adversarial Neural Architecture Search for GANs
• UNAS: Differentiable Architecture Search Meets Reinforcement Learning
• Rethinking Performance Estimation in Neural Architecture Search
• EcoNAS: Finding Proxies for Economical Neural Architecture Search
• Hit-Detector: Hierarchical Trinity Architecture Search for Object Detection
• MTL-NAS: Task-Agnostic Neural Architecture Search towards General-Purpose Multi-Task Learning
• GP-NAS: Gaussian Process based Neural Architecture Search
• NAS-FCOS: Fast Neural Architecture Search for Object Detection
• MiLeNAS: Efficient Neural Architecture Search via Mixed-Level Reformulation
• Butterfly Transform: An Efficient FFT Based Neural Architecture Design
• DSNAS: Direct Neural Architecture Search without Parameter Retraining
• Network Adjustment: Channel Search Guided by FLOPs Utilization Ratio
• Meta-Learning of Neural Architectures for Few-Shot Learning
• MnasFPN : Learning Latency-aware Pyramid Architecture for Object Detection on Mobile Devices
• Improving One-shot NAS by Suppressing the Posterior Fading
3
NAS in CVPR 2020
NASに関する論⽂が約32本採択されている
4. • Can weight sharing outperform random architecture search? An investigation with TuNAS
• A Semi-Supervised Assessor of Neural Architectures
• CARS: Continuous Evolution for Efficient Neural Architecture Search
• Block-wisely Supervised Neural Architecture Search with Knowledge Distillation
• GreedyNAS: Towards Fast One-Shot NAS with Greedy Supernet
• APQ: Joint Search for Network Architecture, Pruning and Quantization Policy
• MemNAS: Memory-Efficient Neural Architecture Search with Grow-Trim Learning
• All in One Bad Weather Removal using Architectural Search
• Memory-Efficient Hierarchical Neural Architecture Search for Image Denoising
• C2FNAS: Coarse-to-Fine Neural Architecture Search for 3D Medical Image Segmentation
• Graph-guided Architecture Search for Real-time Semantic Segmentation
• Organ at Risk Segmentation for Head and Neck Cancer using Stratified Learning and Neural
Architecture Search
• When NAS Meets Robustness: In Search of Robust Architectures against Adversarial Attacks
• Densely Connected Search Space for More Flexible Neural Architecture Search
• Neural Architecture Search for Lightweight Non-Local Networks
4
NAS in CVPR 2020
7. When NAS Meets Robustness: In Search of Robust Architectures
against Adversarial Attacks
• Minghao Guo1, Yuzhe Yang2, Rui Xu1, Ziwei Liu1, Dahua Lin1
• 1The Chinese University of Hong Kong, 2MIT CSAIL
この論⽂では以下の問いに答えます
• Adversarial attackに頑健なCNNの構造は何なのか?
• パラメータ数に制限がある場合,どのように畳み込み層を配置すべきか?
• 統計的に頑健性を計る指標は何が良いか?
7
紹介する論文
?
8. 既存のOne-shot NAS[1]を利⽤して,多数のCNNを⽣成し,
それらCNNのAdversarial attackに対する頑健性を調査
8
基本的な方針
[1] G. Bender+, Understanding and Simplifying One-Shot Architecture Search, ICML, 2018
Supernet
…
sampling
Finetuning the network with adversarial
training and evaluate it on eval samples
Subnets
…
…
Finetuning the network with adversarial
training and evaluate it on eval samples
12. for ! = 0 … % do
Set all elements in & to 1
Sample a training batch (), +) ),-
.
Randomly set some of the elements of & to 0 and
get the corresponding network parameters /0
for 1 = 1 … 2 do
()
3
← ()
for 5 = 0 … (7 − 1) do
((:;-) ← Π= ()
:
+ ? @ A1BC ∇Eℒ /0, ()
:
, +)
Update /0 using ()
G
, +)
),-
.
by SGD
12
Supernetの学習手順
// PGD adversarial example
Supernet
Subnet /0
22. • Black-box results on CIFAR-10
• 同じネットワークを⽤意して,そちらでadversarial exampleを⽣成
• White-box results across different datasets
• CIFAR-10上で探索したCNNをほかのデータセット上で評価
22
既存のCNNとの比較(Black-box attack, different datasets)
Black-box results White-box results across different datasets
23. 頑健性を向上させる既存⼿法[6]をRobNetに適⽤することで,ResNetと⽐べて
さらなる頑健性の向上が可能
• [6] C. Xie+, Feature denoising for improving adversarial robustness, CVPR, 2019
23
既存手法との関係
ほかにも頑健性を向上させる既存⼿法はあるため,それらとの関係性もみたい
• [7] C. Xie+, Adversarial Examples Improve Image Recognition, CVPR, 2020
• [8] C. Xie+, Smooth Adversarial Training, arXiv:2006.14536, 2020