Pattern classifiers have been widely used in adversarial settings like spam and malware detection, although they have not been originally designed to cope with intelligent attackers that manipulate data at test time to evade detection. While a number of adversary-aware learning algorithms have been proposed, they are computationally demanding and aim to counter specific kinds of adversarial data manipulation. In this work, we overcome these limitations by proposing a multiple classifier system capable of improving security against evasion attacks at test time by learning a decision function that more tightly encloses the legitimate samples in feature space, without significantly compromising accuracy in the absence of attack. Since we combine a set of one-class and two-class classifiers to this end, we name our approach one-and-a-half-class (1.5C) classification. Our proposal is general and it can be used to improve the security of any classifier against evasion attacks at test time, as shown by the reported experiments on spam and malware detection.

1. Pa#ern
Recogni-on
and
Applica-ons
Lab
University
of
Cagliari,
Italy
Department
of
Electrical
and
Electronic
Engineering
1.5-class MCSs for Secure Learning against
Evasion Attacks at Test Time
Ba#sta
Biggio1,
Igino
Corona1,
Zhi-­‐min
He2,
Patrick
P.K.
Chan2,
Giorgio
Giacinto1,
Daniel
Yeung2,
Fabio
Roli1
(1)
Dept.
Of
Electrical
and
Electronic
Engineering,
University
of
Cagliari,
Italy
(2)
School
of
Computer
Science
and
Eng.,
South
China
University
of
Technology,
China
Guenzburg,
Germany,
Jun
29
-­‐
Jul
1,
2015
MCS
2015

2.
http://pralab.diee.unica.it
Machine Learning in Adversarial Settings
• Pattern recognition in security applications
– spam filtering, malware detection, biometrics
• Attackers manipulate data to evade detection at test time
2
legitimate
malicious
x1
x2
f(x)
…cheap…
…che4p…
a(x)

3.
http://pralab.diee.unica.it
a(x)
Simplified Risk Analysis under Attack
• Malicious data distribution is not stationary (TR/TS)
3
f
Rts ( f )− Rtr ( f ) = Ex,y l(y, f (a(x)))−l(y, f (x)){ }
x
p(x, y)

4.
http://pralab.diee.unica.it
a(x)
Simplified Risk Analysis under Attack
• Malicious data distribution is not stationary (TR/TS)
4
Rts ( f )− Rts ( f *
) = Ex,y l(y, f (a(x)))−l(y, f *
(a(x))){ }
x
p(x, y)
f *
Be+er
enclosing
legi4mate
data
in
feature
space
may
improve
classifier
security
…
at
the
expense
of
more
false
alarms

5.
http://pralab.diee.unica.it
1.5-class Classification
The Rationale Behind
5
2−class classification
−5 0 5
−5
0
5
1−class classification (legitimate)
−5 0 5
−5
0
5
• 2-­‐class
classifica-on
is
usually
more
accurate
in
the
absence
of
a#ack
• …
but
poten-ally
more
vulnerable
under
a#ack
(not
enclosing
legi-mate
data)
1.5C classification (MCS)
−5 0 5
−5
0
5
1.5-­‐class
classifica4on
aims
at
retaining
high
accuracy
and
security
under
a+ack

6.
http://pralab.diee.unica.it
Secure 1.5-class Classification with MCSs
• Heuristic approach to 1.5-class classification
• Base classifiers
– 2-class classifier: good accuracy in the absence of attacks
– 1-class classifiers: detect anomalous patterns (no support in TR)
• Combiner
– 1-class classifier on legitimate data to improve classifier security
6
data
1C Classifier
(malicious)
Feature
Extraction
malicious
1C Classifier
(legitimate)
2C Classifier
1C Classifier
(legitimate)
legitimate
x
g1(x)
g2(x)
g3(x)
g(x) ≥ t
g(x)
true
false

7.
http://pralab.diee.unica.it
Classifier Security against Evasion Attacks
7
• How to evaluate classifier security
against evasion attacks?
• Attack strategy:
• Non-linear, constrained optimization
– Gradient descent: approximate
solution for smooth functions
• Gradients of g(x) can be analytically
computed in many cases
– SVMs, Neural networks
−2−1.5−1−0.500.51
x
f (x) = sign g(x)( )=
+1, malicious
−1, legitimate
"
#
$
%$
min
x'
g(x')
s.t. d(x, x') ≤ dmax
x '
B. Biggio et al., Evasion attacks against machine learning at test time, ECML-PKDD 2013

8.
http://pralab.diee.unica.it
Computing Descent Directions
Support vector machines
1.5-class MCS
g(x) = αi yik(x,
i
∑ xi )+ b, ∇g(x) = αi yi∇k(x, xi )
i
∑
RBF kernel gradient: ∇k(x,xi
) = −2γ exp −γ || x − xi
||2
{ }(x − xi
)
8
1C Classifier
(malicious)
e
on
malicious
1C Classifier
(legitimate)
2C Classifier
1C Classifier
(legitimate)
legitimate
x
g1(x)
g2(x)
g3(x)
g(x) ≥ t
g(x)
true
false
z(x) = g1
(x), g2
(x), g3
(x)!
"
#
$
T
∇g(x) = −2γ αi
exp −γ z(x)− z(xi
)
2
{ }i
∑ z(x)− z(xi
)( )
Τ δz
δx
B. Biggio et al., Evasion attacks against machine learning at test time, ECML-PKDD 2013

9.
http://pralab.diee.unica.it
Bounding the Adversary’s Knowledge
Limited-knowledge attacks
• Only feature representation and learning algorithm are known
• Surrogate data sampled from the same distribution as the
classifier’s training data
• Classifier’s feedback to label surrogate data
PD(X,Y)data
Surrogate
training data
f(x)
Send queries
Get labels
Learn
surrogate
classifier
f’(x)
9
B. Biggio et al., Evasion attacks against machine learning at test time, ECML-PKDD 2013

10.
http://pralab.diee.unica.it
Experimental Analysis
• Two case studies: spam and PDF malware detection
– Perfect-knowledge (PK) and limited-knowledge (LK) attacks
• Spam data (TREC ’07)
– 25,220 ham and 50,199 spam emails
• we used the first 5,000 emails in chronological order
– 2-class linear SVM, 1-class RBF SVMs
• PDF data
– 2,000 samples collected from the web and public malware
databases (e.g., Contagio)
– 2-class RBF SVM, 1-class RBF SVMs
• Experimental setup
– 50% TR/TS splits, 20% TR for surrogate learning
– 5-fold cross-validation to tune
10
C,γ ∈ 2−10
,2−9
,...,2+10
{ }

11.
http://pralab.diee.unica.it
Spam Filtering
• Features: presence/absence of words
• Attacks: bad word obfuscation / good word insertion
• Attack strategy:
11
Start 2007
with a bang!
Make WBFS
YOUR
PORTFOLIO’s
first winner
of the year
...
start
bang
portfolio
winner
year
...
university
campus
1
1
1
1
1
...
0
0
x
x’
St4rt 2007
with a b4ng!
Make WBFS
YOUR
PORTFOLIO’s
first winner
of the year
... campus
start
bang
portfolio
winner
year
...
university
campus
0
0
1
1
1
...
0
1
min
x'
g(x')
s.t. d(x, x') ≤ dmax
L1-­‐distance
counts
the
number
of
modified
words
in
each
spam

12.
http://pralab.diee.unica.it
Experiments on PDF Malware Detection
• PDF: hierarchy of interconnected objects (keyword/value pairs)
• Attack strategy
– adding up to dmax objects to the PDF
– removing objects may
compromise the PDF file
(and embedded malware code)!
/Type
2
/Page
1
/Encoding
1
…
13
0
obj
<<
/Kids
[
1
0
R
11
0
R
]
/Type
/Page
...
>>
end
obj
17
0
obj
<<
/Type
/Encoding
/Differences
[
0
/C0032
]
>>
endobj
Features:
keyword
count
min
x'
g(x')
s.t. d(x, x') ≤ dmax
x ≤ x'
12

13.
http://pralab.diee.unica.it
Experimental Results
13
0 5 10 15 20 25 30
0
0.2
0.4
0.6
0.8
1
maximum number of modified words
AUC
1%
(PK)
2C SVM
1C SVM (L)
1C SVM (M)
1.5C MCS
0 5 10 15 20 25 30
0
0.2
0.4
0.6
0.8
1
maximum number of modified words
AUC
1%
(LK)
2C SVM
1C SVM (L)
1C SVM (M)
1.5C MCS
Spam
filtering
0 5 10 15 20 25 30
0
0.2
0.4
0.6
0.8
1
maximum number of added keywords
AUC
1%
(PK)
2C SVM
1C SVM (L)
1C SVM (M)
1.5C MCS
0 5 10 15 20 25 30
0
0.2
0.4
0.6
0.8
1
maximum number of added keywords
AUC
1%
(LK)
2C SVM
1C SVM (L)
1C SVM (M)
1.5C MCS
PDF
Malware
Detec-on

14.
http://pralab.diee.unica.it
Conclusions and Future Work
• 1.5-class MCSs
– to improve classifier security under attack (enclosing legitimate data)
– to retain good accuracy in the absence of attack
• General approach
– Suitable for any learning/classification algorithm (in principle)
– No specific assumption on adversarial data manipulation
• Future work
– Formal characterization of trade-off between security and accuracy
– Robustness to poisoning attacks (training data contamination)
14

15.
http://pralab.diee.unica.it
?
Any questions
Thanks
for
your
a#en-on!
15