Presentation given at Strata 2015 at San Jose

7.  Problems
 sensors/detections
 Ranking

9. What is Lateral Movement?
Source: Pass-the-Hash Mitigation

10. Why is this Important?





11. Why is this difficult?

12. Problem # 1 - Independent Alert Streams

13. Problem #2: Burden of triage
Attacks are
complex. Need
more
detections!
So, Now I
have to
triage all of
them?

14. Problem #3: Feedback not captured

15. Problem 4: Interpretability of alerts

17. Windows Security Events Data
On average, an online service in O365 produces 30 billion
sessions/day; 82 TB/day
Data: Sequences of Windows security event IDs from user
sessions
• Examples: User logs into machine, process start, credential
switch, etc.
• 367 unique security event IDs

18. - We built separate models to detect
our goal of compromised
account/machines
- The models, independently assess if
the account is acting suspiciously

19.  probability of logging

sequences of events

credential elevation
 auto-generated

20. .
𝑃1 𝑃2 𝑃𝑑
𝑃1(𝑥)
…
𝑃2(𝑥) 𝑃𝑑(𝑥)
. .
𝑥Session
𝑤1 𝑤2 𝑤 𝑑
Combined Score

22. 





Burges, Chris, et al. "Learning to rank using
gradient descent.” 2005.

23. 𝑃1 𝑃2 𝑃𝑑
𝑃1(𝑚)
…𝑃2(𝑚) 𝑃𝑑(𝑚)
m
𝑃1 𝑃2 𝑃𝑑
𝑃1(𝑏)
…𝑃2(𝑏) 𝑃𝑑(𝑏)
bPm>b
…
𝑤1 𝑤2 𝑤 𝑑

24. Putting it together
.
𝑃1 𝑃2 𝑃𝑑
−𝑙𝑜𝑔𝑃1(𝑥)
…
−𝑙𝑜𝑔𝑃2(𝑥) −𝑙𝑜𝑔𝑃𝑑(𝑥)
. .
𝑥Session
𝑤1 𝑤2 𝑤 𝑑
Rank Score = 𝑤 𝑇 𝑃

25. Testing the system
• Wargame with the red team
• Blind experiment
• 8 out of 12 top-ranked sessions on day
1 among ~28 billion sessions are pen
testers, precision at 12 is 96%

26. …𝑤′1 𝑤′2 𝑤′ 𝑑

27. Alert Score  Weights
Higher Weight, more
contributing factor to alert
Tells the user, what is
probable cause of the alert

28.  extensible



30. Reality
 Constantly changing environment…
….but you can account for it during training
and adding metadata
 In the beginning, there will be false positives…
….but you will reduce your attack surface
 No labelled data…
….but you can get away with a good red team

32. Takeaways
 Combine alert streams
 Make your alerts interpretable
 Capture feedback and close the last mile
 Check out ranking algorithms – they are
powerful!