Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Clamdigging: Leveraging ClamAV for Malware Analysis and Detection

It's 2017: Weren't signature based technologies supposed to be dead by now? To the chagrin of many, signature based detection isn't an antiquated practice just yet. This presentation focuses on a signature based technology that has been around for a very long time: ClamAV, and how it can be practical / useful in today's security space. This talk will explore the current ClamAV capabilities, use cases, signatures, all with an emphasis on detecting malicious Office documents and exploits. Additionally, there will be an introduction to the open source project: 'clam-punch' which is a curated set of signatures for ClamAV to punch miscreants as well as a tool 'clamdigger' to assist analysts in creating their own signatures.

  • Login to see the comments

Clamdigging: Leveraging ClamAV for Malware Analysis and Detection

  1. 1. Clamdigging: Leveraging ClamAV for Malware Analysis and Detection 2017-05-12 BSides Denver
  2. 2. Signature based detection • Dead, right? • No, it’s not *totally*useless • Of course, not enough on its own • Defense in depth • sigh, I know.. • but srsly • Hunting! Tracking! Punching!
  3. 3. ClamAV • Its 2017, why are we talking about ClamAV • Developed by Sourcefire, acquired by Cisco in 2013 • Free, open source (GPL) anti virus solution for mail, endpoint and on demand scanning • Linux / BSD • OSX • Windows • Target filetypes / structures • Decompression / unrolling of filetypes • UPX/NSPack, ZIP/RAR/OLE, LZMA, etc
  4. 4. ClamAV 0.99+ • Yay, good reasons to use it! • Updated functionality • Decompression / unrolling of more filetypes! • PCRE! • Yara functionality! • Yara -> ClamAV (with some exceptions) • ClamAV decompression / unrolling + YARA logic == win • Expanded sigtool functionality!
  5. 5. So.. why ClamAV? • Why not? I just gave you some good reasons • It’s free • Some MTAs have Clam functionality built in • Similar functionality to Yara, just does some other stuff better • Quick triage of attachments • Evil documents, macros, exploits, etc. • Detection capabilities • Track threats across the board • Classify threats • Known evil doc vs known evil macro inside a doc vs known Flash exploit • Actor tracking • APTz, Builders, Gangs, campaigns, etc
  6. 6. Integration for Malware Analysis • Automated analysis with Cuckoo Sandbox
  7. 7. Integration for Malware Analysis • Pipe into your favorite tools for sorting results • Know when things update and you start missing detection • EKs • Track the use of known dropped exploits (Flash, Silverlight, etc) • Macros / Documents • Track Actors / Campaigns • Hunting! • Write generic sigs to detect evil conditions and start to hunt
  8. 8. ClamAV for Malware Analysis • Multiple lures / payloads, same macros
  9. 9. ClamAV for Malware Analysis • Hancitor document keyword
  10. 10. ClamAV for Malware Analysis • APT maldoc detected by generic XOR exe sig
  11. 11. ClamAV for Malware Analysis • Detecting various image lures embedded in docs
  12. 12. ClamAV for Malware Analysis • Determine common doc social engineering lure text • Used ~1,000 malicious docs • Determined common strings using statistical analysis • Most common: • Document created in earlier version of Microsoft Office (Word|Excel) • To (view|decrypt|display) this (content|document), please click "Enable Editing" (form|from) the yellow bar and then click "Enable Content” • To properly display the document, please Enable Content. • To display the contents of the document click on Enable Content button.
  13. 13. ClamAV for Malware Analysis
  14. 14. ClamAV for Malware Analysis • Detecting JS droppers/downloaders
  15. 15. ClamAV for Malware Analysis • Malicious PDF
  16. 16. ClamAV for Malware Analysis • Embedded objects in docs
  17. 17. ClamAV for Malware Analysis • CVE-2017-0199
  18. 18. ClamAV for Analysts • Run locally, using command line scanner • Dissect raw macros within documents • How does *ClamAV* see things? • What if you don’t want to beautify/deobfuscate the macros? • Built in tools • sigtool • clamscan • External tools • oletools • viper • Cuckoo • lots more
  19. 19. clamscan • Command line version of ClamAV • $clamscan –d rules.ldb attachment • What I find most useful: • --debug • Provides a ton of useful information on the target file • Also will provide information when a file is scanned • --info • Provides a ton of useful information on the target file • --leave-temps • Saves temp files generated during analysis • UPX packed file -> saves compressed file
  20. 20. sigtool • Swiss-army knife for ClamAV • What I find most useful are: • --vba • Dumps out the raw macros inside of a document • --html-normalise • Creates an ascii output html/js file which is ‘normalized’ for how ClamAV will read it • Removes case, whitespaces, et • --decode-sigs • Feeds in sigs from stdin and decodes the signatures contents • --hex-dump • Reads in from stdin and spits out hex dump
  21. 21. sigtool • $ sigtool --vba ‘evildoc.bin’ | sigtool --hex-dump | less
  22. 22. sigtool • $ cat local-rules.ldb | sigtool --decode
  23. 23. Detour: Sound Rule Writing Theory • Know what you want to detect: • Super specific content, ideally only one version of something (think hash, or very specific contents) • Generic signature • Detects a specific type of content, but with some wiggle room as far as things like variable names, or domain names • Loose / Heuristical Signature • Detects abnormalities, or the presence of something in particular • Sometimes INFO level, but other times Jumping off point to hunt • i.e this APT uses this specific control to auto-run macros when this document opens • Ideally, the perfect rule will be loose enough to allow slight changes, but specific enough to capture the right amount of maliciousness indicative of that specific threat
  24. 24. ClamAV Signatures • Have you used YARA? It’s pretty similar • Various types of ClamAV signatures, not limited to, but: • hdb (hash database) • Hash based signatures– simple • ldb (logical database) • Logical signatures– getting trickier now • Boolean logic • Regex • Content modifiers • Other stuff I’m not getting into because these I find most useful
  25. 25. hdb signatures • 7d8d7e1b9b4e54a113769fae842cc279:48237:EnableMacroLureImg.9.170412 md5 hash file size message • sigtool --md5 <file>
  26. 26. Suggestions for sound signatures • Naming • MiscreantPunch.EvilMacro.MultiPSD L.170425 • Content • Utilize conditional features • <, >, =, |, & for logical expressions • i, a, w, | for contents • * , ?? (wild card bytes) • !(hex), negate bytes • {x-y}, byte ranges • Offsets • 0:[hex], etc • PCREs! • Anchor appropriately to a content (i, s, g, etc) • Use the appropriate flags • ldb logic • Take advantage of • <,>,=,|,& • Target type • OLE / Flash / html/js / etc
  27. 27. Target Types src: github.com/vrtadmin/clamav-devel/blob/master/docs/signatures.pdf
  28. 28. ldb signatures • MiscreantPunch.EvilMacro.VBDL.170404; Engine:81-255,Target:2; (0&1&2&3&4&(5|6)&7&8&9&10); 4174747269627574652056425f::i;55524c446f776e6c6f6164546f46696c65::i;5c50726f6772616d73::i;5c537 46172747570::i;2e657865::i;2e72756e::i;7368656c6c::i;68747470::i;7/x3a//[^x22x27]+.exeb/si;433a:: i;9/[^x22x27]+.exeb/si Signature name Engine compatibility, and Target type Logical expression Desired contents used for detection as well as PCRE 0: Attribute VB_ 1: URLDownloadToFile 2: Programs
  29. 29. More examples... MiscreantPunch.EvilMacro.MultiPSDL.170501;Engine:81- 255,Target:2;(0&1&2&3&4&5&6);4174747269627574652056425f::i;54686973446f63756d656e742e426f6f6b6 d61726b732e436f756e74::i;55426f756e64::i;436872::i;5368656c6c::i;466f726d2e45646974::i;52656d20{3- 20}0d0a466f726d2e45646974??2e{3-20}203d2022220d0a456c73650d0a52656d20{3- 20}0d0a466f726d2e45646974??2e{3-20}203d2022??220d0a::i MiscreantPunch.RTF.EvilRTF.CVE-2017-0199-Obfus;Engine:81-255,Target:0;(0);0:7b5c7274!(66)* 5c6f626a757064617465 MiscreantPunch.SWF.MultiEKFlashExploit.CommonConStruct.20170505;Engine:81- 255,Target:11;(0&1&2&3&4);d030{0-500}2404c6732a;fcff7f;fcffffff07;0/xd0x30.{0,300}(?P<var1>([xd4- xd7]|x63.))x2e.(?:[xd0-xd3]|x62.){2}x46.{2}xa0x73x2a(?P=var1)x37(?:[xd4-xd7]|x63.)(?:[xd0- xd3]|x62.)x2e.xa1x73x2a(?P=var1)x37(?:[xd4-xd7]|x63.)(?:[xd0- xd3]|x62.)x24x04xc6x73x2a(?P=var1)x37(?:[xd4-xd7]|x63.)(?:[xd0- xd3]|x62.)/s;0/xa0x73(?P<s1>([xd4-xd7]|x63.))(?:[xd0-xd3]|x62.)(?P<g1>([xd0- xd3]|x62.))x3c(?P=g1)x2e.xa1x73x2a(?P=s1)x37(?:[xd4-xd7]|x63.)(?P=g1)x24x04xc6x73x2a/s
  30. 30. QAing your Sigs • It’s important to test signatures before either deploying to prod or an analysis environment • Twofold: • Checking your sigs for: • Misspellings • Formatting • Errors • Test sigs against evil docs and benign docs– ensure what is supposed to happen.. does
  31. 31. I wrote a sig and it didn’t fire • Getting things to fire on first go isn’t easy • Probably is your pcre • Debug! • --debug • $ clamscan –d <rule-file>.ldb <file> --debug • Spits out a ton of information with regards to how the engine inspected the file, what it found, and what conditions exist • Useful for determining why a sig didn’t fire or had problems
  32. 32. clam-punch • Owned / Created by the rule druid himself, Will Metcalf • Github repo containing buckets and shovels • Several rulesets updated p much daily • MiscreantPunch099-Low.ldb • miscreantpunch.hdb • MiscreantPunch099-INFO-Low.ldb • exexor99.ldb • Clamdigger • Signature generation tool
  33. 33. clamdigger.py • Python script for generating ldb ClamAV signatures • Does the heavy lifting • Converts content -> hex • Logically formats it • Appends macro auto* stuff (if you want) • Add/remove modifiers
  34. 34. Limitations • Encrypted docs • Observed in recent campaigns • Use some tool to bruteforce (or enter if known) -> send to clam for processing • EPS Files detected as PostScript are blanket ignored • Multimatch can be unreliable
  35. 35. Resources • ClamAV Signature Docs (really important!) • github.com/vrtadmin/clamav-devel/blob/master/docs/signatures.pdf • Miscreant Punch Sigs • github.com/wmetcalf/clam-punch • Tutorial on sigwriting I made • www.malwarefor.me/writing-signatures-for-clam-av-0-99-a-tutorial • SaneSecurity 3rd party Sigs • sanesecurity.com/usage/signatures • ClamAV Mailing List • lists.clamav.net/cgi-bin/mailman/listinfo/clamav-users • Mal(?:doc|ware) Samples • malware-traffic-analysis.net • hybrid-analysis.com • malwr.com
  36. 36. come hang w me (trainings) • OISF Training– Denver, CO: June 20-21st • Mix of Suricata User training with a kicker of signature development • DerbyCon 7.0 – Lousiville, KY: September 20-21st • Practical Signature Development for Open Source IDS • TL;DR how to write Suricata/Snort sigs that don’t suck J • SuriCon 2017 – Prague, CZ: November 15-17th • https://suricon.net/ • CFP open (talk to me!) • Sponsors needed! (support Open Source!)
  37. 37. Questions? • don’t tweet me: @malwareforme • don’t email me: jack@malwarefor.me • tks wmetcalf • tks jwilliams • tks clamav • tks Steve B / SaneSecurity • tks BSides Denver!

×