NETRESEC Network Security Blog - Tag : malware
This short video demonstrates how you can search through PCAP files with regular expressions (regex) using CapLoader and how this can be leveraged in order to improve IDS signatures.
The EmergingThreats snort/suricata rule mentioned in the video is SID 2014411 “ET TROJAN Fareit/Pony Downloader Checkin 2”.
The header accept-encoding header with quality factor 0 used by the Pony malware is:
And here is the regular expression used to search for that exact header:
After recording the video I noticed that the leaked source code for Pony 2.0 actually contains this accept-encoding header as a
hard-coded string. Have a look in the
redirect.php file, where they set
Wanna learn more about the intended use of quality factors in HTTP accept headers?
Then have a look at
section 14.1 of RFC 2616section 5.3.4 of RFC 7231, which defines how to use qvalues (i.e. quality factors) in the Accept-Encoding header.
I submitted a snort/suricata signature to the Emerging-Sigs mailinglist after publishing this blog post, which resulted in the Emerging Threats signature 2014411 being updated on that same day to include:content:"|0d 0a|Accept-Encoding|3a 20|identity,|20 2a 3b|q=0|0d 0a|"; http_header;
Thank you @EmergingThreats for the fast turnaround!
Posted by Erik Hjelmvik on Wednesday, 04 July 2018 07:39:00 (UTC/GMT)
This network forensics video tutorial covers analysis of a malware redirect chain, where a PC is infected through the RIG Exploit Kit. A PCAP file, from Brad Duncan's malware-traffic-analysis.net website, is opened in NetworkMiner Professional in order to follow a redirect chain via a couple of hacked websites before delivering malware to the PC.
Meadgive on VirusTotal
CVE-2014-0569 Flash Exploit on VirusTotal
CVE-2012-0507 Java Exploit on VirusTotal
Check out our series of network forensic video tutorials for more tips and tricks on how to analyze captured network traffic.
Posted by Erik Hjelmvik on Monday, 26 February 2018 11:19:00 (UTC/GMT)
series of network forensic video tutorials covers a quick and crude way to scan a PCAP file for malware. It's all done locally without having to run the PCAP through an IDS. Kudos to Lenny Hanson for showing me this little trick!
Antivirus Scanning of a PCAP File
- SWF/Neclu.B on VirusTotal
- CVE-2015-0311 (Flash Player vulnerability exploited by Neclu)
- Win32/Simda.AT on VirusTotal (Kryptik)
Posted by Erik Hjelmvik on Monday, 12 February 2018 08:00:00 (UTC/GMT)
Jason Reaves gave a talk titled “Malware C2 over x509 certificate exchange” at BSides Springfield 2017, where he demonstrated that the SSL handshake can be abused by malware as a covert command-and-control (C2) channel.
He got the idea while analyzing the Vawtrak malware after discovering that it read multiple fields in the X.509 certificate provided by the server before proceeding. Jason initially thought these fields were used as a C2 channel, but then realized that Vawtrak performed a variant of certificate pinning in order to discover SSL man-in-the-middle attempts.
Nevertheless, Jason decided to actually implement a proof-of-concept (PoC) that uses the X.509 certificate as a C2 channel. Jason’s code is now available on GitHub along with a PCAP file demonstrating this covert C2 channel. Of course I couldn’t resist having a little look at this PCAP file in NetworkMiner.
The first thing I noticed was that the proof-of-concept PCAP ran the SSL session on TCP 4433, which prevented NetworkMiner from parsing the traffic as SSL. However, I was able to parse the SSL traffic with NetworkMiner Professional just fine thanks to the port-independent-protocol-identification feature (a.k.a Dynamic Port Detection), which made the Pro-version parse TCP 4433 as SSL/TLS.
Image: X.509 certificates extracted from PCAP with NetworkMiner
A “normal” x509 certificate size is usually around 1kB, so certificates that are 11kB should be considered as anomalies. Also, opening one of these .cer files reveals an extremely large value in the Subject Key Identifier field.
Not only is this field very large, it also starts with the familiar “4D 5A” MZ header sequence.
NetworkMiner additionally parses details from the certificates that it extracts from PCAP files, so the Subject Key Identifier field is actually accessible from within NetworkMiner, as shown in the screenshot below.
You can also see that NetworkMiner validates the certificate using the local trusted root certificates.
Not surprisingly, this certificates is not trusted
Extracting the MZ Binary from the Covert X.509 Channel
Even though NetworkMiner excels at pulling out files from PCAPs, this is definitively an occasion where manual handling is required. Jason’s PoC implementation actually uses a whopping 79 individual certificates in order to transfer this Mimikatz binary, which is 785 kB.
Here’s a tshark oneliner you can use to extract the Mimikatz binary from Jason's example PCAP file.
tshark -r mimikatz_sent.pcap -Y 'ssl.handshake.certificate_length gt 2000' -T fields -e x509ce.SubjectKeyIdentifier -d tcp.port==4433,ssl | tr -d ':\n' | xxd -r -p > mimikatz.exe
Detecting x509 Anomalies
Even though covert channels using x509 certificates isn’t a “thing” (yet?) it’s still a good idea to think about how this type of covert signaling can be detected. Just looking for large Subject Key Identifier fields is probably too specific, since there are other fields and extensions in X.509 that could also be used to transmit data. A better approach would be to alert on certificates larger than, let’s say, 3kB. Multiple certificates can also be chained together in a single TLS handshake certificate record, so it would also make sense to look for handshake records larger than 8kB (rough estimate).
This type of anomaly-centric intrusion detection is typically best done using the Bro IDS, which provides easy programmatic access to the X.509 certificate and SSL handshake.
There will be false positives when alerting on large certificates in this manner, which is why I recommend to also check if the certificates have been signed by a trusted root or not. A certificate that is signed by a trusted root is very unlikely to contain malicious data.
Posted by Erik Hjelmvik on Tuesday, 06 February 2018 12:13:00 (UTC/GMT)
series of network forensics video tutorials throughout the next few weeks. First up is this analysis of a PCAP file containing network traffic from the "Zyklon H.T.T.P." malware.
Analyzing a Zyklon Trojan with Suricata and NetworkMiner
Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:18.104.22.168pre) Gecko/20070302 BonEcho/22.214.171.124pre
Posted by Erik Hjelmvik on Monday, 05 February 2018 07:30:00 (UTC/GMT)
In November last year Alexa admitted in a tweet that they had stopped releasing their CSV file with the one million most popular domains.
Members of the Internet measurement and infosec research communities were outraged, surprised and disappointed since this domain list had become the de-facto tool for evaluating the popularity of a domain. As a result of this Cisco Umbrella (previously OpenDNS) released a free top 1 million list of their own in December the same year. However, by then Alexa had already announced that their “top-1m.csv” file was back up again.
The Alexa list was unavailable for just about a week but this was enough for many researchers, developers and security professionals to make the move to alternative lists, such as the one from Umbrella. This move was perhaps fueled by Alexa saying that the “file is back for now”, which hints that they might decide to remove it again later on.
We’ve been leveraging the Alexa list for quite some time in NetworkMiner and CapLoader in order to do DNS whitelisting, for example when doing threat hunting with Rinse-Repeat. But we haven’t made the move from Alexa to Umbrella, at least not yet.
Malware Domains in the Top 1 Million Lists
Threat hunting expert Veronica Valeros recently pointed out that there are a great deal of malicious domains in the Alexa top one million list.
I often recommend analysts to use the Alexa list as a whitelist to remove “normal” web surfing from their PCAP dataset when doing threat hunting or network forensics. And, as previously mentioned, both NetworkMiner and CapLoader make use of the Alexa list in order to simplify domain whitelisting. I therefore decided to evaluate just how many malicious domains there are in the Alexa and Umbrella lists.
hpHosts EMD (by Malwarebytes)
- URL: https://hosts-file.net/emd.txt
- Total malicious domains: 154144
|Whitelisted malicious domains:||1365||1458|
|Percent of malicious domains whitelisted:||0.89%||0.95%|
Malware Domain Blocklist
- URL: http://www.malwaredomains.com/?page_id=66
- Total malicious domains: 18270
|Whitelisted malicious domains:||84||63|
|Percent of malicious domains whitelisted:||0.46%||0.34%|
- URL: http://cybercrime-tracker.net/ Total malicious domains: 7861
|Whitelisted malicious domains:||15||10|
|Percent of malicious domains whitelisted:||0.19%||0.13%|
The results presented above indicate that Alexa and Umbrella both contain roughly the same number of malicious domains. The percentages also reveal that using Alexa or Umbrella as a whitelist, i.e. ignore all traffic to the top one million domains, might result in ignoring up to 1% of the traffic going to malicious domains. I guess this is an acceptable number of false negatives since techniques like Rinse-Repeat Intrusion Detection isn’t intended to replace traditional intrusion detection systems, instead it is meant to be use as a complement in order to hunt down the intrusions that your IDS failed to detect. Working on a reduced dataset containing 99% of the malicious traffic is an acceptable price to pay for having removed all the “normal” traffic going to the one million most popular domains.
One significat difference between the two lists is that the Umbrella list contains subdomains (such as www.google.com, safebrowsing.google.com and accounts.google.com) while the Alexa list only contains main domains (like “google.com”). In fact, the Umbrella list contains over 1800 subdomains for google.com alone! This means that the Umbrella list in practice contains fewer main domains compared to the one million main domains in the Alexa list. We estimate that roughly half of the domains in the Umbrella list are redundant if you only are interested in main domains. However, having sub domains can be an asset if you need to match the full domain name rather than just the main domain name.
Data Sources used to Compile the Lists
Image: The Alexa Extension for Firefox
The two lists are compiled in different ways, which can be important to be aware of depending on what type of traffic you are analyzing. Alexa primarily receives web browsing data from users who have installed one of Alexa’s many browser extensions (such as the Alexa browser toolbar shown above). They also gather additional data from users visiting web sites that include Alexa’s tracker script.
Cisco Umbrella, on the other hand, compile their data from “the actual world-wide usage of domains by Umbrella global network users”. We’re guessing this means building statistics from DNS queries sent through the OpenDNS service that was recently acquired by Cisco.
This means that the Alexa list might be better suited if you are only analyzing HTTP traffic from web browsers, while the Umbrella list probably is the best choice if you are analyzing non-HTTP traffic or HTTP traffic that isn’t generated by browsers (for example HTTP API communication).
As noted by Greg Ferro, the Umbrella list contains test domains like “www.example.com”. These domains are not present in the Alexa list.
We have also noticed that the Umbrella list contains several domains with non-authorized gTLDs, such as “.home”, “.mail” and “.corp”. The Alexa list, on the other hand, only seem to contain real domain names.
Resources and Raw Data
Both the Alexa and Cisco Umbrella top one million lists are CSV files named “top-1m.csv”. The CSV files can be downloaded from these URL’s:
- Alexa: http://s3.amazonaws.com/alexa-static/top-1m.csv.zip
- Umbrella: http://s3-us-west-1.amazonaws.com/umbrella-static/top-1m.csv.zip
The analysis results presented in this blog post are based on top-1m.csv files downloaded from Alexa and Umbrella on March 31, 2017. The malware domain lists were also downloaded from the three respective sources on that same day.
We have decided to share the “false negatives” (malware domains that were present in the Alexa and Umbrella lists)
for transparency. You can download the lists with all false negatives from here:
Hands-on Practice and Training
If you wanna learn more about how a list of common domains can be used to hunt down intrusions in your network, then please register for one of our network forensic trainings. The next training will be a pre-conference training at 44CON in London.
Posted by Erik Hjelmvik on Monday, 03 April 2017 14:47:00 (UTC/GMT)
One of the features included in the newly released version 2.0 of NetworkMiner Professional is a new tab called “Browsers”. This tab shows web browsing requests and reponses in a hierarchical tree view, with the identified web browsers as root nodes.
The idea of tracking browser activity this way was suggested to me by Steffen Thorkildsen way back in 2009. I'm therefore happy to finally have this feature implemented in NetworkMiner!
At first glance, the Browser tab looks somewhat like the Hosts tab. One difference is that there can be multiple browsers per host, since each unique HTTP User-Agent is considered a separate browser.
The web pages (URLs) visited by a browser can be analyzed by expanding the node of that browser. The URLs are organized in a hierarchical structure, so that all URLs visited by clicking a link on a web page are placed under the node of that web page. This enables the analyst to see how a user ended up at a particular URL. NetworkMiner primarily uses the HTTP referer header (the misspelling of referrer stems back to RFC1945) to backtrack the pages visited before landing at a particular page.
Image: Bing search for “create bitcoin address” that led the user to www.btcpedia.com
The browser tree view also shows HTTP redirects, such as “302 Found” and “301 moved permanently”. These redirects can be used in order to see encrypted HTTPS URLs that a user is redirected to, for example when logging in at a website.
Image: Microsoft responding with a “302 Moved Temporarily" redirect
The icons that show up at some web servers are favicon images that have been passively extracted from the analyzed PCAP file.
Image: Website icons extracted from favicon.ico downloads
We hope the Browser tab can be of help in criminal investigations in order to show whether or not a suspect visited a particular website intentionally. This feature can also be used to track the activity of malware that uses HTTP for command-and-control (C2) as well as to analyze redirect chains used for malware downloads.
Image: PCAP file containing a redirect chain leading to malware downloads
The PCAP file loaded in the screenshot above originally comes from malware-traffic-analysis.net. Note that our analysis was done by running NetworkMiner in Linux to prevent accidental malware infection. The events shown in NetworkMiner's browser tab matches the description of the redirect chain provided at malware-traffic-analysis.net:
126.96.36.199 port 80 - www.crowdfundingformybusiness.com - Compromised website
188.8.131.52 port 80 - goog1eanalitics.pw - First redirect
184.108.40.206 port 80 - 220.127.116.11 - Second redirect
18.104.22.168 port 80 - osooraudie.co.vu - Nuclear EK
The redirect chain leads to a Nuclear Exploit Kit (SWF file with MD5 695a07cbcac3ca64010e168fe495ff4a VirusTotal). Later on the Nuclear EK retrieves the file “kernel1.exe”, which seems to be related to the Kelihos botnet.
Posted by Erik Hjelmvik on Thursday, 18 February 2016 13:37:00 (UTC/GMT)
A new version of our heavy-duty PCAP parser tool CapLoader is now available. There are many new features and improvements in this release, such as the ability to filter flows with BPF, domain name extraction via passive DNS parser and matching of domain names against a local white list.
Filtering with BPF
The main focus in the work behind CapLoader 1.3 has been to fully support the Rinse-Repeat Intrusion Detection methodology. We've done this by improving the filtering capabilities in CapLoader. For starters, we've added an input filter, which can be used to specify IP addresses, IP networks, protocols or port numbers to be parsed or ignored. The input filter uses the Berkeley Packet Filter (BPF) syntax, and is designed to run really fast. So if you wanna analyze only HTTP traffic you can simply write “port 80” as your input filter to have CapLoader only parse and display flows going to or from port 80. We have also added a display filter, which unlike Wireshark also uses BPF. Thus, once a set of flows is loaded one can easily apply different display filters, like “host 22.214.171.124” or “net 192.168.1.0/24”, to apply different views on the parsed data.
Image: CapLoader with input filter "port 80 or port 443" and display filter "not net 126.96.36.199/16".
The main differences between the input filter and display filter are:
- Input filter is much faster than the display filter, so if you know beforehand what ports, protocols or IP addresses you are interested in then make sure to apply them as an input filter. You will notice a delay when applying a display filter to a view of 10.000 flows or more.
- In order to apply a new input filter CapLoader has to reload all the opened PCAP files (which is done by pressing F5). Modifying display filters, on the other hand, only requires you to press Enter or hit the “Apply” button.
- Previously applied display filters are accessible in a drop-down menu in the GUI, but no history is kept of previous input filters.
NetFlow + DNS == true
The “Flows” view in CapLoader gives a great overview of all TCP, UDP and SCTP flows in the loaded PCAP files. However, it is usually not obvious to an analyst what every IP address is used for. We have therefore added a DNS parser to CapLoader, so that all DNS packets can be parsed in order to map IP addresses to domain names. The extracted domain names are displayed for each flow, which is very useful when performing Rinse-Repeat analysis in order to quickly remove “known good servers” from the analysis.
Leveraging the Alexa top 1M list
As we've show in in our previous blog post “DNS whitelisting in NetworkMiner”, using a list of popular domain names as a whitelist can be an effective method for finding malware. We often use this approach in order to quickly remove lots of known good servers when doing Rinse-Repeat analysis in large datasets.
Therefore, just as we did for NetworkMiner 1.5, CapLoader now includes Alexa's list of the 1 million most popular domain names on the Internet. All domain names, parsed from DNS traffic, are checked against the Alexa list. Domains listed in the whitelist are shown in CapLoader's “Server_Alexa_Domian” column. This makes it very easy to sort on this column in order to remove (hide) all flows going to “normal” servers on the Internet. After removing all those flows, what you're left with is pretty much just:
- Local traffic (not sent over the Internet)
- Outgoing traffic to either new or obscure domains
Manually going through the remaining flows can be very rewarding, as it can reveal C2 traffic from malware that has not yet been detected by traditional security products like anti-virus or IDS.
Image: CapLoader with malicious flow to 1.web-counter[.]info (Miuref/Boaxxe Trojan) singled out due to missing Alexa match.
Many new features in CapLoader 1.3
The new features highlighted above are far from the only additions made to CapLoader 1.3. Here is a more complete list of improvements in this release:
- Support for “Select Flows in PCAP” to extract and select 5-tuples from a PCAP-file. This can be a Snort PCAP with packets that have triggered IDS signatures. This way you can easily extract the whole TCP or UDP flow for each signature match, instead of just trying to make sense of one single packet per alert.
- Improved packet carver functionality to better carve IP, TCP and UPD packets from any file. This includes memory dumps as well as proprietary and obscure packet capture formats.
- Support for SCTP flows.
- DNS parser.
- Alexa top 1M matching.
- Input filter and display filter with BPF syntax.
- Flow Producer-Consumer-Ratio PCR.
- Flow Transcript can be opened simply by double-clicking a flow.
- Find form updated with option to hide non-matching flows instead of just selecting the flows that matched the keyword search criteria.
- New flow transcript encoding with IP TTL, TCP flags and sequence numbers to support analysis of Man-on-the-Side attacks.
- Faster loading of previously opened files, MD5 hashes don't need to be recalculated.
- A selected set of flows in the GUI can be inverted simply by right-clicking the flow list and selecting “Invert Selection” or by hitting Ctrl+I.
Downloading CapLoader 1.3
All these new features, except for the Alexa lookup of domain names, are available in our free trial version of CapLoader.
So to try out these new features in CapLoader, simply grab a trial download here:
https://www.netresec.com/?page=CapLoader#trial (no registration needed)
All paying customers with an older version of CapLoader can grab a free update for version 1.3 at our customer portal.
Posted by Erik Hjelmvik on Monday, 28 September 2015 07:30:00 (UTC/GMT)
One of the new features in NetworkMiner Professional 1.5 is the ability to check if domain names in DNS requests/responses are “normal” or malicious ones. This lookup is performed offline using a local copy of Alexa's top 1 million domain name list.
We got the idea for this feature via Jarno Niemelä's great presentation titled “Making Life Difficult for Malware”. Despite working for F-Secure Jarno presents several smart ideas for avoiding malware infections without having to install an AV-product.
One of Jarno's slides contains the following suggestions:
Block Traffic To Sites Your Users Don’t Go ToBlock subdomain hosting TLDs
Block domains that provide dynamic DNS
- co.cc, co.tv, ce.ms, rr.nu, cu.cc, cz.cc, vv.cc, cw.cm, cx.cc, etc
Block file sharing sites, some malware use them
- *dyndns*, *no-ip*, 8866.org, thescx.info, 3322.org, sock8.com
For strict policy, allow DNS resolving only to Alexa top 1M
- fileleave.com, dropbox.com, rapidshare.com, megafiles.com
- Tip: Instead of null routing domains set up landing page
- Either with a link that allows domain or IT ticket
Preventing users from visiting sites outside of the top 1 million websites (according to Alexa) sounds a bit harsh. In fact, we at Netresec just recently made it into the top 1M list (the current rank for netresec.com is 726 922). There are also many good and legit sites that are not yet on this list. Our idea is, however, to give analysts a heads up on queried DNS names that are not on the top 1M list by displaying this information in NetworkMiner's DNS tab.
The screenshot above contains a lookup for the domain “office.windowupdate.com” (note the missing “s” in “windows”). This domain name was previously used by the C2 protocol Lurk (see Command Five's report “Command and Control in the Fifth Domain” for more details). The “Alexa Top 1M” column in NetworkMiner's DNS tab indicates whether or not the domain name is a well known domain. The malicious “office.windowupdate.com” is marked with “No”, while the proper “www.update.microsoft.com” is indeed on the list. It is, however, important to note that only the second-level domain is checked by NetworkMiner; i.e. in this case “windowupdate.com” and “microsoft.com”.
The DNS whitelisting technique can also come in handy when dealing with malware that employs domain generation algorithms (DGAs) (see the Damballa blog for additional info regarding DGAs). It is probably safe to say that these auto-generated domains should never show up in the Alexa Top 1M list.
Posted by Erik Hjelmvik on Wednesday, 02 October 2013 22:30:00 (UTC/GMT)