NETRESEC Network Security Blog
An increasing number of malware families employ SSL/TLS encryption in order to evade detection by Network Intrusion Detection Systems (NIDS). In this blog post I’m gonna have a look at Adwind, which is a cross-platform Remote Access Trojan (RAT) that has been using SSL to conceal it’s traffic for several years. AdwindRAT typically connects SSL sessions to seemingly random TCP ports on the C2 servers. Hence, a heuristic that could potentially be used to hunt for Adwind RAT malware is to look for SSL traffic going to TCP ports that normally don’t use SSL. However, relying on ONLY that heuristic would generate way too many false positives.
I saw the same certificate information used last week, and it continues this week.
Currently, this may be the best way to identify Adwind-based post-infection traffic. Look for SSL traffic on a non-standard TCP port using that particular certificate.
- commonName = assylias
- organizationName = assylias.Inc
- countryName = FR
Unfortunately, Adwind RAT has evolved to use other CN’s in their new certificates, so looking for “assylias.Inc” will not cut it anymore. However, looking for SSL traffic on non-standard TCP ports still holds on the latest Adwind RAT samples that we’ve analyzed.
“We offer one of the ways to detect malicious AdwindRAT software inside the encrypted traffic. Recently, the detection of this malicious program in network traffic is significantly reduced due to encryption. As a result of the research, a stable structure of data fragments was created.”
Not only is it awesome that they were able to detect static patterns in the encrypted data, they also provided 25 PCAP files containing AdwindRAT traffic. I loaded these PCAP files into NetworkMiner Professional in order to have a look at the X.509 certificates. NetworkMiner Professional supports Port-Independent Protocol Identification (PIPI), which means that it will automatically identify the C2 sessions as SSL, regardless of which port that is used. It will also automatically extract the X.509 certificates along with any other parameters that can be extracted from the SSL handshake before the session goes encrypted.Image: Files extracted from ADT’s PCAP files that mach “Oracle” and “cer”.
In this recent campaign the attackers used X.509 certificates claiming to be from Oracle. The majory of the extracted certificates were exactly 1237 bytes long, so maybe they’re all identical? This is what the first extracted X.509 certificate looks like:
The cert claims to be valid for a whopping 100 years!
Self-signed, not trusted.
However, after opening a few of the other certificates it's clear that each C2 server is using a unique X.509 certificate. This can be quickly confirmed by opening the parameters tab in NetworkMiner Pro and showing only the Certificate Hash or Subject Key Identifier values.Image: Certificate Hash values found in Adwind RAT’s SSL traffic
I also noted that the CN of the certificates isn’t constant either; these samples use CN’s such as “Oracle America”, “Oracle Tanzania”, “Oracle Arusha Inc.”, “Oracle Leonardo” and “Oracle Heaven”.
The CN field is normally used to specify which domain(s) the certificate is valid for, together with any additinoal Subject Alternative Name field. However, Adwind RAT’s certificates don’t contain any domain name in the CN field and they don’t have an Alternative Name record. This might very well change in future versions of this piece of malware though, but I don’t expect the malware authors to generate a certificate with a CN matching the domain name used by each C2 server. I can therefore use this assumption in order to better hunt for Adwind RAT traffic.
But how do I know what public domain name the C2 server has? One solution is to use passive DNS, i.e. to capture all DNS traffic in order to do passive lookups locally. Another solution is to leverage the fact that the Adwind RAT clients use the Server Name Indication (SNI) when connecting to the C2 servers.
Image: TLS Server Name (aka SNI) and Subject CN values don’t match for AdwindRATImage: TLS Server Name (SNI) with matching Subject CN from Google.
My conclusion is therefore that Brad’s recommendations from 2015 are still pretty okay, even for the latest wave of Adwind RAT traffic. However, instead of looking for a fix CN string I’d prefer to use the following heuristics to hunt for this type of C2 traffic:
- SSL traffic to non-standard SSL port
- Self signed X.509 certificate
- The SNI domain name in the Client Hello message does not match the CN or Subject Alternative Name of the certificate.
These heuristics will match more than just Adwind RAT traffic though. You’ll find that the exact same heuristics will also help identify other pieces of SSL-enabled malware as well as Tor traffic.
Posted by Erik Hjelmvik on Monday, 04 September 2017 19:01:00 (UTC/GMT)
NetworkMiner 2.2 is faster, better and stronger than ever before! The PCAP parsing speed has more than doubled and even more details are now extracted from analyzed packet capture files.
The improved parsing speed of NetworkMiner 2.2 can be enjoyed regardless if NetworkMiner is run in Windows or Linux, additionally the user interface is more responsive and flickers way less when capture files are being loaded.
User Interface Improvements
The keyword filter available in the Files, Messages, Sessions, DNS and Parameters tabs has been improved so that the rows now can be filtered on a single column of choice by selecting the desired column in a drop-down list. There is also an “Any column” option, which can be used to search for the keyword in all columns.
The Messages tab has also received an additional feature, which allows the filter keyword to be matched against the text in the message body as well as email headers when the “Any column” option is selected. This allows for an efficient analysis of messages (such as emails sent/received through SMTP, POP3 and IMAP as well as IRC messages and some HTTP based messaging platforms), since the messages can be filtered just like in a normal e-mail client.
We have also given up on using local timestamp formats; timestamps are now instead shown using the
NetworkMiner 2.2 comes with a parser for the Remote Desktop Protocol (RDP), which rides on top of COTP and TPKT. The RDP parser is primarily used in order to extract usernames from RDP cookies and show them on the Credentials tab. This new version also comes with better extraction of SMB1 and SMB2 details, such as NTLM SSP usernames.
One big change that has been made behind the scenes of NetworkMiner is the move from .NET Framework 2.0 to version 4.0. This move doesn’t require any special measures to be taken for most Microsoft Windows users since the 4.0 Framework is typically already installed on these machines. If you’re running NetworkMiner in Linux, however, you might wanna check out our updated blog post on how to install NetworkMiner in Linux.
We have also added an automatic check for new versions of NetworkMiner, which runs every time the tool is started.
This update check can be disabled by adding a
NetworkMiner.exe --noupdatecheck capturefile.pcap
Even though NetworkMiner 2.2 now uses ISO-like time representations NetworkMiner still has to decide which time zone to use
for the timestamps. The default decision has always been to use the same time zone as the local machine, but
NetworkMiner Professional now additionally comes with an option that allows the user to select whether to use UTC
(as nature intended),
the local time zone or some other custom time zone for displaying timestamps. The time zone setting can be found in the
The Port-Independent-Protocol-Detection (PIPI) feature in NetworkMiner Pro has been improved for more reliable identification of HTTP, SSH, SOCKS, FTP and SSL sessions running on non-standard port numbers.
CASE / JSON-LD Export
We are happy to announce that the professional edition of NetworkMiner 2.2 now has support for exporting extracted details using the Cyber-investigation Analysis Standard Expression (CASE) format, which is a JSON-LD format for digital forensics data. The CASE export is also available in the command line tool NetworkMinerCLI.
We would like to thank Europol for recommending us to implement the CASE export format in their effort to adopt CASE as a standard digital forensic format. Several other companies in the digital forensics field are currently looking into implementing CASE in their tools, including AccessData, Cellebrite, Guidance, Volatility and XRY. We believe the CASE format will become a popular format for exchanging digital forensic data between tools for digital forensics, log correlation and SIEM solutions.
We will, however, still continue supporting and maintaining the CSV and XML export formats in NetworkMiner Professional and NetworkMinerCLI alongside the new CASE format.
I would like to thank Sebastian Gebhard and Clinton Page for reporting bugs in the Credentials tab and TFTP parsing code that now have been fixed. I would also like to thank Jeff Carrell for providing a capture file that has been used to debug an issue in NetworkMiner’s OpenFlow parser. There are also a couple of users who have suggested new features that have made it into this release of NetworkMiner. Marc Lindike suggested the powerful deep search of extracted messages and Niclas Hirschfeld proposed a new option in the PCAP-over-IP functionality that allows NetworkMiner to receive PCAP data via a remote netcat listener.
Upgrading to Version 2.2
Users who have purchased a license for NetworkMiner Professional 2.x can download a free update to version 2.2 from our customer portal.
Those who instead prefer to use the free and open source version can grab the latest version of NetworkMiner from the official NetworkMiner page.
Posted by Erik Hjelmvik on Tuesday, 22 August 2017 11:37:00 (UTC/GMT)
People sometimes ask me when I will teach my network forensics class in the United States. The US is undoubtedly the country with the most advanced and mature DFIR community, so it would be awesome to be able to give my class there. However, not being a U.S. person and not working for a U.S. company makes it rather difficult for me to teach in the United States (remember what happened to Halvar Flake?).
So if you’re from the Americas and would like to take my network forensics class, then please don’t wait for me to teach my class at a venue close to you – because I probably won’t. My recommendation is that you instead attend my upcoming training at 44CON in London this September.
The network forensics training in London will cover topics such as:
- Analyzing a web defacement
- Investigating traffic from a remote access trojan (njRAT)
- Analyzing a Man-on-the-Side attack (much like QUANTUM INSERT)
- Finding a backdoored application
- Identifying botnet traffic through whitelisting
- Rinse-Repeat Threat Hunting
The first day of training will focus on analysis using only open source tools. The second day will primarily cover training on commercial software from Netresec, i.e. NetworkMiner Professional and CapLoader. All students enrolling in the class will get a full 6 month license for both these commercial tools.
Hope to see you at the 44CON training in London!
Posted by Erik Hjelmvik on Tuesday, 25 April 2017 14:33:00 (UTC/GMT)
In November last year Alexa admitted in a tweet that they had stopped releasing their CSV file with the one million most popular domains.
Members of the Internet measurement and infosec research communities were outraged, surprised and disappointed since this domain list had become the de-facto tool for evaluating the popularity of a domain. As a result of this Cisco Umbrella (previously OpenDNS) released a free top 1 million list of their own in December the same year. However, by then Alexa had already announced that their “top-1m.csv” file was back up again.
The Alexa list was unavailable for just about a week but this was enough for many researchers, developers and security professionals to make the move to alternative lists, such as the one from Umbrella. This move was perhaps fueled by Alexa saying that the “file is back for now”, which hints that they might decide to remove it again later on.
We’ve been leveraging the Alexa list for quite some time in NetworkMiner and CapLoader in order to do DNS whitelisting, for example when doing threat hunting with Rinse-Repeat. But we haven’t made the move from Alexa to Umbrella, at least not yet.
Malware Domains in the Top 1 Million Lists
Threat hunting expert Veronica Valeros recently pointed out that there are a great deal of malicious domains in the Alexa top one million list.
I often recommend analysts to use the Alexa list as a whitelist to remove “normal” web surfing from their PCAP dataset when doing threat hunting or network forensics. And, as previously mentioned, both NetworkMiner and CapLoader make use of the Alexa list in order to simplify domain whitelisting. I therefore decided to evaluate just how many malicious domains there are in the Alexa and Umbrella lists.
hpHosts EMD (by Malwarebytes)
- URL: https://hosts-file.net/emd.txt
- Total malicious domains: 154144
|Whitelisted malicious domains:||1365||1458|
|Percent of malicious domains whitelisted:||0.89%||0.95%|
Malware Domain Blocklist
- URL: http://www.malwaredomains.com/?page_id=66
- Total malicious domains: 18270
|Whitelisted malicious domains:||84||63|
|Percent of malicious domains whitelisted:||0.46%||0.34%|
- URL: http://cybercrime-tracker.net/ Total malicious domains: 7861
|Whitelisted malicious domains:||15||10|
|Percent of malicious domains whitelisted:||0.19%||0.13%|
The results presented above indicate that Alexa and Umbrella both contain roughly the same number of malicious domains. The percentages also reveal that using Alexa or Umbrella as a whitelist, i.e. ignore all traffic to the top one million domains, might result in ignoring up to 1% of the traffic going to malicious domains. I guess this is an acceptable number of false negatives since techniques like Rinse-Repeat Intrusion Detection isn’t intended to replace traditional intrusion detection systems, instead it is meant to be use as a complement in order to hunt down the intrusions that your IDS failed to detect. Working on a reduced dataset containing 99% of the malicious traffic is an acceptable price to pay for having removed all the “normal” traffic going to the one million most popular domains.
One significat difference between the two lists is that the Umbrella list contains subdomains (such as www.google.com, safebrowsing.google.com and accounts.google.com) while the Alexa list only contains main domains (like “google.com”). In fact, the Umbrella list contains over 1800 subdomains for google.com alone! This means that the Umbrella list in practice contains fewer main domains compared to the one million main domains in the Alexa list. We estimate that roughly half of the domains in the Umbrella list are redundant if you only are interested in main domains. However, having sub domains can be an asset if you need to match the full domain name rather than just the main domain name.
Data Sources used to Compile the Lists
Image: The Alexa Extension for Firefox
The two lists are compiled in different ways, which can be important to be aware of depending on what type of traffic you are analyzing. Alexa primarily receives web browsing data from users who have installed one of Alexa’s many browser extensions (such as the Alexa browser toolbar shown above). They also gather additional data from users visiting web sites that include Alexa’s tracker script.
Cisco Umbrella, on the other hand, compile their data from “the actual world-wide usage of domains by Umbrella global network users”. We’re guessing this means building statistics from DNS queries sent through the OpenDNS service that was recently acquired by Cisco.
This means that the Alexa list might be better suited if you are only analyzing HTTP traffic from web browsers, while the Umbrella list probably is the best choice if you are analyzing non-HTTP traffic or HTTP traffic that isn’t generated by browsers (for example HTTP API communication).
As noted by Greg Ferro, the Umbrella list contains test domains like “www.example.com”. These domains are not present in the Alexa list.
We have also noticed that the Umbrella list contains several domains with non-authorized gTLDs, such as “.home”, “.mail” and “.corp”. The Alexa list, on the other hand, only seem to contain real domain names.
Resources and Raw Data
Both the Alexa and Cisco Umbrella top one million lists are CSV files named “top-1m.csv”. The CSV files can be downloaded from these URL’s:
- Alexa: http://s3.amazonaws.com/alexa-static/top-1m.csv.zip
- Umbrella: http://s3-us-west-1.amazonaws.com/umbrella-static/top-1m.csv.zip
The analysis results presented in this blog post are based on top-1m.csv files downloaded from Alexa and Umbrella on March 31, 2017. The malware domain lists were also downloaded from the three respective sources on that same day.
We have decided to share the “false negatives” (malware domains that were present in the Alexa and Umbrella lists)
for transparency. You can download the lists with all false negatives from here:
Hands-on Practice and Training
If you wanna learn more about how a list of common domains can be used to hunt down intrusions in your network, then please register for one of our network forensic trainings. The next training will be a pre-conference training at 44CON in London.
Posted by Erik Hjelmvik on Monday, 03 April 2017 14:47:00 (UTC/GMT)
We are today happy to announce the release of CapLoader 1.5.
This new version of CapLoader parses pcap and
Support for ICMP Flows
CapLoader is designed to group packets together that belong to the same bi-directional flow, i.e. all UDP, TCP and SCTP packets with the same 5-tuple (regardless of direction) are considered being part of the same flow.
- /fʌɪv ˈtjuːp(ə)l/
- A combination of source IP, destination IP, source port, destination port and transport protocol (TCP/UDP/SCTP) used to uniquely identify a flow or layer 4 session in computer networking.
The flow concept in CapLoader 1.5 has been extended to also include ICMP. Since there are no port numbers in the ICMP protocol CapLoader sets the source and destination port of ICMP flows to 0. The addition of ICMP in CapLoader also allows input filters and display filters like “icmp” to be leveraged.
Image: CapLoader 1.5 showing only ICMP flows due to display filter 'icmp'.
TCP Stream Reassembly
One of the foundations for making CapLoader a super fast tool for reading and filtering PCAP files is that it doesn’t attempt to reassemble TCP streams. This means that CapLoader’s Transcript view will show out-of-order segments in the order they were received and retransmitted segments will be displayed twice.
The steps required to reassemble a TCP stream to disk with Wireshark are:
- Right-click a TCP packet in the TCP session of interest.
- Select “Follow > TCP Stream”.
- Choose direction in the first drop-down-list (client-to-server or server-to-client).
- Change format from “ASCII” to “Raw” in the next drop-down-menu.
- Press the “Save as...” button to save the reassembled TCP stream to disk.
Unfortunately Wireshark fails to properly reassemble some TCP streams. As an example the current stable release of Wireshark (version 2.2.5) shows duplicate data in “Follow TCP Stream” when there are retransmissions with partially overlapping segments. We have also noticed some additional bugs related to TCP stream reassembly in other recent releases of Wireshark. However, we’d like to stress that Wireshark does perform a correct reassembly of most TCP streams; it is only in some specific situations that Wireshark produces a broken reassembly. Unfortunately a minor bug like this can cause serious consequences, for example when the TCP stream is analyzed as part of a digital forensics investigation or when the extracted data is being used as input for further processing. We have therefore decided to include a TCP stream reassembly engine in CapLoader 1.5. The steps required to reassemble a TCP stream in CapLoader are:
- Double click a TCP flow of interest in the “Flows” tab to open a flow transcript.
- Click the “Save Client Byte Stream” or “Save Server Byte Stream” button to save the data stream for the desired direction to disk.
Extracting TCP streams from PCAP files this way not only ensures that the data stream is correctly reassembled, it is also both faster and simpler than having to pivot through Wireshark’s Follow TCP Stream feature.
PCAP Icon Context Menu
The PCAP icon in CapLoader is designed to allow easy drag-and-drop operations in order to open a set of selected flows in an external packet analysis tool, such as Wireshark or NetworkMiner. Right-clicking this PCAP icon will bring up a context menu, which can be used to open a PCAP with the selected flows in an external tool or copy the PCAP to the clipboard. This context menu has been extended in CapLoader 1.5 to also include a “Save As” option. Previous versions of CapLoader required the user to drag-and-drop from the PCAP icon to a folder in order to save filtered PCAP data to disk.
Faster Parsing with Protocol Identification
CapLoader can identify over 100 different application layer protocols, including HTTP, SSL, SSH, RTP, RTCP and SOCKS, without relying on port numbers. The protocol identification has previously slowed down the analysis quite a bit, which has caused many users to disable this powerful feature. This new release of of CapLoader comes with an improved implementation of the port-independent protocol identification feature, which enables PCAP files to be loaded twice as fast as before with the “Identify protocols” feature enabled.
Works in Linux and macOS
One major improvement in CapLoader 1.5 is that this release is compatible with the Mono framework, which makes CapLoader platform independent. This means that you can now run CapLoader on your Mac or Linux machine if you have Mono installed. Please refer to our previous blog posts about how to run NetworkMiner in various flavors of Linux and macOS to find out how to install Mono on your computer. You will, however, notice a performance hit when running CapLoader under Mono instead of using Windows since the Mono framework isn't yet as fast as Microsoft's .NET Framework.
Image: CapLoader 1.5 running in Linux (Xubuntu).
We’d like to thank Sooraj for reporting a bug in the “Open With” context menu of CapLoader’s PCAP icon. This bug has been fixed in CapLoader 1.5 and Sooraj has been awarded an official “PCAP or it didn’t happen” t-shirt for reporting the bug.
Image: PCAP or it didn't happen t-shirt
Have a look at our Bug Bounty Program if you also wanna get a PCAP t-shirt!
Downloading CapLoader 1.5
Everything mentioned in this blog post, except for the protocol identification feature, is available in our free trial version of CapLoader.
To try it out, simply grab a copy here:
https://www.netresec.com/?page=CapLoader#trial (no registration needed)
All paying customers with an older version of CapLoader can download a free update to version 1.5 from our customer portal.
Posted by Erik Hjelmvik on Tuesday, 07 March 2017 09:11:00 (UTC/GMT)