NETRESEC Network Security Blog
The free and open source network forensics tool NetworkMiner now comes with improved extraction of files and metadata from several protocols as well as a few GUI updates. But the biggest improvements for version 2.3 are in the commercial tool NetworkMiner Professional, which now supports VoIP call audio extraction and playback as well as OSINT lookups of file hashes, IP addresses, domain names and URLs.
I’m happy to announce that NetworkMiner 2.3 now does an even better job than before at extracting files and metadata from several protocols. Improvements have been made in the parsers for the following protocols: HTTP, IEC-104, IPv4, Modbus, SIP, SMB, SMB2, SMTP and SSL/TLS.
We have also added support for the SNMP protocol in NetworkMiner 2.3, so that SNMP community strings can be extracted and displayed on the Parameters and Credentials tabs.
Another change is that timestamps are now displayed using the UTC time zone instead of using the local time zone. We have also fixed a few GUI quirks in order to further improve the usability of the tool.
The commercial version of NetworkMiner, i.e. NetworkMiner Professional, comes with several additional improvements which are presented below.
VoIP Call Playback
NetworkMiner Professional has received a new tab called “VoIP”, which enables audio playback of VoIP calls that are using SIP and RTP with G.711 μ-law or A-law encoding (u-Law is primarily used in North America and Japan while A-law is used in Europe and most other parts of the world).
Video: Audio playback and extraction to WAV from the “SIP_CALL_RTP_G711” PCAP file in the Wireshark Sample Captures.
The audio streams from the VoIP calls are also extracted to disk as .WAV files when codecs G.729 or G.711 (u-Law and A-Law) is used. NetworkMiner Professional also attempts to reassemble RTP streams encoded with G.722 to .au files.
OSINT Lookups of IP Addresses, Domains, URLs and File Hashes
Right-clicking a network host in NetworkMiner Professional’s Hosts tab brings up a context menu with options for performing lookups of IP and domain names using external sources. We refer to this method as open-source intelligence (OSINT) because the accessed data resides at publicly available sources.
Clicking on an OSINT provider brings up a webpage with more detailed information about the selected IP address, such as IBM X-Force, mnemonic Passive DNS, Shodan, UrlQuery or VT. However, if you’re lazy like me, then you’ll probably click the “All above!” option instead, which will bring up all of the sources in separate tabs in your browser.
The full list of OSINT providers available for IP lookups includes APNIC Whois, BFK Passive DNS, Censys, Cymon, DNSTrails, ExoneraTor, Google Public DNS, GreenSnow.co, Hurricane Electric, IBM X-Force, Internet Storm Center, mnemonic Passive DNS, PacketTotal, Shodan, ThreatCrowd, ThreatMiner, UrlQuery and VirusTotal.
The domain name lookup menu contains a similar set of providers: BFK Passive DNS, Cymon, DNSTrails, Google Public DNS, Google Safe Browsing, Hybrid Analysis, IBM X-Force Exchange, mnemonic Passive DNS, MXToolBox, MyWOT, Norton Safe Web, PacketTotal, ThreatCrowd, ThreatMiner, URL Void, UrlQuery, VirusTotal, Website Informer, Webutation and Whoisology.
Right-clicking a URL in the Browsers tab brings up a similar context menu, which additionally includes the following services for URL lookups: Google Safe Browsing, IBM X-Force, ThreatMiner, URLhaus and UrlQuery.
Finally, right-clicking on one of the files that NetworkMiner has extracted from a PCAP file brings up a menu for doing OSINT lookups based on the MD5 or SHA256 hash of the file. The sources used for lookups of hashes include IBM X-Force, PacketTotal, ThreatCrowd, TotalHash, UrlQuery, VirScan.org, Comodo Valkyrie, AlienVault OTX, Hybrid Analysis, ThreatMiner and VirusTotal.
Hybrid Analysis API Integration
Did you know that the malware analysis service Hybrid Analysis provides free API keys to people in the IT security community?
This is a great move by the Hybrid Analysis team, and we’re happy to announce that we have leveraged their API in NetworkMiner Professional in order to submit files for analysis directly from within the NetworkMiner GUI. The API integration also enables you to query for an IP on Hybrid Analysis to see which previously submitted samples has communicated with that particular IP address.
Here are the steps required to enable the Hybrid Analysis API integration:
- Create a HybridAnaysis account
- Create an API key
- Upgrade the API key by completing a vetting process
- Start NetworkMiner Pro, open the Tools > Settings menu and input your API key
I would like to thank Chris Sistrunk, Mats Karlsson and Michael Nilsson for suggesting several of the protocol and GUI improvements that have been incorporated into this new release. I’d also like to thank Doug Green and Ahmad Nawawi for discovering and reporting bugs in the IP and SSL parser respectively.
Upgrading to Version 2.3
Users who have purchased a license for NetworkMiner Professional 2.x can download a free update to version 2.3 from our customer portal.
Those who instead prefer to use the free and open source version can grab the latest version of NetworkMiner from the official NetworkMiner page.
⛏ FOR GREAT JUSTICE! ⛏
Posted by Erik Hjelmvik on Tuesday, 03 April 2018 06:27:00 (UTC/GMT)
This network forensics video tutorial covers analysis of a malware redirect chain, where a PC is infected through the RIG Exploit Kit. A PCAP file, from Brad Duncan's malware-traffic-analysis.net website, is opened in NetworkMiner Professional in order to follow a redirect chain via a couple of hacked websites before delivering malware to the PC.
Meadgive on VirusTotal
CVE-2014-0569 Flash Exploit on VirusTotal
CVE-2012-0507 Java Exploit on VirusTotal
Check out our series of network forensic video tutorials for more tips and tricks on how to analyze captured network traffic.
Posted by Erik Hjelmvik on Monday, 26 February 2018 11:19:00 (UTC/GMT)
This network forensics video tutorial covers how to analyze SPAM email traffic from the Kelihos botnet. The analyzed PCAP file comes from the Stratosphere IPS project, where Sebastian Garcia and his colleagues execute malware samples in sandboxes. The particular malware sample execution we are looking at this time is from the CTU-Malware-Capture-Botnet-149-2 dataset.
- Stratosphere IPS dataset: CTU-Malware-Capture-Botnet-149-2
- Alexa's Top 1 Milion Domain List
- Cisco Ubrella Popularity List
- Emerging Threats' TROJAN Win32/Kelihos.F Checkin Signature
6c55 5545 0310 4840
Check out our series of network forensic video tutorials for more tips and tricks on how to analyze captured network traffic.
Posted by Erik Hjelmvik on Monday, 19 February 2018 06:37:00 (UTC/GMT)
series of network forensic video tutorials covers a quick and crude way to scan a PCAP file for malware. It's all done locally without having to run the PCAP through an IDS. Kudos to Lenny Hanson for showing me this little trick!
Antivirus Scanning of a PCAP File
- SWF/Neclu.B on VirusTotal
- CVE-2015-0311 (Flash Player vulnerability exploited by Neclu)
- Win32/Simda.AT on VirusTotal (Kryptik)
Posted by Erik Hjelmvik on Monday, 12 February 2018 08:00:00 (UTC/GMT)
Jason Reaves gave a talk titled “Malware C2 over x509 certificate exchange” at BSides Springfield 2017, where he demonstrated that the SSL handshake can be abused by malware as a covert command-and-control (C2) channel.
He got the idea while analyzing the Vawtrak malware after discovering that it read multiple fields in the X.509 certificate provided by the server before proceeding. Jason initially thought these fields were used as a C2 channel, but then realized that Vawtrak performed a variant of certificate pinning in order to discover SSL man-in-the-middle attempts.
Nevertheless, Jason decided to actually implement a proof-of-concept (PoC) that uses the X.509 certificate as a C2 channel. Jason’s code is now available on GitHub along with a PCAP file demonstrating this covert C2 channel. Of course I couldn’t resist having a little look at this PCAP file in NetworkMiner.
The first thing I noticed was that the proof-of-concept PCAP ran the SSL session on TCP 4433, which prevented NetworkMiner from parsing the traffic as SSL. However, I was able to parse the SSL traffic with NetworkMiner Professional just fine thanks to the port-independent-protocol-identification feature (a.k.a Dynamic Port Detection), which made the Pro-version parse TCP 4433 as SSL/TLS.
Image: X.509 certificates extracted from PCAP with NetworkMiner
A “normal” x509 certificate size is usually around 1kB, so certificates that are 11kB should be considered as anomalies. Also, opening one of these .cer files reveals an extremely large value in the Subject Key Identifier field.
Not only is this field very large, it also starts with the familiar “4D 5A” MZ header sequence.
NetworkMiner additionally parses details from the certificates that it extracts from PCAP files, so the Subject Key Identifier field is actually accessible from within NetworkMiner, as shown in the screenshot below.
You can also see that NetworkMiner validates the certificate using the local trusted root certificates.
Not surprisingly, this certificates is not trusted
Extracting the MZ Binary from the Covert X.509 Channel
Even though NetworkMiner excels at pulling out files from PCAPs, this is definitively an occasion where manual handling is required. Jason’s PoC implementation actually uses a whopping 79 individual certificates in order to transfer this Mimikatz binary, which is 785 kB.
Here’s a tshark oneliner you can use to extract the Mimicatz binary from Jason's example PCAP file.
tshark -r mimikatz_sent.pcap -Y 'ssl.handshake.certificate_length gt 2000' -T fields -e x509ce.SubjectKeyIdentifier -d tcp.port==4433,ssl | tr -d ':\n' | xxd -r -p > mimikatz.exe
Detecting x509 Anomalies
Even though covert channels using x509 certificates isn’t a “thing” (yet?) it’s still a good idea to think about how this type of covert signaling can be detected. Just looking for large Subject Key Identifier fields is probably too specific, since there are other fields and extensions in X.509 that could also be used to transmit data. A better approach would be to alert on certificates larger than, let’s say, 3kB. Multiple certificates can also be chained together in a single TLS handshake certificate record, so it would also make sense to look for handshake records larger than 8kB (rough estimate).
This type of anomaly-centric intrusion detection is typically best done using the Bro IDS, which provides easy programmatic access to the X.509 certificate and SSL handshake.
There will be false positives when alerting on large certificates in this manner, which is why I recommend to also check if the certificates have been signed by a trusted root or not. A certificate that is signed by a trusted root is very unlikely to contain malicious data.
Posted by Erik Hjelmvik on Tuesday, 06 February 2018 12:13:00 (UTC/GMT)
series of network forensics video tutorials throughout the next few weeks. First up is this analysis of a PCAP file containing network traffic from the "Zyklon H.T.T.P." malware.
Analyzing a Zyklon Trojan with Suricata and NetworkMiner
Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:220.127.116.11pre) Gecko/20070302 BonEcho/18.104.22.168pre
Posted by Erik Hjelmvik on Monday, 05 February 2018 07:30:00 (UTC/GMT)
We are happy to release TrimPCAP today! TrimPCAP is a free open source tool that reduces the size of capture files in an intelligent way.
The retention period of a packet capture solution is typically limited by either legal requirements or available disk space. In the latter case the oldest capture files are simply removed when the storage starts getting full. This means that if there is a long ongoing session, such as a download of a large ISO file, streamed video or a reverse shell backdoor, then the start of this session will likely be removed.
I know from experience that it’s painful to analyze network traffic where the start of a session is missing. The most important and interesting stuff generally happens in the beginning of each session, such as the HTTP GET request for an ISO file. As an analyst you don’t need to look at all the other packets in that ISO download (unless you believe the ISO contains malware), it’s enough to see that there is a GET request for the file and the server responds with a “200 OK”.Image: CapLoader transcript of an ISO download
If that download had been truncated, so that only the last few packets were remaining, then it would be really difficult to know what was being downloaded. The same is true also for other protocols, including proprietary C2 protocols used by botnets and other types of malware.
TrimPCAP is designed to overcome the issue with truncated sessions by removing data from the end of sessions rather than from the beginning. This also comes with a great bonus when it comes to saving on disk usage, since the majority of the bytes transferred across the Internet are made up of big sessions (a.k.a “Elephant Flows”). Thus, by trimming a PCAP file so that it only contains the first 100kB of each TCP and UDP session it’s possible to significantly reduce required storage for that data.
The following command reduces the PCAP dataset used in our Network Forensics Training from 2.25 GB to just 223 MB:
user@so$ python trimpcap.py 102400 /nsm/sensor_data/so-eth1/dailylogs/*/*
Trimming capture files to max 102400 bytes per flow.
Dataset reduced by 90.30% = 2181478771 bytes
A maximum session size (or "flow cutoff") of 100kB enables trimpcap.py to reduce the required storage for that dataset to about 10% of its original size, which will significantly extend the maximum retention period.
Putting TrimPCAP Into Practice
Let’s assume that your organization currently has a maximum full content PCAP retention period of 10 days and that trimming sessions to 100kB reduces the required storage to 10%. TrimPCAP will then enable you to store 5 days with full session data plus 50 additional days with trimmed sessions using the same disk space as before!
A slightly more advanced scheme would be to have multiple trim limits, such as trimming to 1MB after 3 days, 100kB after 6 days and 10kB after 30 days. Such a setup would probably extend your total retention period from 10 days to over 100 days.
An even more advanced trimming scheme is implemented in our packet capture agent PacketCache. PacketCache constantly trims its PCAP dataset because it is designed to use only 1 percent of a PC’s RAM to store observed packets, in case they are needed later on for incident response. PacketCache uses a trim limit which declines individually for each observed TCP and UDP session depending on when they were last observed.
TrimPCAP is open source software and is released under the GNU General Public License version 2 (GPLv2).
You can download your own copy of TrimPCAP from the official TrimPCAP page:
✂- - - - - - - - -
Posted by Erik Hjelmvik on Tuesday, 05 December 2017 12:40:00 (UTC/GMT)
CapLoader is designed to simplify complex tasks, such as digging through gigabytes of PCAP data looking for traffic that sticks out or shouldn’t be there. Improved usability has therefore been the primary goal, when developing CapLoader 1.6, in order to help our users do their work even more efficiently than before.
Some of the new features in CapLoader 1.6 are:
- Context aware selection and filter suggestions when right-clicking a flow, session or host.
- Support for IPv6 addresses in the BPF syntax for Input Filter as well as Display Filter.
Flows that are inactive for more than 60 minutes are considered closed. This timeout is configurable in
Tools > Settings.
CapLoader 1.6 also introduces a new column in the Flows tab labeled “Initial_RTT”, which shows the Round Trip Time (RTT) measured during the start of a session. The RTT is defined as “the time it takes for a signal to be sent plus the time it takes for an acknowledgment of that signal to be received”. RTT is often called “ping time” because the ping utility computes the RTT by sending ICMP echo requests and measuring the delay until a reply is received.
Image: CapLoader 1.6 showing ICMP and TCP round trip times.
But using a PCAP file to measure the RTT between two hosts isn’t as straight forward as one might think. One complicating factor is that the PCAP might be generated by the client, server or by any device in between. If we know that the sniffing point is at the client then things are simple, because we can then use the delta-time between an ICMP echo request and the returning ICMP echo response as RTT. In lack of ping traffic the same thing can be achieved with TCP by measuring the time between a SYN and the returning SYN+ACK packet. However, consider the situation when the sniffer is located somewhere between the client and server. The previously mentioned method would then ignore the latency between the client and sniffer, the delta-time will therefore only show the RTT between the sniffer and the server.
This problem is best solved by calculating the Initial RTT (iRTT) as the delta-time between the SYN packet and the final ACK packet in a TCP three-way handshake, as shown here:
Image: Initial RTT is the total time of the black/bold packet traversal paths.
Jasper Bongertz does a great job of explaining why and how to use the iRTT in his blog post “Determining TCP Initial Round Trip Time”, so I will not cover it in any more detail here. However, keep in mind that iRTT can only be calculated this way for TCP sessions. CapLoader therefore falls back on measuring the delta time between the first packet in each direction when it comes to transport protocols like UDP and ICMP.
Exclusive Features Not Available in the Free Trail
The new features mentioned so far are all available in the free 30 day CapLoader trial, which can be downloaded from our CapLoader product page (no registration required). But we’ve also added features that are only available in the commercial/professional edition of CapLoader. One such exclusive feature is the matching of hostnames against the Cisco Umbrella top 1 million domain list. CapLoader already had a feature for matching domain names against the Alexa top 1 million list, so the addition of the Umbrella list might seem redundant. But it’s actually not, the two lists are compiled using different data sources and therefore complement each other (see our blog post “Domain Whitelist Benchmark: Alexa vs Umbrella” for more details). Also, the Umbrella list contains subdomains (such as www.google.com, safebrowsing.google.com and accounts.google.com) while the Alexa list only contains main domains (like “google.com”). CapLoader can therefore do more fine-granular domain matching with the Umbrella list (requiring a full match of the Umbrella domain), while the Alexa list enables a more rough “catch ‘em all” approach (allowing *.google.com to be matched).
CapLoader 1.6 also comes with an ASN lookup feature, which presents the autonomous system number (ASN) and organization name for IPv4 and IPv6 addresses in a PCAP file (see image above). The ASN lookup is built using the GeoLite database created by MaxMind. The information gained from the MaxMind ASN database is also used to provide intelligent display filter CIDR suggestions in the context menu that pops up when right-clicking a flow, service or host.
Image: Context menu suggests Display Filter BPF “net 22.214.171.124/17” based on the server IP in the right-clicked flow.
Users who have previously purchased a license for CapLoader can download a free update to version 1.6 from our customer portal.
Credits and T-shirts
We’d like to thank Christian Reusch for suggesting the Initial RTT feature and Daan from the Dutch Ministry of Defence for suggesting the ASN lookup feature. We’d also like to thank David Billa, Ran Tohar Braun and Stephen Bell for discovering and reporting bugs in CapLoader which now have been fixed. These three guys have received a “PCAP or it didn’t happen” t-shirt as promised in our Bug Bounty Program.
Posted by Erik Hjelmvik on Monday, 09 October 2017 08:12:00 (UTC/GMT)