NETRESEC Network Security Blog - Tag : Tor

NetworkMiner 2.6 Released

NetworkMiner 2.6

We are happy to announce the release of NetworkMiner 2.6 today! The network forensic tool is now even better at extracting emails, password hashes, FTP transfers and artifacts from HTTP and HTTP/2 traffic than before.

Some of the major improvements in this new release are related to extraction and presentation of emails from SMTP, POP3 and IMAP traffic. On that note, we’d like to thank Mandy van Oosterhout for reporting a bug in our email parser!

Emails extracted with NetworkMiner 2-6
Image: Emails extracted from SMTP and IMAP traffic

I have previously blogged about how to extract John-the-Ripper hashes from Kerberos network traffic with NetworkMiner. We have now added support for presenting LANMAN and NTLM credentials as JtR hashes as well.

NTLMv2 and Kerberos hashes in NetworkMiner 2.6
Image: JtR formatted NTLMv2 and Kerberos hashes in NetworkMiner 2.6

We have also improved NetworkMiner’s Linux support. Files, images and folders can now be opened in external tools directly from the NetworkMiner GUI also when running NetworkMiner in Linux using Mono 6 (or later). Linux users previously got a “System.ComponentModel.Win32Exception” error message saying something like “Cannot find the specified file” or “Access denied” due to a breaking change introduced in Mono version 6.

NetworkMiner running in Ubuntu 20.04
Image: NetworkMiner 2.6 running in Ubuntu 20.04 with Mono

The new release also comes with several updates of how HTTP and HTTP/2 traffic is handled and presented. We have, for example, added better extraction of data sent in HTTP (or HTTP/2) POST requests. Posted JSON formatted parameters are also extracted even if the JSON data has been gzip compressed. The “Accept-Language” header values in HTTP and HTTP/2 are extracted as “Host Details” in order to support forensic analysis of user language settings, as shown by Fox-IT in their “Operation Wocao - Shining a light on one of China’s hidden hacking groups” report.

NetworkMiner has supported decapsulation of tunneling protocols and protocols for network virtualization, like 802.1Q, GRE, PPPoE, VXLAN, OpenFlow, MPLS and EoMPLS, since version 2.1. We have now improved our GRE parser to also support NVGRE (RFC 7637) by adding support for Transparent Ethernet Bridging.

Jan Hesse sent us a feature request on Twitter earlier this year, where asked about support for FritzBox captures. We are happy to announce that NetworkMiner now supports the modified pcap format you get when sniffing network traffic with a FritzBox gateway.


NetworkMiner 2.6 can now also parse and extract SIP chat messages (RFC 3428) to the “Messages” tab. Audio extraction of VoIP calls is still a feature that is exclusively available only in NetworkMiner Professional though.

NetworkMiner Professional

Our commercial tool NetworkMiner Professional has received a few additional updates, such as support for analysis of HTTP/2 traffic in the “Browsers tab”. However, please note that NetworkMiner does not perform TLS decryption, so the HTTP/2 traffic will have to be decrypted by a TLS proxy like PolarProxy prior to being saved to a PCAP file.

HTTP/2 traffic in NetworkMiner Professional's Browsers tab

We have added a few new great online services to NetworkMiner Pro’s OSINT lookup as well, such as, Browserling, MalwareDomainList and VirusTotal lookups of URL’s in the “Browsers” tab. We have also added some additional external OSINT sources for lookups of IP addresses and domain names, such as MalwareDomainList and mnemonic ACT. The JA3 hash lookup menu in NetworkMiner Professional’s “Hosts” tab has also been extended to include GreyNoise.

URL lookup menu in NetworkMiner Professional's Browsers tab

NetworkMiner Pro previously played back G.722 VoIP audio at half speed. This issue has now been fixed, so that G.722 RTP audio is extracted and played back in 16k samples/s. The bug was due to an error in RFC 1890 that was later corrected in RFC 3551. Thanks to Michael "MiKa" Kafka for teaching us about this!

Excerpt from RFC 3551:

Even though the actual sampling rate for G.722 audio is 16,000 Hz, the RTP clock rate for the G722 payload format is 8,000 Hz because that value was erroneously assigned in RFC 1890 and must remain unchanged for backward compatibility. The octet rate or sample-pair rate is 8,000 Hz.

We’d also like to mention that NetworkMiner Professional now comes with improved analytical support to help investigators detect Tor traffic.

Upgrading to Version 2.6

Users who have purchased a license for NetworkMiner Professional 2.x can download a free update to version 2.6 from our customer portal, or use the “Help > Check for Updates” feature. Those who instead prefer to use the free and open source version can grab the latest version of NetworkMiner from the official NetworkMiner page.

Posted by Erik Hjelmvik on Wednesday, 23 September 2020 09:10:00 (UTC/GMT)

Tags: #NetworkMiner #SMTP #POP3 #IMAP #email #FTP #JtR #John #Mono #Linux #HTTP #HTTP/2 #GRE #SIP #VoIP #Tor #PCAP

More... Share  |  Facebook   Twitter   Reddit   Hacker News Short URL:

TorPCAP - Tor Network Forensics


Unencrypted network traffic, destined for the Tor network, is sent between localhost TCP sockets on computers running Tor clients, such as the Tor Browser. In this blog post I show how anonymous Tor browsing can be visualized, by loading a PCAP file with localhost traffic into NetworkMiner. We call this technique TorPCAP.

Tor is a secure platform that enables users to browse the web anonymously. The Tor Project website describes the tool as:

“Tor is free software and an open network that helps you defend against traffic analysis”

It is also possible to host anonymous “onion services” on the Dark Web using Tor:

“Tor makes it possible for users to hide their locations while offering various kinds of services, such as web publishing or an instant messaging server. Using Tor "rendezvous points," other Tor users can connect to these onion services, formerly known as hidden services, each without knowing the other's network identity.”

Capturing Tor Traffic Before it gets Encrypted

Tor installations include a SOCKS proxy listening on TCP port 9150 on localhost ( This local SOCKS proxy is used by the Tor Browser, which connects to the proxy in order to have its traffic encrypted and forwarded to the Tor network. This means that by sniffing traffic on localhost it’s actually possible to create a solid forensic trail of all traffic a PC sends to and from the Tor network.

Tor Browser and SOCKS

You can use tcpdump to capture the localhost traffic on PCs running the Tails OS or Tor Browser in MacOS or Linux. If you’re running the Tor Browser in Windows, then we recommend using RawCap to sniff the localhost traffic (RawCap is a portable standalone tool that doesn’t need WinPcap or NDIS drivers to work).

In order to make sense of the captured traffic you need a tool that can parse the SOCKS protocol (RFC 1928). NetworkMiner includes a SOCKS parser since version 2.1, which can be used to extract and reassemble data going to and from the Tor network.

   Image Credit: Ken Edge    Eldon by @kenedgeiscool

Demo: Analysing TorPCAP Network Traffic

A user, let’s call him “Eldon”, used Tor for some dark-web activity on November 30, 2018. Eldon was using the Tor Browser on a Windows PC and RawCap was used to capture the localhost network traffic from Eldon’s computer. A PCAP file with the captured packets from Eldon’s PC can be accessed here. Please feel free to open this capture file with NetworkMiner, in order to follow along in this analysis.

File   : rawcap-localhost-tor.pcap
Size   : 1.47 MB
SHA256 : 9134FA542B388498C2A58A2E1424FCD4AF466CE7117DBE9AAFD0A031CC8209B8

The “Files” tab in NetworkMiner contains a list of all files that have been reassembled from the analyzed PCAP file. This file listing reveals that Eldon used the “not Evil” search engine (hss3uro2hsxfogfq[.]onion) to search for “buy fake passports” in frame 1136.

NetworkMiner's Files tab with not Evil search

The search result page from not Evil has been reassemled by NetworkMiner as “index.php.CB66877E.html”. By opening this HTML document in a browser we can see which search results Eldon got (no Internet connection is needed to open the reassembled html).

not Evil search in Tor

The “Browsers” tab in NetworkMiner Professional shows that Eldon followed the link for entry #2 in his search results (BUY FAKE PASSPORTS [...]), leading him to the “fakeimz[...].onion” website.

HTML document in Edge reassembled by NetworkMiner

Eldon then proceeded to list the available passports (see the reassembled file “novelty_fake_id_samples.shtml” in frame 1837) and chose the UK passport (“pp-uk-open-big.jpg”).

novelty_fake_id_samples.shtml NetworkMiner Professional Images tab with pp-uk-open-big.jpg

As Eldon proceeded he got a price list for the fake passports offered at this site (“novelty_fake_id_pricing.shtml”), but we don’t see any evidence of him actually completing a purchase of a fake UK passport.

HTML file reassembled by NetworkMiner opened in Edge browser

If we go back to the Images tab in NetworkMiner, and scroll a bit further down we see a picture of a gun. Let’s see where it comes from.

NetworkMiner Images tab with gun pic

It turns out Eldon also searched for “buy guns for bitcoin UK”. You can list all search engine queries by looking for entries in the “Parameters” tab with parameter name “q”. This technique is applicable for the “not Evil” search engine as well as most clearnet search engines, like Google, Bing, Yahoo! and DuckDuckGo (disregarding the fact that they use TLS).

NetworkMiner Parameters tab with web searches

The Browsers tab shows us that Eldon clicked on a link to the “UK Guns and Ammo Store” (tuu66[...].onion).

not Evil search in NetworkMiner Professional Browsers tab

This website has also been passively reassembled by NetworkMiner and can be opened offline in a browser (see “index[2].html”).

UK Guns and Ammo Store (dark web)

The Credentials tab in NetworkMiner shows the username and password used by Eldon to log into the website:

Credentials tab in NetworkMiner Professional 2.3.1 showing username and password sent over Tor to an onion service

After logging in, Eldon puts two items in his shopping cart (see “cart.php[1].html”), but gets a message saying “Not enough balance for this order” when clicking the “Continue to Checkout” link. It seems Eldon’s account at the dark-web weapons store doesn’t have any Bitcoins (see “wallet.php.html”)

UK Guns and Ammo Store - Shopping Cart (dark web) UK Guns and Ammo Store - Bitcoin Wallet (dark web)

Side Note - Web Trackers and Tor

It is considered bad practice to use clear-net tracking services, like Google Analytics, to track users visiting an onion service. However, we noticed that the fake passports website uses a Google Analytics script with tracking ID “UA-19359933-1”.

Dark Web HTML with Googla Analytics ID UA-19359933-1

Googling this ID led us to this very similar website:


Posted by Leon Kowalski on Wednesday, 12 December 2018 09:33:00 (UTC/GMT)

Tags: #Tor #PCAP #NetworkMiner #RawCap #SOCKS #

More... Share  |  Facebook   Twitter   Reddit   Hacker News Short URL:

Port Independent Protocol Detection

Protocol Alphabet Soup by ThousandEyes

Our heavy-duty PCAP analyzer CapLoader comes with a feature called ”Port Independent Protocol Identification”, a.k.a. PIPI (see Richard Bejtlich's PIPI blog post from 2006). Academic research in the Traffic Measurement field often use the term ”Traffic Classification”, which is similar but not the same thing. Traffic Classification normally group network traffic in broad classes, such as Email, Web, Chat or VoIP. CapLoader, on the other hand, identifies the actual application layer protocol used in each flow. So instead of classifying a flow as ”VoIP” CapLoader will tell you if the flow carries SIP, Skype, RTP or MGCP traffic. This approach is also known as “Dynamic Protocol Detection”.

Being able to identify application layer protocols without relying on the TCP or UDP port number is crucial when analyzing malicious traffic, such as malware Command-and-Control (C2) communication, covert backdoors and rouge servers, since such communication often use services on non-standard ports. Some common examples are:

  • Many botnet C2 protocols communicate over port TCP 443, but using a proprietary protocol rather than HTTP over SSL.
  • Backdoors on hacked computers and network devices typically wither run a standard service like SSH on a port other than 22 in order to hide.
  • More advanced backdoors use port knocking to run a proprietary C2 protocol on a standard port (SYNful knock runs on TCP 80).

This means that by analyzing network traffic for port-protocol anomalies, like an outgoing TCP connection to TCP 443 that isn't SSL, you can effectively detect intrusions without having IDS signatures for all C2 protocols. This analysis technique is often used when performing Rinse-Repeat Intrusion Detection, which is a blacklist-free approach for identifying intrusions and other form of malicious network traffic. With CapLoader one can simply apply a BPF filter like “port 443” and scroll through the displayed flows to make sure they are all say “SSL” in the Protocol column.

CapLoader detects non-SSL traffic to Image: Miuref/Boaxxe Trojan C2 traffic to "1.web-counter[.]info" on TCP 443 doesn't use SSL (or HTTPS)

Statistical Analysis

CapLoader relies on statistical analysis of each TCP, UDP and SCTP session's behavior in order to compare it to previously computed statistical models for known protocols. These statistical models are generated using a multitude of metrics, such as inter-packet delays, packet sizes and payload data. The port number is, on the other hand, a parameter that is intentionally not used by CapLoader to determine the application layer protocol.

The PIPI/Dynamic Protocol Detection feature in CapLoader has been designed to detect even encrypted and obfuscated binary protocols, such as Tor and Encrypted BitTorrent (MSE). These protocols are designed in order to deceive protocol detection mechanisms, and traditional signature based protocol detection algorithms can't reliably detect them. The statistical approach employed by CapLoader can, on the other hand, actually detect even these highly obfuscated protocols. It is, however, important to note that being a statistical method it will never be 100% accurate. Analysts should therefore not take for granted that a flow is using the protocol stated by CapLoader. There are some situations when it is very difficult to accurately classify an encrypted protocol, such as when the first part of a TCP session is missing in the analyzed data. This can occur when there is an ongoing session that was established before the packet capture was started.

Identified Protocols

The following protocols are currently available for detection in CapLoader's protocol database:

AOL Instant Messenger
BitTorrent Encrypted - MSE
DropBox LSP
eDonkey Obfuscated
Gh0st RAT
Groove LAN DPP
MikroTik NDP
Modbus TCP
MSN Messenger
NetBIOS Datagram Service
NetBIOS Name Service
NetBIOS Session Service
Poison Ivy RAT
Reverse Shell
SopCast P2P
Spotify P2P
Spotify Server
TeamViewer UDP
XMPP Jabber
Zeus TCP
Zeus UDP

The list of implemented protocols is constantly being increased with new protocols.

PIPI in NetworkMiner

NetworkMiner Logo

NetworkMiner Professional, which is the commercial version of NetworkMiner, also comes with an implementation of our protocol detection mechanism. Even though NetworkMiner Professional doesn't detect as many protocols as CapLoader, the PIPI feature built into NetworkMiner Pro still helps a lot when analyzing HTTP traffic on ports other that 80 or 8080 as well as in order to reassemble files downloaded from FTP or TFTP servers running on non-standard ports.


Posted by Erik Hjelmvik on Tuesday, 06 October 2015 09:05:00 (UTC/GMT)

Tags: #Protocol Identification #CapLoader #VoIP #SIP #RTP #TOR #SSL #PIPI #PCAP #NetworkMiner

More... Share  |  Facebook   Twitter   Reddit   Hacker News Short URL:

Detecting TOR Communication in Network Traffic

The anonymity network Tor is often misused by hackers and criminals in order to remotely control hacked computers. In this blog post we explain why Tor is so well suited for such malicious purposes, but also how incident responders can detect Tor traffic in their networks.

Yellow onions with cross section. Photo taken by Andrew c

The privacy network Tor (originally short for The Onion Router) is often used by activists and whistleblowers, who wish to preserve their anonymity online. Tor is also used by citizens of countries with censored Internet (like in China, Saudi Arabia and Belarus), in order to evade the online censorship and surveillance systems. Authorities in repressive regimes are therefore actively trying to detect and block Tor traffic, which makes research on Tor protocol detection a sensitive subject.

Tor is, however, not only used for good; a great deal of the traffic in the Tor networks is in fact port scans, hacking attempts, exfiltration of stolen data and other forms of online criminality. Additionally, in December last year researchers at Rapid7 revealed a botnet called “SkyNet” that used Tor for its Command-and-Control (C2) communication. Here is what they wrote about the choice of running the C2 over Tor:

"Common botnets generally host their Command & Control (C&C) infrastructure on hacked, bought or rented servers, possibly registering domains to resolve the IP addresses of their servers. This approach exposes the botnet from being taken down or hijacked. The security industry generally will try to take the C&C servers offline and/or takeover the associated domains
What the Skynet botnet creator realized, is that he could build a much stronger infrastructure at no cost just by utilizing Tor as the internal communication protocol, and by using the Hidden Services functionality that Tor provides."

Tor disguised as HTTPS

Tor doesn't just provide encryption, it is also designed to look like normal HTTPS traffic. This makes Tor channels blend in quite well with normal web surfing traffic, which makes Tor communication difficult to identify even for experienced incident responders. As an example, here is how tshark interprets a Tor session to port TCP 443:

$ tshark -nr tbot_2E1814CCCF0.218EB916.pcap | head
1 0.000000 -> TCP 62 1565 > 443 [SYN] Seq=0 Win=64240 Len=0 MSS=1460 SACK_PERM=1
2 0.126186 -> TCP 60 443 > 1565 [SYN, ACK] Seq=0 Ack=1 Win=64240 Len=0 MSS=1460
3 0.126212 -> TCP 54 1565 > 443 [ACK] Seq=1 Ack=1 Win=64240 Len=0
4 0.127964 -> SSL 256 Client Hello
5 0.128304 -> TCP 60 443 > 1565 [ACK] Seq=1 Ack=203 Win=64240 Len=0
6 0.253035 -> TLSv1 990 Server Hello, Certificate, Server Key Exchange, Server Hello Done
7 0.259231 -> TLSv1 252 Client Key Exchange, Change Cipher Spec, Encrypted Handshake Message
8 0.259408 -> TCP 60 443 > 1565 [ACK] Seq=937 Ack=401 Win=64240 Len=0
9 0.379712 -> TLSv1 113 Change Cipher Spec, Encrypted Handshake Message
10 0.380009 -> TLSv1 251 Encrypted Handshake Message

A Tor session to TCP port 443, decoded by tshark as if it was HTTPS

The thsark output above looks no different from when a real HTTPS session is being analyzed. So in order to detect Tor traffic one will need to apply some sort of traffic classification or application identification. However, most implementations for protocol identification rely on either port number inspection or protocol specification validation. But Tor often communicate over TCP 443 and it also follows the TLS protocol spec (RFC 2246), because of this most products for intrusion detection and deep packet inspection actually fail at identifying Tor traffic. A successful method for detecting Tor traffic is to instead utilize statistical analysis of the communication protocol in order to tell different SSL implementations apart. One of the very few tools that has support for protocol identification via statistical analysis is CapLoader.

CapLoader provides the ability to differentiate between different types of SSL traffic without relying on port numbers. This means that Tor sessions can easily be identified in a network full of HTTPS traffic.

Analyzing the tbot PCAPs from Contagio

@snowfl0w provides some nice analysis of the SkyNet botnet (a.k.a. Trojan.Tbot) at the Contagio malware dump, where she also provides PCAP files with the network traffic generated by the botnet.

The following six PCAP files are provided via Contagio:

  1. tbot_191B26BAFDF58397088C88A1B3BAC5A6.pcap (7.55 MB)
  2. tbot_23AAB9C1C462F3FDFDDD98181E963230.pcap (3.24 MB)
  3. tbot_2E1814CCCF0C3BB2CC32E0A0671C0891.pcap (4.08 MB)
  4. tbot_5375FB5E867680FFB8E72D29DB9ABBD5.pcap (5.19 MB)
  5. tbot_A0552D1BC1A4897141CFA56F75C04857.pcap (3.97 MB) [only outgoing packets]
  6. tbot_FC7C3E087789824F34A9309DA2388CE5.pcap (7.43 MB)

Unfortunately the file “tbot_A055[...]” only contains outgoing network traffic. This was likely caused by an incorrect sniffer setup, such as a misconfigured switch monitor port (aka SPAN port) or failure to capture the traffic from both monitor ports on a non-aggregating network tap (we recommend using aggregation taps in order to avoid these types of problems, see our sniffing tutorial for more details). The analysis provided here is therefore based on the other five pcap files provided by Contagio.

Here is a timeline with relative timestamps (the frame timestamps in the provided PCAP files were way of anyway, we noticed an offset of over 2 months!):

  • 0 seconds : Victim boots up and requests an IP via DHCP
  • 5 seconds : Victim perform a DNS query for
  • 6 seconds : Victim gets time via NTP
    ---{malware most likely gets executed here somewhere}---
  • 22 seconds : Victim performs DNS query for
  • 22 seconds : Victim gets its external IP via an HTTP GET request to
  • 23 seconds : Victim connects to the Tor network, typically on port TCP 9001 or 443
    ---{lots of Tor traffic from here on}---

This is what it looks like when one of the tbot pcap files has been loaded into CapLoader with the “Identify protocols” feature activated:
CapLoader detecting Tor protocol
CapLoader with protocol detection in action - see “TOR” in the “Sub_Protocol” column

Notice how the flows to TCP ports 80, 9101 and 443 are classified as Tor? The statistical method for protocol detection in CapLoader is so effective that CapLoader actually ignores port numbers altogether when identifying the protocol. The speed with which CapLoader parses PCAP files also enables analysis of very large capture files. A simple way to detect Tor traffic in large volumes of network traffic is therefore to load a capture file into CapLoader (with “Identify protocols” activated), sort the flows on the “Sub_Protocol” column, and scroll down to the flows classified as Tor protocol.

Beware of more Tor backdoors

Most companies and organizations allow traffic on TCP 443 to pass through their firewalls without content inspection. The privacy provided by Tor additionally makes it easy for a botnet herder to control infected machines without risking his identity to be revealed. These two factors make Tor a perfect fit for hackers and online criminals who need to control infected machines remotely.

Here is what Claudio Guarnieri says about the future use of Tor for botnets in his Rapid7 blog post:

“The most important factor is certainly the adoption of Tor as the main communication channel and the use of Hidden Services for protecting the backend infrastructure. While it’s surprising that not more botnets adopt the same design, we can likely expect more to follow the lead in the future.”

Incident responders will therefore need to learn how to detect Tor traffic in their networks, not just in order to deal with insiders or rogue users, but also in order to counter malware using it as part of their command-and-control infrastructure. However, as I've shown in this blog post, telling Tor apart from normal SSL traffic is difficult. But making use of statistical protocol detection, such as the Port Independent Protocol Identification (PIPI) feature provided with CapLoader, is in fact an effective method to detect Tor traffic in your networks.

Posted by Erik Hjelmvik on Saturday, 06 April 2013 20:55:00 (UTC/GMT)

Tags: #CapLoader #TOR #Protocol Identification #SSL #TLS #HTTPS #PCAP #PIPI

More... Share  |  Facebook   Twitter   Reddit   Hacker News Short URL:


NETRESEC on Twitter

Follow @netresec on twitter:


Recommended Books

» The Practice of Network Security Monitoring, Richard Bejtlich (2013)

» Applied Network Security Monitoring, Chris Sanders and Jason Smith (2013)

» Network Forensics, Sherri Davidoff and Jonathan Ham (2012)

» The Tao of Network Security Monitoring, Richard Bejtlich (2004)

» Practical Packet Analysis, Chris Sanders (2017)

» Windows Forensic Analysis, Harlan Carvey (2009)

» TCP/IP Illustrated, Volume 1, Kevin Fall and Richard Stevens (2011)

» Industrial Network Security, Eric D. Knapp and Joel Langill (2014)