NETRESEC Network Security Blog - Tag : Wireshark
One of the highlights at this year’s SEC-T conference in Stockholm was Steve Miller’s talk titled "Reversing the TriStation Network Protocol". In this talk Steve covered his quest to better understand the TRITON malware, which had been used in a targeted attack of an industrial control system (ICS). Steve didn’t disclose the type or location of the plant, saying “Don’t ask me who it was, ‘cause I can’t say” when the Q&A started. However, an article in the Wall Street Journal points out that it was a petrochemical plant in Saudi Arabia that had been hacked.
Targeting Safety Instrumented System
The TRITON malware (also called TRISIS) was used to target a safety instrumented system (SIS) from Schneider Electric called Triconex. A SIS is typically not used to control the process of a plant, but rather to detect abnormal operating conditions and safely shut down the industrial process if needed.
I could elaborate a lot regarding the consequences of attacking the SIS, but the good guys from Dragos have already done a great job explaining this in their “TRISIS Malware” report.
Reverse Engineering the ICS Protocol
The communication protocol used by the Triconex controllers is called TriStation, which is a proprietary protocol. This means that there were no publicly available specifications available for the protocol at that time. There was also no Wireshark dissector that could parse TriStation traffic. Nevertheless, Steve’s initial reaction to this was “Awesome, undocumented things are my favorite things!”
Unfortunately Steve wasn’t able to get hold of a single PCAP file with the TriStation network protocol, which made it really difficult to reverse engineer the protocol implementation in the TRITON malware. The only piece of actual TriStation network traffic he was able to get hold of was a hex dump of a TriStation packet in an academic paper.
Armed with only the hexdump and Wireshark’s text2pcap Steve managed to piece together an actual PCAP file containing a single frame with a TriStation packet inside.
As you can see in the image above, Wireshark doesn’t decode any of the application layer data coming from TCP port 1502 (which TriStation uses). He therefore implemented a Wireshark Lua dissector for the TriStation protocol. And some time later the people from Nozomi Networks even implemented a proper Wireshark dissector for the TriStation protocol.
BSI’s ICS-SEC team have now also created Snort IDS rules specifically for the TriStation protocol. These IDS rules trigger on events like:
- Packets sent to the controller from an unauthorized host
- Malicious commands used by the TRITON malware to read and write to the RAM of the SIS controller as well as to execute code
The Importance of Sniffing ICS Traffic
I’ve been trying to convince asset owners, who use ICS in their power plants, factories, water treatment facilities etc, to start capturing the network traffic and storing it as PCAP files for many years now. However, asset owners sometimes try to argue that there is no point in capturing their traffic since it is using a proprietary protocol. Even Ralph Langner has opposed to the idea of capturing ICS network traffic in a blog post, which I have criticized. So, how difficult is it to write a parser for a proprietary protocol?
I have personally implemented support for over 30 application layer protocols in NetworkMiner, but unlike Steve I’ve always had access to at least one PCAP file and some form of documentation of the protocol. However, I’ve found that many real-world protocol implementations don’t follow specifications properly. In these cases I’ve found that having access to PCAP files with real-world network traffic is more important than having a full protocol specification.
Even complex proprietary protocols like the old proprietary Skype protocol has been reverse engineered, so with access to network traffic of a protocol combined with a binary that uses this protocol I’d say that pretty much any network protocol can be reverse engineered.
Steve’s SEC-T talk also proves that ICS protocols are no different, since they too can be reverse engineered without having a protocol specification or RFC.
Capturing network traffic in ICS networks is never wrong. There might not be parsers available today for all the protocols you’re using. But once a parser or IDS signature becomes available for the protocol you’re using, you can simply use that to analyze previously captured network traffic from your ICS network. Also, in the wake of an incident you might actually end up writing a parser (as in the TRITON case) or a custom IDS rule, in which case having historical network traffic from your plant in invaluable!
For more information on this topic I’d suggest reading my blog post titled “Monitor those Control System Networks!” from 2011, which still is highly relevant.
I’m also happy to announce that two PCAP files containing TriStation network traffic have been linked from our list of publicly accessible PCAP files today (see the “SCADA/ICS Network Captures” section).
And remember: PCAP or it didn’t happen!
Posted by Erik Hjelmvik on Friday, 21 September 2018 14:20:00 (UTC/GMT)
We are today happy to announce the release of CapLoader 1.5.
This new version of CapLoader parses pcap and
Support for ICMP Flows
CapLoader is designed to group packets together that belong to the same bi-directional flow, i.e. all UDP, TCP and SCTP packets with the same 5-tuple (regardless of direction) are considered being part of the same flow.
- /fʌɪv ˈtjuːp(ə)l/
- A combination of source IP, destination IP, source port, destination port and transport protocol (TCP/UDP/SCTP) used to uniquely identify a flow or layer 4 session in computer networking.
The flow concept in CapLoader 1.5 has been extended to also include ICMP. Since there are no port numbers in the ICMP protocol CapLoader sets the source and destination port of ICMP flows to 0. The addition of ICMP in CapLoader also allows input filters and display filters like “icmp” to be leveraged.
Image: CapLoader 1.5 showing only ICMP flows due to display filter 'icmp'.
TCP Stream Reassembly
One of the foundations for making CapLoader a super fast tool for reading and filtering PCAP files is that it doesn’t attempt to reassemble TCP streams. This means that CapLoader’s Transcript view will show out-of-order segments in the order they were received and retransmitted segments will be displayed twice.
The steps required to reassemble a TCP stream to disk with Wireshark are:
- Right-click a TCP packet in the TCP session of interest.
- Select “Follow > TCP Stream”.
- Choose direction in the first drop-down-list (client-to-server or server-to-client).
- Change format from “ASCII” to “Raw” in the next drop-down-menu.
- Press the “Save as...” button to save the reassembled TCP stream to disk.
Unfortunately Wireshark fails to properly reassemble some TCP streams. As an example the current stable release of Wireshark (version 2.2.5) shows duplicate data in “Follow TCP Stream” when there are retransmissions with partially overlapping segments. We have also noticed some additional bugs related to TCP stream reassembly in other recent releases of Wireshark. However, we’d like to stress that Wireshark does perform a correct reassembly of most TCP streams; it is only in some specific situations that Wireshark produces a broken reassembly. Unfortunately a minor bug like this can cause serious consequences, for example when the TCP stream is analyzed as part of a digital forensics investigation or when the extracted data is being used as input for further processing. We have therefore decided to include a TCP stream reassembly engine in CapLoader 1.5. The steps required to reassemble a TCP stream in CapLoader are:
- Double click a TCP flow of interest in the “Flows” tab to open a flow transcript.
- Click the “Save Client Byte Stream” or “Save Server Byte Stream” button to save the data stream for the desired direction to disk.
Extracting TCP streams from PCAP files this way not only ensures that the data stream is correctly reassembled, it is also both faster and simpler than having to pivot through Wireshark’s Follow TCP Stream feature.
PCAP Icon Context Menu
The PCAP icon in CapLoader is designed to allow easy drag-and-drop operations in order to open a set of selected flows in an external packet analysis tool, such as Wireshark or NetworkMiner. Right-clicking this PCAP icon will bring up a context menu, which can be used to open a PCAP with the selected flows in an external tool or copy the PCAP to the clipboard. This context menu has been extended in CapLoader 1.5 to also include a “Save As” option. Previous versions of CapLoader required the user to drag-and-drop from the PCAP icon to a folder in order to save filtered PCAP data to disk.
Faster Parsing with Protocol Identification
CapLoader can identify over 100 different application layer protocols, including HTTP, SSL, SSH, RTP, RTCP and SOCKS, without relying on port numbers. The protocol identification has previously slowed down the analysis quite a bit, which has caused many users to disable this powerful feature. This new release of of CapLoader comes with an improved implementation of the port-independent protocol identification feature, which enables PCAP files to be loaded twice as fast as before with the “Identify protocols” feature enabled.
Works in Linux and macOS
One major improvement in CapLoader 1.5 is that this release is compatible with the Mono framework, which makes CapLoader platform independent. This means that you can now run CapLoader on your Mac or Linux machine if you have Mono installed. Please refer to our previous blog posts about how to run NetworkMiner in various flavors of Linux and macOS to find out how to install Mono on your computer. You will, however, notice a performance hit when running CapLoader under Mono instead of using Windows since the Mono framework isn't yet as fast as Microsoft's .NET Framework.
Image: CapLoader 1.5 running in Linux (Xubuntu).
We’d like to thank Sooraj for reporting a bug in the “Open With” context menu of CapLoader’s PCAP icon. This bug has been fixed in CapLoader 1.5 and Sooraj has been awarded an official “PCAP or it didn’t happen” t-shirt for reporting the bug.
Image: PCAP or it didn't happen t-shirt
Have a look at our Bug Bounty Program if you also wanna get a PCAP t-shirt!
Downloading CapLoader 1.5
Everything mentioned in this blog post, except for the protocol identification feature, is available in our free trial version of CapLoader.
To try it out, simply grab a copy here:
https://www.netresec.com/?page=CapLoader#trial (no registration needed)
All paying customers with an older version of CapLoader can download a free update to version 1.5 from our customer portal.
Posted by Erik Hjelmvik on Tuesday, 07 March 2017 09:11:00 (UTC/GMT)
time machine? Then your best chance is to install PacketCache, which allows you to read OLD packets with Wireshark.
We recently released a free tool for keeping a cache of recently sent/received network traffic in Windows. The tool, called PacketCache, is actually a Windows service that saves a copy of recent packets in RAM. The cached packets can be read simply by connecting to a named pipe called “PacketCache”, for example by using a PowerShell script as shown on the PacketCache page.
After talking to some Wireshark core developers at
last week we managed to get Wireshark to read packets from PacketCache's named pipe stream.
However, you will need to use Wireshark 2.3 or later to properly read from a named pipe.
Unfortunately version 2.3 isn't scheduled for release until next summer (2017),
so until then you'll have to use one of the automated builds instead.
I usually go for the latest WiresharkPortable build, since it doesn't require installation.
You can download the portable version of Wireshark 2.3 here:
Look for a file called “WiresharkPortable_2.3.[something].paf.exe”.
Follow these steps in order to read packets captured by PacketCache:
- Make sure you have Wireshark 2.3.0 (or later)
- Start Wireshark with admin rights (right-click > “Run as administrator”)
- Press: Capture > Options
- Click “Manage Interfaces...”
- Select the “Pipes” tab
- Press the “+” button to add a named pipe
- Name the pipe “\\.\pipe\PacketCache” and press ENTER to save it
- Press “OK” in the Manage Interface window.
- Press “Start” to read the packets from PacketCache
The status field in Wireshark will say “Live capture in progress”, which is somewhat true. Wireshark will be updating the GUI live as packets are read from PacketCache, but the packets displayed can be several hours or even days old depending on when they were captured by PacketCache. The “live” capture will stop once all packets have been read from the PacketCache.
Posted by Erik Hjelmvik on Friday, 28 October 2016 14:50:00 (UTC/GMT)
Lets say you've collected around 100 GB of PCAP files in a network monitoring installation. How would you approach the task of looking at the application layer data of a few of the captured sessions or flows?
For much smaller datasets, in the order of 100 MB, one would typically load the PCAP into Wirehsark and perform ”Follow TCP Stream” on a few sessions to see what's going on. But loading gigabyte datasets into Wireshark doesn't scale very well, in fact Wireshark will typically run out of RAM and crash saying “Out Of Memory!” or just “Wireshark has stopped working”. Ulf Lamping explains why on the Wireshark Wiki:
“Wireshark uses memory to store packet meta data (e.g. conversation and fragmentation related data) and to display this info on the screen.
I need memory about ten times the actual capture file size”
The solution I'm proposing is to instead download the free version of CapLoader, load the PCAP files into CapLoader and perform ”Flow Transcript” on a few of the flows. So how long time would it take to do this on 100 GB of PCAP files? I did a quick test and loaded the 85 GB dataset from ISCX 2012 into CapLoader on an ordinary laptop computer. After just 1 hour 47 minutes all of the PCAP files from ISCX 2012 were loaded and indexed by CapLoader! Also, please note that datasets this large can be parsed in less than 30 minutes with a more powerful PC.
After having loaded all the PCAP files CapLoader presents a list of all the 2.066.653 indexed flows from the ISCX 2012 dataset. Right-clicking a UDP or TCP flow brings up a context menu that allows you to generate a “Flow Transcript”, this feature is basically the same thing as Wireshark's “Follow TCP Stream”.
CapLoader's Flow Transcript View
You can, of course, always extract the frames from any flow directly to Wirehsark if you aren't ready to abandon Wireshark's Follow TCP Stream just yet. A flow is extracted simply by selecting a flow in the list and then doing drag-and-drop from CapLoader's PCAP icon (at the top right) onto Wireshark.
The fact that CapLoader parses and indexes large PCAP files very fast and that the analyst is provided with powerful tools, like the Transcript feature, to look at the raw packet data makes it possible to perform big-data network traffic analysis using an ordinary PC. This means that you do NOT need to upload your network traffic to the Cloud, or build a 100-machine cluster, in order to let a Hadoop instance parse though your multi-gigabyte packet captures. All you need is an ordinary PC and a copy of CapLoader.
This blog post makes use of the UNB ISCX 2012 Intrusion Detection Evaluation Dataset, which is created by Ali Shiravi, Hadi Shiravi, and Mahbod Tavallaee from University of New Brunswick
With the release of CapLoader 1.4 it is now possible to perform flow transcripts not only from the Flows tab, but also from the Services and Hosts tab. In these cases the transcript will be that of the first flow of the selected service or host.
Posted by Erik Hjelmvik on Thursday, 24 January 2013 12:20:00 (UTC/GMT)
There was recently a question on the Wireshark users mailing list about “how to get the query name from a dns request packet with tshark”. This is a problem that many network analysts run into, so I decided to write a blog post instead of just replying to the mailing list.
Note: the pcap file used in this blog post is from the DFRWS 2009 Challenge.
Who queried for a particular domain?
Tshark can easily be used in order to determine who queried for a particular domain, such as google.com, by using the following command:
tshark -r nssal-capture-1.pcap -T fields -e ip.src -e dns.qry.name -R "dns.flags.response eq 0 and dns.qry.name contains google.com"
List all queries
A list of ALL queries can be built with the same command, but without filtering on a particular domain:
tshark -r nssal-capture-1.pcap -T fields -e ip.src -e dns.qry.name -R "dns.flags.response eq 0"
DNS lists in NetworkMiner
There is a DNS tab in NetworkMiner, which displays a nice list of all DNS queries and responses in a pcap file. Loading the same nssal-capture-1.pcap into NetworkMiner generates the following list:
DNS tab with nssal-capture-1.pcap loaded
NetworkMiner Professional also has the ability to export this data to a CSV file. The command line tool NetworkMinerCLI can also generate such a CSV file without a GUI, which is perfect if you wanna integrate it in a customized script.
Posted by Erik Hjelmvik on Sunday, 17 June 2012 17:45:00 (UTC/GMT)
Below is a short video tutorial showing some of the cool features in CapLoader 1.0.
The functionality showed in the video includes:
- Loading multiple pcap files into a single flow view
- Port Independent Protocol Identification (PIPI)
- Fast extraction of packets related to one or several flows
- Exporting packets to Wireshark and NetworkMiner
- Drag-and-dropping packets to Wireshark
- Selecting a flow based on an IDS alert from Snort
- Extracting packets from a selected flow to a new pcap file
The video can also be seen on YouTube at the following URI:
The three pcap files loaded in the video tutorial are from the DFRWS 2009 Challenge.
Posted by Erik Hjelmvik on Monday, 30 April 2012 14:35:00 (UTC/GMT)
This blog post is the second part of a two-part tutorial that shows how to sniff network traffic. The first part covered how to intercept the traffic, i.e. how to get the packets to arrive to your network card. This second part covers how to best capture the network traffic to disk once you've managed to have them sent to your network card.
The first thing I recommend you to do before you start sniffing is to ensure that your sniffer computer stays quiet, so that you don't pollute the sniffed traffic with packets from your sniffer computer. How to do this varies between Windows, Linux and FreeBSD, so I will briefly cover how to silence all three OS's.
Windows is normally quite chatty on the network, but there is a simple trick that makes it shut up without disconnecting it from the network. Simply bring up the properties window for your network adapter and un-check all clients, services and protocols from the list of used items.
I don't know any good way to disable the TCP/IP stack in Linux similar to what we just did to Windows. But by disabling ARP and not giving the interface any IP address the risk of leaking packets can be reduced significantly. To achieve this you'll also need to set the interface to promiscuous mode and disable IPv6. Here's an example of how this can be done in Debian/Ubuntu:
# ifconfig eth0 -arp upHowever, in general it is better to configure the sniffer interface by modifying /etc/network/interfaces to look as follows:
# ifconfig eth0 promisc
# ip addr del 192.168.3.4/24 dev eth0
# ifconfig eth0 del fe80::1234:ddff:fe3f:1337/64
iface eth0 inet manual
up ifconfig $IFACE -arp up
up ip link set $IFACE promisc on
down ip link set $IFACE promisc off
down ifconfig $IFACE down
For more details on shuting up your Linux machine I suggest you take a look at the “Undetectable Sniffing On Ethernet” blog post on the Ask Apache blog. I also recommend reading the Ubuntu instructions at the NetworkConfiguration wiki page for Security Onion.
One of the many great things about FreeBSD is that it is really easy to configure a network interface to be silent. All you need to do is to issue this single command:
# ifconfig em0 monitor up
Read Richard Bejtlich's blog post “Ifconfig Monitor Option” for more info about this command.
Sniff with dumpcap
The most common software used for sniffing traffic is undoubtedly Wireshark, other common tools are tshark and tcpdump. All these three tools parse the packets as they come in and show data about the sniffed packets in the GUI or on the command line. This isn't desirable when doing network forensics since these application could use up too much resources on parsing the incoming packets, which during traffic peaks can lead to packets being dropped. There might also be bugs in the packet parsing code that can cause the application to crash completely, there have been plenty of such bugs in Wireshark for example.
A much more reliable way to sniff traffic is therefore to use an application that is designed to reliably dump traffic to disk, without trying to parse or analyze the contents of the sniffed frames. One such application is dumpcap, which is a command line tool developed by the Wireshark crew. In fact, as of version 1.4.0 of Wireshark both tshark and Wireshark run dumpcap in the background rather than doing the sniffing themselves. Another great thing about dumpcap is that it is available on Windows as well as on Linux and FreeBSD. My recommendation is to run it under FreeBSD since I consider it to be more reliable than the other operating systems. In order to get dumcap you need to download either Wirehsark or tshark since dumpcap doesn't have an installation package of its own.
Dumpcap supports capture filtering with BPF, but I recommend that you sniff ALL packets if you are authorized to do so and have enough disk space. It is much better to do the filtering later on in the analysis phase. The feature I like best in dumpcap is the ring buffer functionality, where it will continue writing incoming frames to a new pcap file once the old one has reached a specific size limit. In order to have dumpcap write sniffed frames to disk and roll over to a new file when it reaches 100.000 kB you'd run this command:
# dumpcap -i em0 -w wiretap.pcap -b filesize:100000
When you've dumped the traffic to disk is when the fun starts, i.e. analyzing the pcap files. You can read more analyzing pcap files in my blog posts about the TCP/IP Weapons School Sample Lab, DFRWS 2009 Challenge, Webmail Information Leakage and Command-line forensics.
Posted by Erik Hjelmvik on Friday, 11 March 2011 16:56:00 (UTC/GMT)