Showing blog posts from March 2017
We are today happy to announce the release of CapLoader 1.5.
This new version of CapLoader parses pcap and
Support for ICMP Flows
CapLoader is designed to group packets together that belong to the same bi-directional flow, i.e. all UDP, TCP and SCTP packets with the same 5-tuple (regardless of direction) are considered being part of the same flow.
- /fʌɪv ˈtjuːp(ə)l/
- A combination of source IP, destination IP, source port, destination port and transport protocol (TCP/UDP/SCTP) used to uniquely identify a flow or layer 4 session in computer networking.
The flow concept in CapLoader 1.5 has been extended to also include ICMP. Since there are no port numbers in the ICMP protocol CapLoader sets the source and destination port of ICMP flows to 0. The addition of ICMP in CapLoader also allows input filters and display filters like “icmp” to be leveraged.
Image: CapLoader 1.5 showing only ICMP flows due to display filter 'icmp'.
TCP Stream Reassembly
One of the foundations for making CapLoader a super fast tool for reading and filtering PCAP files is that it doesn’t attempt to reassemble TCP streams. This means that CapLoader’s Transcript view will show out-of-order segments in the order they were received and retransmitted segments will be displayed twice.
The steps required to reassemble a TCP stream to disk with Wireshark are:
- Right-click a TCP packet in the TCP session of interest.
- Select “Follow > TCP Stream”.
- Choose direction in the first drop-down-list (client-to-server or server-to-client).
- Change format from “ASCII” to “Raw” in the next drop-down-menu.
- Press the “Save as...” button to save the reassembled TCP stream to disk.
Unfortunately Wireshark fails to properly reassemble some TCP streams. As an example the current stable release of Wireshark (version 2.2.5) shows duplicate data in “Follow TCP Stream” when there are retransmissions with partially overlapping segments. We have also noticed some additional bugs related to TCP stream reassembly in other recent releases of Wireshark. However, we’d like to stress that Wireshark does perform a correct reassembly of most TCP streams; it is only in some specific situations that Wireshark produces a broken reassembly. Unfortunately a minor bug like this can cause serious consequences, for example when the TCP stream is analyzed as part of a digital forensics investigation or when the extracted data is being used as input for further processing. We have therefore decided to include a TCP stream reassembly engine in CapLoader 1.5. The steps required to reassemble a TCP stream in CapLoader are:
- Double click a TCP flow of interest in the “Flows” tab to open a flow transcript.
- Click the “Save Client Byte Stream” or “Save Server Byte Stream” button to save the data stream for the desired direction to disk.
Extracting TCP streams from PCAP files this way not only ensures that the data stream is correctly reassembled, it is also both faster and simpler than having to pivot through Wireshark’s Follow TCP Stream feature.
PCAP Icon Context Menu
The PCAP icon in CapLoader is designed to allow easy drag-and-drop operations in order to open a set of selected flows in an external packet analysis tool, such as Wireshark or NetworkMiner. Right-clicking this PCAP icon will bring up a context menu, which can be used to open a PCAP with the selected flows in an external tool or copy the PCAP to the clipboard. This context menu has been extended in CapLoader 1.5 to also include a “Save As” option. Previous versions of CapLoader required the user to drag-and-drop from the PCAP icon to a folder in order to save filtered PCAP data to disk.
Faster Parsing with Protocol Identification
CapLoader can identify over 100 different application layer protocols, including HTTP, SSL, SSH, RTP, RTCP and SOCKS, without relying on port numbers. The protocol identification has previously slowed down the analysis quite a bit, which has caused many users to disable this powerful feature. This new release of of CapLoader comes with an improved implementation of the port-independent protocol identification feature, which enables PCAP files to be loaded twice as fast as before with the “Identify protocols” feature enabled.
Works in Linux and macOS
One major improvement in CapLoader 1.5 is that this release is compatible with the Mono framework, which makes CapLoader platform independent. This means that you can now run CapLoader on your Mac or Linux machine if you have Mono installed. Please refer to our previous blog posts about how to run NetworkMiner in various flavors of Linux and macOS to find out how to install Mono on your computer. You will, however, notice a performance hit when running CapLoader under Mono instead of using Windows since the Mono framework isn't yet as fast as Microsoft's .NET Framework.
Image: CapLoader 1.5 running in Linux (Xubuntu).
We’d like to thank Sooraj for reporting a bug in the “Open With” context menu of CapLoader’s PCAP icon. This bug has been fixed in CapLoader 1.5 and Sooraj has been awarded an official “PCAP or it didn’t happen” t-shirt for reporting the bug.
Image: PCAP or it didn't happen t-shirt
Have a look at our Bug Bounty Program if you also wanna get a PCAP t-shirt!
Downloading CapLoader 1.5
Everything mentioned in this blog post, except for the protocol identification feature, is available in our free trial version of CapLoader.
To try it out, simply grab a copy here:
https://www.netresec.com/?page=CapLoader#trial (no registration needed)
All paying customers with an older version of CapLoader can download a free update to version 1.5 from our customer portal.
Posted by Erik Hjelmvik on Tuesday, 07 March 2017 09:11:00 (UTC/GMT)
NetworkMiner can reassemble files transferred over protocols such as HTTP, FTP, TFTP, SMB, SMB2, SMTP, POP3 and IMAP simply by reading a PCAP file. NetworkMiner stores the extracted files in a directory called “AssembledFiles” inside of the NetworkMiner directory.
Files extracted by NetworkMiner from the DFRWS 2008 challenge file suspect.pcap
NetworkMiner is a portable tool that is delivered as a zip file. The tool doesn’t require any installation, you simply just extract the zip file to your PC. We don’t provide any official guidance regarding where to place NetworkMiner, users are free to place it wherever they find it most fitting. Some put the tool on the Desktop or in “My Documents” while others prefer to put it in “C:\Program Files”. However, please note that normal users usually don’t have write permissions to sub-directories of %programfiles%, which will prevent NetworkMiner from performing file reassembly.
Unfortunately, previous versions of NetworkMiner didn’t alert the user when it failed to write to the AssembledFiles directory. This means that the tool would silently fail to extract any files from a PCAP file. This behavior has been changed with the release of NetworkMiner 2.1. Now the user gets a windows titled “Insufficient Write Permissions” with a text like this:
User is unauthorized to access the following file:
File(s) will not be extracted!
Follow these steps to set adequate write permissions to the AssembledFiles directory in Windows:
- Open the Properties window for the AssembledFiles directory
- Open the “Security” tab
- Press “Edit” to change permissions
- Select the user who will be running NetworkMiner
- Check the “Allow”checkbox for Write permissions
- Press the OK button
If you are running NetworkMiner under macOS (OS X) or Linux, then please make sure to follow our installation instructions, which include this command:
sudo chmod -R go+w AssembledFiles/
Once you have set up the appropriate write permissions you should be able to start NeworkMiner and open a PCAP file in order to have the tool automatically extract files from the captured network traffic.
Posted by Erik Hjelmvik on Friday, 03 March 2017 09:44:00 (UTC/GMT)