Comparison of tools that extract files from PCAP

One of the premier features in NetworkMiner is the ability to extract files from captured network traffic in PCAP files. NetworkMiner reassembles the file contents by parsing protocols that are used to transfer files across a network.

But there are other tools that also can extract files from PCAP files, such as Wireshark and Zeek. The file extraction support in these alternative solutions sometimes complement and sometimes overlap with that of NetworkMiner. Either way it is good that there are multiple tools that are designed to perform the same task. This allows us to compare the output from the different implementations, for example if the results from one tool seems strange or is suspected to be incorrect or incomplete.

Tools that can reassemble and extract files from network traffic or PCAP files:

• Chaosreader (hasn't been updated since 2014)
• NetworkMiner
• Suricata
• tcpflow (-e all)
• Wireshark's Export Objects
• Zeek's extract-all-files.zeek

All of these tools can extract files from HTTP and FTP, but when it comes to other protocols the support varies. The following table summarizes which protocols each tool supports:

I’ve been quite forgiving when compiling the table above. Tools are listed as supporting a protocol even if they only work under very specific conditions. I don’t want to name-and-shame any tool, but I strongly recommend that you verify the tools you’re using by comparing what they extract to one or two alternative tools. As an example, some tools only support a few specific commands for the protocol they claim to support. Additionally, some tools only support file extraction in one direction for protocols like HTTP or FTP, even though these protocols are regularly used to download as well as upload files.

Posted by Erik Hjelmvik on Monday, 05 May 2025 16:05:00 (UTC/GMT)

Tags: #Extract​ #PCAP​ #NetworkMiner​ #Suricata​ #tcpflow​ #Wireshark​ #Zeek​ #FTP​ #HTTP​ #IEC-104​ #IMAP​ #LPD​ #LPR​ #njRAT​ #POP3​ #SMB​ #SMB2​ #SMTP​

Decoding njRAT traffic with NetworkMiner

I investigate network traffic from a Triage sandbox execution of njRAT in this video. The analysis is performed using NetworkMiner in Linux (REMnux to be specific).

About njRAT / Bladabindi

njRAT is a Remote Access Trojan (RAT) that can be used to remotely control a hacked computer. It has been around since 2013, but despite being over 10 years old it still remains one of the most popular backdoors used by malicious actors. Anti virus vendors usually refer to njRAT as Bladabindi.

njRAT Artefacts Extracted by NetworkMiner

NetworkMiner has a built-in parser for the njRAT Command-and-Control (C2) protocol. This njRAT parser kicks in whenever there is traffic to a well-known njRAT port, such as TCP 1177 or 5552, plus a few extra ports (like TCP 14817 that was used by the analysed sample). You’ll need NetworkMiner Professional to decode njRAT traffic to other ports, since it comes with a port-independent-protocol-identification (PIPI) feature that automatically detects the protocol regardless which port the server runs on.

As demonstrated in the video, NetworkMiner can extract the following types of artefacts from njRAT network traffic:

• Screenshots of victim computer
• Transferred files
• Commands from C2 server
• Replies from bot
• Stolen credentials/passwords
• Keylog data

Covered njRAT Commands and Plugins

These njRAT commands and plugins are mentioned in the video:

• CAP = Screen Capture
• ret = Get Passwords
• inv = Invoke Plugin
• PLG = Plugin Delivery
• kl = Key Logger
• Ex = Execute Plugin
• Ex proc = Process List
• Ex fm = File Manager

IOC List

• Sample (a.exe): cca1e0b65d759f4c58ce760f94039a0a
• C2 server: 5.tcp.eu.ngrok[.]io:14817
• njRAT inv (dll): 2d65bc3bff4a5d31b59f5bdf6e6311d7
• njRAT PLG (dll): c179e212316f26ce9325a8d80d936666
• njRAT ret (dll): ac43720c43dcf90b2d57d746464ad574
• Splitter: Y262SUCZ4UJJ

Posted by Erik Hjelmvik on Monday, 28 April 2025 06:00:00 (UTC/GMT)

Tags: #njRAT​ #NetworkMiner​ #REMnux​ #Video​ #videotutorial​​

NetworkMiner 3.0 Released

I am very proud to announce the release of NetworkMiner 3.0 today!

This version brings several new protocols as well as user interface improvements to NetworkMiner. We have also made significant changes under the hood, such as altering the default location to where NetworkMiner extracts files from network traffic.

Some of the major changes in this new release are:

• New protocols: QUIC, CIP (EtherNet/IP), UMAS and Remcos RAT.
• Improved passive OS fingerprinting.
• Additional filtering capabilities.
• User interface adapted for Linux.

Filtering of Displayed Artefacts

A tooltip text is temporarily displayed when a filter is activated. The tooltip shows the number of visible items after the filter is applied. This tooltip can also be shown at a later point by hovering with the mouse over the filter text or the Apply button.

Right-clicking on an item or artefact in any of NetworkMiner’s tabs brings up a context menu. We’ve now added an “Apply as Filter” option to this context menu, which can be used to let NetworkMiner automatically generate a filter based on the clicked item. This feature saves time for the analyst and reduces risk of misspellings.

We have also added a keyword filter to the Credentials tab and updated the image filename filter to ignore case.

Other User Interface Improvements

The File Details window, which shows metadata and contents of an extracted file, now has a “Show as” menu that can be used to preview the contents of a file as ASCII, Hex, Unicode or UTF-8.

This file details window can now also be accessed directly from the Images tab by right-clicking on a thumbnail of an extracted image.

NetworkMiner 3.0 extracts Maximum Segment Size (MSS) values from TCP handshakes and show them under Host Details for each respective IP address. This value can help with determining if a host is behind a VPN. An MSS value below 1400 indicates that the traffic might have passed through some form of overlay network, such as a tunnel or VPN.

Other indicators that can help identify VPN and tunnelled traffic is IP TTL and latency, which NetworkMiner already extracts.

The screenshot above also shows that the operating system was identified as Windows, both with help of p0f as well as based on the client’s web browser user-agent. The user-agent based OS fingerprinting is a new feature that we added in NetworkMiner 3.0. This is a nice complement to the TCP and DHCP based OS fingerprinting features that NetworkMiner already performs. We’ve configured this feature to also detect operating systems of user-agent strings sent over UPnP/SSDP.

The text on a few of NetworkMiner’s buttons were not visible on some Linux distros, depending on how much button padding the respective window manager and theme added. Button sizes have therefore been increased in this release to reduce the risk of text not being visible when NetworkMiner is run in Linux.

New Protocol: QUIC

NetworkMiner 3.0 parses initial packets from the QUIC protocol (RFC 9000), which is the UDP based protocol used to transport HTTP/3. The QUIC parser allows NetworkMiner to extract TLS handshakes from UDP 443 traffic, from which the server’s hostname can be read if the client uses the SNI extension. The extracted TLS handshakes from QUIC are also used to generate JA3 and JA4 fingerprints for clients.

New Protocol: CIP and EtherNet/IP

We added parsers for the industrial control system protocols CIP and EtherNet/IP. The implementation does not cover all of the CIP and EtherNet/IP specifications, instead we focused on extracting device information, such as product vendor, product name, bulletin name, serial number and hostname. Such device information is crucial when performing passive asset identification of PLC’s and other industrial devices on OT/ICS networks, such as in factories and power plants. The CIP parser also supports extraction of tag data from Rockwell's proprietary version of CIP.

New Protocol: UMAS

A parser for the industrial control system protocol Modbus/TCP was added to NetworkMiner 2.0 back in 2016. In today’s 3.0 release we’ve enhanced the Modbus implementation to also parse out commands from Schneider Electric's proprietary UMAS protocol, which runs on top of Modbus by using the special function code 90 (0x5a). Our implementation unfortunately doesn’t have full coverage of UMAS, since we don’t have a protocol specification for this proprietary protocol. Nevertheless, our implementation recognizes 40 different UMAS commands (aka UMAS function codes) and can extract fields and parameters from at least 6 of them. The parsed UMAS commands can be viewed in NetworkMiner’s Parameters tab.

New Protocol: REMCOS C2

We started adding parsers for proprietary malicious Command-and-Control (C2) protocols, like StealC, njRAT, BackConnect and RMS, to NetworkMiner a couple of years ago. These malware C2 and backdoor protocol parsers enable security researchers to study what actions threat actors perform when accessing victim computers or honeypot systems.

We’re continuing on our endeavour of creating parsers for malicious protocol by adding support for the Remcos RAT C2 protocol to NetworkMiner 3.0.

Naturally, NetworkMiner’s Remcos parser can’t extract the C2 comms if Remcos uses TLS. Another limitation is that the free version of NetworkMiner is only able to parse Remcos traffic when the C2 server is running on a standard port like TCP 2404. The port-independent-protocol-identification feature in the Professional edition of NetworkMiner, however, identifies and parses Remcos traffic regardless of which port the C2 server listens on (the C2 server in the screenshot above was running on TCP port 1961).

Improved Protocol Parsers

We have also improved several of NetworkMiner’s existing protocol parsers. NetworkMiner’s parser for the trojan/backdoor njRAT (Bladabindi) protocol has, for example, been extended to reassemble full desktop screenshots from njRAT’s Remote Desktop feature.

NetworkMiner’s parser for Modbus has also been extended to support additional function codes and the NTLMSSP parser (for SMB/SMB2) is now better at extracting hostnames to NetworkMiner’s Hosts tab.

Bugs Fixes

A bug in NetworkMiner’s timestamp comparison code previously caused items to be sorted incorrectly when the Timestamp column header was clicked. This bug has now been fixed. We have also fixed a bug relating to extraction of parameters sent in JSON encoded HTTP POST requests.

Breaking Changes

Some of the changes introduced in the 3.0 release might require some users to adapt their workflow. One such change is that the default output path for extracted files and captured packets has changed from NetworkMiner’s directory to %LocalAppData%\NetworkMiner\ in Windows and ~/.local/share/NetworkMiner/ in Linux. This means that you no longer need to add write permissions to the NetworkMiner application directory or subdirectories thereof, since NetworkMiner no longer creates or modifies files there.

Another breaking change is that we have removed the Anomalies tab from the user interface. Windows users can still see alerts by starting NetworkMiner with --filelog, while Linux can use --debug to print debug, warning and error messages to stderr. Use --loglevel warning to suppress info and debug messages.

A change that only affects users of NetworkMiner Professional is that the command line tool NetworkMinerCLI now requires a Corporate License. If you currently have a single-user license, then you will still be able to use the command line tool in your 2.x version of NetworkMiner Professional, but not in the new 3.0 release.

NetworkMiner Professional

There are several improvements in the 3.0 release that only affect users of NetworkMiner Professional. One noteworthy update is that the Pro release has become significantly faster, especially for capture files containing many short TCP sessions. NetworkMiner Professional now saves around two milliseconds in parsing time for every TCP session. This might not sound as much, but it actually makes a huge difference when parsing capture files containing thousands of small TCP sessions.

NetworkMiner’s support for the TLS fingerprinting method JA4 has also been extended even further in the 3.0 release. NetworkMiner Professional now leverages FoxIO’s JA4 database to identify operating systems as well as applications based on client TLS handshake packets.

Other improvement of NetworkMiner Professional include:

• Network operator and AS number displayed on Hosts tab.
• File OSINT lookup includes Censys body_hash lookups.
• IP and domain OSINT lookups added to NetworkMiner’s DNS tab.
• PcapNG packet comments displayed in the Parameters tab.

Upgrading to Version 3.0

Users who have purchased NetworkMiner Professional can download version 3.0 from our customer portal, or use the “Check for Updates” feature from NetworkMiner's Help menu. Those who instead prefer to use the free and open source version can grab the latest release of NetworkMiner from the official NetworkMiner page.

Posted by Erik Hjelmvik on Friday, 04 April 2025 10:53:00 (UTC/GMT)

Tags: #NetworkMiner​ #JA3​ #JA4​ #njRAT​

NetworkMiner 2.8.1 Released

I am happy to announce the release of NetworkMiner 2.8.1 today!

This new release brings a VNC parser to NetworkMiner, so that screenshots, keystrokes and clipboard data can be extracted from unencrypted VNC traffic. NetworkMiner 2.8.1 additionally includes parsers for command-and-control (C2) protocols used by njRAT, IcedID, QakBot and Bazar. We have also added extraction of files sent using a HTTP PUT request, which is the rare cousin to the more common POST request. NetworkMiner’s Images tab has also been updated to enable filtering based on filename or pixel count.

VNC

NetworkMiner now supports the remote framebuffer (RFB) protocol (RFC 6143), which is the underlying protocol used in VNC software. VNC is used to remotely control another computer with help of a real-time video feed of the remote computer’s screen and the ability to control the remote desktop using a mouse and keyboard.

NetworkMiner reassembles the desktop graphics that get transferred between a VNC client and server to build a local representation of the remote desktop. Screenshots of that desktop then get extracted to disk by NetworkMiner as it gets updated. These desktop screenshots can be viewed in NetworkMiner’s Images tab. NetworkMiner also extracts keystrokes and clipboard data from the VNC traffic and displays them in the Parameters tab.

Limitations in VNC Extraction

Essential configuration of the VNC session is set up in the beginning of each TCP session. NetworkMiner therefore requires analyzed traffic to include this initial setup in order to successfully extract graphics from a VNC session.

Only traffic to TCP ports 5900 and 5901 will be parsed as VNC traffic by the free version of NetworkMiner. Our commercial tool NetworkMiner Professional has the ability to identify VNC traffic regardless of port number, but the free version will fail to extract screenshots from VNC traffic running on non-standard ports.

NetworkMiner’s VNC implementation is currently at an early stage, which is why it might fail to extract screenshots or other data from VNC sessions. If you encounter VNC traffic that NetworkMiner fails to parse, then we’d be very grateful if you would contact us and let us know so that we can improve NetworkMiner’s VNC extraction in future releases.

BackConnect

I have previously outlined the BackConnect protocol used by IcedID, QakBot as well as Bazar. This BackConnect protocol is used by botnet operators to remotely control a victim computer using a form of remote access trojan (RAT). This BackConnect protocol supports features such as tunneling network traffic through the victim’s computer, uploading files and the ability to control the victim’s computer using VNC.

NetworkMiner now has a parser for this BackConnect protocol, which allows analysts to see screenshots, keystrokes and file uploads from when the attacker interacts with the hacked computer. As you might have guessed, NetworkMiner’s VNC parser is reused in order to also parse reverse VNC traffic from BackConnect sessions.

I would like to thank Maxime Thiebaut for releasing his PCAPeek tool, which also reassembles reverse VNC traffic from BackConnect traffic. Maxime’s tool proved that it was possible to extract graphics from this C2 traffic, which motivated me to create a BackConnect VNC parser for NetworkMiner.

BackConnect VNC Replay

To demonstrate NetworkMiner’s reverse VNC reassembly I’d like to use the file “2023-07-26-IcedID-with-Keyhole-VNC-traffic.pcap” from Brad Duncan’s fantastic malware-traffic-analysis.net website. In this traffic the IcedID BackConnect C2 server listens to TCP port 443 on 165.232.175.216.

The screenshot thumbnails from the BackConnect VNC traffic shows that the attacker opens up the Amazon website on the victim’s computer. A copy of the full 1920x1152 screenshot, which NetworkMiner extracted from the VNC traffic, can be viewed here: VNC_73F81C83_230726142247.jpg

The parameters tab in NetworkMiner additionally confirms that the attacker manually typed “amazon.com” into the hacked computer’s web browser.

Limitations in BackConnect Parser

The BackConnect implementation in NetworkMiner 2.8.1 is pretty well tested and even supports the new “encrypted” version of BackConnect that IcedID has been deploying lately (we call it “BackConnect XOR”). NetworkMiner can parse the BackConnect C2 traffic as well as the traffic from modules for reverse VNC, reverse SOCKS, reverse shell and the file manager.

Unfortunately the BackConnect traffic typically runs on TCP ports like 443 or 8080, which are normally used for TLS or web proxies. The free open source version of NetworkMiner normally attempts to interpret traffic to port 443 as TLS and 8080 as HTTP proxy, even though it might be BackConnect. A hardcoded list of 32 verified BackConnect C2 servers, such as 165.232.175.216:443, 137.184.172.23:443 and 185.99.132.16:8080, has therefore been added to NetworkMiner’s source code. This enables NetworkMiner to interpret traffic to those IP:port combinations as BackConnect. Our commercial tool NetworkMiner Professional can identify protocols without relying on the port number though, this enables the “Pro” version to also parse BackConnect traffic to C2 servers that aren’t among the 32 hard coded ones in the source code.

njRAT

njRAT is an old, yet still very popular remote access trojan (RAT). It has been around for at least 10 years (since 2013) and can be used to do things like:

• Steal credentials
• Take screenshots of the desktop
• Start the camera
• Record audio with the microphone
• Start a reverse shell
• Upload or download files
• Execute a file or module
• Capture keystrokes with a keylogger
• Create, read or delete registry keys
• Run a command

I have been teaching students in my network forensics classes how to manually parse njRAT traffic since 2015, but from now on I’ll instead be able to tell them to load the njRAT traffic into NetworkMiner to see what the attacker did on the victim’s computer.

The NetworkMiner parameters and victim PC screenshot above were extracted from 2015-04-22_capture-win4.pcap in StratosphereIPS’ dataset CTU-Malware-Capture-Botnet-120-1.

Limitations in njRAT Parser

njRAT comes in many different versions, forks and flavors and the protocol even has a field delimiter sequence that can be customized for each njRAT backdoor. I can therefore not guarantee that NetworkMiner will be able to parse traffic from all njRAT sessions, but so far I’ve seen very good results – even when custom field delimiters are being used.

Nevertheless, this trojan has an abundance of features but only the most commonly used ones are implemented in NetworkMiner. More specifically NetworkMiner primarily supports extraction of screenshots, credentials, keylog data, file uploads and downloads. NetworkMiner also extracts exfiltrated metadata about the victim machine and bot, such as the hostname, operating system, logged in user, bot ID, botnet, njRAT version and njRAT install date.

Also, as previously mentioned, NetworkMiner doesn’t yet stitch together tiles of desktop screenshots correctly.

Another limitation is that an njRAT C2 server can be deployed on any TCP port, which might prevent the free version from interpreting the C2 traffic as njRAT. The free version of NetworkMiner currently tries to parse traffic to ports like 1177, 5050, 5552 and a few others as njRAT, while NetworkMiner Professional will automatically recognize the njRAT protocol regardless which port it runs on.

Bug Fixes

We’ve added support for SIP keepalive messages (RFC 5626) in NetworkMiner 2.8.1. NetworkMiner Professional would previously fail to extract VoIP calls, or text messages sent through SIP, if a keepalive packet was sent in the session prior to the call or text message. We have also fixed a bug in NetworkMiner’s CapWap parser.

Upgrading to Version 2.8.1

Users who have purchased NetworkMiner Professional can download a free update to version 2.8.1 from our customer portal, or use the “Check for Updates” feature from NetworkMiner's Help menu. Those who instead prefer to use the free and open source version can grab the latest version of NetworkMiner from the official NetworkMiner page.

Posted by Erik Hjelmvik on Monday, 02 October 2023 11:13:00 (UTC/GMT)

Tags: #NetworkMiner​ #VNC​ #BackConnect​ #IcedID​ #njRAT​

CapLoader 1.9.5 Alerts on Malicious Traffic

CapLoader 1.9.5 was released today!

The most important addition in the 1.9.5 release is the new Alerts tab, in which CapLoader warns about malicious network traffic such as command-and-control protocols. The alerts tab also shows information about network anomalies that often are related to malicious traffic, such as periodic connections to a particular service or long running sessions.

Other additions in this new version are:

• BPF support for “vlan” keyword, for example “vlan”, “not vlan” or “vlan 121”
• Support for nanosecond PCAP files (magic 0xa1b23c4d)
• Support for FRITZ!Box PCAP files (magic 0xa1b2cd34)
• Decapsulation of CAPWAP protocol, so that flows inside CAPWAP can be viewed and filtered on
• Domain names extracted from TLS SNI extensions

Alerts for Malicious Network Traffic

As you can see in the video at the end of this blog post, the Alert tab is a fantastic addition for everyone who wants to detect malicious activity in network traffic. Not only can it alert on over 30 different malicious command-and-control (C2) protocols — including Cerber, Gozi ISFB, IcedID, RedLine Stealer, njRAT and QakBot — it also alerts on generic behavior that is typically seen in malware traffic. Examples of such generic behavior are periodic connections to a C2 server or long running TCP connections. This type of behavioral analysis can be used to detect C2 and backdoor traffic even when the protocol is unknown. There are also signatures that detect “normal” protocols, such as HTTP, TLS or SSH running on non-standard ports as well as the reverse, where a standard port like TCP 443 is carrying a protocol that isn’t TLS.

Many of CapLoader’s alert signatures are modeled after threat hunting techniques, which can be used to detect malicious activities that traditional alerting mechanisms like antivirus, EDR’s and IDS’s might have missed. By converting the logic involved in such threat hunting tasks into signatures a great deal of the analysts’ time can be saved. In this sense part of CapLoader’s alerting mechanism is a form of automated threat hunting, which saves several steps in the process of finding malicious network traffic in a packet haystack.

Watch my Hunting for C2 Traffic video for a demonstration on the steps required to perform manual network based threat hunting without CapLoader's alerts tab. In that video I identify TLS traffic to a non-TLS port (TCP 2222) as well as non-TLS traffic to TCP port 443. As of version 1.9.5 CapLoader automatically generates alerts for that type of traffic. More specifically, the alert types will be Protocol-port mismatch (TLS on TCP 2222) and Port-protocol mismatch (non-TLS on TCP 443). Below is a screenshot of CapLoader’s new Alerts tab after having loaded the capture files analyzed in the Hunting for C2 Traffic video.

Image: Alerts for malicious traffic in CapLoader 1.9.5.

Video Demonstration of CapLoader's Alerts Tab

The best way to explain the power of CapLoader’s Alerts tab is probably by showing it in action. I have therefore recorded the following video demonstration.

The PCAP file analyzed in the video can be downloaded from here:
https://media.netresec.com/pcap/McDB_150724-18-22_FpF90.pcap

This capture file is a small snippet of the network traffic analyzed in one of my old network forensics classes. It contains malicious traffic from njRAT and Kovter mixed with a great deal of legitimate web traffic.

Posted by Erik Hjelmvik on Thursday, 09 February 2023 14:30:00 (UTC/GMT)

Tags: #CapLoader​ #Video​ #njRAT​ #Threat Hunting​