NetworkMiner 2.6 Released

We are happy to announce the release of NetworkMiner 2.6 today! The network forensic tool is now even better at extracting emails, password hashes, FTP transfers and artifacts from HTTP and HTTP/2 traffic than before.

Some of the major improvements in this new release are related to extraction and presentation of emails from SMTP, POP3 and IMAP traffic. On that note, we’d like to thank Mandy van Oosterhout for reporting a bug in our email parser!

I have previously blogged about how to extract John-the-Ripper hashes from Kerberos network traffic with NetworkMiner. We have now added support for presenting LANMAN and NTLM credentials as JtR hashes as well.

We have also improved NetworkMiner’s Linux support. Files, images and folders can now be opened in external tools directly from the NetworkMiner GUI also when running NetworkMiner in Linux using Mono 6 (or later). Linux users previously got a “System.ComponentModel.Win32Exception” error message saying something like “Cannot find the specified file” or “Access denied” due to a breaking change introduced in Mono version 6.

The new release also comes with several updates of how HTTP and HTTP/2 traffic is handled and presented. We have, for example, added better extraction of data sent in HTTP (or HTTP/2) POST requests. Posted JSON formatted parameters are also extracted even if the JSON data has been gzip compressed. The “Accept-Language” header values in HTTP and HTTP/2 are extracted as “Host Details” in order to support forensic analysis of user language settings, as shown by Fox-IT in their “Operation Wocao - Shining a light on one of China’s hidden hacking groups” report.

NetworkMiner has supported decapsulation of tunneling protocols and protocols for network virtualization, like 802.1Q, GRE, PPPoE, VXLAN, OpenFlow, MPLS and EoMPLS, since version 2.1. We have now improved our GRE parser to also support NVGRE (RFC 7637) by adding support for Transparent Ethernet Bridging.

Jan Hesse sent us a feature request on Twitter earlier this year, where asked about support for FritzBox captures. We are happy to announce that NetworkMiner now supports the modified pcap format you get when sniffing network traffic with a FritzBox gateway.

NetworkMiner 2.6 can now also parse and extract SIP chat messages (RFC 3428) to the “Messages” tab. Audio extraction of VoIP calls is still a feature that is exclusively available only in NetworkMiner Professional though.

NetworkMiner Professional

Our commercial tool NetworkMiner Professional has received a few additional updates, such as support for analysis of HTTP/2 traffic in the “Browsers tab”. However, please note that NetworkMiner does not perform TLS decryption, so the HTTP/2 traffic will have to be decrypted by a TLS proxy like PolarProxy prior to being saved to a PCAP file.

We have added a few new great online services to NetworkMiner Pro’s OSINT lookup as well, such as shouldiclick.org, Browserling, MalwareDomainList and VirusTotal lookups of URL’s in the “Browsers” tab. We have also added some additional external OSINT sources for lookups of IP addresses and domain names, such as MalwareDomainList and mnemonic ACT. The JA3 hash lookup menu in NetworkMiner Professional’s “Hosts” tab has also been extended to include GreyNoise.

NetworkMiner Pro previously played back G.722 VoIP audio at half speed. This issue has now been fixed, so that G.722 RTP audio is extracted and played back in 16k samples/s. The bug was due to an error in RFC 1890 that was later corrected in RFC 3551. Thanks to Michael "MiKa" Kafka for teaching us about this!

Excerpt from RFC 3551:

Even though the actual sampling rate for G.722 audio is 16,000 Hz, the RTP clock rate for the G722 payload format is 8,000 Hz because that value was erroneously assigned in RFC 1890 and must remain unchanged for backward compatibility. The octet rate or sample-pair rate is 8,000 Hz.

We’d also like to mention that NetworkMiner Professional now comes with improved analytical support to help investigators detect Tor traffic.

Upgrading to Version 2.6

Users who have purchased a license for NetworkMiner Professional 2.x can download a free update to version 2.6 from our customer portal, or use the “Help > Check for Updates” feature. Those who instead prefer to use the free and open source version can grab the latest version of NetworkMiner from the official NetworkMiner page.

Posted by Erik Hjelmvik on Wednesday, 23 September 2020 09:10:00 (UTC/GMT)

Tags: #NetworkMiner​ #SMTP​ #POP3​ #IMAP​ #email​ #FTP​ #JtR​ #John​ #Mono​ #Linux​ #HTTP​ #HTTP/2​ #GRE​ #SIP​ #VoIP​ #Tor​ #PCAP​

NetworkMiner 2.4 Released

We are proud to announce the release of NetworkMiner 2.4 today! The new version comes with several improvements, such as username extraction from Kerberos traffic, better OS fingerprinting and even better Linux support.

Protocol Updates

The Kerberos v5 implementation in NetworkMiner 2.4 can be used to to extract usernames, hostnames and realms (domains) from unencrypted Kerberos requests/responses on port 88. NetworkMiner also parses and extracts usernames etc. from HTTP auth headers and SMB security blobs when they use Kerberos for authentication.

NetworkMiner also automatically attempts to parse traffic to TCP port 11371 as HTTP in order to extract GPG keys sent using the HKP protocol.

MAC Address Magic

We’ve added two new features related to MAC addresses to this release. One of them is the “MAC Age” field (showing “2000-11-09” in the previous screenshot), which is a guesstimate of how hold a device/host is based on its MAC address. This functionality uses HD Moore’s mac-ages database, which contains approximate dates for when hardware address ranges were allocated by IEEE (original concept from DeepMac).

The second MAC feature is a simple yet useful feature that adds links between hosts that share the same MAC address. This feature is useful for linking a host's IPv6 and IPv4 addresses with each other, but it can also be used to track if a physical host has changed its IP address. The MAC address links can be accessed by expanding the MAC address node in NetworkMiner’s Hosts tab.

ICS Asset Inventory

We’ve put in some ground work in order to create OS fingerprinting signatures for several Industrial Control System (ICS) devices. Our signatures have been submitted and merged into Eric Kollmann’s Satori TCP database, which NetworkMiner uses to passively fingerprint hosts by examining various TCP and IP fields in the initial SYN/SYN+ACK packets of TCP sessions. The ICS devices we’ve added include PLCs, RTUs as well as rugged network equipment from vendors like ABB, Allen-Bradley, Modicon, Moxa, Phoenix Contact and Siemens. Some ICS vendors even got an icon showing their logo in the Hosts tab (see the Siemens/RUGGEDCOM device in the screenshot below) while the others got a yellow hard hat.

EternalBlue

NetworkMiner isn’t designed to be used as an IDS. Nevertheless we decided to add detection for the EternalBlue exploit to NetworkMiner 2.4. The fact that NetworkMiner parses NetBIOS and SMB makes it pretty straightforward to identify when an attacker is attempting to allocate a large non-paged pool in srvnet.sys by using a vulnerability in Microsoft’s SMB implementation (see MS17-010 for reference). This type of detection is difficult to perform using a standard IDS solution that cannot parse the NetBIOS and SMB protocols. Detected EternalBlue exploit attempts are listed in NetworkMiner's “Anomalies” tab. Example PCAP files with attackers/malware using the EternalBlue exploit can be found here:

• https://www.malware-traffic-analysis.net/2018/10/04/index.html
• https://www.malware-traffic-analysis.net/2018/08/17/index.html
• https://packettotal.com/app/analysis?id=c8850b1fe07572c82c6fac5db5aae0c9
• https://packettotal.com/app/analysis?id=8d2fdb7698c27e32e88bde809c89216f

NetworkMiner in Linux

NetworkMiner is a Windows tool, but it actually runs just fine also in other operating systems with help of the Mono Framework (see our guide “HowTo install NetworkMiner in Ubuntu Fedora and Arch Linux”). However, there are a few pitfalls that must be avoided to get the software running smoothly using Mono. With this release we’ve implemented workarounds for two bugs in Mono’s GUI implementation (System.Windows.Forms).

The first workaround handles a Mono bug that sometimes could be triggered by Drag-and-Dropping a file or image from NetworkMiner to another application, such as a browser, text editor or image viewer. Doing so would previously trigger a NullReferenceException in System.Windows.Forms.X11Dnd+TextConverter.SetData under certain conditions. We’re happy to report that you can now reliably drag and drop files extracted by NetworkMiner to other tools, even when running Linux.

The second workaround handles a bug in Mono’s GDIPlus implementation related to rendering of Unicode characters. We were unfortunately not able to reliably get Mono to render Unicode characters, NetworkMiner will therefore convert all Unicode MIME data to ASCII when using Mono (typically in Linux). Windows users will still get the proper Unicode representations of exotic characters and emojis in NetworkMiner though. ☺

NetworkMiner Professional

The commercial version of NetworkMiner, i.e. NetworkMiner Professional, comes with a few additional improvements. One of them is is that the following additional online sources have been added to the OSINT lookup feature:

• Wayback Machine (URL lookup)
• Alexa Rank (DNS lookup)
• Kaspersky Application Advisor (file hash lookup)

The CSV export from NetworkMinerCLI has been updated to use the ISO 8601 format with explicit time zone for timestamps. An exported timestamp now look something like this:

2019-01-08T13:37:00.4711000+02:00

NetworkMiner Professional 2.4 also identifies application layer protocols regardless of port number (a.k.a. PIPI) with much better precision than earlier versions. It also extracts audio from VoIP calls (SIP) more reliably than before.

Credits

I would like to thank Chris Sistrunk for requesting GUI support to link IPv4 and IPv6 hosts with the same MAC address and Jonas Lejon for the HKP GPG key extraction idea. I would also like to thank Phil Hagen for notifying us about the issue with Unicode in emails when running NetworkMiner under Mono and Ahmad Nawawi for notifying us about the protocol identification shortages in the previous version.

Upgrading to Version 2.4

Users who have purchased a license for NetworkMiner Professional 2.x can download a free update to version 2.4 from our customer portal. Those who instead prefer to use the free and open source version can grab the latest version of NetworkMiner from the official NetworkMiner page.

⛏ FOR GREAT JUSTICE! ⛏

Posted by Erik Hjelmvik on Thursday, 10 January 2019 14:20:00 (UTC/GMT)

Tags: #NetworkMiner​ #ICS​ #Kerberos​ #SIP​ #VoIP​ #IPv6​ #Mono​ #Linux​ #Satori​ #OSINT​ #PIPI​

CapLoader 1.5 Released

We are today happy to announce the release of CapLoader 1.5. This new version of CapLoader parses pcap and pcap-ng files even faster than before and comes with new features, such as a built-in TCP stream reassembly engine, as well as support for Linux and macOS.

Support for ICMP Flows

CapLoader is designed to group packets together that belong to the same bi-directional flow, i.e. all UDP, TCP and SCTP packets with the same 5-tuple (regardless of direction) are considered being part of the same flow.

5-tuple

/fʌɪv ˈtjuːp(ə)l/

noun

A combination of source IP, destination IP, source port, destination port and transport protocol (TCP/UDP/SCTP) used to uniquely identify a flow or layer 4 session in computer networking.

The flow concept in CapLoader 1.5 has been extended to also include ICMP. Since there are no port numbers in the ICMP protocol CapLoader sets the source and destination port of ICMP flows to 0. The addition of ICMP in CapLoader also allows input filters and display filters like “icmp” to be leveraged.

TCP Stream Reassembly

One of the foundations for making CapLoader a super fast tool for reading and filtering PCAP files is that it doesn’t attempt to reassemble TCP streams. This means that CapLoader’s Transcript view will show out-of-order segments in the order they were received and retransmitted segments will be displayed twice.

The basic concept has been to let other tools do the TCP reassembly, for example by exporting a PCAP for a flow from CapLoader to Wireshark or NetworkMiner.

The steps required to reassemble a TCP stream to disk with Wireshark are:

1. Right-click a TCP packet in the TCP session of interest.
2. Select “Follow > TCP Stream”.
3. Choose direction in the first drop-down-list (client-to-server or server-to-client).
4. Change format from “ASCII” to “Raw” in the next drop-down-menu.
5. Press the “Save as...” button to save the reassembled TCP stream to disk.

Unfortunately Wireshark fails to properly reassemble some TCP streams. As an example the current stable release of Wireshark (version 2.2.5) shows duplicate data in “Follow TCP Stream” when there are retransmissions with partially overlapping segments. We have also noticed some additional  bugs related to TCP stream reassembly in other recent releases of Wireshark. However, we’d like to stress that Wireshark does perform a correct reassembly of most TCP streams; it is only in some specific situations that Wireshark produces a broken reassembly. Unfortunately a minor bug like this can cause serious consequences, for example when the TCP stream is analyzed as part of a digital forensics investigation or when the extracted data is being used as input for further processing. We have therefore decided to include a TCP stream reassembly engine in CapLoader 1.5. The steps required to reassemble a TCP stream in CapLoader are:

1. Double click a TCP flow of interest in the “Flows” tab to open a flow transcript.
2. Click the “Save Client Byte Stream” or “Save Server Byte Stream” button to save the data stream for the desired direction to disk.

Extracting TCP streams from PCAP files this way not only ensures that the data stream is correctly reassembled, it is also both faster and simpler than having to pivot through Wireshark’s Follow TCP Stream feature.

PCAP Icon Context Menu

The PCAP icon in CapLoader is designed to allow easy drag-and-drop operations in order to open a set of selected flows in an external packet analysis tool, such as Wireshark or NetworkMiner. Right-clicking this PCAP icon will bring up a context menu, which can be used to open a PCAP with the selected flows in an external tool or copy the PCAP to the clipboard. This context menu has been extended in CapLoader 1.5 to also include a “Save As” option. Previous versions of CapLoader required the user to drag-and-drop from the PCAP icon to a folder in order to save filtered PCAP data to disk.

Faster Parsing with Protocol Identification

CapLoader can identify over 100 different application layer protocols, including HTTP, SSL, SSH, RTP, RTCP and SOCKS, without relying on port numbers. The protocol identification has previously slowed down the analysis quite a bit, which has caused many users to disable this powerful feature. This new release of of CapLoader comes with an improved implementation of the port-independent protocol identification feature, which enables PCAP files to be loaded twice as fast as before with the “Identify protocols” feature enabled.

Works in Linux and macOS

One major improvement in CapLoader 1.5 is that this release is compatible with the Mono framework, which makes CapLoader platform independent. This means that you can now run CapLoader on your Mac or Linux machine if you have Mono installed. Please refer to our previous blog posts about how to run NetworkMiner in various flavors of Linux and macOS to find out how to install Mono on your computer. You will, however, notice a performance hit when running CapLoader under Mono instead of using Windows since the Mono framework isn't yet as fast as Microsoft's .NET Framework.

Credits

We’d like to thank Sooraj for reporting a bug in the “Open With” context menu of CapLoader’s PCAP icon. This bug has been fixed in CapLoader 1.5 and Sooraj has been awarded an official “PCAP or it didn’t happen” t-shirt for reporting the bug.

Have a look at our Bug Bounty Program if you also wanna get a PCAP t-shirt!

Downloading CapLoader 1.5

Everything mentioned in this blog post, except for the protocol identification feature, is available in our free trial version of CapLoader. To try it out, simply grab a copy here:
https://www.netresec.com/?page=CapLoader#trial (no registration needed)

All paying customers with an older version of CapLoader can download a free update to version 1.5 from our customer portal.

Posted by Erik Hjelmvik on Tuesday, 07 March 2017 09:11:00 (UTC/GMT)

Tags: #CapLoader​ #PCAP​ #pcap-ng​ #Follow TCP Stream​ #Wireshark​ #Linux​ #Mac​ #macOS​ #RTP​ #SOCKS​ #Mono​ #Flow​

Running NetworkMiner on Mac OS X

The following is a guest blog post written by Jonas Lejon from the Swedish IT security company Triop, which specialize in crypto, reverse engineering and penetration testing.

This guide describes how to get NetworkMiner running on Mac OS X Mavericks (version 10.9.3).

First of all, download NetworkMiner and then go to the Mono downloads page and get the latest version for Mac OS X

After the download of “Mono MRE installer” has completed, just run the installer:

Press “Continue” to proceed installing the Mono Framework using the guided installer.

When the Mono Framework has been installed you can extract the downloaded NetworkMiner zip archive. Then start NetworkMiner from the terminal like this:

$ mono --arch=32 NetworkMiner.exe

Live sniffing with NetworkMiner on Mac OS X

Live sniffing with WinPcap or Raw Sockets is only available when running NetworkMiner in Windows. However, live sniffing can still be achieved on Mac OSX (as well as in Linux) by using the PCAP-over-IP functionality. Simply select [File > Receive PCAP over IP] or press [Ctrl]+R and select a TCP port to listen on (TCP 57012 is the default port).

Press the “Start Receiving” button and then use tcpdump to do live sniffing and forward all captured packets to NetworkMiner like this:

$ sudo tcpdump -i en0 -s0 -U -w - | nc localhost 57012

The preferred way to use NetworkMiner is, however, to load previously captured packets in a PCAP file and let NetworkMiner dig out all interesting details like transmitted files, images, messages, SSL certificates etc.

For more info on how to run NetworkMiner on other operating systems, please see our previous blog posts HowTo install NetworkMiner in Ubuntu Fedora and Arch Linux and No more Wine - NetworkMiner in Linux with Mono.

UPDATE

Microsoft .NET Windows.Forms GUI applications don't run on 64 bit macOS systems running Mono. This will cause the application to hang/freeze during startup when the GUI window is about to be rendered, throwing errors such as:

• Unable to start NetworkMiner: An exception was thrown by the type initializer for System.Windows.Forms.WindowsFormsSynchronizationContext
• Unhandled Exception: System.TypeInitializationException: An exception was thrown by the type initializer for System.Windows.Forms.ThemeEngine

$ mono --arch=32 /opt/NetworkMiner/NetworkMiner.exe

We'd like to thank Fredrik Pettai for reporting this issue and Joel Langill for suggesting the workaround.

UPDATE 190627

You can also use homebrew to install mono on macOS like this:

brew update && brew install mono

Posted by Jonas Lejon on Tuesday, 24 June 2014 21:25:00 (UTC/GMT)

Tags: #Mac​ #macOS​ #NetworkMiner​ #Mono​ #tcpdump​ #PCAP-over-IP​