NetworkMiner 2.6 Released

We are happy to announce the release of NetworkMiner 2.6 today! The network forensic tool is now even better at extracting emails, password hashes, FTP transfers and artifacts from HTTP and HTTP/2 traffic than before.

Some of the major improvements in this new release are related to extraction and presentation of emails from SMTP, POP3 and IMAP traffic. On that note, we’d like to thank Mandy van Oosterhout for reporting a bug in our email parser!

I have previously blogged about how to extract John-the-Ripper hashes from Kerberos network traffic with NetworkMiner. We have now added support for presenting LANMAN and NTLM credentials as JtR hashes as well.

We have also improved NetworkMiner’s Linux support. Files, images and folders can now be opened in external tools directly from the NetworkMiner GUI also when running NetworkMiner in Linux using Mono 6 (or later). Linux users previously got a “System.ComponentModel.Win32Exception” error message saying something like “Cannot find the specified file” or “Access denied” due to a breaking change introduced in Mono version 6.

The new release also comes with several updates of how HTTP and HTTP/2 traffic is handled and presented. We have, for example, added better extraction of data sent in HTTP (or HTTP/2) POST requests. Posted JSON formatted parameters are also extracted even if the JSON data has been gzip compressed. The “Accept-Language” header values in HTTP and HTTP/2 are extracted as “Host Details” in order to support forensic analysis of user language settings, as shown by Fox-IT in their “Operation Wocao - Shining a light on one of China’s hidden hacking groups” report.

NetworkMiner has supported decapsulation of tunneling protocols and protocols for network virtualization, like 802.1Q, GRE, PPPoE, VXLAN, OpenFlow, MPLS and EoMPLS, since version 2.1. We have now improved our GRE parser to also support NVGRE (RFC 7637) by adding support for Transparent Ethernet Bridging.

Jan Hesse sent us a feature request on Twitter earlier this year, where asked about support for FritzBox captures. We are happy to announce that NetworkMiner now supports the modified pcap format you get when sniffing network traffic with a FritzBox gateway.

NetworkMiner 2.6 can now also parse and extract SIP chat messages (RFC 3428) to the “Messages” tab. Audio extraction of VoIP calls is still a feature that is exclusively available only in NetworkMiner Professional though.

NetworkMiner Professional

Our commercial tool NetworkMiner Professional has received a few additional updates, such as support for analysis of HTTP/2 traffic in the “Browsers tab”. However, please note that NetworkMiner does not perform TLS decryption, so the HTTP/2 traffic will have to be decrypted by a TLS proxy like PolarProxy prior to being saved to a PCAP file.

We have added a few new great online services to NetworkMiner Pro’s OSINT lookup as well, such as shouldiclick.org, Browserling, MalwareDomainList and VirusTotal lookups of URL’s in the “Browsers” tab. We have also added some additional external OSINT sources for lookups of IP addresses and domain names, such as MalwareDomainList and mnemonic ACT. The JA3 hash lookup menu in NetworkMiner Professional’s “Hosts” tab has also been extended to include GreyNoise.

NetworkMiner Pro previously played back G.722 VoIP audio at half speed. This issue has now been fixed, so that G.722 RTP audio is extracted and played back in 16k samples/s. The bug was due to an error in RFC 1890 that was later corrected in RFC 3551. Thanks to Michael "MiKa" Kafka for teaching us about this!

Excerpt from RFC 3551:

Even though the actual sampling rate for G.722 audio is 16,000 Hz, the RTP clock rate for the G722 payload format is 8,000 Hz because that value was erroneously assigned in RFC 1890 and must remain unchanged for backward compatibility. The octet rate or sample-pair rate is 8,000 Hz.

We’d also like to mention that NetworkMiner Professional now comes with improved analytical support to help investigators detect Tor traffic.

Upgrading to Version 2.6

Users who have purchased a license for NetworkMiner Professional 2.x can download a free update to version 2.6 from our customer portal, or use the “Help > Check for Updates” feature. Those who instead prefer to use the free and open source version can grab the latest version of NetworkMiner from the official NetworkMiner page.

Posted by Erik Hjelmvik on Wednesday, 23 September 2020 09:10:00 (UTC/GMT)

Tags: #NetworkMiner​ #SMTP​ #POP3​ #IMAP​ #email​ #FTP​ #JtR​ #John​ #Mono​ #Linux​ #HTTP​ #HTTP/2​ #GRE​ #SIP​ #VoIP​ #Tor​ #PCAP​

NetworkMiner 2.4 Released

We are proud to announce the release of NetworkMiner 2.4 today! The new version comes with several improvements, such as username extraction from Kerberos traffic, better OS fingerprinting and even better Linux support.

Protocol Updates

The Kerberos v5 implementation in NetworkMiner 2.4 can be used to to extract usernames, hostnames and realms (domains) from unencrypted Kerberos requests/responses on port 88. NetworkMiner also parses and extracts usernames etc. from HTTP auth headers and SMB security blobs when they use Kerberos for authentication.

NetworkMiner also automatically attempts to parse traffic to TCP port 11371 as HTTP in order to extract GPG keys sent using the HKP protocol.

MAC Address Magic

We’ve added two new features related to MAC addresses to this release. One of them is the “MAC Age” field (showing “2000-11-09” in the previous screenshot), which is a guesstimate of how hold a device/host is based on its MAC address. This functionality uses HD Moore’s mac-ages database, which contains approximate dates for when hardware address ranges were allocated by IEEE (original concept from DeepMac).

The second MAC feature is a simple yet useful feature that adds links between hosts that share the same MAC address. This feature is useful for linking a host's IPv6 and IPv4 addresses with each other, but it can also be used to track if a physical host has changed its IP address. The MAC address links can be accessed by expanding the MAC address node in NetworkMiner’s Hosts tab.

ICS Asset Inventory

We’ve put in some ground work in order to create OS fingerprinting signatures for several Industrial Control System (ICS) devices. Our signatures have been submitted and merged into Eric Kollmann’s Satori TCP database, which NetworkMiner uses to passively fingerprint hosts by examining various TCP and IP fields in the initial SYN/SYN+ACK packets of TCP sessions. The ICS devices we’ve added include PLCs, RTUs as well as rugged network equipment from vendors like ABB, Allen-Bradley, Modicon, Moxa, Phoenix Contact and Siemens. Some ICS vendors even got an icon showing their logo in the Hosts tab (see the Siemens/RUGGEDCOM device in the screenshot below) while the others got a yellow hard hat.

EternalBlue

NetworkMiner isn’t designed to be used as an IDS. Nevertheless we decided to add detection for the EternalBlue exploit to NetworkMiner 2.4. The fact that NetworkMiner parses NetBIOS and SMB makes it pretty straightforward to identify when an attacker is attempting to allocate a large non-paged pool in srvnet.sys by using a vulnerability in Microsoft’s SMB implementation (see MS17-010 for reference). This type of detection is difficult to perform using a standard IDS solution that cannot parse the NetBIOS and SMB protocols. Detected EternalBlue exploit attempts are listed in NetworkMiner's “Anomalies” tab. Example PCAP files with attackers/malware using the EternalBlue exploit can be found here:

• https://www.malware-traffic-analysis.net/2018/10/04/index.html
• https://www.malware-traffic-analysis.net/2018/08/17/index.html
• https://packettotal.com/app/analysis?id=c8850b1fe07572c82c6fac5db5aae0c9
• https://packettotal.com/app/analysis?id=8d2fdb7698c27e32e88bde809c89216f

NetworkMiner in Linux

NetworkMiner is a Windows tool, but it actually runs just fine also in other operating systems with help of the Mono Framework (see our guide “HowTo install NetworkMiner in Ubuntu Fedora and Arch Linux”). However, there are a few pitfalls that must be avoided to get the software running smoothly using Mono. With this release we’ve implemented workarounds for two bugs in Mono’s GUI implementation (System.Windows.Forms).

The first workaround handles a Mono bug that sometimes could be triggered by Drag-and-Dropping a file or image from NetworkMiner to another application, such as a browser, text editor or image viewer. Doing so would previously trigger a NullReferenceException in System.Windows.Forms.X11Dnd+TextConverter.SetData under certain conditions. We’re happy to report that you can now reliably drag and drop files extracted by NetworkMiner to other tools, even when running Linux.

The second workaround handles a bug in Mono’s GDIPlus implementation related to rendering of Unicode characters. We were unfortunately not able to reliably get Mono to render Unicode characters, NetworkMiner will therefore convert all Unicode MIME data to ASCII when using Mono (typically in Linux). Windows users will still get the proper Unicode representations of exotic characters and emojis in NetworkMiner though. ☺

NetworkMiner Professional

The commercial version of NetworkMiner, i.e. NetworkMiner Professional, comes with a few additional improvements. One of them is is that the following additional online sources have been added to the OSINT lookup feature:

• Wayback Machine (URL lookup)
• Alexa Rank (DNS lookup)
• Kaspersky Application Advisor (file hash lookup)

The CSV export from NetworkMinerCLI has been updated to use the ISO 8601 format with explicit time zone for timestamps. An exported timestamp now look something like this:

2019-01-08T13:37:00.4711000+02:00

NetworkMiner Professional 2.4 also identifies application layer protocols regardless of port number (a.k.a. PIPI) with much better precision than earlier versions. It also extracts audio from VoIP calls (SIP) more reliably than before.

Credits

I would like to thank Chris Sistrunk for requesting GUI support to link IPv4 and IPv6 hosts with the same MAC address and Jonas Lejon for the HKP GPG key extraction idea. I would also like to thank Phil Hagen for notifying us about the issue with Unicode in emails when running NetworkMiner under Mono and Ahmad Nawawi for notifying us about the protocol identification shortages in the previous version.

Upgrading to Version 2.4

Users who have purchased a license for NetworkMiner Professional 2.x can download a free update to version 2.4 from our customer portal. Those who instead prefer to use the free and open source version can grab the latest version of NetworkMiner from the official NetworkMiner page.

⛏ FOR GREAT JUSTICE! ⛏

Posted by Erik Hjelmvik on Thursday, 10 January 2019 14:20:00 (UTC/GMT)

Tags: #NetworkMiner​ #ICS​ #Kerberos​ #SIP​ #VoIP​ #IPv6​ #Mono​ #Linux​ #Satori​ #OSINT​ #PIPI​

NetworkMiner 2.3.2 Released!

NetworkMiner 2.3.2 was released this morning, and there was much rejoicing!

This new release primarily fixes bugs related to extraction of emails and VoIP calls. We have also corrected a bug affecting the json/CASE export function in NetworkMiner Professional.

The OSINT domain name lookup in NetworkMiner Professional has also been extended with the crt.sh Certificate Search and DNSTrails has been replaced with SecurityTrails.

Credits

We’d like to thank Carlos Kasprzykowski for notifying us about the VoIP bug, which caused lots of files to be written to the %TEMP% directory when there were more than 50 simultaneous SIP+RTP calls. We also wanna thank Josh Wilczek for reporting a bug in the “User Defined Port-to-Protocol Mappings” in NetworkMiner Professional’s Settings window, which also has been fixed in the 2.3.2 release.

Upgrading to Version 2.3.2

Users who have purchased a license for NetworkMiner Professional 2.x can download a free update to version 2.3.2 from our customer portal. Those who instead prefer to use the free and open source version can grab the latest version of NetworkMiner from the official NetworkMiner page.

⛏ FOR GREAT JUSTICE! ⛏

Posted by Erik Hjelmvik on Monday, 27 August 2018 09:23:00 (UTC/GMT)

Tags: #Netresec​ #NetworkMiner​ #RTP​ #VoIP​ #SIP​ #email​ #CASE​

NetworkMiner 2.3 Released!

The free and open source network forensics tool NetworkMiner now comes with improved extraction of files and metadata from several protocols as well as a few GUI updates. But the biggest improvements for version 2.3 are in the commercial tool NetworkMiner Professional, which now supports VoIP call audio extraction and playback as well as OSINT lookups of file hashes, IP addresses, domain names and URLs.

I’m happy to announce that NetworkMiner 2.3 now does an even better job than before at extracting files and metadata from several protocols. Improvements have been made in the parsers for the following protocols: HTTP, IEC-104, IPv4, Modbus, SIP, SMB, SMB2, SMTP and SSL/TLS.

We have also added support for the SNMP protocol in NetworkMiner 2.3, so that SNMP community strings can be extracted and displayed on the Parameters and Credentials tabs.

Another change is that timestamps are now displayed using the UTC time zone instead of using the local time zone. We have also fixed a few GUI quirks in order to further improve the usability of the tool.

NetworkMiner Professional

The commercial version of NetworkMiner, i.e. NetworkMiner Professional, comes with several additional improvements which are presented below.

VoIP Call Playback

NetworkMiner Professional has received a new tab called “VoIP”, which enables audio playback of VoIP calls that are using SIP and RTP with G.711 μ-law or A-law encoding (u-Law is primarily used in North America and Japan while A-law is used in Europe and most other parts of the world).

Video: Audio playback and extraction to WAV from the “SIP_CALL_RTP_G711” PCAP file in the Wireshark Sample Captures.

The audio streams from the VoIP calls are also extracted to disk as .WAV files when codecs G.729 or G.711 (u-Law and A-Law) is used. NetworkMiner Professional also attempts to reassemble RTP streams encoded with G.722 to .au files.

OSINT Lookups of IP Addresses, Domains, URLs and File Hashes

Right-clicking a network host in NetworkMiner Professional’s Hosts tab brings up a context menu with options for performing lookups of IP and domain names using external sources. We refer to this method as open-source intelligence (OSINT) because the accessed data resides at publicly available sources.

Clicking on an OSINT provider brings up a webpage with more detailed information about the selected IP address, such as IBM X-Force, mnemonic Passive DNS, Shodan, UrlQuery or VT. However, if you’re lazy like me, then you’ll probably click the “All above!” option instead, which will bring up all of the sources in separate tabs in your browser.

The full list of OSINT providers available for IP lookups includes APNIC Whois, BFK Passive DNS, Censys, Cymon, DNSTrails, ExoneraTor, Google Public DNS, GreenSnow.co, Hurricane Electric, IBM X-Force, Internet Storm Center, mnemonic Passive DNS, PacketTotal, Shodan, ThreatCrowd, ThreatMiner, UrlQuery and VirusTotal.

The domain name lookup menu contains a similar set of providers: BFK Passive DNS, Cymon, DNSTrails, Google Public DNS, Google Safe Browsing, Hybrid Analysis, IBM X-Force Exchange, mnemonic Passive DNS, MXToolBox, MyWOT, Norton Safe Web, PacketTotal, ThreatCrowd, ThreatMiner, URL Void, UrlQuery, VirusTotal, Website Informer, Webutation and Whoisology.

Right-clicking a URL in the Browsers tab brings up a similar context menu, which additionally includes the following services for URL lookups: Google Safe Browsing, IBM X-Force, ThreatMiner, URLhaus and UrlQuery.

Finally, right-clicking on one of the files that NetworkMiner has extracted from a PCAP file brings up a menu for doing OSINT lookups based on the MD5 or SHA256 hash of the file. The sources used for lookups of hashes include IBM X-Force, PacketTotal, ThreatCrowd, TotalHash, UrlQuery, VirScan.org, Comodo Valkyrie, AlienVault OTX, Hybrid Analysis, ThreatMiner and VirusTotal.

Hybrid Analysis API Integration

Did you know that the malware analysis service Hybrid Analysis provides free API keys to people in the IT security community?

This is a great move by the Hybrid Analysis team, and we’re happy to announce that we have leveraged their API in NetworkMiner Professional in order to submit files for analysis directly from within the NetworkMiner GUI. The API integration also enables you to query for an IP on Hybrid Analysis to see which previously submitted samples has communicated with that particular IP address.

Here are the steps required to enable the Hybrid Analysis API integration:

• Create a HybridAnaysis account
• Create an API key
• Upgrade the API key by completing a vetting process
• Start NetworkMiner Pro, open the Tools > Settings menu and input your API key

Credits

I would like to thank Chris Sistrunk, Mats Karlsson and Michael Nilsson for suggesting several of the protocol and GUI improvements that have been incorporated into this new release. I’d also like to thank Doug Green and Ahmad Nawawi for discovering and reporting bugs in the IP and SSL parser respectively.

Upgrading to Version 2.3

Users who have purchased a license for NetworkMiner Professional 2.x can download a free update to version 2.3 from our customer portal.

Those who instead prefer to use the free and open source version can grab the latest version of NetworkMiner from the official NetworkMiner page.

⛏ FOR GREAT JUSTICE! ⛏

Posted by Erik Hjelmvik on Tuesday, 03 April 2018 06:27:00 (UTC/GMT)

Tags: #NetworkMiner​ #PCAP​ #OSINT​ #SMTP​ #SIP​ #RTP​ #VoIP​ #Network Forensics​ #extract​ #Netresec​

Port Independent Protocol Detection

Our heavy-duty PCAP analyzer CapLoader comes with a feature called ”Port Independent Protocol Identification”, a.k.a. PIPI (see Richard Bejtlich's PIPI blog post from 2006). Academic research in the Traffic Measurement field often use the term ”Traffic Classification”, which is similar but not the same thing. Traffic Classification normally group network traffic in broad classes, such as Email, Web, Chat or VoIP. CapLoader, on the other hand, identifies the actual application layer protocol used in each flow. So instead of classifying a flow as ”VoIP” CapLoader will tell you if the flow carries SIP, Skype, RTP or MGCP traffic. This approach is also known as “Dynamic Protocol Detection”.

Being able to identify application layer protocols without relying on the TCP or UDP port number is crucial when analyzing malicious traffic, such as malware Command-and-Control (C2) communication, covert backdoors and rouge servers, since such communication often use services on non-standard ports. Some common examples are:

• Many botnet C2 protocols communicate over port TCP 443, but using a proprietary protocol rather than HTTP over SSL.
• Backdoors on hacked computers and network devices typically wither run a standard service like SSH on a port other than 22 in order to hide.
• More advanced backdoors use port knocking to run a proprietary C2 protocol on a standard port (SYNful knock runs on TCP 80).

This means that by analyzing network traffic for port-protocol anomalies, like an outgoing TCP connection to TCP 443 that isn't SSL, you can effectively detect intrusions without having IDS signatures for all C2 protocols. This analysis technique is often used when performing Rinse-Repeat Intrusion Detection, which is a blacklist-free approach for identifying intrusions and other form of malicious network traffic. With CapLoader one can simply apply a BPF filter like “port 443” and scroll through the displayed flows to make sure they are all say “SSL” in the Protocol column.

Statistical Analysis

CapLoader relies on statistical analysis of each TCP, UDP and SCTP session's behavior in order to compare it to previously computed statistical models for known protocols. These statistical models are generated using a multitude of metrics, such as inter-packet delays, packet sizes and payload data. The port number is, on the other hand, a parameter that is intentionally not used by CapLoader to determine the application layer protocol.

The PIPI/Dynamic Protocol Detection feature in CapLoader has been designed to detect even encrypted and obfuscated binary protocols, such as Tor and Encrypted BitTorrent (MSE). These protocols are designed in order to deceive protocol detection mechanisms, and traditional signature based protocol detection algorithms can't reliably detect them. The statistical approach employed by CapLoader can, on the other hand, actually detect even these highly obfuscated protocols. It is, however, important to note that being a statistical method it will never be 100% accurate. Analysts should therefore not take for granted that a flow is using the protocol stated by CapLoader. There are some situations when it is very difficult to accurately classify an encrypted protocol, such as when the first part of a TCP session is missing in the analyzed data. This can occur when there is an ongoing session that was established before the packet capture was started.

Identified Protocols

The following protocols are currently available for detection in CapLoader's protocol database:

AOL Instant Messenger

BACnet

BitTorrent

BitTorrent Encrypted - MSE

CCCam

CUPS

DAYTIME

DHCP

DHCPv6

Diameter

DirectConnect

DNS

Dockster

DropBox LSP

eDonkey

eDonkey Obfuscated

EtherNet-IP

FTP

Gh0st RAT

Gnutella

Groove LAN DPP

HSRP

HTTP

IMAP

IRC

ISAKMP

iSCSI

JavaRMI

Kelihos

Kerberos

L2TP

LDAP

LLC

Meterpreter

MgCam

MGCP

MikroTik NDP

Modbus TCP

MSN Messenger

MS RPC

MS-SQL

MySQL

NAT-PMP

NetBIOS Datagram Service

NetBIOS Name Service

NetBIOS Session Service

NetFlow

NTP

OsCam

Pcap-over-IP

Poison Ivy RAT

POP3

QUIC

Ramnit

Reverse Shell

RTCP

RTP

RTSP

Shell

SIP

Skype

SLP

SMTP

SNMP

Socks

SopCast P2P

Spotify P2P

Spotify Server

SSH

SSL

Syslog

TeamViewer

TeamViewer UDP

Telnet

Teredo

TFTP

TFTP Data

TPKT

VNC

WS-Discovery

XMPP Jabber

ZeroAccess

Zeus TCP

Zeus UDP

The list of implemented protocols is constantly being increased with new protocols.

PIPI in NetworkMiner

NetworkMiner Professional, which is the commercial version of NetworkMiner, also comes with an implementation of our protocol detection mechanism. Even though NetworkMiner Professional doesn't detect as many protocols as CapLoader, the PIPI feature built into NetworkMiner Pro still helps a lot when analyzing HTTP traffic on ports other that 80 or 8080 as well as in order to reassemble files downloaded from FTP or TFTP servers running on non-standard ports.

Posted by Erik Hjelmvik on Tuesday, 06 October 2015 09:05:00 (UTC/GMT)

Tags: #Protocol Identification​ #CapLoader​ #VoIP​ #SIP​ #RTP​ #TOR​ #SSL​ #PIPI​ #PCAP​ #NetworkMiner​