NETRESEC Network Security Blog - Tag : Linux


NetworkMiner 2.4 Released

NetworkMiner 2.4

We are proud to announce the release of NetworkMiner 2.4 today! The new version comes with several improvements, such as username extraction from Kerberos traffic, better OS fingerprinting and even better Linux support.


Protocol Updates

The Kerberos v5 implementation in NetworkMiner 2.4 can be used to to extract usernames, hostnames and realms (domains) from unencrypted Kerberos requests/responses on port 88. NetworkMiner also parses and extracts usernames etc. from HTTP auth headers and SMB security blobs when they use Kerberos for authentication.

Kerberos username (Administrator) and realm (DENYDC.COM) in NetworkMiner's Host tab
Image: NetworkMiner showing extracted username (Administrator) and realm (DENYDC.COM) from the Wireshark sample capture file “Krb-contrained-delegation.cap”.

NetworkMiner also automatically attempts to parse traffic to TCP port 11371 as HTTP in order to extract GPG keys sent using the HKP protocol.


MAC Address Magic

We’ve added two new features related to MAC addresses to this release. One of them is the “MAC Age” field (showing “2000-11-09” in the previous screenshot), which is a guesstimate of how hold a device/host is based on its MAC address. This functionality uses HD Moore’s mac-ages database, which contains approximate dates for when hardware address ranges were allocated by IEEE (original concept from DeepMac).

The second MAC feature is a simple yet useful feature that adds links between hosts that share the same MAC address. This feature is useful for linking a host's IPv6 and IPv4 addresses with each other, but it can also be used to track if a physical host has changed its IP address. The MAC address links can be accessed by expanding the MAC address node in NetworkMiner’s Hosts tab.

IPv4 and IPv6 address with the same MAC address
Image: NetworkMiner with a PCAP file from ISTS 2012

ICS Asset Inventory

Hard Hat

We’ve put in some ground work in order to create OS fingerprinting signatures for several Industrial Control System (ICS) devices. Our signatures have been submitted and merged into Eric Kollmann’s Satori TCP database, which NetworkMiner uses to passively fingerprint hosts by examining various TCP and IP fields in the initial SYN/SYN+ACK packets of TCP sessions. The ICS devices we’ve added include PLCs, RTUs as well as rugged network equipment from vendors like ABB, Allen-Bradley, Modicon, Moxa, Phoenix Contact and Siemens. Some ICS vendors even got an icon showing their logo in the Hosts tab (see the Siemens/RUGGEDCOM device in the screenshot below) while the others got a yellow hard hat.

Asset inventory list with ICS devices
Image: Asset inventory list generated by NetworkMiner using PCAP files from the 4SICS 2015 ICS Lab.

EternalBlue

NetworkMiner isn’t designed to be used as an IDS. Nevertheless we decided to add detection for the EternalBlue exploit to NetworkMiner 2.4. The fact that NetworkMiner parses NetBIOS and SMB makes it pretty straightforward to identify when an attacker is attempting to allocate a large non-paged pool in srvnet.sys by using a vulnerability in Microsoft’s SMB implementation (see MS17-010 for reference). This type of detection is difficult to perform using a standard IDS solution that cannot parse the NetBIOS and SMB protocols. Detected EternalBlue exploit attempts are listed in NetworkMiner's “Anomalies” tab. Example PCAP files with attackers/malware using the EternalBlue exploit can be found here:


NetworkMiner in Linux

NetworkMiner Loves Linux

NetworkMiner is a Windows tool, but it actually runs just fine also in other operating systems with help of the Mono Framework (see our guide “HowTo install NetworkMiner in Ubuntu Fedora and Arch Linux”). However, there are a few pitfalls that must be avoided to get the software running smoothly using Mono. With this release we’ve implemented workarounds for two bugs in Mono’s GUI implementation (System.Windows.Forms).

The first workaround handles a Mono bug that sometimes could be triggered by Drag-and-Dropping a file or image from NetworkMiner to another application, such as a browser, text editor or image viewer. Doing so would previously trigger a NullReferenceException in System.Windows.Forms.X11Dnd+TextConverter.SetData under certain conditions. We’re happy to report that you can now reliably drag and drop files extracted by NetworkMiner to other tools, even when running Linux.

The second workaround handles a bug in Mono’s GDIPlus implementation related to rendering of Unicode characters. We were unfortunately not able to reliably get Mono to render Unicode characters, NetworkMiner will therefore convert all Unicode MIME data to ASCII when using Mono (typically in Linux). Windows users will still get the proper Unicode representations of exotic characters and emojis in NetworkMiner though. ☺


NetworkMiner Professional

The commercial version of NetworkMiner, i.e. NetworkMiner Professional, comes with a few additional improvements. One of them is is that the following additional online sources have been added to the OSINT lookup feature:

OSINT lookup of file hash in NetworkMiner Professional
Image: OSINT lookup menu for .exe file extracted from Malware-Traffic-Analysis.net’s 2018-10-16-trickbot.pcap.

The CSV export from NetworkMinerCLI has been updated to use the ISO 8601 format with explicit time zone for timestamps. An exported timestamp now look something like this:

2019-01-08T13:37:00.4711000+02:00

NetworkMiner Professional 2.4 also identifies application layer protocols regardless of port number (a.k.a. PIPI) with much better precision than earlier versions. It also extracts audio from VoIP calls (SIP) more reliably than before.


Credits

I would like to thank Chris Sistrunk for requesting GUI support to link IPv4 and IPv6 hosts with the same MAC address and Jonas Lejon for the HKP GPG key extraction idea. I would also like to thank Phil Hagen for notifying us about the issue with Unicode in emails when running NetworkMiner under Mono and Ahmad Nawawi for notifying us about the protocol identification shortages in the previous version.


Upgrading to Version 2.4

Users who have purchased a license for NetworkMiner Professional 2.x can download a free update to version 2.4 from our customer portal. Those who instead prefer to use the free and open source version can grab the latest version of NetworkMiner from the official NetworkMiner page.

⛏ FOR GREAT JUSTICE! ⛏

Posted by Erik Hjelmvik on Thursday, 10 January 2019 14:20:00 (UTC/GMT)

Tags: #NetworkMiner #ICS #SIP #VoIP #IPv6 #Mono #Linux #Satori #OSINT #PIPI

More... Share  |  Facebook   Twitter   Reddit   Hacker News Short URL: http://netres.ec/?b=191BD02


CapLoader 1.5 Released

CapLoader 1.5 Logo

We are today happy to announce the release of CapLoader 1.5. This new version of CapLoader parses pcap and pcap-ng files even faster than before and comes with new features, such as a built-in TCP stream reassembly engine, as well as support for Linux and macOS.

Support for ICMP Flows

CapLoader is designed to group packets together that belong to the same bi-directional flow, i.e. all UDP, TCP and SCTP packets with the same 5-tuple (regardless of direction) are considered being part of the same flow.

5-tuple
/fʌɪv ˈtjuːp(ə)l/
noun

A combination of source IP, destination IP, source port, destination port and transport protocol (TCP/UDP/SCTP) used to uniquely identify a flow or layer 4 session in computer networking.

The flow concept in CapLoader 1.5 has been extended to also include ICMP. Since there are no port numbers in the ICMP protocol CapLoader sets the source and destination port of ICMP flows to 0. The addition of ICMP in CapLoader also allows input filters and display filters like “icmp” to be leveraged.

Flows tab in CapLoader 1.5 with display filter BPF 'icmp'
Image: CapLoader 1.5 showing only ICMP flows due to display filter 'icmp'.

TCP Stream Reassembly

One of the foundations for making CapLoader a super fast tool for reading and filtering PCAP files is that it doesn’t attempt to reassemble TCP streams. This means that CapLoader’s Transcript view will show out-of-order segments in the order they were received and retransmitted segments will be displayed twice.

The basic concept has been to let other tools do the TCP reassembly, for example by exporting a PCAP for a flow from CapLoader to Wireshark or NetworkMiner.

The steps required to reassemble a TCP stream to disk with Wireshark are:

  1. Right-click a TCP packet in the TCP session of interest.
  2. Select “Follow > TCP Stream”.
  3. Choose direction in the first drop-down-list (client-to-server or server-to-client).
  4. Change format from “ASCII” to “Raw” in the next drop-down-menu.
  5. Press the “Save as...” button to save the reassembled TCP stream to disk.

Follow TCP Stream window in Wireshark

Unfortunately Wireshark fails to properly reassemble some TCP streams. As an example the current stable release of Wireshark (version 2.2.5) shows duplicate data in “Follow TCP Stream” when there are retransmissions with partially overlapping segments. We have also noticed some additional  bugs related to TCP stream reassembly in other recent releases of Wireshark. However, we’d like to stress that Wireshark does perform a correct reassembly of most TCP streams; it is only in some specific situations that Wireshark produces a broken reassembly. Unfortunately a minor bug like this can cause serious consequences, for example when the TCP stream is analyzed as part of a digital forensics investigation or when the extracted data is being used as input for further processing. We have therefore decided to include a TCP stream reassembly engine in CapLoader 1.5. The steps required to reassemble a TCP stream in CapLoader are:

  1. Double click a TCP flow of interest in the “Flows” tab to open a flow transcript.
  2. Click the “Save Client Byte Stream” or “Save Server Byte Stream” button to save the data stream for the desired direction to disk.

Transcript window in CapLoader 1-5

Extracting TCP streams from PCAP files this way not only ensures that the data stream is correctly reassembled, it is also both faster and simpler than having to pivot through Wireshark’s Follow TCP Stream feature.

PCAP Icon Context Menu

CapLoader 1.5 PCAP icon Save As...

The PCAP icon in CapLoader is designed to allow easy drag-and-drop operations in order to open a set of selected flows in an external packet analysis tool, such as Wireshark or NetworkMiner. Right-clicking this PCAP icon will bring up a context menu, which can be used to open a PCAP with the selected flows in an external tool or copy the PCAP to the clipboard. This context menu has been extended in CapLoader 1.5 to also include a “Save As” option. Previous versions of CapLoader required the user to drag-and-drop from the PCAP icon to a folder in order to save filtered PCAP data to disk.

Faster Parsing with Protocol Identification

CapLoader can identify over 100 different application layer protocols, including HTTP, SSL, SSH, RTP, RTCP and SOCKS, without relying on port numbers. The protocol identification has previously slowed down the analysis quite a bit, which has caused many users to disable this powerful feature. This new release of of CapLoader comes with an improved implementation of the port-independent protocol identification feature, which enables PCAP files to be loaded twice as fast as before with the “Identify protocols” feature enabled.

Works in Linux and macOS

One major improvement in CapLoader 1.5 is that this release is compatible with the Mono framework, which makes CapLoader platform independent. This means that you can now run CapLoader on your Mac or Linux machine if you have Mono installed. Please refer to our previous blog posts about how to run NetworkMiner in various flavors of Linux and macOS to find out how to install Mono on your computer. You will, however, notice a performance hit when running CapLoader under Mono instead of using Windows since the Mono framework isn't yet as fast as Microsoft's .NET Framework.

CapLoader 1.5 running in Linux with Mono
Image: CapLoader 1.5 running in Linux (Xubuntu).

Credits

We’d like to thank Sooraj for reporting a bug in the “Open With” context menu of CapLoader’s PCAP icon. This bug has been fixed in CapLoader 1.5 and Sooraj has been awarded an official “PCAP or it didn’t happen” t-shirt for reporting the bug.

PCAP or it didn't happen t-shirt
Image: PCAP or it didn't happen t-shirt

Have a look at our Bug Bounty Program if you also wanna get a PCAP t-shirt!

Downloading CapLoader 1.5

Everything mentioned in this blog post, except for the protocol identification feature, is available in our free trial version of CapLoader. To try it out, simply grab a copy here:
https://www.netresec.com/?page=CapLoader#trial (no registration needed)

All paying customers with an older version of CapLoader can download a free update to version 1.5 from our customer portal.

Posted by Erik Hjelmvik on Tuesday, 07 March 2017 09:11:00 (UTC/GMT)

Tags: #CapLoader #PCAP #pcap-ng #Follow TCP Stream #Wireshark #Linux #Mac #RTP #SOCKS #Mono #Flow

More... Share  |  Facebook   Twitter   Reddit   Hacker News Short URL: http://netres.ec/?b=1738EBA


HowTo install NetworkMiner in Ubuntu Fedora and Arch Linux

NetworkMiner Loves Linux NetworkMiner is a network forensics tool primarily developed for Windows OS's, but it actually runs just fine also in other operating systems with help of the Mono Framework. This guide shows how to install NetworkMiner in three different Linux distros (Ubuntu, Fedora and Arch Linux).

STEP 1: Install Mono

Ubuntu (also other Debian based distros like Xubuntu and Kali Linux)

sudo apt-get install libmono-system-windows-forms4.0-cil
sudo apt-get install libmono-system-web4.0-cil
sudo apt-get install libmono-system-net4.0-cil
sudo apt-get install libmono-system-runtime-serialization4.0-cil
sudo apt-get install libmono-system-xml-linq4.0-cil
If you're on an old version of Debian/Ubuntu (e.g. Ubuntu 14.04) then you first need to add the Mono Project GPG signing key and the package repository.

Fedora (credit Renegade0x6)

sudo yum -y install mono-core
sudo yum -y install mono-basic mono-winforms expect

ArchLinux (credit: Tyler Fisher)

sudo pacman -Sy mono

Other Operating Systems
Users of other Linux flavors as well as Mac OS X can download and install the Mono Framwork from www.mono-project.com/Downloads


STEP 2: Install NetworkMiner

wget www.netresec.com/?download=NetworkMiner -O /tmp/nm.zip
sudo unzip /tmp/nm.zip -d /opt/
cd /opt/NetworkMiner*
sudo chmod +x NetworkMiner.exe
sudo chmod -R go+w AssembledFiles/
sudo chmod -R go+w Captures/

STEP 3: Run NetworkMiner

mono NetworkMiner.exe

NetworkMiner in Ubuntu Linux
NetworkMiner 1.2 running under Ubuntu Linux, with “day12-1.dmp” from the M57-Patents Scenario loaded.

Live CD's

Another way to try out NetworkMiner in Linux is to spin up one of the Live CD's that has the tool installed, such as Security Onion, REMnux or NST.

Live sniffing with NetworkMiner

In order to capture packets (sniff traffic) in Linux you will have to use the “PCAP-over-IP” feature. NetworkMiner is, however, not really designed for packet capturing; it is primarily a tool for parsing and analyzing PCAP files containing previously sniffed traffic.

We recommend using other tools such as tcpdump, dumpcap or netsniff-ng in order to reliably capture packets to a PCAP file. You can read more on how to sniff traffic in our Sniffing Tutorial.

Posted by Erik Hjelmvik on Saturday, 01 February 2014 20:45:00 (UTC/GMT)

Tags: #NetworkMiner #Linux #mono #network forensics

More... Share  |  Facebook   Twitter   Reddit   Hacker News Short URL: http://netres.ec/?b=142AA47


Install NetworkMiner with apt-get

Doug Burks has done a great work integrating NetworkMiner into Security Onion. One really cool thing he has done is to build a Debian repository that includes NetworkMiner. This means that NetworkMiner (and its dependencies) can be installed on Debian / Ubuntu machines by using apt-get!

How to install NetworkMiner with a one-liner:

sudo add-apt-repository -y ppa:securityonion/stable && sudo apt-get update && sudo apt-get -y install securityonion-networkminer

You can then start NetworkMiner with the following syntax:

/opt/networkminer/networkminer [optional-pcap-file]

NetworkMiner running on a vanilla Xubuntu machine

NetworkMiner running on a vanilla Xubuntu machine

Users of non-Debian OS's (i.e. other Linux flavors as well as Mac OS X and FreeBSD) will, however, still need to install NetworkMiner and Mono manually.

UPDATE 2014-06-25

NetworkMiner is now available in a few additional Linux packet handlers, such as Fedora's yum and Arch Linux's pacman. For more details, please see our HowTo install NetworkMiner in Ubuntu Fedora and Arch Linux blog post.
There are also instructions available for how to install NetworkMiner on Mac OS X.

Posted by Erik Hjelmvik on Monday, 10 September 2012 13:15:00 (UTC/GMT)

Tags: #NetworkMiner #Linux #Debian #Ubuntu #Security Onion #SecurityOnion

More... Share  |  Facebook   Twitter   Reddit   Hacker News Short URL: http://netres.ec/?b=12951F9


Security Onion includes NetworkMiner

Doug Burks released a new ISO of his popular IDS/NSM Linux distro "Security Onion" a couple of days ago. This new ISO includes an installation of NetworkMiner straight out of the box. You can retrieve the Live install CD of Security Onion here:
http://sourceforge.net/projects/security-onion/files/security-onion-live-20120125.iso/download

You can also read more about how to install Security Onion here: http://code.google.com/p/security-onion/wiki/Installation

Starting NetworkMiner in Security Onion NetworkMiner can be started from: Applications > Security Onion > NetworkMiner

Doug has also, as part of the work of including NetworkMiner into Security Onion, built a debian package for NetworkMiner. This means that you can install NetworkMiner on a Debian Linux machine (including Ubuntu) with these commands:

apt-get install libmono-winforms2.0-cil
wget http://sourceforge.net/projects/security-onion/files/20111228/securityonion-networkminer_20111228_i386.deb
dpkg -i securityonion-networkminer_20111228_i386.deb 
mono /opt/networkminer/NetworkMiner.exe

We do, however, still recommend Linux users to install NetworkMiner “manually” with the commands supplied in our blog post “No more Wine - NetworkMiner in Linux with Mono

Finally, Security Onion is one of the great tools you can vote for in Russ McRee's 2011 Toolsmith Tool of the Year survey. Other great tools you can vote for are PacketFence and Volatility. Make sure you submit your vote before the survey ends on Tuesday (January 31):
http://holisticinfosec.blogspot.com/2011/12/choose-2011-toolsmith-tool-of-year.html

Posted by Erik Hjelmvik on Friday, 27 January 2012 20:45:00 (UTC/GMT)

Tags: #Linux #NetworkMiner #Debian

More... Share  |  Facebook   Twitter   Reddit   Hacker News Short URL: http://netres.ec/?b=12160DB


No more Wine - NetworkMiner in Linux with Mono

UPDATE
See our blog post HowTo install NetworkMiner in Ubuntu Fedora and Arch Linux for a more up to date installation guide.

British Street, E3 sticker close-up by George Rex NetworkMiner is a network forensics tool that is primarily designed to run under Windows. But it is now (as of version 1.2 of NetworkMiner) also possible to run NetworkMiner on non-Windows OS's like Linux, Mac, FreeBSD etc. with help of Mono. This means that there is no longer any need to run NetworkMiner under Wine, since Mono is a much better alternative.

So what is Mono? Here is a description from the mono-project's website:

“Mono is a software platform designed to allow developers to easily create cross platform applications. Sponsored by Xamarin, Mono is an open source implementation of Microsoft's .NET Framework based on the ECMA standards for C# and the Common Language Runtime. A growing family of solutions and an active and enthusiastic contributing community is helping position Mono to become the leading choice for development of Linux applications.”
(emphasis added)

Here are some of the features in NetworkMiner that work better under Mono compared to Wine:

  • Drag-and-drop pcap files onto NetworkMiner works under Mono
  • Extracted/reassembled files are put in OS-native folders (under the NetworkMiner/AssembledFiles folder)
  • Right-clicking an extracted file or image and selecting “Open file” or “Open containing folder” works under Mono
  • No big Wine install required, the Mono framework only requires 32 MB to install

Here are the commands required to install Mono and NetworkMiner on Ubuntu Linux:

sudo apt-get install libmono-winforms2.0-cil
wget www.netresec.com/?download=NetworkMiner -O /tmp/networkminer.zip
sudo unzip /tmp/networkminer.zip -d /opt/
cd /opt/NetworkMiner_*
sudo chmod +x NetworkMiner.exe
sudo chmod -R go+w AssembledFiles/
sudo chmod -R go+w Captures/
mono NetworkMiner.exe
The reason for setting write permission to the AssembledFiles folder is because this is the directory to where extracted files are written. If you prefer to instead have the files extracted to /tmp or the user's home directory, then simply move the AssembledFiles directory to your desired location and create a symlink to it in the NetworkMiner directory (hat tip to Lenny Zeltser for this idea).


NetworkMiner 1.2 running under Ubuntu Linux with Mono, with “day12-1.dmp” from the M57-Patents Scenario loaded.

Update: See our blog post HowTo install NetworkMiner in Ubuntu Fedora and Arch Linux for an installation guide for other linux flavors.


Posted by Erik Hjelmvik on Monday, 26 December 2011 20:30:00 (UTC/GMT)

Tags: #NetworkMiner #Mono #Wine #Linux #Ubuntu

More... Share  |  Facebook   Twitter   Reddit   Hacker News Short URL: http://netres.ec/?b=11C135E


REMnux now includes NetworkMiner

REMnux logo

Lenny Zeltser recently released version 3 of his Reverse-Engineering Malware Linux distro REMnux.

Here are a few of the improvements in REMnux 3 compared to the previous version:

  • The REMnux distro is now based on Ubuntu
  • Updated versions of Volatility and Origami
  • NetworkMiner is included for forensic analysis of network traffic

As of version 1.2 of NetworkMiner it is possible to use mono to run it on non-Windows OS's like Linux, Mac and FreeBSD. Lenny used this functionality in order to run NetworkMiner under mono instead of using Wine, which I think is a great decision since NetworkMiner integrates much better with the OS when it is run with mono.

NetworkMiner running on REMnux

NetworkMiner running on REMnux

There is, however, one caveat to be aware of when running NetworkMiner under REMnux; you either have to run it as root (as in the screenshot above) or add write permissions to the AssembledFiles directory with:

sudo chmod -R go+w /usr/local/NetworkMiner/AssembledFiles

NetworkMiner will otherwise not be able to extract any files from the analyzed pcap files to disk since it won't have right to write them to the AssembledFiles folder.

Luckily, Lenny has already confirmed to me that he will have this fixed in the next release of REMnux.

Posted by Erik Hjelmvik on Friday, 16 December 2011 21:46:00 (UTC/GMT)

Tags: #Linux #NetworkMiner #mono

More... Share  |  Facebook   Twitter   Reddit   Hacker News Short URL: http://netres.ec/?b=11CCD67


Running NetworkMiner on Linux with Wine

UPDATE : We no longer recommend running NetworkMiner under Wine, please see our blog post on HowTo install NetworkMiner in Ubuntu Fedora and Arch Linux instead.

Joshua Smith has written a great blog post on toastresearch.com about how to get NetworkMiner running on BackTrack Linux. C. S. Lee (a.k.a. geek00l) has also written a blog post a couple of years ago explaining how to install NetworkMiner on Ubuntu Linux.

Unfortunately both these blog posts point to URLs with old versions of NetworkMiner (now that version 1.1 is released). I'm therefore posting a simple walkthrough of the required commands in order to install the latest version of NetworkMiner on an Ubuntu machine:

sudo apt-get install winetricks
winetricks corefonts dotnet20 gdiplus
cd /opt
wget www.netresec.com/?download=NetworkMiner
unzip latest
cd NetworkMiner_1-1/
wine NetworkMiner.exe
NetworkMiner in Linux with Wine

I hope this will help you get NetworkMiner running on your Ubuntu analyst station!

We will also be looking into having NetworkMiner fully compatible with mono in a future release. This would allow you to run NetworkMiner “natively” on Linux, Mac OS X as well as BSD (OpenBSD, FreeBSD, NetBSD).

Posted by Erik Hjelmvik on Thursday, 13 October 2011 16:51:00 (UTC/GMT)

Tags: #Netresec #Linux #Wine #Ubuntu

More... Share  |  Facebook   Twitter   Reddit   Hacker News Short URL: http://netres.ec/?b=11A3324

twitter

NETRESEC on Twitter

Follow @netresec on twitter:
» twitter.com/netresec


book

Recommended Books

» The Practice of Network Security Monitoring, Richard Bejtlich (2013)

» Applied Network Security Monitoring, Chris Sanders and Jason Smith (2013)

» Network Forensics, Sherri Davidoff and Jonathan Ham (2012)

» The Tao of Network Security Monitoring, Richard Bejtlich (2004)

» Practical Packet Analysis, Chris Sanders (2017)

» Windows Forensic Analysis, Harlan Carvey (2009)

» TCP/IP Illustrated, Volume 1, Kevin Fall and Richard Stevens (2011)

» Industrial Network Security, Eric D. Knapp and Joel Langill (2014)