NETRESEC Network Security Blog - Tag : DFRWS

Command-line Network Forensics with NetworkMinerCLI

NetworkMinerCLI is a Command Line Interface version of NetworkMiner Professional. Apart from being faster than the GUI version NetworkMinerCLI also has the benefit of being easy to integrate into scripts of various kinds (Batch / perl / python / PowerShell / etc).

Running NetworkMinerCLI.exe from your NetworkMiner Professional USB flash drive shows the syntax and arguments to use:

Usage: F:\NetworkMinerProfessional_1-0\NetworkMinerCLI.exe [OPTIONS]

 -r <input_file>        : Set the pcap file to read
 -w <output_directory>  : Directory to store output files in
 -b <frame_buffer_size> : Number of frames to buffer in memory (5000 = default)
 -noHeader              : Disables column headers for CSV files

Example: F:\NetworkMinerProfessional_1-0\NetworkMinerCLI.exe -r evidence.pcap -w D:\exported_data\

When a pcap is loaded by NetworkMinerCLI it will generate eight CSV files containing data about Sessions, Parameters, Credentials, CleartextWords, DnsRecords, FileInfos, Messages and Hosts. All assembled files will also be written to disk. The assembled files, as well as the seven CSV files, are written to the current working directory or to the directory specified with the -w argument.

This is what it looks like when loading suspect.pcap from the DFRWS 2008 challenge into NetworkMinerCLI (don't let my Swedish cmd.exe shell confuse you):

F:\pcap_files\DFRWS\Challenge 2008>F:\NetworkMinerProfessional_1-0\NetworkMinerCLI.exe -r suspect.pcap
Closing file handles...
10243 frames parsed in 56,084 seconds.

F:\pcap_files\DFRWS\Challenge 2008>dir
 Volymen i enhet F har etiketten NETRESEC
 Volymens serienummer är 7017-B488

 Innehåll i katalogen F:\pcap_files\DFRWS\Challenge 2008

2011-02-23 21:19 <KAT>        .
2011-02-23 21:19 <KAT>        ..
2007-12-16 23:32    5 110 493 suspect.pcap
2011-03-04 17:20 <KAT>        AssembledFiles
2011-03-04 17:21       15 098 suspect.pcap.Sessions.csv
2011-03-04 17:21      459 868 suspect.pcap.Parameters.csv
2011-03-04 17:21       83 034 suspect.pcap.Credentials.csv
2011-03-04 17:21    3 271 213 suspect.pcap.CleartextWords.csv
2011-03-04 17:21       23 748 suspect.pcap.DnsRecords.csv
2011-03-04 17:21      182 382 suspect.pcap.FileInfos.csv
2011-03-04 17:21          837 suspect.pcap.Messages.csv
2011-03-04 17:21        8 009 suspect.pcap.Hosts.csv
             9 fil(er)       9 154 682 byte
             3 katalog(er) 711 819 264 byte ledigt

F:\pcap_files\DFRWS\Challenge 2008>

I've uploaded the generated suspect.pcap.Parameters.csv file so that you can have a look at what the CSV files from NetworkMiner look like. Corey Harrell has by the way recently written a great blog post on how to do forensic work with CSV files in OpenOffice Calc on his Journey into Incident Response Methodology blog.

The CSV files can also be parsed directly from command line with help of some script-fu. I admit being a Windows nerd, but I have still not gone through the pain of learning Windows PowerShell properly. I therefore prefer to use Cygwin, or actually xterm in Cygwin/X, in order to assemble the power of the GNU bash shell while running Windows. I can for example use the following command in order to get a list of all detected gmail accounts and the logged in clients' IPs:

cat suspect.pcap.Parameters.csv | grep gmailchat | awk -F',' '{print $1" "$9}' | sort -u

Here is a short explanation of the command above for people without UNIX fingers:

  • grep gmailchat : selects all lines in the csv file containing the string “gmailchat”
  • awk -F',' '{print $1" "$9}' : tokenize text between commas and display first token (IP address) and ninth token (gmail account)
  • sort -u : sort each outputed line and only display unique lines (remove duplicates)

The same method can also be used in order to quickly filter out Google searches from a pcap file (remember that the search strings are sent to Google with the 'q' parameter):

cat suspect.pcap.Parameters.csv | grep 'HTTP QueryString,q,'|awk -F',' '{print $1" "$9}' overseas credit card payments hurricane

Or to get all the frame numbers (in the pcap file) where the word “bank” has been sent in clear text:

grep bank suspect.pcap.CleartextWords.csv -i | awk -F',' '{print$1" "$2}' | sort -u
Bank 1551
Bank 1553
Bank 1559
Bank 1566
Bank 1568
Bank 1617
Bank 1619
Bank 1629
Bank 1708
Bank 1710
Bank 1716
Bank 1724
bank 2063
banks 5797

It should also be noted that NetworkMinerCLI extracts and writes all identified files to disk, just as the normal GUI version of NetworkMiner Professional does. A good overview of these extracted files can be found in the suspect.pcap.FileInfos.csv CSV file. So if you are looking for a particular file you can search the suspect.pcap.FileInfos.csv for the MD5 sum like this:

grep e516cd1dbad131024693d31155a6577f suspect.pcap.FileInfos.csv | awk -F',' '{print $9" "$5}'
e516cd1dbad131024693d31155a6577f XRamp Security Services G.cer
e516cd1dbad131024693d31155a6577f XRamp Security Services G[1].cer
e516cd1dbad131024693d31155a6577f XRamp Security Services G[2].cer

In order to get your hands on a copy of the NetworkMinerCLI tool you need to buy a license for NetworkMiner Professional, since the USB flash drive contains the GUI as well as the CLI version of NetworkMiner Professional.

Posted by Erik Hjelmvik on Friday, 04 March 2011 17:12:00 (UTC/GMT)

Tags: #NetworkMinerCLI #DFRWS

More... Share  |  Facebook   Twitter   Reddit   Hacker News Short URL:

DFRWS 2009 Network Forensics

I noticed that Juan Leaniz recently posted his analysis of the DFRWS challenge from 2009 on the SANS Computer Forensic Investigations and Incident Response Blog.

I actually submitted a contest entry to this challenge back in 2009, titled "Nothing but Network Forensics". The idea behind my entry was to see just how much information that could be extracted from the pcap files included in the challenge without even looking at the physical memory dump or filesystem images that also were provided as part of the challenge.

I will here provide some highlights from my analysis of the 2009 DFRWS challenge pcap files:

There were two persons involved in the case, I call them "NSSAL" and "JHUISI". I managed to extract the avatar images that were downloaded by these persons when they logged into their online PlayStation accounts.

NSSAL Avatar JHUISU Avatar

The actual URL these avatar images were extracted from was Yes, that's right, HTTP over TCP port 10060. Extracting these images was much easier to do with NetworkMiner Professional, since it would automatically detect this TCP session as running HTTP with help of the built in port-independent protocol identification feature.

NetworkMiner Professional Screenshot with Images tab

NetworkMiner Professional Screenshot with Files tab

I also used NetworkMiner to see what Google queries that NSSAL performed. By opening the Parameters tab, sorting contents on "Parameter name" column and scrolling down to parameter name "q" i found that NSSAL had searched for:

  • mardi gras
  • mardi gras pictures
  • mardi gras pictures k00l
Search queries sent to Google and many other search engines use parameter name "q" to denote the query. A google search for "netresec" would for example have an URL such as

The Hosts tab in NetworkMiner also provides more detailed info about the machines involved in this case. The computer used by NSSAL was actually a PS3 console (with MAC 001FA7B21ADE) that was running Sony's own CellOS with IP address The OS fingerprinting feature of NetworkMiner does not contain any OS class for the PS3 CellOS, the console therefore gets fingerprinted as FreeBSD. I suppose FreeBSD is quite similar to CellOS since the CellOS is in fact believed to be a branch from the FreeBSD/Darwin development tree.

NetworkMiner Professional with Hosts tab from DFRWS 2009

NSSAL's PS3 was later rebooted into Ubuntu Linux with IP (notice that the MAC address in the screenshot above remained unchanged). I could tell that NSSAL's PS3 was running Ubuntu Linux since the OS fingerprinting functionality provided by Satori and p0f both show that the TCP/IP stack behaves as Linux. The SSH banner-grabbing functionality included in NetworkMiner also says that NSSAL's machine was running "OpenSSH_5.1p1 Debian-3ubuntu1".

I should probably also mention the backdoor I found being used by JHUISI to get into NSSAL's machine. The initial commands sent in the remote shell session from when this backdoor was used looks as follows:


Backdoor by darkXside

Enter the second password.

Password accepted!
[backdoor]# ls
[backdoor]# ls
[backdoor]# rm backd00r

This backdoor sure looks very much like a modified version of darkXside's backd00r.c to me.

By the way, I will try to provide a similar analysis of the DFRWS challenge from 2008 on this blog sometime in the near future. The 2008 challenge was in fact much more interesting to investigate, from a network forensics perspective, than the 2009 challenge that we have looked at in this blog post.

Posted by Erik Hjelmvik on Wednesday, 26 January 2011 20:26:00 (UTC/GMT)

Tags: #DFRWS #Forensics

More... Share  |  Facebook   Twitter   Reddit   Hacker News Short URL:


NETRESEC on Twitter

Follow @netresec on twitter:


Recommended Books

» The Practice of Network Security Monitoring, Richard Bejtlich (2013)

» Applied Network Security Monitoring, Chris Sanders and Jason Smith (2013)

» Network Forensics, Sherri Davidoff and Jonathan Ham (2012)

» The Tao of Network Security Monitoring, Richard Bejtlich (2004)

» Practical Packet Analysis, Chris Sanders (2017)

» Windows Forensic Analysis, Harlan Carvey (2009)

» TCP/IP Illustrated, Volume 1, Kevin Fall and Richard Stevens (2011)

» Industrial Network Security, Eric D. Knapp and Joel Langill (2014)