I've released version 1.6 of a tool called SplitCap today. SplitCap is a really fast PCAP file splitter, which can be used to split large pcap files based on for example IP addresses or sessions. You can also use SplitCap in order to filter out traffic to/from specific IP addresses or port numbers from a large pcap file.
The best thing about SplitCap is that it is FAST, much faster than the alternatives.
Split frames in a PCAP file into one file per IP
The -s switch can be used with arguments flow, host, hostpair or nosplit and tells SplitCap how the frames should be separated into the output pcap files.
SplitCap.exe -r huge.pcap -s hostWhen running the command above SplitCap creates a directory called "huge.pcap" and fills it with lots of pcap files -- one pcap file per identified IP-address (i.e. host). The generated files are named according to the IP address they contain traffic for. Here is how the files from huge.pcap are named:
Split frames based on session
What I believe is a unique feature of SplitCap is the ability to split a pcap file based on session, i.e. the frames from each TCP or UDP session are placed in a separate pcap file. The session split mode is the default one, but you can also use the -s session argument to tell SplitCap to use the session split mode.
SplitCap.exe -r huge.pcap -s sessionThe generated files get names describing the 5-tuple (proto, src_ip, src_port, dst_ip, dst_port) for which they contain traffic. Here is an example showing how the files can be named:
huge.pcap.TCP_128-183-104-74_80_192-168-21-249_32788.pcapNote that the session parameter tells SplitCap to join frames going both to and from the server into the same pcap file. If you instead wanna separate the two directions into separate pcap files, then you can use the -s flow switch.
Filter PCAP file on IP address
You can filter a pcap file based on address with the -ip switch like this:
SplitCap.exe -r huge.pcap -ip 220.127.116.11 -s nosplitThe -s nosplit argument is used to tell SplitCap not to split the pcap into one file per session. The generated file "huge.pcap.NoSplit.pcap" will only contain frames going to or from the IP address 18.104.22.168. You can also specify multiple IP addresses if you are interested in traffic to/from more than one IP address:
SplitCap.exe -r huge.pcap -ip 22.214.171.124 -ip 126.96.36.199 -s nosplit
Filter PCAP file on port number
What if you are only interested in DNS (UDP+TCP 53) and HTTP (TCP 80) traffic in a PCAP file? Well, then you can specify these port numbers as arguments to SplitCap like this:
SplitCap.exe -r huge.pcap -port 53 -port 80 -s nosplitThis command creates a file named "huge.pcap.NoSplit.pcap" that only contains the frames going to or from ports 53 and 80.
Extract application layer (L7) contents
A pretty cool thing with SplitCap is the ability to extract application (i.e. layer 7) data from a pcap file with the -y L7 argument. I'll use the file "dump.eth0.1059726000.pcap" from Defcon 11 here to demonstrate how to extract layer 7 payload from SMTP (TCP 25) traffic to files (one per session).
SplitCap.exe -r dump.eth0.1059726000.pcap -s session -port 25 -y L7This command creates 10 files with a ".bin" file extention in the output directory. Each such bin file contains the application layer data for both directions (server->client and vice versa) from an SMTP session.
One of the generated SMTP session files is called "dump.eth0.1059726000.pcap.TCP_64-48-248-30_25_192-168-17-79_33443.bin" and looks something like this (I've redacted some of the contents):
220 smtp102.mail.sc5.yahoo.com ESMTP
[SMTP LOGIN REDACTED]
MAIL FROM: <[REDACTED]@nytimes.com>
RCPT TO: <[REDACTED]@[REDACTED].senate.gov>
354 go ahead
From: "[REDACTED]" <[REDACTED]@nytimes.com>
To: "[REDACTED]" <[REDACTED]@[REDACTED].senate.gov>
Subject: RE: Just checking in
Date: Fri, 1 Aug 2003 16:28:25 -0400
X-Priority: 3 (Normal)
X-Mailer: Microsoft Outlook IMO, Build 9.0.2416 (9.0.2911.0)
X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2800.1106
This is a multi-part message in MIME format.
i am going to gulfport mississippi for the story. am in las vegas now. will
be in mississippi on sunday.
the guy who is really good on this and helped me actually was [REDACTED]
from uspirg. he put me in contact with a whistle blower.
From: [REDACTED] [mailto:[REDACTED]@[REDACTED].senate.gov]
Sent: Friday, August 01, 2003 4:17 PM
Subject: Just checking in
I wanted to make sure you are getting everything you need from us and to
check in on the story. Let me know if/when you want to sit down with
[REDACTED] on this. Hope all is well, [REDACTED]
SplitCap is FAST, really FAST!
So how fast is SplitCap at splitting or filtering a pcap file? Let's do a simple benchmark and compare it to the well known tool Tshark. Again, lets use the 189 MB file "dump.eth0.1059726000.pcap" from Defcon 11 for this example.
Filtering the file based on IP with Tshark takes 50 seconds (4 MB/s).
tshark.exe -r dump.eth0.1059726000.pcap -R "ip.addr eq 188.8.131.52" -w defcon11_tshark_filtered.pcapPerforming an equivalent filtering with SplitCap takes 3 seconds (63 MB/s). That is 16 times faster than running Tshark!
SplitCap.exe -r dump.eth0.1059726000.pcap -ip 184.108.40.206 -s nosplit
You can read more about other command line tools from Netresec in the following posts:
SplitCap can be downloaded from here: http://www.netresec.com/?page=SplitCap
Posted by Erik Hjelmvik on Sunday, 08 May 2011 17:41:00 (UTC/GMT)