NETRESEC Network Security Blog - Tag : editcap


HowTo handle PcapNG files

Users of Wireshark 1.8.0 (or later) have most likely noticed that the default output file format has changed from libpcap (.pcap) to Pcap-NG (.pcapng). So what does this mean other than a longer file extension?

New Features in PcapNG

PcapNG is short for “PCAP Next Generation Dump File Format”. This new format isn't just an update of the old PCAP specification, but a total rewrite of the whole spec. Here are a few of the features that are available in the new PcapNG format:

  • Traffic captured from multiple interfaces can be stored in a single file. This even works when the interfaces have different data link types, such as Ethernet, 802.11 (WiFi) and PPP.
  • PcapNG trace files can be tagged with metadata info about what OS, hardware and sniffer application that was used to capture the traffic. Wireshark and dumpcap automatically tags generated PcapNG files this way.
  • There is a long list of metadata attributes that can be stored about each interface. This list includes attributes like interface name, dropped packets and used capture filter.
  • Text comments can be added and saved to individual frames. These comments, which are called “annotations”, are available in Wireshark and Tshark via the display filter named “pkt_comment”.

You can, for example, use tshark to list all annotations in a PcapNG file with the following command:

tshark -r dump.pcapng -T fields -e pkt_comment -R pkt_comment

You might also have seen improved timestamp resolution as a new feature of the PcapNG format. It is true that the PcapNG format allows for more precise timestamps compared to the microsecond resolution provided by the old PCAP format. This does, however, not mean that you will get nanosecond resolution in your capture files just because you switch to the PcapNG format. The default timestamp resolution in PcapNG files is still microseconds, and tools like Wireshark and dumpcap additionally only get microsecond resolution timestamps from the library they rely on for packet capturing (i.e. libpcap or WinPcap).

Compatibility Issues

Many tools are not yet able to load PcapNG files, instead they'll spit out error messages like “bad dump file format”. These error messages typically appear when a tool parses PCAP files with help of a libpcap version prior to 1.1.0 (you can find the error message in savefile.c from early 2010). If you instead load a PcapNG file into NetworkMiner you'll currently get the following error message:
NetworkMiner 1.4.1 with error message: Error opening pcap file: The stream is not a PCAP file. Magic number is A0D0D0A or A0D0D0A but should be A1B2C3D4.
NetworkMiner 1.4.1 with error message while trying to open a PcapNG file

Convert PcapNG to PCAP

Convert PcapNG to PCAP

If you end up with a capture file in the PcapNG format, which you need to parse with a tool that does not yet support the “NG” format, then you'll have to convert it to the legacy PCAP format. You can use editcap to do the conversion like this:

editcap -F libpcap dump.pcapng dump.pcap
However, if the PcapNG file contains traffic from multiple interfaces with different data link types then you'll get an error message saying:
editcap: Can't open or create dump.pcap: That file format doesn't support per-packet encapsulations

One way to convert such multi-encapsulation PcapNG files back to PCAP format is to first split the capture file into multiple PcapNG files, by using the display filters “frame.interface_id” or “frame.dlt”, and then convert each individual PcapNG file back to PCAP using editcap.

Another option is to submit your PcapNG file to the conversion service provided at pcapng.com. This free service only lets you convert file sizes up to 1MB, but it's really easy to use and you don't need to sign up or register to use it. Just choose a file and press “Convert to PCAP”, the site will then provide you with PCAP formatted capture files for each interface.

Sniff to PCAP

If you want to avoid conversion issues altogether then it's probably safest to stay with the good ol' PCAP format. But since dumpcap defaults to PcapNG nowadays you'll have to tell it to use the legacy PCAP format with the “-P” switch like this:

dumpcap -P -i eth0 -w dump.pcap

Posted by Erik Hjelmvik on Wednesday, 05 December 2012 12:15:00 (UTC/GMT)

Tags: #PcapNG #PCAP #convert #editcap #tshark #NetworkMiner #dumpcap

More... Share  |  Facebook   Twitter   Reddit   Hacker News Short URL: http://netres.ec/?b=12CB2A2


Convert Endace ERF capture files to PCAP

Chili Time

A customer recently contacted us because he wanted to load ERF capture files from their Endace probes into NetworkMiner Professional. In order to do so they would first need to convert the ERF file into the libpcap format. The obvious solution is to use editcap and specify the output capture type with “-F libpcap”. However, it turns out that by doing so the captured data in each frame will still be preceded by an ERF header. This causes a problem for tools that handle pcap files but do not have a parser for the ERF header format.

The Solution

In order to convert capture files from the ERF format into PCAP without any ERF headers you need to specify not only the capture type but also the encapsulation type of the data inside the ERF headers. One way to see what encapsulation type to use is to run the Protocol Hierarchy Statistics (PHS) function in tshark. Here is what the PHS look like on the publicly available capture file erf-ethernet-example.erf:

tshark -r erf-ethernet-example.erf -q -z io,phs

===================================================================
Protocol Hierarchy Statistics
Filter:

erfframes:19 bytes:7269
 ethframes:19 bytes:7269
  ipframes:19 bytes:7269
   tcpframes:19 bytes:7269
    httpframes:4 bytes:2077
     image-gifframes:2 bytes:655
      tcp.segmentsframes:2 bytes:655
===================================================================

The PHS output show that every ERF frame contains an Ethernet frame (eth). We can therefore specify the output filetype to be libpcap and encapsulation type to be and Ethernet like this:

editcap -F libpcap -T ether erf-ethernet-example.erf erf-ethernet-example.pcap

Let's check the PHS for the pcap file we've now generated:

tshark -q -z io,phs -r erf-ethernet-example.pcap

===================================================================
Protocol Hierarchy Statistics
Filter:

 ethframes:19 bytes:7269
  ipframes:19 bytes:7269
   tcpframes:19 bytes:7269
    httpframes:4 bytes:2077
     image-gifframes:2 bytes:655
      tcp.segmentsframes:2 bytes:655
===================================================================

Success! All ERF headers are now gone and the output pcap file contains plain old Ethernet frames. You can now open the pcap in NetworkMiner or whichever pcap parsing tool you wish.

NetworkMiner with erf-ethernet-example.pcap loaded

NetworkMiner with erf-ethernet-example.pcap loaded

UPDATE 2014-03-17

Another way to convert an ERF file to the PCAP or PcapNG format is to carve packets from the ERF file with CapLoader.

Posted by Erik Hjelmvik on Thursday, 22 November 2012 13:11:00 (UTC/GMT)

Tags: #PCAP #editcap #tshark #convert

More... Share  |  Facebook   Twitter   Reddit   Hacker News Short URL: http://netres.ec/?b=12B3A4A

twitter

NETRESEC on Twitter

Follow @netresec on twitter:
» twitter.com/netresec


book

Recommended Books

» The Practice of Network Security Monitoring, Richard Bejtlich (2013)

» Applied Network Security Monitoring, Chris Sanders and Jason Smith (2013)

» Network Forensics, Sherri Davidoff and Jonathan Ham (2012)

» The Tao of Network Security Monitoring, Richard Bejtlich (2004)

» Practical Packet Analysis, Chris Sanders (2017)

» Windows Forensic Analysis, Harlan Carvey (2009)

» TCP/IP Illustrated, Volume 1, Kevin Fall and Richard Stevens (2011)

» Industrial Network Security, Eric D. Knapp and Joel Langill (2014)