NETRESEC Network Security Blog - Tag : convert
Users of Wireshark 1.8.0 (or later) have most likely noticed that the default output file format has changed from libpcap (.pcap) to Pcap-NG (.pcapng). So what does this mean other than a longer file extension?
New Features in PcapNG
PcapNG is short for “PCAP Next Generation Dump File Format”. This new format isn't just an update of the old PCAP specification, but a total rewrite of the whole spec. Here are a few of the features that are available in the new PcapNG format:
- Traffic captured from multiple interfaces can be stored in a single file. This even works when the interfaces have different data link types, such as Ethernet, 802.11 (WiFi) and PPP.
- PcapNG trace files can be tagged with metadata info about what OS, hardware and sniffer application that was used to capture the traffic. Wireshark and dumpcap automatically tags generated PcapNG files this way.
- There is a long list of metadata attributes that can be stored about each interface. This list includes attributes like interface name, dropped packets and used capture filter.
- Text comments can be added and saved to individual frames. These comments, which are called “annotations”, are available in Wireshark and Tshark via the display filter named “pkt_comment”.
You can, for example, use tshark to list all annotations in a PcapNG file with the following command:
tshark -r dump.pcapng -T fields -e pkt_comment -R pkt_comment
You might also have seen improved timestamp resolution as a new feature of the PcapNG format. It is true that the PcapNG format allows for more precise timestamps compared to the microsecond resolution provided by the old PCAP format. This does, however, not mean that you will get nanosecond resolution in your capture files just because you switch to the PcapNG format. The default timestamp resolution in PcapNG files is still microseconds, and tools like Wireshark and dumpcap additionally only get microsecond resolution timestamps from the library they rely on for packet capturing (i.e. libpcap or WinPcap).
Many tools are not yet able to load PcapNG files, instead they'll spit out error messages like “bad dump file format”.
These error messages typically appear when a tool parses PCAP files with help of a libpcap version
prior to 1.1.0 (you can find the error message in
savefile.c from early 2010).
If you instead load a PcapNG file into NetworkMiner you'll currently get the following error message:
NetworkMiner 1.4.1 with error message while trying to open a PcapNG file
Convert PcapNG to PCAP
If you end up with a capture file in the PcapNG format, which you need to parse with a tool that does not yet support the “NG” format, then you'll have to convert it to the legacy PCAP format. You can use editcap to do the conversion like this:
editcap -F libpcap dump.pcapng dump.pcapHowever, if the PcapNG file contains traffic from multiple interfaces with different data link types then you'll get an error message saying:
editcap: Can't open or create dump.pcap: That file format doesn't support per-packet encapsulations
One way to convert such multi-encapsulation PcapNG files back to PCAP format is to first split the capture file into multiple PcapNG files, by using the display filters “frame.interface_id” or “frame.dlt”, and then convert each individual PcapNG file back to PCAP using editcap.
Another option is to submit your PcapNG file to the conversion service provided at pcapng.com. This free service only lets you convert file sizes up to 1MB, but it's really easy to use and you don't need to sign up or register to use it. Just choose a file and press “Convert to PCAP”, the site will then provide you with PCAP formatted capture files for each interface.
Sniff to PCAP
If you want to avoid conversion issues altogether then it's probably safest to stay with the good ol' PCAP format. But since dumpcap defaults to PcapNG nowadays you'll have to tell it to use the legacy PCAP format with the “-P” switch like this:
dumpcap -P -i eth0 -w dump.pcap
Posted by Erik Hjelmvik on Wednesday, 05 December 2012 12:15:00 (UTC/GMT)
A customer recently contacted us because he wanted to load ERF capture files from their Endace probes into NetworkMiner Professional. In order to do so they would first need to convert the ERF file into the libpcap format. The obvious solution is to use editcap and specify the output capture type with “-F libpcap”. However, it turns out that by doing so the captured data in each frame will still be preceded by an ERF header. This causes a problem for tools that handle pcap files but do not have a parser for the ERF header format.
In order to convert capture files from the ERF format into PCAP without any ERF headers you need to specify not only the capture type but also the encapsulation type of the data inside the ERF headers. One way to see what encapsulation type to use is to run the Protocol Hierarchy Statistics (PHS) function in tshark. Here is what the PHS look like on the publicly available capture file erf-ethernet-example.erf:
tshark -r erf-ethernet-example.erf -q -z io,phs
Protocol Hierarchy Statistics
erf frames:19 bytes:7269 eth frames:19 bytes:7269 ip frames:19 bytes:7269 tcp frames:19 bytes:7269 http frames:4 bytes:2077 image-gif frames:2 bytes:655 tcp.segments frames:2 bytes:655
The PHS output show that every ERF frame contains an Ethernet frame (eth). We can therefore specify the output filetype to be libpcap and encapsulation type to be and Ethernet like this:
editcap -F libpcap -T ether erf-ethernet-example.erf erf-ethernet-example.pcap
Let's check the PHS for the pcap file we've now generated:
tshark -q -z io,phs -r erf-ethernet-example.pcap
Protocol Hierarchy Statistics
eth frames:19 bytes:7269 ip frames:19 bytes:7269 tcp frames:19 bytes:7269 http frames:4 bytes:2077 image-gif frames:2 bytes:655 tcp.segments frames:2 bytes:655
Success! All ERF headers are now gone and the output pcap file contains plain old Ethernet frames. You can now open the pcap in NetworkMiner or whichever pcap parsing tool you wish.
NetworkMiner with erf-ethernet-example.pcap loaded
Another way to convert an ERF file to the PCAP or PcapNG format is to carve packets from the ERF file with CapLoader.
Posted by Erik Hjelmvik on Thursday, 22 November 2012 13:11:00 (UTC/GMT)