Showing blog posts from February 2013

Extracting Metadata from PcapNG files

Photo by Meighan O'Toole

In our blog post about the Chinese MITM of GitHub we revealed the identity of the anonymous capture file uploader by analyzing metadata available in the PCAP-NG file format. In this blog post we explain what type of meta-data that can be found in PcapNG files, and how to extract it.

The old libpcap format (a.k.a. PCAP) is designed to only store captured network frames. The new PCAP-NG format, however, additionally includes the ability to store meta-data in the capture files. In fact only about 20% of the PcapNG file specification concerns storage of captured frames, the remaining 80% is focused on various types of metadata.

So what types of metadata can a PcapNG file contain? Well, the spec allows features such as MAC address of capturing interface as well as interface speed (in bps) to be stored. But the most commonly used metadata types are:

  • Operating system of the sniffer machine
  • Software used to create the capture file (application name and version)
  • Name of interface from where packets are captured
  • Description of interface from where packets are captured
  • Capture filter used when sniffing packets
  • Cached name resolution entries (mappings from IPv4 or IPv6 addresses to host names)
  • User comments on individual frames (a.k.a. “annotations”),

Extracting Metadata

Some of these metadata types can be displayed in Wireshark by clicking “Statistics > Summary”. Here's the information provided when displaying a Summary in Wireshark on MachineB.pcapng from CloudShark:

Wireshark Summary of MachineB.pcapng

However, what is not shown in this summary view is that MachineB.pcapng additionally contains cached name resolution information for two machines. The easiest way to extract this extra metadata is to upload the capture file to, which will display a list of the metadata that was found in the uploaded file as well as convert the capture file to the old libpcap format (without metadata).

Here's the list of metadata information extracted from MachineB.pcapng by

Type Value
shb_os Windows Server 2003 Service Pack 2, build 3790
shb_userappl Dumpcap 1.8.4 (SVN Rev 46250 from /trunk-1.8)
if_name \Device\NPF_{D007FF28-3F24-4C1B-8EF5-069A6FB811ED}
if_tresol 0x06
if_os Windows Server 2003 Service Pack 2, build 3790
nres_ip6record fe80::9a03:d8ff:fedb:904b = iPhone-von-b1.local
nres_ip4record = iPhone-von-b1.local
nres_ip6record fe80::8685:6ff:fe23:c95d = iPhone-von-Gurkan.local
nres_ip4record = iPhone-von-Gurkan.local

The metadata apparently contains name resolution info for two iPhones, and we've got their internal IPv4 as well as IPv6 addresses. This would normally imply that the capture file contains frames to or from these IP addresses. However, as shown in the following screenshot from CloudShark, the capture file contains 21 frames but not a single one of them is from or to “iPhone-von-Gurkan” or “iPhone-von-b1”:

CloudShark's view of MachineB.pcapng

Thus, it seems as if the Wireshark pcap-ng host name disclosure bug wasn't fixed in Wireshark 1.8.4 after all.

Megecap Leaks File Paths

It is, however, not only Wireshark that might leak private information in PcapNG metadata. The command line tool Mergecap additionally stores the complete path and filename of the individual capture files that have been combined with the tool.

Type Value
opt_comment File created by merging: File1: /home/erik/captures/customer_X/capture_tap0_p5p1.cap File2: /home/erik/captures/customer_X/capture_tap1_p1p1.cap
shb_userappl mergecap
if_tresol 0x06

Metadata available in PcapNG file created with mergecap

This is most likely not a bug, but rather information leakage by design, so don't expect this to be mitigated in a future Mergecap release.


As we pointed out in our blog post “HowTo handle PcapNG files”, the best way to avoid information leakage via PcapNG files is to stay with the good ol' PCAP format by using the “-P” switch like this:

dumpcap -P -i eth0 -w dump.pcap

Another alternative is to convert your PcapNG files to PCAP format before sharing them.

More... Share  |  Facebook   Twitter   Reddit   Hacker News Short URL:

Posted by Erik Hjelmvik on Thursday, 14 February 2013 19:25:00 (UTC/GMT)

Forensics of Chinese MITM on GitHub

Great Wall of China by beggs

On January 26 several users in China reported  SSL  problems while connecting to the software development site The reports indicated that the Great Firewall of China (GFW) was used to perform a Man-in-the-Middle (MITM) attack against users in China who were visiting GitHub. This attack was most likely conducted in order to track which users that were reading or contributing to the list of persons behind the GFW, which is hosted on GitHub.

There is a good writetup on the attack by, which says:

”At around 8pm, on January 26, reports appeared on Weibo and Twitter that users in China trying to access were getting warning messages about invalid SSL certificates. The evidence, listed further down in this post, indicates that this was caused by a man-in-the-middle attack.
It was very crude, in that the fake certificate was signed by an unknown authority and bound to be detected quickly. The attack stopped after about an hour.”

SSL error at by bitinn
Screenshot of SSL error at by @bitinn

An important part of evidence, which was used  in  the  coverage of this incident, was a packet capture file named github.pcapng that was anonymously uploaded to CloudShark.

A self-signed X.509 certificate file, named, was also used as evidence. It is, however, easy to extract X.509 certificates from PCAP files, so the file might come from the github.pcapng capture file at CloudShark.

We have no reason not to trust the reports coming from users in China, who have been victims of this MITM attack. However, in the spirit of “Trust, but verify” I decided to perform some network forensic analysis on the capture file. More specifically I set out to answer the following questions:

Q1 : Is the user a victim of a real attack rather than having staged and recorded a MITM attack set up by himself?
Q2 : Is the user located in China?
Q3 : What can we say about the technology being used for the MITM?

Time-to-Live Analysis

My first step was to load the PcapNG file from CloudShark in CapLoader:

File > Open URL > Enter:

CapLoader with github.pcapng loaded

The “Hosts” tab shows that the capture file only contained traffic between two network hosts; the user's machine and the GitHub server. I also noticed that the GitHub server ( had an IP TTL of 58 and TCP Window Size of 5840. This values are a typical indication of the host being a Linux machine (see our passive OS fingerprinting blog post for more details). However, when connecting to from here (Sweden) the same server has an IP TTL of 128 and Window Size 64240. This looks more like a Windows machine, and is significantly different from what the Chinese user got.

It can also be noted that a TTL of 58, which the Chinese user got, would mean that the SSL connection was terminated just six router hops away from the user. However, a traceroute from (in Beijing, China) to requires a whopping 16 hops. Additionally, this traceroute hasn't even left Beijing after six hops.

traceroute to (, 30 hops max, 38 byte packets
1 gw ( 0.231 ms 0.175 ms 0.178 ms
2 ( 0.719 ms 0.626 ms 0.652 ms
3 ( 0.509 ms 0.485 ms 0.459 ms
4 8.131 ( 0.516 ms 0.488 ms 0.496 ms
5 8.198 ( 11.981 ms 0.913 ms 0.829 ms
6 8.192 ( 40.477 ms 40.614 ms 40.524 ms
7 ( 53.062 ms 40.610 ms 40.617 ms
8 ( 42.459 ms 43.133 ms 43.765 ms
9 ( 76.603 ms 76.586 ms 76.648 ms
10 ( 195.939 ms 195.934 ms 196.090 ms
11 ( 180.966 ms 181.058 ms 181.339 ms
12 ( 342.431 ms 340.882 ms 338.639 ms
13 ( 253.975 ms 253.407 ms 253.742 ms
14 ( 253.468 ms 252.947 ms 253.639 ms
15 ( 253.767 ms 253.862 ms 253.010 ms
16 ( 254.104 ms 253.789 ms 253.638 ms

MAC Address

NetworkMiner with PCAP file from GitHub MITM loaded

As shown in the NetworkMiner screenshot above, the Ethernet MAC address of the client's default gateway is ec:17:2f:15:23:b0, which indicates that the user's router was a device from TP-LINK. TP-LINK is a large provider of networking devices who make more than half of their sales within China. This supports the theory that the user was located in China.

PcapNG Metadata

As we've mentioned in a previous blog post, PcapNG files can contain a great deal of metadata. I therefore wrote a simple parser to extract all the juicy metadata that was available in the github.pcapng file that was anonymously uploaded to CloudShark. The metadata revealed that the user was running “64-bit Windows 7 Service Pack 1, build 7601” and sniffing with “Dumpcap 1.8.2 (SVN Rev 44520 from /trunk-1.8)”, which normally means that Wireshark 1.8.2 was being used (since it uses dumpcap for all packet capturing tasks).

The PcapNG file additionally contained a great deal of Name Resolution Blocks, i.e. cached DNS entries. Among these entries was an entry that mapped the IP to the hostname “SHAOJU-IPAD.local”. So why was there a name resolution entry for an IP address that wasn't part of the capture file? Well, cached name resolution entries aren't properly cleaned from PcapNG files written with Wireshark 1.8.0 to 1.8.3 due to a vulnerability discovered by Laura Chappell.

So, who is “Shaoju”? Well, a Google query for shaoju github will bring you to the GitHub user Chen Shaoju, who also tweeted a screenshot of the SSL error he received when accessing

SSL error received by Chen Shaoju when accessing


Q1 : Is the user a victim of a real attack rather than having staged and recorded a MITM attack set up by himself?
A1 : Yes, this is most likely a REAL attack. I'd find it unlikely that a user trying to fake such a MITM attack would use an environment with six router hops between the client and MITM machine.
Q2 : Is the user located in China?
A2 : Yes, I'm convinced that the PcapNG file was sniffed by Chen Shaoju, who lives in Wuxi, China according to his GitHub profile.
Q3 : What can we say about the technology being used for the MITM?
A3 : The fact that the MITM machine was six hops away from the user indicates that the MITM is taking place at some fairly central position in China's internet infrastructure, as opposed to being done locally at the ISP. Additionally, the machine doing the MITM seems to be running Linux and having an initial IP TTL of 64 and a TCP Window Size of 5840.

UPDATE (February 5) : Privacy / Anonymity

The fact that this blog post reveals the identity of the anonymous github.pcapng uploader seems to have caused some  reactions online.

I did, however, contact Chen Shaoju before publishing this blog post. In fact, I even sent him an email as soon as we discovered that it was possible to identify him through the capture file's metadata. Here's what I wrote:

“I'm working on a blog post about the GitHub pcap file on CloudShark. My analysis indicates that you sniffed the traffic. Is it OK with you if I publish this blog post?

If not, then I'd recommend that you remove the pcap file from couldshark in order to prevent others from identifying you via your pcap file.”
Shaoju responded the same day and said it was OK.

Once the text for the blog post was finished I also sent him a copy for verification before we published it online. Again, Shaoju responded swiftly and said that it looked good.

Also, I'm sure that Shaoju (@chenshaoju) can confirm that this blog post was published with his consent.

Finally, I'd like to add a few recommendations to users who wish to preserve their anonymity when sharing capture files:

  1. Filter out any traffic that isn't relevant to what you wish to share. CapLoader is a handy tool for selecting and filtering capture files based on flows or IP addresses.
  2. Save the capture file in the old libpcap (PCAP) format or convert your PcapNG file to PCAP format. This will remove the metadata available in Pcap-NG option fields.
  3. Anonymize frame headers, i.e Ethernet MAC addresses or even IP addresses (if needed), with a tool like: tcprewrite, Bit-Twist or TraceWrangler.

More... Share  |  Facebook   Twitter   Reddit   Hacker News Short URL:

Posted by Erik Hjelmvik on Saturday, 02 February 2013 22:10:00 (UTC/GMT)


NETRESEC on Twitter

Follow @netresec on twitter:


Recommended Books

» The Practice of Network Security Monitoring, Richard Bejtlich (2013)

» Applied Network Security Monitoring, Chris Sanders and Jason Smith (2013)

» Network Forensics, Sherri Davidoff and Jonathan Ham (2012)

» The Tao of Network Security Monitoring, Richard Bejtlich (2004)

» Practical Packet Analysis, Chris Sanders (2011)

» Windows Forensic Analysis, Harlan Carvey (2009)

» TCP/IP Illustrated, Volume 1, Kevin Fall and Richard Stevens (2011)

» Industrial Network Security, Eric D. Knapp and Joel Langill (2014)