NETRESEC Network Security Blog - Tag : DNS


Robust Indicators of Compromise for SUNBURST

Were you targeted by SUNBURST? Image credit: NASA

There has been a great deal of confusion regarding what network based Indicators of Compromise (IOC) SolarWinds Orion customers can use to self assess whether or not they have been targeted after having installed a software update with the SUNBURST backdoor. Many of the published IOCs only indicate that a backdoored SolarWinds Orion update has been installed, but the question that many security teams are trying to answer is whether or not the installed backdoor has been been used by the threat actor.

Dont trust everything you read!

There is a widespread misunderstanding that receiving a so-called “NetBios” DNS A record (for example an address in 8.18.144.0/23) in response to a *.avsvmcloud.com DNS query would mean that you’ve been targeted. Our analysis of the decompiled SUNBURST code and passive DNS data show that that receiving a “NetBios” response does not necessarily mean that the client has been targeted. Unfortunately this misunderstanding has lead to various sensationalist stories being published with long lists of companies and organizations that are claimed to be “singled out by the hacking group for the second stage of the attack” or “victims where attackers escalated access”.

Another common misunderstanding is that clients sending *.avsvmcloud.com DNS queries with encoded timestamps, and optionally a list of installed/running AV products, have been actively targeted. Our analysis of the decompiled SUNBURST code show that the timestamped “Pings” or AV service status reports get exfiltrated in DNS traffic after the client’s internal AD domain has been sent, but before the perpetrators decide whether or not they want to activate the backdoor. Additionally, our analysis of publicly available passive DNS traffic indicate that there are almost as many unique clients transmitting timestamps and AV products in avsvmcloud.com DNS queries (409) as there are clients leaking their internal AD domains (513).

Indicators of a Targeted Attack

So what network based IOC’s can incident responders, blue teams and SOC analysts use in order to see if they have been targeted by the SUNBURST operators?

The following network based events indicate that a client has been actively targeted and the SUNBURST backdoor has progressed beyond the initial mode of operation:

  • Received a DNS A record for an *.avsvmcloud.com query, that points to an IP address in any of the following three networks: 18.130.0.0/16, 99.79.0.0/16 or 184.72.0.0/15
  • Sent an *.avsvmcloud.com DNS query with the STAGE2 flag encoded in the subdomain.
  • Received a CNAME record for a query to *.avsvmcloud.com
These three indicators are DNS based, so organizations will need to have a full historical backlog of DNS transactions ranging back to April 2020 in order to use them reliably.

Another network based IOC is HTTPS communication to one of the known STAGE3 C2 domains. However, please note that the C2 domain list might not be complete. It is even possible that a unique C2 domain is used for each victim. Nevertheless, here’s a list of the SUNBURST STAGE3 C2 domains we are currently aware of:

  • avsvmcloud[.]com
  • databasegalore[.]com
  • deftsecurity[.]com
  • digitalcollege[.]org
  • freescanonline[.]com
  • globalnetworkissues[.]com
  • highdatabase[.]com
  • incomeupdate[.]com
  • kubecloud[.]com
  • lcomputers[.]com
  • mobilnweb[.]com
  • panhardware[.]com
  • seobundlekit[.]com
  • solartrackingsystem[.]net
  • thedoccloud[.]com
  • virtualwebdata[.]com
  • webcodez[.]com
  • websitetheme[.]com
  • zupertech[.]com

Palo Alto was a Targeted SUNBURST Victim

We can now verify that Palo Alto was among the targeted SUNBURST victims, because their DNS request for "5qbtj04rcbp3tiq8bo6t.appsync.api.us.east.1.avsvmcloud.com" contains an encoded STAGE2 flag. The attack took place on September 29 at around 04:00 UTC, according to the timestamp that was also encoded into the avsvmcloud subdomain.

paloaltonetworks SUNBURST STAGE2 detected by SunburstDomainDecoder

Image: Parsing passive DNS data from Dancho Danchev with SunburstDomainDecoder v1.9 and filtering on GUID “22334A7227544B1E”.

Palo Alto's CEO Nikesh Arora has confirmed that they were hit by SUNBURST (or "SolarStorm" as they call it), but they don’t provide much details. Here’s what Nikesh wrote on December 17:

Recently, we experienced an attempt to download Cobalt Strike on one of our IT SolarWinds servers. [...]

We thought this was an isolated incident, however, on Dec. 13, we became aware that the SolarWinds software supply chain was compromised and it became clear that the incident we prevented was an attempted SolarStorm attack.

Our SUNBURST STAGE2 Victim Table has now been updated to include Palo Alto along side the other targeted victims.

Posted by Erik Hjelmvik on Monday, 11 January 2021 10:30:00 (UTC/GMT)

Tags: #SUNBURST #SolarWinds #22334A7227544B1E #SolarStorm #avsvmcloud #STAGE2 #DNS #DNS

More... Share  |  Facebook   Twitter   Reddit   Hacker News Short URL: https://netresec.com/?b=211f30f


Extracting Security Products from SUNBURST DNS Beacons

The latest version of our SunburstDomainDecoder (v1.7) can be used to reveal which endpoint protection applications that are installed on trojanized SolarWinds Orion deployments. The security application info is extracted from DNS queries for "avsvmcloud.com" subdomains, which is used by SUNBURST as a beacon and C2 channel.

Here's an example showing that City of Kingston, Ontario, Canada were running Windows Defender on their trojanized SolarWinds deployment back in June:

C:\> SunburstDomainDecoder.exe < uniq-hostnames.txt | findstr F9A9387F7D252842
F9A9387F7D252842 2020-06-16T00:00:00.0000000Z,​WindowsDefender_RUNNING,WindowsDefender_STOPPED lt5ai41qh5d53qoti3mkmc0
F9A9387F7D252842 on.ca olc62cocacn7u2q22v02eu
F9A9387F7D252842 2020-06-17T00:00:00.0000000Z q94idf4sjbem0rait7gv
F9A9387F7D252842 city.kingston. r1qshoj05ji05ac6eoip02jovt6i2v0c
F9A9387F7D252842 city.kingston.on.ca

The "F9A9387F7D252842" value is the victim's unique SUNBURST GUID. See our blog post Reassembling Victim Domain Fragments from SUNBURST DNS for more info about how the GUID value is encoded into the DNS traffic.

You can also run SunburstDomainDecoder in Linux, with help of Mono, like this:

$ mono SunburstDomainDecoder.exe < uniq-hostnames.txt | grep 76330B4D49BF7EC4
76330B4D49BF7EC4 LABELMAR e8fh1ravufms0qpt00gudir2951udivf
76330B4D49BF7EC4 2020-05-30T12:30:00.0000000Z,​ESET_RUNNING,ESET_STOPPED gp27ssesmvnpkgff7rc0eok
76330B4D49BF7EC4 nde5gaefm oiltaoj08jjd8h12vnr4tur5h
76330B4D49BF7EC4 LABELMARKET.ES

The file "uniq-hostnames.txt" is a publicly available SUNBURST passive DNS repository created by Bambenek Consulting.

Time Analysis of SUNBURST Beacons

This bash one-liner indicates that the passive DNS data shared by Bambenek contains queries posted between April and October 2020.

$ mono SunburstDomainDecoder.exe < uniq-hostnames.txt | awk '{print $2}' | grep 00000Z | sort | (head -1 && tail -1)
2020-04-04T06:30:00.0000000Z
2020-10-06T23:30:00.0000000Z,​WindowsDefender_RUNNING,​ESET_RUNNING,​ESET_STOPPED

The April 4 date here might indicate that this is when the first backdoored installer was released in the wild, but we only see SUNBURST DNS queries from a single GUID (CB28867A08967B43) on that date. The second victim doesn't appear until April 11, with additional victims starting beaconing on April 13, 14 and 15.

The first known SolarWinds Orion update containing the SUNBURST backdoor was "SolarWinds-Core-v2019.4.5220-Hotfix5.msp" (02af7cec58b9a5da1c542b5a32151ba1), which was signed on March 24. This hotfix was released publicly on March 26, according to SolarWind's Orion Platform Hotfix Release Notes. Both these dates are well before April 4, but the SUNBURST code was actually hardcoded not to start until at least 288 hours (12 days) have passed since the executing assembly was written to disk (it actually picks a random wait interval between 288 and 336 hours).

This means that an organization installing the trojanized Hotfix 5 update, when it was released on March 26, will not start sending SUNBURST DNS beacons until at least April 7. Hence the mystery GUID CB28867A08967B43, which was sendng SUNBURST DNS beacons already on April 4, is most likely not a regular SolarWinds customer.

We did unfortunately not find any SUNBURST DNS beacon with an encoded domain name for the mystery CB28867A08967B43 GUID. Nevertheless, here's a list of victim GUIDs, with corresponding domain names, that were sent in SUNBURST DNS beacons during April this year:

18039E2C39E8469D kk.dk
1B33246AC9917060 tx.org
29964E4A8F627CA1 aerioncorp.com
3247C6644BE3F231 detmir-group.ru
369080B3E59A4EE1 rccf.ru
40A97F7746D6BA4D edg.net
4A2986E3161612C4 gnb.local
4AF99133CB8E23F2 bok.com
59E0EB67DCE7CF9B WASHOE.WCSD
5A107058A310ADEA *TED.com
6776C1C2C729F869 ciena.com
6B740B9519FCAB6B btb.az
72E2D872130A33F8 calsb.org
831DBA83CED9C7D4 uont.com
8D3B008A2532D350 bok.com
92CFB39FA70AF6C5 RCWFactory.local
AA53764C15581A1A pageaz.gov
AFB5B6D3337C8448 LOGOSTECH.NET
B956E216974A17ED rai.com
CA7D468F9242EB3C fortsmithlibrary.org
D9EF8CDC3A72F7FA MOC.local
E6B2E46C5ED604DD christieclinic.com
F5D6AA262381B084 glu.com

Security Product Statistics

It is also possible to use the passive DNS data shared by Bambenek, Joe Słowik and others to compute statistics of which security products that are popular among SolarWinds' customers.

Application Count
Windows Defender 150
Windows Defender ATP 1
MS Azure ATP /
Defender for Identity
0
Carbon Black 21
CrowdStrike Falcon 25
FireEye 9
ESET 32
F-Secure 0
SUNBURST Security Applications Chart

It is worth mentioning that SUNBURST does not report status for several other major endpoint protection vendors, such as Kaspersky, McAfee and Symantec, Sophos and Trend Micro.

Download SunburstDomainDecoder

Our tool SunburstDomainDecoder is released under a Creative Commons CC-BY license, and can be downloaded here: https://www.netresec.com/files/SunburstDomainDecoder.zip

You can also read more about SunburstDomainDecoder in our blog post Reassembling Victim Domain Fragments from SUNBURST DNS.

Posted by Erik Hjelmvik on Tuesday, 29 December 2020 09:38:00 (UTC/GMT)

Tags: #SunburstDomainDecoder #SUNBURST #SolarWinds #Solorigate #DNS #Windows Defender #Carbon Black #FireEye #ESET #F-Secure #C2 #beacon

More... Share  |  Facebook   Twitter   Reddit   Hacker News Short URL: https://netresec.com/?b=20C1c3b


Reassembling Victim Domain Fragments from SUNBURST DNS

We are releasing a free tool called SunburstDomainDecoder today, which is created in order to help CERT organizations identify victims of the trojanized SolarWinds software update, known as SUNBURST or Solorigate.

SunburstDomainDecoder.exe output showing innout.corp nswhealth.net cisco.com fa.lcl int.lukoil-international.uz tr.technion.ac.il bisco.int phabahamas.org banccentral.com bk.local htwanmgmt.local

SunburstDomainDecoder can be fed with DNS queries to avsvmcloud.com in order to reveal the full internal domain names of infected companies and organizations.

UPDATE December 18, 2020 (v1.1)

SunburstDomainDecoder has now been updated to automatically reassemble fragmented domain name segments in order to show the full domain in the output.

UPDATE December 19, 2020 (v1.2)

Domain names that have been base32 encoded, such as domain names with uppercase letters, can now be extracted with SunburstDomainDecoder. The queried SUNBURST subdomains are now also included in the output.

UPDATE December 21, 2020 (v1.6)

Improved parsing of base32 encoded domain names. SUNBURST victim domains like "LKDataCenter.com", "Sunkistgrowers.com" and "BrokenArrow.Local" can now be extracted.

UPDATE December 27, 2020 (v1.7)

Improved reassembly of long domain names, like "CIMBMY.CIMBDomain.com" and "BE.AJINOMOTO-OMNICHEM.AD", that get segmented into multiple parts. Extraction of time stamps and security applications, including "Windows Defender", "Carbon Black", "CrowdStrike", "FireEye", "ESET" and "F-Secure". See Sergei Shevchenko's blog post Sunburst Backdoor, Part III: DGA & Security Software for more details.

UPDATE January 4, 2021 (v1.8)

Security products (WinDefend, ESET etc.) are now included in the summary output at the end. SUNBURST stage2 victims, which accept C2 domains in CNAME responses, are indicated with a "STAGE2" tag. The previous release marked stage2 queries with a "DNSSEC" tag. Improved extraction of truncated base32 domains, such as "*TED.com".

UPDATE January 12, 2021 (v1.9)

DNS queries with encoded timestamps are tagged with either "AVProducts" or "Ping", depending on if they include an update of the installed/running security products and services or not. The summary data at the end has been modified to also show partial domain names, such as "paloaltonetworks*".

Download SunburstDomainDecoder.zip

 

SUNBURST DNS Traffic

SUNBURST victims, who have installed one of the trojanized SolarWinds Orion software updates, will query for domain names formatted like this:

<SUBDOMAIN>.appsync-api.eu-west-1.avsvmcloud.com <SUBDOMAIN>.appsync-api.us-west-2.avsvmcloud.com <SUBDOMAIN>.appsync-api.us-east-1.avsvmcloud.com <SUBDOMAIN>.appsync-api.us-east-2.avsvmcloud.com

The "SUBDOMAIN" string has different values for each victim and the second half of this string actually contains an encoded domain name (encrypted with a simple substitution cipher).

RedDrip's decode.py

The RedDrip Team published a SUNBURST DGA decoding script yesterday, which can be used to identify SUNBURST victim organizations like CISCO and Belkin by decoding the domain names encoded in the outgoing DNS queries for subdomains of avsvmcloud.com.

This is what it looks like when RedDrip's decode.py script is fed with domain names from John Bambenek's uniq-hostnames.txt file.

cat uniq-hostnames.txt | python decode.py
02m6hcopd17p6h450gt3.appsync-api.us-west-2.avsvmcloud.com .gh
039n5tnndkhrfn5cun0y0sz02hij0b12.appsync-api.us-west-2.avsvmcloud.com ad001.mtk.lo
04spiistorug1jq5o6o0.appsync-api.us-west-2.avsvmcloud.com isi
060mpkprgdk087ebcr1jov0te2h.appsync-api.us-east-1.avsvmcloud.com belkin.com
06o0865eliou4t0btvef0b12eu1.appsync-api.us-east-1.avsvmcloud.com gncu.local
07605jn8l36uranbtvef0b12eu1.appsync-api.us-east-1.avsvmcloud.com gncu.local
07q2aghbohp4bncce6vi0odsovertr2s.appsync-api.us-east-1.avsvmcloud.com csnt.princegeor
07ttndaugjrj4pcbtvef0b12eu1.appsync-api.us-east-1.avsvmcloud.com gncu.local
08amtsejd02kobtb6h07ts2fd0b12eu1.appsync-api.eu-west-1.avsvmcloud.com sm-group.local
0b0fbhp20mdsv4scwo11r0oirssrc2vv.appsync-api.us-east-2.avsvmcloud.com ville.terrebonn
[...]

The beauty of this approach is that passive DNS data can be used in order to reliably identify the victims. This is great news for national CERTs, because they typically have readily access to passive DNS data and can use the decoded domain names in order to identify and reach out to victims in their country.

After using the python script provided by ReadDrip Team I noticed two things:

  1. The leaked domain names were internal domain names used on the victim organizations' corporate networks. Many of the domains were using the ".local" suffix.
  2. Most of the extracted domains were truncated to around 15 bytes, which make it difficult to identify the victim organization.

Truncated Domains Fragmented Domains

I later learned that what seemed to be truncated domains were actually fragmented domains, where long domain names would be split into multiple queries. This revelation turns the output from RedDrip's python tool into an interesting domain name puzzle. At this point I decided to take a closer look at the malicious SolarWinds update I had downloaded from SolarWind's website a few days ago -- yes, that's right the malicious software update "SolarWinds-Core-v2019.4.5220-Hotfix5.msp" (MD5: 02af7cec58b9a5da1c542b5a32151ba1) was actually available for download from SolarWinds' website long after they had been notified about their software being backdoored!

As an example, lets' take a closer look at this DNS query from John Bambenek's passive DNS data: r1qshoj05ji05ac6eoip02jovt6i2v0c.appsync-api.us-west-2.avsvmcloud.com

This query can be broken down into three parts:

  1. r1qshoj05ji05ac6 : What is encoded here???
  2. eoip02jovt6i2v0c : Base32 encoded string "city.kingston."
  3. .appsync-api.us-west-2.avsvmcloud.com : DNS trailer without encoded data

So, which "City of Kingston", or "Kingston City", should we contact to let them know that they have installed a trojanized SolarWinds update? Is it Kingston Jamaica, City of Kingston NY USA, City of Kingston Ontario Canada, Kingston City Tennessee USA or City of Kingston Australia?

After analyzing the "SolarWinds.Orion.Core.BusinessLayer.dll" file (MD5: b91ce2fa41029f6955bff20079468448) from the "SolarWinds-Core-v2019.4.5220-Hotfix5.msp" I learned that the initial "r1qshoj05ji05ac6" string is representing a unique "GUID" value for the infected machine. This GUID is generated by calculating an MD5 hash of the MAC address of the first active non-Loopback network interface, the domain name and the "MachineGuid" registry key value in "HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Cryptography".

This MD5 hash is then squeezed into a tiny 8 byte array by XOR'ing overlapping bytes. The "CreateSecureString" function in the trojanized SolarWinds update then "encrypts" this hash using XOR with a random key, which is prepended to the data. The XOR key and the XOR'ed data is then finally base32 encoded into what makes up the first part of the subdomain to query for. Don't let the SUNBURST source code below fool you, it is actually using base32 encoding with a custom alphabet even though the function is called "Base64Encode";

CreateSecureString function in SolarWinds.Orion.Core.BusinessLayer.OrionImprovementBusinessLayer.CryptoHelper
Image: SUNBURST source code generates a random value between 1 and 127 as XOR key

Each DNS lookup from an infected machine will query for a unique subdomain because a new XOR key will be generated for each request. Luckily for us, this XOR key is provided in each request, so we can use it in order to "decrypt" the subdomain and get the original 8 bytes derived from the MAC+domain+MachineGuid MD5 hash.

The output from my "SunburstDomainDecoder.exe" tool will print the "decrypted" 8 byte GUID in the first column, the decoded victim domain segment or timestamp in the second column and the queried SUNBURST subdomain in the last column. Each DNS query line read from standard input will generate a "GUID DecodedHostname SunburstSubdomain" line on standard output.

SunburstDomainDecoder.exe < uniq-hostnames.txt
F18613981DEC4D1A 2020-10-02T21:00:00.0000000Z 02m6hcopd17p6h450gt3
BD6DEFBBE9FEA3A9 ad001.mtk.lo 039n5tnndkhrfn5cun0y0sz02hij0b12
2BF8DE15406EA780 2020-08-25T03:00:00.0000000Z 043o9vacvthf0v95t81l
573DEB889FC54130 2020-08-13T21:00:00.0000000Z,​WindowsDefender_RUNNING,CrowdStrike_RUNNING 04jrge684mgk4eq8m8adfg7
518092C8FD571806 2020-06-09T22:30:00.0000000Z 04r0rndp6aom5fq5g6p1
F18613981DEC4D1A 2020-07-06T08:30:00.0000000Z 04spiistorug1jq5o6o0
BC1CB013239B4B92 2020-04-25T10:00:00.0000000Z 05q2sp0v4b5ramdf71l7
3ED2E979D53B2523 belkin.com 060mpkprgdk087ebcr1jov0te2h
4225A5C345C1FC8E gncu.local 06o0865eliou4t0btvef0b12eu1
[...]

The tool then finishes off by outputting the domains that are complete or at least have the last part of their domain intact. Some of these domains are complete because they were short enough to fit in one single SUNBURST DNS query, while others have been pieced together by SunburstDomainDecoder from domain fragments arriving in separate SUNBURST DNS queries.

[...]
F59BBAACBA3493C0 dufferincounty.on.ca
F5D6AA262381B084 glu.com
F9024D5B1E9717C6 gyldendal.local
F90BDDB47E495629 central.pima.gov
F956B5EF56BCF666 coxnet.cox.com
F9A9387F7D252842 city.kingston.on.ca
FB0B50553BC00DED gloucesterva.net
FBB6164BC2B0DFAD ARYZTA.COM
FD04AC52C95A1B0A bmrn.com
FDFCAB8E4C0AB3EE ansc.gob.pe
FE7FF8C9104A0508 thoughtspot.int
FF6760F36DB3D7DC smes.org

We can now see that it was "city.kingston.on.ca", (City of Kingston, Ontario, Canada) who had installed a trojanized SolarWinds update.

Download SunburstDomainDecoder

The C# source code and a compiled Windows binary for SunburstDomainDecoder is available here: https://www.netresec.com/files/SunburstDomainDecoder.zip

Creative Commons CC-BY

The source code and Windows binary is shared under a Creative Commons CC-BY license, which means that you are free to:

  • Share : copy and redistribute the material in any medium or format
  • Adapt : remix, transform, and build upon the material for any purpose, even commercially.
Provided that you give appropriate credit, provide a link to the license, and indicate if changes were made.

Running SunburstDomainDecoder on Linux/MacOS

Wanna run SunburstDomainDecoder.exe but not in Windows? No problems, the tool runs perfectly fine in Mono. Another option is to build SunburstDomainDecoder.cs as a .NET core project in Linux.

.NET Reversing

Would you like to verify my findings or learn more about .NET reverse engineering? Cool, then I'd recommend that you download dnSpy in order to reverse engineer the SUNBURST .NET DLL (which can be extracted from the msp installer with 7zip). Or you can have a look at the already extracted OrionImprovementBusinessLayer.cs on GitHub.

Posted by Erik Hjelmvik on Thursday, 17 December 2020 22:30:00 (UTC/GMT)

Tags: #SunburstDomainDecoder #SUNBURST #SolarWinds #Solorigate #domain #DNS #pDNS #Windows Defender #Carbon Black #FireEye #ESET #F-Secure #Trojan #avsvmcloud

More... Share  |  Facebook   Twitter   Reddit   Hacker News Short URL: https://netresec.com/?b=20C0f71


Installing a Fake Internet with INetSim and PolarProxy

INetSim + PolarProxy

This is a tutorial on how to set up an environment for dynamic malware analysis, which can be used to analyze otherwise encrypted HTTPS and SMTPS traffic without allowing the malware to connect to the Internet. Dynamic malware analysis (or behavioral analysis) is performed by observing the behavior of a malware while it is running. The victim machine, which executes the malware, is usually a virtual machine that can be rolled back to a clean state when the analysis is complete. The safest way to prevent the malware from infecting other machines, or doing other bad things like sending SPAM or taking part in DDoS attacks, is to run the victim machine in an offline environment. However, network traffic analysis of malware is a central part of dynamic malware analysis, which is is why a “fake Internet” is needed in most malware labs.

INetSim and PolarProxy

INetSim is a software suite that simulates common internet services like HTTP, DNS and SMTP, which useful when analyzing the network behavior of malware samples without connecting them to the Internet. INetSim also has basic support for TLS encrypted protocols, like HTTPS, SMTPS, POP3S and FTPS, but requires a pre-defined X.509-certificate to be loaded at startup. This can cause malware to terminate because the Common Names (CN) in the presented certificates don’t match the requested server names. The victim machine will actually get the exact same certificate regardless of which web site it visits. INetSim’s TLS encryption also inhibits analysis of the network traffic captured in the malware lab, such as C2 traffic or SPAM runs, because the application layer traffic is encrypted. PolarProxy can solve both these issues because it generates certificates on the fly, where the CN value is dynamically set to the requested host name, and saves the network traffic in decrypted form to PCAP files. It is therefore a good idea to replace the TLS services in INetSim with PolarProxy, which will be used as a TLS termination proxy that forwards the decrypted traffic to INetSim’s cleartext services.

Malware Lab Setup

Install Linux

The first step is to install a Linux VM, which will act as a fake Internet to the victim machine(s). I'm using Ubuntu Server 18.04.3 LTS in this tutorial, but you can use any 64-bit linux distro. I'm adding two network interfaces to the Linux VM, one interface with Internet access and one that connects to an isolated offline network to which the victim VM's will be connected. The offline interface is configured to use the static IP 192.168.53.19.

Important: Do not bridge, bond or enable IP forwarding between the two interfaces!

Network connection config Ubuntu Server 18.04

Install INetSim

INetSim is available in Ubuntu's repo, so it is possible to install it with "apt install inetsim". However, I recommend installing INetSim as described in the official documentation to get the latest packaged version of INetSim.

sudo -s

echo "deb http://www.inetsim.org/debian/ binary/" > /etc/apt/sources.list.d/inetsim.list

curl https://www.inetsim.org/inetsim-archive-signing-key.asc | apt-key add -

apt update

apt install inetsim

exit

INetSim listens on 127.0.0.1 by default, change this to INetSim's offline IP address by un-commenting and editing the service_bind_address variable in /etc/inetsim/inetsim.conf.

service_bind_address    192.168.53.19

Also configure INetSim's fake DNS server to resolve all domain names to the IP of INetSim with the dns_default_ip setting:

dns_default_ip    192.168.53.19

Finally, disable the "start_service https" and "start_service smtps" lines, because these services will be replaced with PolarProxy:

start_service dns
start_service http
#start_service https
start_service smtp
#start_service smtps

Restart the INetSim service after changing the config.

sudo systemctl restart inetsim.service

Verify that you can access INetSim's HTTP server with curl:

curl http://192.168.53.19

<html>
  <head>
    <title>INetSim default HTML page</title>
  </head>
  <body>
    <p></p>
    <p align="center">This is the default HTML page for INetSim HTTP server fake mode.</p>
    <p align="center">This file is an HTML document.</p>
  </body>
</html>

It looks like INetSim's web server can be accessed alright.

Install PolarProxy

Next step is to install PolarProxy as a systemd service (as instructed here):

sudo adduser --system --shell /bin/bash proxyuser

sudo mkdir /var/log/PolarProxy

sudo chown proxyuser:root /var/log/PolarProxy/

sudo chmod 0775 /var/log/PolarProxy/

sudo su - proxyuser

mkdir ~/PolarProxy

cd ~/PolarProxy/

curl https://www.netresec.com/?download=PolarProxy | tar -xzvf -

exit

sudo cp /home/proxyuser/PolarProxy/PolarProxy.service /etc/systemd/system/PolarProxy.service

We will need to modify the PolarProxy service config file a bit before we start it. Edit the ExecStart setting in /etc/systemd/system/PolarProxy.service to configure PolarProxy to terminate the TLS encryption for HTTPS and SMTPS (implicitly encrypted email submission). The HTTPS traffic should be redirected to INetSim's web server on tcp/80 and the SMTPS to tcp/25.

ExecStart=/home/proxyuser/PolarProxy/PolarProxy -v -p 10443,80,80 -p 10465,25,25 -x /var/log/PolarProxy/polarproxy.cer -f /var/log/PolarProxy/proxyflows.log -o /var/log/PolarProxy/ --certhttp 10080 --terminate --connect 192.168.53.19 --nosni nosni.inetsim.org

Here's a break-down of the arguments sent to PolarProxy through the ExecStart setting above:

  • -v : verbose output in syslog (not required)
  • -p 10443,80,80 : listen for TLS connections on tcp/10443, save decrypted traffic in PCAP as tcp/80, forward traffic to tcp/80
  • -p 10465,25,25 : listen for TLS connections on tcp/10465, save decrypted traffic in PCAP as tcp/25, forward traffic to tcp/25
  • -x /var/log/PolarProxy/polarproxy.cer : Save certificate to be imported to clients in /var/log/PolarProxy/polarproxy.cer (not required)
  • -f /var/log/PolarProxy/proxyflows.log : Log flow meta data in /var/log/PolarProxy/proxyflows.log (not required)
  • -o /var/log/PolarProxy/ : Save PCAP files with decrypted traffic in /var/log/PolarProxy/
  • --certhttp 10080 : Make the X.509 certificate available to clients over http on tcp/10080
  • --terminate : Run PolarProxy as a TLS termination proxy, i.e. data forwarded from the proxy is decrypted
  • --connect 192.168.53.19 : forward all connections to the IP of INetSim
  • --nosni nosni.inetsim.org : Accept incoming TLS connections without SNI, behave as if server name was "nosni.inetsim.org".

Finally, start the PolarProxy systemd service:

sudo systemctl enable PolarProxy.service

sudo systemctl start PolarProxy.service

Verify that you can reach INetSim through PolarProxy's TLS termination proxy using curl:

curl --insecure --connect-to example.com:443:192.168.53.19:10443 https://example.com

<html>
  <head>
    <title>INetSim default HTML page</title>
  </head>
  <body>
    <p></p>
    <p align="center">This is the default HTML page for INetSim HTTP server fake mode.</p>
    <p align="center">This file is an HTML document.</p>
  </body>
</html>

Yay, it is working! Do the same thing again, but also verify the certificate against PolarProxy's root CA this time. The root certificate is downloaded from PolarProxy via the HTTP service running on tcp/10080 and then converted from DER to PEM format using openssl, so that it can be used with curl's "--cacert" option.

curl http://192.168.53.19:10080/polarproxy.cer > polarproxy.cer

openssl x509 -inform DER -in polarproxy.cer -out polarproxy-pem.crt

curl --cacert polarproxy-pem.crt --connect-to example.com:443:192.168.53.19:10443 https://example.com

<html>
  <head>
    <title>INetSim default HTML page</title>
  </head>
  <body>
    <p></p>
    <p align="center">This is the default HTML page for INetSim HTTP server fake mode.</p>
    <p align="center">This file is an HTML document.</p>
  </body>
</html>

Yay #2!

Now let's set up routing to forward all HTTPS traffic to PolarProxy's service on tcp/10443 and SMTPS traffic to tcp/10465. I'm also adding a firewall rule to redirect ALL other incoming traffic to INetSim, regardless of which IP it is destined to, with the final REDIRECT rule. Make sure to replace "enp0s8" with the name of your interface.

sudo iptables -t nat -A PREROUTING -i enp0s8 -p tcp --dport 443 -j REDIRECT --to 10443

sudo iptables -t nat -A PREROUTING -i enp0s8 -p tcp --dport 465 -j REDIRECT --to 10465

sudo iptables -t nat -A PREROUTING -i enp0s8 -j REDIRECT

Verify that the iptables port redirection rule is working from another machine connected to the offline 192.168.53.0/24 network:

curl --insecure --resolve example.com:443:192.168.53.19 https://example.com

<html>
  <head>
    <title>INetSim default HTML page</title>
  </head>
  <body>
    <p></p>
    <p align="center">This is the default HTML page for INetSim HTTP server fake mode.</p>
    <p align="center">This file is an HTML document.</p>
  </body>
</html>

Yay #3!

curl --insecure --resolve example.com:465:192.168.53.19 smtps://example.com

214-Commands supported:
214- HELO MAIL RCPT DATA
214- RSET NOOP QUIT EXPN
214- HELP VRFY EHLO AUTH
214- ETRN STARTTLS
214 For more info use "HELP <topic>".

Yay #4!

It is now time to save the firewall rules, so that they will survive reboots.

sudo apt-get install iptables-persistent

Install the Victim Windows PC

Configure a static IP address on the victim Windows host by manually setting the IP address. Set the INetSim machine (192.168.53.19) as the default gateway and DNS server.

Windows IPv4 Properties

Download the X.509 root CA certificate from your PolarProxy installation here: http://192.168.53.19:10080/polarproxy.cer

  1. Double-click on "polarproxy.cer"
  2. Click [Install Certificate...]
  3. Select 🔘 Local Machine and press [Next]
  4. Select 🔘 Place all certificates in the following store and press [Browse...]
  5. Choose "Trusted Root Certification Authorities" and press [OK], then [Next]
  6. Press [Finish]

You might also want to install the PolarProxy certificate in your browser. This is how you install it to Firefox:

  1. Options / Preferences
  2. Press [Privacy & Security]
  3. Scroll down to "Certificates" and press [View Certificates...]
  4. In the "Authorities" tab, press [Import...]
  5. Open "polarproxy.cer"
  6. ☑ Trust this CA to identify websites. (check the box)
  7. Press [OK]

Now, open a browser and try visiting some websites over HTTP or HTTPS. If you get the following message regardless of what domain you try to visit, then you've managed to set everything up correctly:

This is the default HTML page for INetSim HTTP server fake mode.

This file is an HTML document.

Accessing the Decrypted Traffic

PCAP files with decrypted HTTPS and SMTPS traffic are now available in /var/log/PolarProxy/

PolarProxy will start writing to a new capture file every 60 minutes. However, the captured packets are not written to disk instantly because PolarProxy uses buffered file writing in order to improve performance. You can restart the proxy service if you wish to flush the buffered packets to disk and have PolarProxy rotate to a new capture file.

sudo systemctl restart PolarProxy

I also recommend capturing all network traffic sent to INetSim with a sniffer like netsniff-ng. This way you’ll get PCAP files with traffic from INetSim’s cleartext services (like DNS and HTTP) as well.

PCAP or it didn’t happen!

Credits

I'd like to thank Thomas Hungenberg and Patrick Desnoyers for providing valuable feedback for this blog post!

Posted by Erik Hjelmvik on Monday, 09 December 2019 08:40:00 (UTC/GMT)

Tags: #PolarProxy #HTTPS #SMTPS #HTTP #SMTP #DNS #Malware #TLS #PCAP #tutorial

More... Share  |  Facebook   Twitter   Reddit   Hacker News Short URL: https://netresec.com/?b=19Ce12f


CapLoader 1.8 Released

CapLoader 1.8

We are happy to announce the release of CapLoader 1.8 today!

CapLoader is primarily used to filter, slice and dice large PCAP datasets into smaller ones. This new version contains several new features that improves this filtering functionality even further. To start with, the “Keyword Filter” can now be used to filter the rows in the Flows, Services or Hosts tabs using regular expressions. This enables the use of matching expressions like this:

  • amazon|akamai|cdn
    Show only rows containing any of the strings “amazon” “akamai” or “cdn”.
  • microsoft\.com\b|windowsupdate\.com\b
    Show only servers with domain names ending in “microsoft.com” or “windowsupdate.com”.
  • ^SMB2?$
    Show only SMB and SMB2 flows.
  • \d{1,3}\.\d{1,3}\.\d{1,3}\.255$
    Show only IPv4 address ending with “.255”.

For a reference on the full regular expression syntax available in CapLoader, please see Microsoft’s regex “Quick Reference”.

One popular workflow supported by CapLoader is to divide all flows (or hosts) into two separate datasets, for example one “normal” and one “malicious” set. The user can move rows between these two sets, where only one set is visible while the rows in the other set are hidden. To switch which dataset that is visible versus hidden the user needs to click the [Invert Hiding] button (or use the [Ctrl]+[Tab] key combination). With this new release we’ve also made the “Invert Hiding” functionality available by clicking the purple bar, which shows the number of rows present in the currently viewed set.

CapLoader Invert Hiding GIF

Readers with a keen eye might also notice that the purple bar charts are now also accompanied by a number, indicating how many rows that are visible after each filter is applied. The available filters are: Set Selection, BPF and Keyword Filter.

NetFlow + DNS = Great Success!

CapLoader’s main view presents the contents of the loaded PCAP files as a list of netflow records. Since the full PCAP is available, CapLoader also parses the DNS packets in the capture files in order to enrich the netflow view with hostnames. Recently PaC shared a great idea with us, why not show how many failed DNS lookups each client does? This would enable generic detection of DGA botnets without using blacklists. I’m happy to announce that this great idea made it directly into this new release! The rightmost column in CapLoader’s hosts tab, called “DNS_Fails”, shows how many percent of a client’s DNS requests that have resulted in an NXRESPONSE or SRVFAIL response.

CapLoader 1.8

Two packet capture files are loaded into CapLoader in the screenshot above; one PCAP file from a PC infected with the Shifu malware and one PCAP file with “normal traffic” (thanks @StratosphereIPS for sharing these capture files). As you can see, one of the clients (10.0.2.107) has a really high DNS failure ratio (99.81%). Unsurprisingly, this is also the host that was infected with the Shifu, which uses a domain generation algorithm (DGA) to locate its C2 servers.

Apart from parsing A and CNAME records from DNS responses CapLoader now also parses AAAA DNS records (IPv6 addresses). This enables CapLoader to map public domain names to hosts with IPv6 addresses.

Additional Updates

The new CapLoader release also comes with several other new features and updates, such as:

  • Added urlscan.io service for domain and IP lookups (right-click a flow or host to bring up the lookup menu).
  • Flow ID coloring based on 5-tuple, and clearer colors in timeline Gantt chart.
  • Extended default flow-timeout from 10 minutes to 2 hours for TCP flows.
  • Changed flow-timout for non-TCP flows to 60 seconds.
  • Upgraded to .NET Framework 4.7.2.

Updating to the Latest Release

Users who have previously purchased a license for CapLoader can download a free update to version 1.8 from our customer portal. All others can download a free 30 day trial from the CapLoader product page (no registration required).

Credits

We’d like to thank Mikael Harmark, Mandy van Oosterhout and Ulf Holmström for reporting bugs that have been fixed in this release. We’d also like to thank PaC for the DNS failure rate feature request mentioned in this blog post.

Posted by Erik Hjelmvik on Tuesday, 28 May 2019 10:45:00 (UTC/GMT)

Tags: #CapLoader #NetFlow #regex #DNS #DGA #Stratosphere

More... Share  |  Facebook   Twitter   Reddit   Hacker News Short URL: https://netresec.com/?b=1950482


Domain Whitelist Benchmark: Alexa vs Umbrella

Alexa vs Umbrella

In November last year Alexa admitted in a tweet that they had stopped releasing their CSV file with the one million most popular domains.

Yes, the top 1m sites file has been retired

Members of the Internet measurement and infosec research communities were outraged, surprised and disappointed since this domain list had become the de-facto tool for evaluating the popularity of a domain. As a result of this Cisco Umbrella (previously OpenDNS) released a free top 1 million list of their own in December the same year. However, by then Alexa had already announced that their “top-1m.csv” file was back up again.

The file is back for now. We'll post an update before it changes again.

The Alexa list was unavailable for just about a week but this was enough for many researchers, developers and security professionals to make the move to alternative lists, such as the one from Umbrella. This move was perhaps fueled by Alexa saying that the “file is back for now”, which hints that they might decide to remove it again later on.

We’ve been leveraging the Alexa list for quite some time in NetworkMiner and CapLoader in order to do DNS whitelisting, for example when doing threat hunting with Rinse-Repeat. But we haven’t made the move from Alexa to Umbrella, at least not yet.


Malware Domains in the Top 1 Million Lists

Threat hunting expert Veronica Valeros recently pointed out that there are a great deal of malicious domains in the Alexa top one million list.

Researchers using Alexa top 1M as legit, you may want to think twice about that. You'd be surprised how many malicious domains end there.

I often recommend analysts to use the Alexa list as a whitelist to remove “normal” web surfing from their PCAP dataset when doing threat hunting or network forensics. And, as previously mentioned, both NetworkMiner and CapLoader make use of the Alexa list in order to simplify domain whitelisting. I therefore decided to evaluate just how many malicious domains there are in the Alexa and Umbrella lists.

hpHosts EMD (by Malwarebytes)

Alexa Umbrella
Whitelisted malicious domains: 1365 1458
Percent of malicious domains whitelisted: 0.89% 0.95%

Malware Domain Blocklist

Alexa Umbrella
Whitelisted malicious domains: 84 63
Percent of malicious domains whitelisted: 0.46% 0.34%

CyberCrime Tracker

Alexa Umbrella
Whitelisted malicious domains: 15 10
Percent of malicious domains whitelisted: 0.19% 0.13%

The results presented above indicate that Alexa and Umbrella both contain roughly the same number of malicious domains. The percentages also reveal that using Alexa or Umbrella as a whitelist, i.e. ignore all traffic to the top one million domains, might result in ignoring up to 1% of the traffic going to malicious domains. I guess this is an acceptable number of false negatives since techniques like Rinse-Repeat Intrusion Detection isn’t intended to replace traditional intrusion detection systems, instead it is meant to be use as a complement in order to hunt down the intrusions that your IDS failed to detect. Working on a reduced dataset containing 99% of the malicious traffic is an acceptable price to pay for having removed all the “normal” traffic going to the one million most popular domains.


Sub Domains

One significat difference between the two lists is that the Umbrella list contains subdomains (such as www.google.com, safebrowsing.google.com and accounts.google.com) while the Alexa list only contains main domains (like “google.com”). In fact, the Umbrella list contains over 1800 subdomains for google.com alone! This means that the Umbrella list in practice contains fewer main domains compared to the one million main domains in the Alexa list. We estimate that roughly half of the domains in the Umbrella list are redundant if you only are interested in main domains. However, having sub domains can be an asset if you need to match the full domain name rather than just the main domain name.


Data Sources used to Compile the Lists

The Alexa Extension for Firefox
Image: The Alexa Extension for Firefox

The two lists are compiled in different ways, which can be important to be aware of depending on what type of traffic you are analyzing. Alexa primarily receives web browsing data from users who have installed one of Alexa’s many browser extensions (such as the Alexa browser toolbar shown above). They also gather additional data from users visiting web sites that include Alexa’s tracker script.

Cisco Umbrella, on the other hand, compile their data from “the actual world-wide usage of domains by Umbrella global network users”. We’re guessing this means building statistics from DNS queries sent through the OpenDNS service that was recently acquired by Cisco.

This means that the Alexa list might be better suited if you are only analyzing HTTP traffic from web browsers, while the Umbrella list probably is the best choice if you are analyzing non-HTTP traffic or HTTP traffic that isn’t generated by browsers (for example HTTP API communication).


Other Quirks

As noted by Greg Ferro, the Umbrella list contains test domains like “www.example.com”. These domains are not present in the Alexa list.

We have also noticed that the Umbrella list contains several domains with non-authorized gTLDs, such as “.home”, “.mail” and “.corp”. The Alexa list, on the other hand, only seem to contain real domain names.


Resources and Raw Data

Both the Alexa and Cisco Umbrella top one million lists are CSV files named “top-1m.csv”. The CSV files can be downloaded from these URL’s:

The analysis results presented in this blog post are based on top-1m.csv files downloaded from Alexa and Umbrella on March 31, 2017. The malware domain lists were also downloaded from the three respective sources on that same day.

We have decided to share the “false negatives” (malware domains that were present in the Alexa and Umbrella lists) for transparency. You can download the lists with all false negatives from here:
https://www.netresec.com/files/alexa-umbrella-malware-domains_170331.zip


Hands-on Practice and Training

If you wanna learn more about how a list of common domains can be used to hunt down intrusions in your network, then please register for one of our network forensic trainings. The next training will be a pre-conference training at 44CON in London.

Posted by Erik Hjelmvik on Monday, 03 April 2017 14:47:00 (UTC/GMT)

Tags: #Alexa #Umbrella #domain #DNS #malware

More... Share  |  Facebook   Twitter   Reddit   Hacker News Short URL: https://netresec.com/?b=1743fae


CapLoader 1.3 Released

CapLoader Logo

A new version of our heavy-duty PCAP parser tool CapLoader is now available. There are many new features and improvements in this release, such as the ability to filter flows with BPF, domain name extraction via passive DNS parser and matching of domain names against a local white list.


Filtering with BPF

The main focus in the work behind CapLoader 1.3 has been to fully support the Rinse-Repeat Intrusion Detection methodology. We've done this by improving the filtering capabilities in CapLoader. For starters, we've added an input filter, which can be used to specify IP addresses, IP networks, protocols or port numbers to be parsed or ignored. The input filter uses the Berkeley Packet Filter (BPF) syntax, and is designed to run really fast. So if you wanna analyze only HTTP traffic you can simply write “port 80” as your input filter to have CapLoader only parse and display flows going to or from port 80. We have also added a display filter, which unlike Wireshark also uses BPF. Thus, once a set of flows is loaded one can easily apply different display filters, like “host 194.9.94.80” or “net 192.168.1.0/24”, to apply different views on the parsed data.

CapLoader BPF Input Filter and Display Filter
Image: CapLoader with input filter "port 80 or port 443" and display filter "not net 74.125.0.0/16".

The main differences between the input filter and display filter are:

  • Input filter is much faster than the display filter, so if you know beforehand what ports, protocols or IP addresses you are interested in then make sure to apply them as an input filter. You will notice a delay when applying a display filter to a view of 10.000 flows or more.
  • In order to apply a new input filter CapLoader has to reload all the opened PCAP files (which is done by pressing F5). Modifying display filters, on the other hand, only requires you to press Enter or hit the “Apply” button.
  • Previously applied display filters are accessible in a drop-down menu in the GUI, but no history is kept of previous input filters.


NetFlow + DNS == true

The “Flows” view in CapLoader gives a great overview of all TCP, UDP and SCTP flows in the loaded PCAP files. However, it is usually not obvious to an analyst what every IP address is used for. We have therefore added a DNS parser to CapLoader, so that all DNS packets can be parsed in order to map IP addresses to domain names. The extracted domain names are displayed for each flow, which is very useful when performing Rinse-Repeat analysis in order to quickly remove “known good servers” from the analysis.


Leveraging the Alexa top 1M list

As we've show in in our previous blog post “DNS whitelisting in NetworkMiner”, using a list of popular domain names as a whitelist can be an effective method for finding malware. We often use this approach in order to quickly remove lots of known good servers when doing Rinse-Repeat analysis in large datasets.

Therefore, just as we did for NetworkMiner 1.5, CapLoader now includes Alexa's list of the 1 million most popular domain names on the Internet. All domain names, parsed from DNS traffic, are checked against the Alexa list. Domains listed in the whitelist are shown in CapLoader's “Server_Alexa_Domian” column. This makes it very easy to sort on this column in order to remove (hide) all flows going to “normal” servers on the Internet. After removing all those flows, what you're left with is pretty much just:

  • Local traffic (not sent over the Internet)
  • Outgoing traffic to either new or obscure domains

Manually going through the remaining flows can be very rewarding, as it can reveal C2 traffic from malware that has not yet been detected by traditional security products like anti-virus or IDS.

Flows in CapLoader with DNS parsing and Alexa lookup
Image: CapLoader with malicious flow to 1.web-counter[.]info (Miuref/Boaxxe Trojan) singled out due to missing Alexa match.

Many new features in CapLoader 1.3

The new features highlighted above are far from the only additions made to CapLoader 1.3. Here is a more complete list of improvements in this release:

  • Support for “Select Flows in PCAP” to extract and select 5-tuples from a PCAP-file. This can be a Snort PCAP with packets that have triggered IDS signatures. This way you can easily extract the whole TCP or UDP flow for each signature match, instead of just trying to make sense of one single packet per alert.
  • Improved packet carver functionality to better carve IP, TCP and UPD packets from any file. This includes memory dumps as well as proprietary and obscure packet capture formats.
  • Support for SCTP flows.
  • DNS parser.
  • Alexa top 1M matching.
  • Input filter and display filter with BPF syntax.
  • Flow Producer-Consumer-Ratio PCR.
  • Flow Transcript can be opened simply by double-clicking a flow.
  • Find form updated with option to hide non-matching flows instead of just selecting the flows that matched the keyword search criteria.
  • New flow transcript encoding with IP TTL, TCP flags and sequence numbers to support analysis of Man-on-the-Side attacks.
  • Faster loading of previously opened files, MD5 hashes don't need to be recalculated.
  • A selected set of flows in the GUI can be inverted simply by right-clicking the flow list and selecting “Invert Selection” or by hitting Ctrl+I.


Downloading CapLoader 1.3

All these new features, except for the Alexa lookup of domain names, are available in our free trial version of CapLoader. So to try out these new features in CapLoader, simply grab a trial download here:
https://www.netresec.com/?page=CapLoader#trial (no registration needed)

All paying customers with an older version of CapLoader can grab a free update for version 1.3 at our customer portal.

Posted by Erik Hjelmvik on Monday, 28 September 2015 07:30:00 (UTC/GMT)

Tags: #CapLoader #BPF #Berkeley Packet Filter #Rinse-Repeat #DNS #Alexa #PCAP #NetFlow #Malware #C2

More... Share  |  Facebook   Twitter   Reddit   Hacker News Short URL: https://netresec.com/?b=15914E3


NetworkMiner 1.6 Released

We've released version 1.6 of NetworkMiner today!

Confetti in Toronto by Winnie Surya Image credits: Confetti in Toronto by Winnie Surya

The new features in NetworkMiner 1.6 include:

  • Drag-and-Drop
    Reassembled files and images can be opened with external tools by drag-and-dropping items from NetworkMiner's Files or Images tabs onto your favorite editor or viewer.

  • Email extraction
    Improved extraction of emails and attachments sent over SMTP.

  • DNS analysis
    Failed DNS lookups that result in NXDOMAIN and SERVFAIL are displayed in the DNS tab along with the flags in the DNS response.

  • Live sniffing
    Improved live sniffing performance.

  • PCAP-over-IP
    Remote live sniffing enabled by bringing the PCAP-over-IP feature into the free open source version of NetworkMiner.


Identifying Malware DNS lookups

NetworkMiner Professional 1.6 with DNS traffic from the Contagio Kuluoz-Asprox

DNS traffic from the Kuluoz-Asprox botnet (PCAP file available via Contagio)

Note the NXDOMAIN responses and “No” in Alexa top 1 million column in the screenshot above; these domains are probably generated by a domain generation algorithm (DGA).

Live Sniffing with Pcap-over-IP

The PCAP-over-IP functionality enables live sniffing also on non-Windows machines, simply by running tcpdump (or dumpcap) and netcat like this:

# tcpdump -i eth0 -s0 -U -w - | nc localhost 57012
For more information about how to run NetworkMiner in Linux, please read our HowTo install NetworkMiner in Ubuntu Fedora and Arch Linux blog post.

To receive the Pcap-over-IP stream in NetworkMiner, simply press Ctrl+R and select a TCP port.

NetworkMiner Pcap-over-IP

For more information about this feature please see our previous blog post about the PCAP‑over‑IP feature.

NetworkMiner Professional

The professional version of NetworkMiner additionally contains the following improvements of the command line tool NetworkMinerCLI:

  • Enabled reading of PCAP and PcapNG data from standard input (STDIN)
  • Full support for PCAP-over-IP
  • More detailed DNS logging in NetworkMinerCLI's CSV export of DNS responses

The ability to read PCAP data from STDIN with NetworkMinerCLI makes it really simple to do live extraction of emails and email attachments. Here's an example showing how to do live SMTP extraction in Linux:

# tcpdump -i eth0 -s0 -w - port 25 or 587 | mono NetworkMinerCLI.exe -r - -w /var/log/smtp_extraction/

The syntax for extracting emails and attachments in Windows is very similar:

C:\>dumpcap.exe -i 1 -f "port 25 or 587" -w - | NetworkMinerCLI.exe -r -

The TCP ports 25 and 587, which are used in the capture filter above, are the standard port numbers for SMTP. In order to do live extraction of files sent over HTTP, simply use “port 80” as capture filter instead. Likewise, X.509 certificates can also be extracted from HTTPS sessions simply by using “port 443” as capture filter.

Download NetworkMiner 1.6

The most recent release of the free (open source) version of NetworkMiner can be downloaded from SourceForge or our NetworkMiner product page. Paying customers can download an update for NetworkMiner Professional from our customer portal.

Credits

We would like to thank Dan Eriksson (FM CERT) and Lenny Hansson (Danish GovCERT) for submitting bug reports and feature requests.

Posted by Erik Hjelmvik on Monday, 16 June 2014 11:00:00 (UTC/GMT)

Tags: #Netresec #NetworkMiner #Professional #SMTP #Extract #DNS #PCAP-over-IP

More... Share  |  Facebook   Twitter   Reddit   Hacker News Short URL: https://netresec.com/?b=1463B56


DNS whitelisting in NetworkMiner

Stack of Papers by Jenni C

One of the new features in NetworkMiner Professional 1.5 is the ability to check if domain names in DNS requests/responses are “normal” or malicious ones. This lookup is performed offline using a local copy of Alexa's top 1 million domain name list.

We got the idea for this feature via Jarno Niemelä's great presentation titled “Making Life Difficult for Malware”. Despite working for F-Secure Jarno presents several smart ideas for avoiding malware infections without having to install an AV-product.

One of Jarno's slides contains the following suggestions:

Block Traffic To Sites Your Users Don’t Go To

Block subdomain hosting TLDs
  • co.cc, co.tv, ce.ms, rr.nu, cu.cc, cz.cc, vv.cc, cw.cm, cx.cc, etc
Block domains that provide dynamic DNS
  • *dyndns*, *no-ip*, 8866.org, thescx.info, 3322.org, sock8.com
Block file sharing sites, some malware use them
  • fileleave.com, dropbox.com, rapidshare.com, megafiles.com
For strict policy, allow DNS resolving only to Alexa top 1M[1]
  • Tip: Instead of null routing domains set up landing page
  • Either with a link that allows domain or IT ticket

Preventing users from visiting sites outside of the top 1 million websites (according to Alexa) sounds a bit harsh. In fact, we at Netresec just recently made it into the top 1M list (the current rank for netresec.com is 726 922). There are also many good and legit sites that are not yet on this list. Our idea is, however, to give analysts a heads up on queried DNS names that are not on the top 1M list by displaying this information in NetworkMiner's DNS tab.

NetworkMiner Professional 1.5 with DNS tab showing Alexa result for office.windowupdate.com

The screenshot above contains a lookup for the domain “office.windowupdate.com” (note the missing “s” in “windows”). This domain name was previously used by the C2 protocol Lurk (see Command Five's report “Command and Control in the Fifth Domain” for more details). The “Alexa Top 1M” column in NetworkMiner's DNS tab indicates whether or not the domain name is a well known domain. The malicious “office.windowupdate.com” is marked with “No”, while the proper “www.update.microsoft.com” is indeed on the list. It is, however, important to note that only the second-level domain is checked by NetworkMiner; i.e. in this case “windowupdate.com” and “microsoft.com”.

The DNS whitelisting technique can also come in handy when dealing with malware that employs domain generation algorithms (DGAs) (see the Damballa blog for additional info regarding DGAs). It is probably safe to say that these auto-generated domains should never show up in the Alexa Top 1M list.

Posted by Erik Hjelmvik on Wednesday, 02 October 2013 22:30:00 (UTC/GMT)

Tags: #Netresec #DNS #domain #DGA #malware #Alexa

More... Share  |  Facebook   Twitter   Reddit   Hacker News Short URL: https://netresec.com/?b=13A66EB


Extracting DNS queries

There was recently a question on the Wireshark users mailing list about “how to get the query name from a dns request packet with tshark”. This is a problem that many network analysts run into, so I decided to write a blog post instead of just replying to the mailing list.

Note: the pcap file used in this blog post is from the DFRWS 2009 Challenge.

Who queried for a particular domain?

Tshark can easily be used in order to determine who queried for a particular domain, such as google.com, by using the following command:

tshark -r nssal-capture-1.pcap -T fields -e ip.src -e dns.qry.name -R "dns.flags.response eq 0 and dns.qry.name contains google.com"
137.30.123.78 google.com
137.30.123.78 www.google.com
137.30.123.78 id.google.com
137.30.123.78 images.google.com
137.30.123.78 tbn2.google.com
137.30.123.78 tbn0.google.com
137.30.123.78 tbn2.google.com
137.30.123.78 tbn1.google.com
137.30.123.78 tbn3.google.com
137.30.123.78 tbn3.google.com

List all queries

A list of ALL queries can be built with the same command, but without filtering on a particular domain:

tshark -r nssal-capture-1.pcap -T fields -e ip.src -e dns.qry.name -R "dns.flags.response eq 0"
137.30.123.78 fp.ps3.us.playstation.com
137.30.123.78 cmt.us.playstation.com
137.30.123.78 google.com
137.30.123.78 www.google.com
137.30.123.78 www.mardigrasday.com
137.30.123.78 pagead2.googlesyndication.com
137.30.123.78 googleads.g.doubleclick.net
137.30.123.78 www.google-analytics.com
137.30.123.78 mardigrasday.makesparties.com
137.30.123.78 images.scanalert.com
137.30.123.78 a248.e.akamai.net
137.30.123.78 ssl-hints.netflame.cc
...

DNS lists in NetworkMiner

There is a DNS tab in NetworkMiner, which displays a nice list of all DNS queries and responses in a pcap file. Loading the same nssal-capture-1.pcap into NetworkMiner generates the following list:


DNS tab with nssal-capture-1.pcap loaded

NetworkMiner Professional also has the ability to export this data to a CSV file. The command line tool NetworkMinerCLI can also generate such a CSV file without a GUI, which is perfect if you wanna integrate it in a customized script.

Posted by Erik Hjelmvik on Sunday, 17 June 2012 17:45:00 (UTC/GMT)

Tags: #domain #tshark #pcap

More... Share  |  Facebook   Twitter   Reddit   Hacker News Short URL: https://netresec.com/?b=126C5CB

twitter

NETRESEC on Twitter

Follow @netresec on twitter:
» twitter.com/netresec


book

Recommended Books

» The Practice of Network Security Monitoring, Richard Bejtlich (2013)

» Applied Network Security Monitoring, Chris Sanders and Jason Smith (2013)

» Network Forensics, Sherri Davidoff and Jonathan Ham (2012)

» The Tao of Network Security Monitoring, Richard Bejtlich (2004)

» Practical Packet Analysis, Chris Sanders (2017)

» Windows Forensic Analysis, Harlan Carvey (2009)

» TCP/IP Illustrated, Volume 1, Kevin Fall and Richard Stevens (2011)

» Industrial Network Security, Eric D. Knapp and Joel Langill (2014)