NETRESEC Network Security Blog - Tag : HTTPS


PolarProxy in Docker

PolarProxy + Docker

Our transparent TLS proxy PolarProxy is gaining lots of popularity due to how effective it is at generating decrypted PCAP files in combination with how easy it is to deploy. In this blog post we will show how to run PolarProxy in Docker.

Installation Instructions

Create a Dockerfile with the following contents:

FROM mcr.microsoft.com/dotnet/core/runtime:2.2
EXPOSE 10443
EXPOSE 10080
EXPOSE 57012
RUN groupadd -g 31337 polarproxy && useradd -m -u 31337 -g polarproxy polarproxy && mkdir -p /var/log/PolarProxy /opt/polarproxy && chown polarproxy:polarproxy /var/log/PolarProxy && curl -s https://www.netresec.com/?download=PolarProxy | tar -xzf - -C /opt/polarproxy
VOLUME ["/var/log/PolarProxy/", "/home/polarproxy/"]
USER polarproxy
WORKDIR /opt/polarproxy/
ENTRYPOINT ["dotnet", "PolarProxy.dll"]
CMD ["-v", "-p", "10443,80,443", "-o", "/var/log/PolarProxy/", "--certhttp", "10080", "--pcapoverip", "0.0.0.0:57012"]

Save the Docker file as "Dockerfile" (no extension) in an empty directory and start a shell in that directory with root privileges. Build the PolarProxy Docker image with:

docker build -t polarproxy-image .

Next, create a Docker container named "polarproxy":

docker create -p 443:10443 -p 10443:10443 -p 10080:10080 --name polarproxy polarproxy-image
The "-p" switches in this command define three DNAT rules that will get activated when the polarproxy container is started. The first DNAT rule forwards incoming TCP port 443 traffic to the polarproxy Docker container's transparent TLS proxy service on TCP port 10443. The second one does the same thing, but for incoming traffic to TCP 10443. The last one forwards TCP port 10080 traffic to a web server that delivers the public X.509 certificate of the proxy.

It is now time to start the polarproxy container:

docker start polarproxy

Verify that PolarProxy is running:

docker ps
docker logs polarproxy

Try fetching PolarProxy's public root CA certificate with curl and then connect to a website over HTTPS through the proxy:

curl -sL http://localhost:10080 | openssl x509 -inform DER -issuer -noout -dates
curl --insecure --connect-to www.netresec.com:443:127.0.0.1:10443 https://www.netresec.com/
curl --insecure --resolve www.netresec.com:443:127.0.0.1 https://www.netresec.com/

Redirect HTTPS and Trust the Root CA

You can now redirect outgoing TCP 443 traffic from your network to your Docker host. Review the "Routing HTTPS Traffic to the Proxy" section on the PolarProxy page for recommendations on how to redirect outgoing traffic to PolarProxy.

Finally, configure the operating system, browsers and other applications that will get their TLS traffic proxied by PolarProxy to trust the root CA of the PolarProxy service running in your Docker container. Follow the steps in the "Trusting the PolarProxy root CA" section of the PolarProxy documentation in order to install the root cert.

Docker Volumes

The Docker file we used in this blog post defines two volumes. The first volume is mounted on "/var/log/PolarProxy" in the container, which is where the decrypted network traffic will be stored as hourly rotated PCAP files. The second volume is the polarproxy home directory, under which PolarProxy will store its private root CA certificate.

The volumes are typically located under "/var/lib/docker/volumes" on the Docker host's file system. You can find the exact path by running:

docker volume ls
docker volume inspect <VOLUME_NAME>

Or use find to list *.pcap files in the Docker volumes directory:

find /var/lib/docker/volumes/ -name *.pcap
/var/lib/docker/volumes/7ebb3f56fd4ceab96[...]/_data/​proxy-201006-095937.pcap/var/lib/docker/volumes/7ebb3f56fd4ceab96[...]/_data/​proxy-201006-105937.pcap/var/lib/docker/volumes/7ebb3f56fd4ceab96[...]/_data/​proxy-201006-115937.pcap

The full path of your private PolarProxy Root CA certificate, which is located under "/home/polarproxy/" in the Docker container, can also be located using find:

find /var/lib/docker/volumes/ -name *.p12
/var/lib/docker/volumes/dcabbbac10e1b1461[...]/_data/​.local/share/PolarProxy/​e249f9c497d7b5c41339f153a31eda1c.p12

We recommend reusing the "/home/polarproxy/" volume, when deploying new PolarProxy instances or upgrading to a new version of PolarProxy, in order to avoid having to re-configure clients to trust a new root CA every time a new PolarProxy container is created.

PolarProxy in Docker on ARM Linux

PolarProxy can also run on ARM Linux installations, such as a Raspberry Pi. However, the Dockerfile must be modified slightly in order to do so.

ARM 32-bit / AArch32 / ARMv7 If you're running an "arm32" Linux OS, then change the download link in the "RUN" instruction to the following URL:
https://www.netresec.com/?download=PolarProxy_linux-arm

ARM 64-bit / AArch64 / ARMv8 If you're running an "arm64" Linux OS, then change the download link in the "RUN" instruction to the following URL:
https://www.netresec.com/?download=PolarProxy_linux-arm64

Don't know if you're running a 32-bit or 64-bit OS? Run "uname -m" and check if the output says "armv7*" (arm32) or "armv8*" (arm64).

See our blog post "Raspberry PI WiFi Access Point with TLS Inspection" for more details about deploying PolarProxy on a Raspberry Pi (without Docker).

Credits

We'd like to thank Jonas Lejon for contacting us back in February about the work he had done to get PolarProxy running in Docker. We used Jonas' work as a starting point when building the installation instructions in this how-to guide.

We also want to thank Erik Ahlström for providing valuable feedback on the instructions in this guide.

ʕ•ᴥ•ʔ + 🐳 = 💜

Posted by Erik Hjelmvik on Wednesday, 07 October 2020 08:09:00 (UTC/GMT)

Tags: #PolarProxy #TLS #HTTPS #TLSI #curl #x509 #X.509 #PCAP #PCAP-over-IP

More... Share  |  Facebook   Twitter   Reddit   Hacker News Short URL: https://netresec.com/?b=20Accbd


Reverse Proxy and TLS Termination

PolarProxy is primarily a TLS forward proxy, but it can also be used as a TLS termination proxy or reverse TLS proxy to intercept and decrypt incoming TLS traffic, such as HTTPS or IMAPS, before it is forwarded to a server. The proxied traffic can be accessed in decrypted form as a PCAP formatted data stream, which allows real-time analysis of the decrypted traffic by an IDS as well as post incident forensics with Wireshark.

PolarProxy version 0.8.15 and later can import an existing X.509 server certificate (aka leaf certificate or end-entity certificate) in order to perform the TLS decryption using a valid certificate signed by a trusted certificate authority. If no server certificate is provided, then PolarProxy falls back to generating server certificates on the fly and signing them with its own root CA certificate.

There are two principal ways to run PolarProxy as a reverse proxy, either as a TLS termination proxy or as a reverse proxy that decrypts and re-encrypts the traffic.

PolarProxy as a TLS Termination Proxy

TLS Termination Proxy

The TLS termination proxy mode is useful in order to offload the task of performing TLS encryption to PolarProxy instead of doing the decryption on the web server. This mode can also be used when the proxied services don’t support TLS encryption, such as legacy web servers or servers hosting other unencrypted services that you want to secure with TLS.

The following command sequence shows how to create a Let’s Encrypt SSL certificate, convert it to the PKCS#12 format, and load the server certificate into PolarProxy to terminate incoming HTTPS connections. In this setup PolarProxy decrypts the TLS traffic and relays the HTTP traffic to the web server on TCP port 80.

sudo certbot certonly --manual --preferred-challenges dns -d example.com,www.example.com

sudo openssl pkcs12 -export -out /etc/example.p12 -inkey /etc/letsencrypt/live/example.com/privkey.pem -in /etc/letsencrypt/live/example.com/fullchain.pem --passout pass:PASSWORD

sudo mkdir /var/log/TlsTerminationProxy/

sudo ./PolarProxy --terminate --connect 10.1.2.3 --nosni www.example.com --servercert example.com,www.example.com:/etc/example.p12:PASSWORD -p 443,80,80 -o /var/log/TlsTerminationProxy/

Here’s a breakdown of the arguments sent to PolarProxy:

  • --terminate : Terminate incoming TLS sessions and forward proxied traffic in unencrypted form.
  • --connect 10.1.2.3 : Forward all proxied traffic to 10.1.2.3 instead of connecting to the host name provided in the SNI extension of the TLS ClientHello message.
  • --nosni www.example.com : Treat incoming TLS sessions that don’t define a host name with the SNI extension as if they wanna to connect to “www.example.com”.
  • --servercert example.com,www.example.com:/etc/example.p12:PASSWORD : Use the server certificate “/etc/example.p12” for incoming connections to “example.com” and “www.example.com”.
  • -p 443,80,80 : Listen on TCP port 443, save decrypted traffic in PCAP file as if it was directed to port 80, forward decrypted traffic to port 80.
  • -o /var/log/TlsTerminationProxy/ : Save decrypted traffic to hourly rotated PCAP files in “/var/log/TlsTerminationProxy/”.

PolarProxy is a generic TLS proxy that doesn’t care what application layer protocol the TLS tunnel carries. So if you want to terminate the TLS encryption of incoming IMAPS sessions as well, then simply append an additional argument saying “-p 993,143,143” to also forward decrypted IMAP sessions to 10.1.2.3. This method can be used in order to wrap almost any TCP based protocol in a TLS tunnel, which can be useful for privacy reasons as well as to prevent network monitoring tools from detecting the actual application layer protocol.

PolarProxy as a Reverse TLS Proxy

Reverse TLS Proxy

There are setups for which it is preferable to also encrypt the internal sessions between PolarProxy and the final server. One such setup is when the server is hosting a web service with support for the HTTP/2 protocol, which in practice always uses TLS. Luckily PolarProxy is designed to decrypt and re-encrypt proxied traffic while also forwarding important TLS parameters, such as ALPN and SNI, between the internal and external TLS sessions.

To use TLS encryption on the inside as well as outside of PolarProxy, simply do as explained in the previous TLS termination section, but remove the “--terminate” argument and change the port argument to “-p 443,80,443” like this:

sudo ./PolarProxy --connect 10.1.2.3 --nosni www.example.com --servercert example.com,www.example.com:/etc/example.p12:PASSWORD -p 443,80,443 -o /var/log/ReverseTlsProxy/

PolarProxy will save the decrypted traffic as cleartext HTTP (or HTTP/2) to PCAP files in the “/var/log/ReverseTlsProxy/” directory.

Real-Time Analysis of Decrypted Traffic

Both the external (client-to-proxy) and internal (proxy-to-server) TCP sessions, in the reverse TLS proxy example above, are encrypted with TLS. This prevents passive network security monitoring tools, such as IDSs, DPI and DLP appliances, from analyzing the application layer data being sent and received. The PCAP files written to “/var/log/ReverseTlsProxy/” can be a valuable forensic asset when investigating an incident, but a real-time stream of the decrypted data is needed in order to swiftly detect and alert on potential security breaches and other incidents.

PolarProxy’s “--pcapoverip” option can be used to provide such a real-time stream of the decrypted data passing through the proxy. This data can easily be sent to a network interface using tcpreplay, as explained in our blog post “Sniffing Decrypted TLS Traffic with Security Onion”.

Security Considerations

The examples shown in this blog post all run PolarProxy with root privileges using sudo, which can be dangerous from a security perspective. PolarProxy is actually designed to be run without root privileges, but doing so prevents it from listening on a port below 1024. Luckily, this issue can easily be overcome with a simple port forwarding or redirect rule. The following iptables redirect rule can be used if PolarProxy is listening on TCP port 20443 and incoming HTTPS request are arriving to the eth0 interface of the proxy:

iptables -t nat -A PREROUTING -i eth0 -p tcp --dport 443 -j REDIRECT --to 20443

PolarProxy does not support loading settings from a config file. The password for the PKCS12 certificate will therefore need to be supplied on the command line, which can make it visible from a process listing. If this is a concern for you, then please consider using “hidepid” to hide processes from other users. You can find instructions on how to use hidepid in hardening guides for Debian, Arch, SUSE and most other Linux flavors.

Posted by Erik Hjelmvik on Thursday, 12 March 2020 15:45:00 (UTC/GMT)

Tags: #PolarProxy #TLS #SSL #PCAP #decrypt #HTTPS #HTTP #HTTP/2 #http2 #IMAPS #decrypt

More... Share  |  Facebook   Twitter   Reddit   Hacker News Short URL: https://netresec.com/?b=2033d38


Sniffing Decrypted TLS Traffic with Security Onion

Wouldn't it be awesome to have a NIDS like Snort, Suricata or Zeek inspect HTTP requests leaving your network inside TLS encrypted HTTPS traffic? Yeah, we think so too! We have therefore created this guide on how to configure Security Onion to sniff decrypted TLS traffic with help of PolarProxy.

Network drawing with Clients, SecurityOnion and the Internet

PolarProxy is a forward TLS proxy that decrypts incoming TLS traffic from clients, re-encrypts it and forwards it to the server. One of the key features in PolarProxy is the ability to export the proxied traffic in decrypted form using the PCAP format (a.k.a. libpcap/tcpdump format). This makes it possible to read the decrypted traffic with external tools, without having to perform the decryption again. It also enables packet analysis using tools that don't have built-in TLS decryption support.

This guide outlines how to configure PolarProxy to intercept HTTPS traffic and send the decrypted HTTP traffic to an internal network interface, where it can be sniffed by an IDS.

STEP 1 ☆ Install Ubuntu

Download and install the latest SecurityOnion ISO image, but don't run the "Setup" just yet.

STEP 2 ☆ Add a Dummy Network Interface

Add a dummy network interface called "decrypted", to which decrypted packets will be sent.

ip link add decrypted type dummy
ip link set decrypted arp off up
Add the commands above to /etc/rc.local before "exit 0" to have the network interface automatically configured after reboots.

dummy interface in rc.local

STEP 3 ☆ Install Updates

Install updates in Security Onion by running "sudo soup".

STEP 4 ☆ Run the Security Onion Setup

Run the Security Onion setup utility by double-clicking the "Setup" desktop shortcut or executing "sudo sosetup" from a terminal. Follow the setup steps in the Production Deployment documentation and select "decrypted" as your sniffing interface.

Sniffing Interface Selection Window

Reboot and run Setup again to continue with the second phase of Security Onion's setup. Again, select "decrypted" as the interface to be monitored.

STEP 5 ☆ Install PolarProxy Service

Download and install PolarProxy:

sudo adduser --system --shell /bin/bash proxyuser
sudo mkdir /var/log/PolarProxy
sudo chown proxyuser:root /var/log/PolarProxy/
sudo chmod 0775 /var/log/PolarProxy/

sudo su - proxyuser
mkdir ~/PolarProxy
cd ~/PolarProxy/
curl https://www.netresec.com/?download=PolarProxy | tar -xzf -
exit

sudo cp /home/proxyuser/PolarProxy/PolarProxy.service /etc/systemd/system/PolarProxy.service

Edit /etc/systemd/system/PolarProxy.service and add "--pcapoverip 57012" at the end of the ExecStart command.

--pcapoverip 57012 in PolarProxy.service

Start the PolarProxy systemd service:

sudo systemctl enable PolarProxy.service
sudo systemctl start PolarProxy.service

STEP 6 ☆ Install Tcpreplay Service

The decrypted traffic can now be accessed via PolarProxy's PCAP-over-IP service on TCP 57012. We can leverage tcpreplay and netcat to replay these packets to our dummy network interface in order to have them picked up by Security Onion.

nc localhost 57012 | tcpreplay -i decrypted -t -
However, it's better to create a systemd service that does this automatically on bootup. We therefore create a file called /etc/systemd/system/tcpreplay.service with the following contents:
[Unit]
Description=Tcpreplay of decrypted traffic from PolarProxy
After=PolarProxy.service

[Service]
Type=simple
ExecStart=/bin/sh -c 'nc localhost 57012 | tcpreplay -i decrypted -t -'
Restart=on-failure
RestartSec=3

[Install]
WantedBy=multi-user.target

Start the tcpreplay systemd service:

sudo systemctl enable tcpreplay.service
sudo systemctl start tcpreplay.service

STEP 7 ☆ Add firewall rules

Security Onion only accepts incoming connections on TCP 22 by default, we also need to allow connections to TCP port 10443 (proxy port), and 10080 (root CA certificate download web server). Add allow rules for these services to the Security Onion machine's firewall:

sudo ufw allow in 10443/tcp
sudo ufw allow in 10080/tcp

Verify that the proxy is working by running this curl command on a PC connected to the same network as the Security Onion machine:

curl --insecure --connect-to www.netresec.com:443:[SecurityOnionIP]:10443 https://www.netresec.com/
Note: You can even perform this test from a Win10 PC, since curl is included with Windows 10 version 1803 and later.

Add the following lines at the top of /etc/ufw/before.rules (before the *filter section) to redirect incoming packets on TCP 443 to PolarProxy on port 10443.

*nat
:PREROUTING ACCEPT [0:0]
-A PREROUTING -i enp0s3 -p tcp --dport 443 -j REDIRECT --to 10443
COMMIT

Note: Replace "enp0s3" with the Security Onion interface to which clients will connect.

After saving before.rules, reload ufw to activate the port redirection:

sudo ufw reload

Verify that you can reach the proxy on TCP 443 with this command:

curl --insecure --resolve www.netresec.com:443:[SecurityOnionIP] https://www.netresec.com/

STEP 8 ☆ Redirect HTTPS traffic to PolarProxy

It's now time to configure a client to run its HTTPS traffic through PolarProxy. Download and install the PolarProxy X.509 root CA certificate from PolarProxy's web service on TCP port 10080:

http://[SecurityOnionIP]:10080/polarproxy.cer

Install the certificate in the operating system and browser, as instructed in the PolarProxy documentation.

You also need to forward packets from the client machine to the Security Onion machine running PolarProxy. This can be done either by configuring a local NAT rule on each monitored client (STEP 8.a) or by configuring the default gateway's firewall to forward HTTPS traffic from all clients to the proxy (STEP 8.b).

STEP 8.a ☆ Local NAT

Use this firewall rule on a Linux client to configure it to forward outgoing HTTPS traffic to the Security Onion machine:

sudo iptables -t nat -A OUTPUT -p tcp --dport 443 -j DNAT --to [SecurityOnionIP]

STEP 8.b ☆ Global NAT Network drawing Firewall, PolarProxy, Clients

If the client isn't running Linux, or if you wanna forward HTTPS traffic from a whole network to the proxy, then apply the following iptables rules to the firewall in front of the client network. See "Routing Option #2" in the PolarProxy documentation for more details.

  1. Add a forward rule on the gateway to allow forwarding traffic to our PolarProxy server:
    sudo iptables -A FORWARD -i eth1 -d [SecurityOnionIP] -p tcp --dport 10443 -m state --state NEW -j ACCEPT
  2. Add a DNAT rule to forward 443 traffic to PolarProxy on port 10443:
    sudo iptables -t nat -A PREROUTING -i eth1 -p tcp --dport 443 -j DNAT --to [SecurityOnionIP]:10443
  3. If the reverse traffic from PolarProxy to the client doesn't pass the firewall (i.e. they are on the same LAN), then we must add this hide-nat rule to fool PolarProxy that we are coming from the firewall:
    sudo iptables -t nat -A POSTROUTING -o eth1 -d [SecurityOnionIP] -p tcp --dport 10443 -j MASQUERADE
For other network configurations, please see the various routing setups in the PolarProxy documentation.

STEP 9 ☆ Inspect traffic in SecurityOnion

Wait for the Elastic stack to initialize, so that the intercepted network traffic becomes available through the Kibana GUI. You can check the status of the elastic initialization with "sudo so-elastic-status".

You should now be able to inspect decrypted traffic in Security Onion using Kibana, Squert, Sguil etc., just as if it was unencrypted HTTP.

Bro HTTP traffic in Kibana Image: Kibana showing HTTP traffic info from decrypted HTTPS sessions

MIME types in Kibana Image: MIME types in Kibana

NIDS alerts in Kibana Image: NIDS alerts from payload in decrypted traffic shown in Kibana

Snort alerts in Squert Image: Snort alerts from decrypted traffic shown in Squert

Security Considerations and Hardening

Security Onion nodes are normally configured to only allow access by SOC/CERT/CSIRT analysts, but the setup described in this blog post requires that "normal" users on the client network can access the PolarProxy service running on the Security Onion node. We therefore recommend installing PolarProxy on a dedicated Security Onion Forward Node, which is configured to only monitor traffic from the proxy.

We also recommend segmenting the client network from the analyst network, for example by using separate network interfaces on the Security Onion machine or putting it in a DMZ. Only the PolarProxy service (TCP 10080 and 10443) should be accessable from the client network.

PolarProxy could be used to pivot from the client network into the analyst network or to access the Apache webserver running on the Security Onion node. For example, the following curl command can be used to access the local Apache server running on the Security Onion machine via PolarProxy:

curl --insecure --connect-to localhost:443:[SecurityOnionIP]:10443 https://localhost/
We therefore recommend adding firewall rules that prevent PolarProxy from accessing the analyst network as well as the local Apache server.

Hardening Steps:

  • Configure the Security Onion node as a Forward Node
  • Segment client network from analyst network
  • Add firewall rules to prevent PolarProxy from accessing services on the local machine and analyst network

For additional info on hardening, please see the recommendations provided by Wes Lambert on the Security-Onion mailing list.

Posted by Erik Hjelmvik on Monday, 20 January 2020 09:40:00 (UTC/GMT)

Tags: #SecurityOnion #Security Onion #PCAP #Bro #PolarProxy #Snort #Suricata #TLS #SSL #HTTPS #tcpreplay #PCAP-over-IP #IDS #NIDS #netcat #curl

More... Share  |  Facebook   Twitter   Reddit   Hacker News Short URL: https://netresec.com/?b=2013af4


Sharing a PCAP with Decrypted HTTPS

Modern malware and botnet C2 protocols use TLS encryption in order to blend in with "normal" web traffic, sometimes even using legitimate services like Twitter or Instagram.

I did a live demo at the CS3Sthlm conference last year, titled "TLS Interception and Decryption", where I showed how TLS interception can be used to decrypt and analyze malicious HTTPS network traffic. During the demo I used DNS-over-HTTPS (DoH) and posted messages to Pastebin and Twitter, pretending to be a malware or malicious actor. The HTTPS network traffic was decrypted and analyzed live as part of my demo. The CS3Sthlm organizers have posted a video recording of the live demo on YouTube.

Erik presenting PolarProxy at CS3Sthlm, photo credit: CS3Sthlm

Image: Erik demoing TLS Interception and Decryption at CS3Sthlm 2019

We are now releasing a PCAP file with the decrypted network traffic captured during this live demo here:

» https://www.netresec.com/files/proxy-191023-091924.pcap «

This blog post provides a step-by-step walk-through of the decrypted HTTPS traffic in the released capture file.

The TLS decryption was performed by connecting a laptop to a custom WiFi access point, which was a Raspberry Pi configured as in our "Raspberry Pi WiFi Access Point with TLS Inspection" blog post. I additionally enabled the PCAP-over-IP feature in PolarProxy by starting it with the "--pcapoverip 57012" option. This allowed me to connect with Wireshark and NetworkMiner to TCP port 57012 on the TLS proxy and stream the decrypted traffic in order to perform live network traffic analysis.

Laptop, Raspberry Pi, PolarProxy, Internet ASCII

Image: Live demo network with Laptop (Browser, NetworkMiner, Wireshark), Raspberry Pi (PolarProxy) and the Internet.

Below follows a breakdown of various significant events of my demo and where you can find these events in the released capture file.

DNS lookup of "www.google.com" using DoH

  • Frame: 833
  • Protocol: DoH using HTTP/2 POST
  • Five tuple: 192.168.4.20:52694 104.16.248.249:80 TCP
DoH lookup of www.google.com shown in NetworkMiner DoH lookup of www.google.com shown in Wireshark

Google search for "tibetan fox psbattle"

  • Frame: 2292
  • Protocol: HTTP/2
  • Five tuple: 192.168.4.20:52716 216.58.211.4:80 TCP
Google search for 'tibetan fox psbattle' in Wireshark Google search for 'tibetan fox psbattle' in NetworkMiner

Tibetan Fox image downloaded from reddit

  • Frame: 3457
  • Protocol: HTTP/2
  • Five tuple: 192.168.4.20:52728 151.101.85.140:80 TCP
Image download from reddit shown in NetworkMiner

Orginal "tibetan fox" image downloaded from this reddit thread.

Tibetan Fox Remix Image HTTP/2 Download

  • Frame: 5805
  • Protocol: HTTP/2
  • Five tuple: 192.168.4.20:52769 151.101.84.193:80 TCP
Images downloaded via HTTP/2

DNS Lookup of "cs3sthlm.se"

  • Frame: 13494
  • Protocol: DoH using HTTP/2 POST
  • Five tuple: 192.168.4.20:52699 104.16.249.249:80 TCP

Images downloaded from CS3Sthlm's website

  • Frame: 14134
  • Protocol: HTTP/1.1
  • Five tuple: 192.168.4.20:52896 192.195.142.160:80 TCP
Images downloaded from CS3Sthlm's website

Data sent in HTTP/2 POST to Pastebin

  • Frame: 18572
  • Protocol: HTTP/2 POST
  • Five tuple: 192.168.4.20:52904 104.22.2.84:80 TCP
Data sent to Pastebin in HTTP/2 POST

The file "post.php.form-data" contains the data sent to Pastebin in the HTTP/2 POST request. Here are the reassembled contents of that file, including the "hello cs3 I am a malware" message:

-----------------------------54168074520069581482009826076
Content-Disposition: form-data; name="csrf_token_post"

MTU3MTgyMjg5OTFwcjBzODJaQ0NuUk9PT1B3ZTl0b20zdFg3ZkhXQ1R4
-----------------------------54168074520069581482009826076
Content-Disposition: form-data; name="submit_hidden"

submit_hidden
-----------------------------54168074520069581482009826076
Content-Disposition: form-data; name="paste_code"

hello cs3 I am a malware
-----------------------------54168074520069581482009826076
Content-Disposition: form-data; name="paste_format"

1
-----------------------------54168074520069581482009826076
Content-Disposition: form-data; name="paste_expire_date"

1H
-----------------------------54168074520069581482009826076
Content-Disposition: form-data; name="paste_private"

0
-----------------------------54168074520069581482009826076
Content-Disposition: form-data; name="paste_name"

malware traffic
-----------------------------54168074520069581482009826076--

Mallory80756920 logs in to Twitter

  • Frame: 24881
  • Protocol: HTTP/2 POST
  • Five tuple: 192.168.4.20:53210 104.244.42.65:80 TCP
Twitter credentials for Mallory80756920

Mallory80756920 posts a Tweet

  • Frame: 26993
  • Protocol: HTTP/2 POST
  • Five tuple: 192.168.4.20:53251 104.244.42.66:80 TCP

Mallory80756920 tweeted "Hello CS3! I'm in you!". The data was sent to twitter using an HTTP/2 POST request.

Twitter post in Wireshark Twitter post in NetworkMiner

Conclusions

A great deal of the interesting TLS traffic in the analyzed capture file is using the HTTP/2 protocol. This doesn't come as a surprise since more than half of all HTTPS traffic is using HTTP/2 nowadays (sources: server protocol statistics, client protocol statistics). It is therefore essential to be able to analyze HTTP/2 traffic if you have a TLS inspection (TLSI) solution in place. Unfortunately many TLSI products don't yet support the HTTP/2 protocol.

Wireshark was one of the first network traffic analysis tools to implement HTTP/2 support, much thanks to Alexis La Goutte. However, Wireshark's excellent "File > Export Objects" doesn't yet support extraction of files from HTTP/2 traffic. There are other ways to extract HTTP/2 file transfers with Wireshark, but they require a few additional steps in order to carve out the file to disk.

Luckily NetworkMiner extracts files from HTTP/2 as of version 2.5. In fact, we believe NetworkMiner is the first open source tool to support automatic HTTP/2 file extraction from PCAP.

Finally, I'd like to stress the point that modern malware use HTTPS, so you need to have a TLSI solution in place to analyze the malicious traffic. As the majority of all HTTPS traffic is using HTTP/2 you also need to ensure that you're able to analyze HTTP/2 traffic passing through your TLSI solution.

Posted by Erik Hjelmvik on Monday, 13 January 2020 12:45:00 (UTC/GMT)

Tags: #HTTP/2 #http2 #DoH #TLS #Google #decrypt #HTTPS #TLSI #TLS Inspection #TLS Interception #PolarProxy #NetworkMiner #Wireshark #CS3Sthlm #CS3 #Forensics #PCAP #Video

More... Share  |  Facebook   Twitter   Reddit   Hacker News Short URL: https://netresec.com/?b=2015d89


Installing a Fake Internet with INetSim and PolarProxy

INetSim + PolarProxy

This is a tutorial on how to set up an environment for dynamic malware analysis, which can be used to analyze otherwise encrypted HTTPS and SMTPS traffic without allowing the malware to connect to the Internet. Dynamic malware analysis (or behavioral analysis) is performed by observing the behavior of a malware while it is running. The victim machine, which executes the malware, is usually a virtual machine that can be rolled back to a clean state when the analysis is complete. The safest way to prevent the malware from infecting other machines, or doing other bad things like sending SPAM or taking part in DDoS attacks, is to run the victim machine in an offline environment. However, network traffic analysis of malware is a central part of dynamic malware analysis, which is is why a “fake Internet” is needed in most malware labs.

INetSim and PolarProxy

INetSim is a software suite that simulates common internet services like HTTP, DNS and SMTP, which useful when analyzing the network behavior of malware samples without connecting them to the Internet. INetSim also has basic support for TLS encrypted protocols, like HTTPS, SMTPS, POP3S and FTPS, but requires a pre-defined X.509-certificate to be loaded at startup. This can cause malware to terminate because the Common Names (CN) in the presented certificates don’t match the requested server names. The victim machine will actually get the exact same certificate regardless of which web site it visits. INetSim’s TLS encryption also inhibits analysis of the network traffic captured in the malware lab, such as C2 traffic or SPAM runs, because the application layer traffic is encrypted. PolarProxy can solve both these issues because it generates certificates on the fly, where the CN value is dynamically set to the requested host name, and saves the network traffic in decrypted form to PCAP files. It is therefore a good idea to replace the TLS services in INetSim with PolarProxy, which will be used as a TLS termination proxy that forwards the decrypted traffic to INetSim’s cleartext services.

Malware Lab Setup

Install Linux

The first step is to install a Linux VM, which will act as a fake Internet to the victim machine(s). I'm using Ubuntu Server 18.04.3 LTS in this tutorial, but you can use any 64-bit linux distro. I'm adding two network interfaces to the Linux VM, one interface with Internet access and one that connects to an isolated offline network to which the victim VM's will be connected. The offline interface is configured to use the static IP 192.168.53.19.

Important: Do not bridge, bond or enable IP forwarding between the two interfaces!

Network connection config Ubuntu Server 18.04

Install INetSim

INetSim is available in Ubuntu's repo, so it is possible to install it with "apt install inetsim". However, I recommend installing INetSim as described in the official documentation to get the latest packaged version of INetSim.

sudo -s

echo "deb http://www.inetsim.org/debian/ binary/" > /etc/apt/sources.list.d/inetsim.list

curl https://www.inetsim.org/inetsim-archive-signing-key.asc | apt-key add -

apt update

apt install inetsim

exit

INetSim listens on 127.0.0.1 by default, change this to INetSim's offline IP address by un-commenting and editing the service_bind_address variable in /etc/inetsim/inetsim.conf.

service_bind_address    192.168.53.19

Also configure INetSim's fake DNS server to resolve all domain names to the IP of INetSim with the dns_default_ip setting:

dns_default_ip    192.168.53.19

Finally, disable the "start_service https" and "start_service smtps" lines, because these services will be replaced with PolarProxy:

start_service dns
start_service http
#start_service https
start_service smtp
#start_service smtps

Restart the INetSim service after changing the config.

sudo systemctl restart inetsim.service

Verify that you can access INetSim's HTTP server with curl:

curl http://192.168.53.19

<html>
  <head>
    <title>INetSim default HTML page</title>
  </head>
  <body>
    <p></p>
    <p align="center">This is the default HTML page for INetSim HTTP server fake mode.</p>
    <p align="center">This file is an HTML document.</p>
  </body>
</html>

It looks like INetSim's web server can be accessed alright.

Install PolarProxy

Next step is to install PolarProxy as a systemd service (as instructed here):

sudo adduser --system --shell /bin/bash proxyuser

sudo mkdir /var/log/PolarProxy

sudo chown proxyuser:root /var/log/PolarProxy/

sudo chmod 0775 /var/log/PolarProxy/

sudo su - proxyuser

mkdir ~/PolarProxy

cd ~/PolarProxy/

curl https://www.netresec.com/?download=PolarProxy | tar -xzvf -

exit

sudo cp /home/proxyuser/PolarProxy/PolarProxy.service /etc/systemd/system/PolarProxy.service

We will need to modify the PolarProxy service config file a bit before we start it. Edit the ExecStart setting in /etc/systemd/system/PolarProxy.service to configure PolarProxy to terminate the TLS encryption for HTTPS and SMTPS (implicitly encrypted email submission). The HTTPS traffic should be redirected to INetSim's web server on tcp/80 and the SMTPS to tcp/25.

ExecStart=/home/proxyuser/PolarProxy/PolarProxy -v -p 10443,80,80 -p 10465,25,25 -x /var/log/PolarProxy/polarproxy.cer -f /var/log/PolarProxy/proxyflows.log -o /var/log/PolarProxy/ --certhttp 10080 --terminate --connect 192.168.53.19 --nosni nosni.inetsim.org

Here's a break-down of the arguments sent to PolarProxy through the ExecStart setting above:

  • -v : verbose output in syslog (not required)
  • -p 10443,80,80 : listen for TLS connections on tcp/10443, save decrypted traffic in PCAP as tcp/80, forward traffic to tcp/80
  • -p 10465,25,25 : listen for TLS connections on tcp/10465, save decrypted traffic in PCAP as tcp/25, forward traffic to tcp/25
  • -x /var/log/PolarProxy/polarproxy.cer : Save certificate to be imported to clients in /var/log/PolarProxy/polarproxy.cer (not required)
  • -f /var/log/PolarProxy/proxyflows.log : Log flow meta data in /var/log/PolarProxy/proxyflows.log (not required)
  • -o /var/log/PolarProxy/ : Save PCAP files with decrypted traffic in /var/log/PolarProxy/
  • --certhttp 10080 : Make the X.509 certificate available to clients over http on tcp/10080
  • --terminate : Run PolarProxy as a TLS termination proxy, i.e. data forwarded from the proxy is decrypted
  • --connect 192.168.53.19 : forward all connections to the IP of INetSim
  • --nosni nosni.inetsim.org : Accept incoming TLS connections without SNI, behave as if server name was "nosni.inetsim.org".

Finally, start the PolarProxy systemd service:

sudo systemctl enable PolarProxy.service

sudo systemctl start PolarProxy.service

Verify that you can reach INetSim through PolarProxy's TLS termination proxy using curl:

curl --insecure --connect-to example.com:443:192.168.53.19:10443 https://example.com

<html>
  <head>
    <title>INetSim default HTML page</title>
  </head>
  <body>
    <p></p>
    <p align="center">This is the default HTML page for INetSim HTTP server fake mode.</p>
    <p align="center">This file is an HTML document.</p>
  </body>
</html>

Yay, it is working! Do the same thing again, but also verify the certificate against PolarProxy's root CA this time. The root certificate is downloaded from PolarProxy via the HTTP service running on tcp/10080 and then converted from DER to PEM format using openssl, so that it can be used with curl's "--cacert" option.

curl http://192.168.53.19:10080/polarproxy.cer > polarproxy.cer

openssl x509 -inform DER -in polarproxy.cer -out polarproxy-pem.crt

curl --cacert polarproxy-pem.crt --connect-to example.com:443:192.168.53.19:10443 https://example.com

<html>
  <head>
    <title>INetSim default HTML page</title>
  </head>
  <body>
    <p></p>
    <p align="center">This is the default HTML page for INetSim HTTP server fake mode.</p>
    <p align="center">This file is an HTML document.</p>
  </body>
</html>

Yay #2!

Now let's set up routing to forward all HTTPS traffic to PolarProxy's service on tcp/10443 and SMTPS traffic to tcp/10465. I'm also adding a firewall rule to redirect ALL other incoming traffic to INetSim, regardless of which IP it is destined to, with the final REDIRECT rule. Make sure to replace "enp0s8" with the name of your interface.

sudo iptables -t nat -A PREROUTING -i enp0s8 -p tcp --dport 443 -j REDIRECT --to 10443

sudo iptables -t nat -A PREROUTING -i enp0s8 -p tcp --dport 465 -j REDIRECT --to 10465

sudo iptables -t nat -A PREROUTING -i enp0s8 -j REDIRECT

Verify that the iptables port redirection rule is working from another machine connected to the offline 192.168.53.0/24 network:

curl --insecure --resolve example.com:443:192.168.53.19 https://example.com

<html>
  <head>
    <title>INetSim default HTML page</title>
  </head>
  <body>
    <p></p>
    <p align="center">This is the default HTML page for INetSim HTTP server fake mode.</p>
    <p align="center">This file is an HTML document.</p>
  </body>
</html>

Yay #3!

curl --insecure --resolve example.com:465:192.168.53.19 smtps://example.com

214-Commands supported:
214- HELO MAIL RCPT DATA
214- RSET NOOP QUIT EXPN
214- HELP VRFY EHLO AUTH
214- ETRN STARTTLS
214 For more info use "HELP <topic>".

Yay #4!

It is now time to save the firewall rules, so that they will survive reboots.

sudo apt-get install iptables-persistent

Install the Victim Windows PC

Configure a static IP address on the victim Windows host by manually setting the IP address. Set the INetSim machine (192.168.53.19) as the default gateway and DNS server.

Windows IPv4 Properties

Download the X.509 root CA certificate from your PolarProxy installation here: http://192.168.53.19:10080/polarproxy.cer

  1. Double-click on "polarproxy.cer"
  2. Click [Install Certificate...]
  3. Select 🔘 Local Machine and press [Next]
  4. Select 🔘 Place all certificates in the following store and press [Browse...]
  5. Choose "Trusted Root Certification Authorities" and press [OK], then [Next]
  6. Press [Finish]

You might also want to install the PolarProxy certificate in your browser. This is how you install it to Firefox:

  1. Options / Preferences
  2. Press [Privacy & Security]
  3. Scroll down to "Certificates" and press [View Certificates...]
  4. In the "Authorities" tab, press [Import...]
  5. Open "polarproxy.cer"
  6. ☑ Trust this CA to identify websites. (check the box)
  7. Press [OK]

Now, open a browser and try visiting some websites over HTTP or HTTPS. If you get the following message regardless of what domain you try to visit, then you've managed to set everything up correctly:

This is the default HTML page for INetSim HTTP server fake mode.

This file is an HTML document.

Accessing the Decrypted Traffic

PCAP files with decrypted HTTPS and SMTPS traffic are now available in /var/log/PolarProxy/

PolarProxy will start writing to a new capture file every 60 minutes. However, the captured packets are not written to disk instantly because PolarProxy uses buffered file writing in order to improve performance. You can restart the proxy service if you wish to flush the buffered packets to disk and have PolarProxy rotate to a new capture file.

sudo systemctl restart PolarProxy

I also recommend capturing all network traffic sent to INetSim with a sniffer like netsniff-ng. This way you’ll get PCAP files with traffic from INetSim’s cleartext services (like DNS and HTTP) as well.

PCAP or it didn’t happen!

Credits

I'd like to thank Thomas Hungenberg and Patrick Desnoyers for providing valuable feedback for this blog post!

Posted by Erik Hjelmvik on Monday, 09 December 2019 08:40:00 (UTC/GMT)

Tags: #PolarProxy #HTTPS #SMTPS #HTTP #SMTP #DNS #Malware #TLS #PCAP #tutorial

More... Share  |  Facebook   Twitter   Reddit   Hacker News Short URL: https://netresec.com/?b=19Ce12f


NetworkMiner 2.5 Released

NetworkMiner 2.5

I am happy to announce the release of NetworkMiner 2.5 today! This new version includes new features like JA3 and parsers for the HTTP/2 and DoH protocols. We have also added support for a few older protocols that are still widely used, such as Kerberos and the CIFS browser protocol. Additionally, NetworkMiner can now parse PCAP files up to twice as fast as before!

Improving Passive TLS Analysis with JA3

Almost all web traffic is TLS encrypted nowadays, which prevents incident responders, analysts and investigators from inspecting otherwise unencrypted HTTP traffic for clues about malicious behavior or criminal intent. This requires analysts to use alternative approaches, such as looking at hostnames and X.509 certificates. This type of analysis is supported by NetworkMiner, since it parses Server Name Indication fields in client TLS handshakes and extracts X.509 certificates automatically when PCAP files are loaded.

In this release we’ve also added support for another passive TLS analysis technique called JA3, which is a method for fingerprinting TLS client implementations.

NetworkMiner leverages the JA3 fingerprint database from Trisul Network Analytics in order to match observed JA3 hashes to hashes of known malware and “normal” applications. This is what it looks like when the capture file “snort.log.1428364808”, from the FIRST 2015 “Hands-on Network Forensics” training (available here), has been loaded into NetworkMiner 2.5:

JA3 fingerprint of a Skype client

Image: JA3 fingerprint of a Skype client in NetworkMiner 2.5

The JA3 hash is also available in the “Parameters” tab, which is useful in order to find out what hosts that particular TLS implementation was reaching out to.

Filtering on JA3 hash 06207a1730b5deeb207b0556e102ded2 in NetworkMiner 2.5

Image: Filtering on JA3 hash 06207a1730b5deeb207b0556e102ded2

HTTP/2 and DoH Support

Passive analysis of TLS traffic, such as HTTPS, often doesn’t give sufficient visibility. Many organizations therefore use TLS proxies in order to decrypt the traffic going in and out from their networks. However, more than half of all HTTPS traffic is actually http2 (RFC 7540) nowadays. This has previously been an issue for users who wanted to analyze decrypted http2 traffic from their TLS intercepting proxies with NetworkMiner. We’re happy to announce that NetworkMiner now can parse http2 traffic, that has been decrypted by a TLS proxy, and extract files from the http2 transfers.

NetworkMiner 2.5 also supports the DNS over HTTPS (DoH) protocol (RFC 8484), which is a technique for sending DNS queries as http2 POST requests and parsing the returned data as DNS responses. We’ve incorporated the DoH data into NetworkMiner’s DNS tab, so that you can analyze it just like normal DNS traffic.

DoH traffic to mozilla.cloudflare-dns.com in NetworkMiner’s DNS tab

Image: DoH traffic to mozilla.cloudflare-dns.com in NetworkMiner’s DNS tab

Please note that NetworkMiner 2.5 does not perform TLS decryption. This means that NetworkMiner can only parse the contents of a TLS stream if it has been decrypted by a TLS proxy, such as PolarProxy.

Extracting Kerberos Hashes from PCAP

NetworkMiner’s support for the Kerberos protocol allows you to passively track which user accounts that are authenticating to what services, simply by monitoring network traffic. This is a feature is essential in order to track credential theft and lateral movement by adversaries inside your networks. After implementing kerberos username and hash extraction we realized that this feature could also be valuable for penetration testers. We therefore decided to present extracted Kerberos credentials in a format that is compatible with tools like hashcat and John the Ripper.

Kerberos krb5pa, krb5asrep and krb5tgs credentials extracted from the Wireshark sample capture file

Image: Kerberos krb5pa, krb5asrep and krb5tgs credentials extracted from the Wireshark sample capture file Krb-contrained-delegation.cap

For more information about Kerberos hashes, please see our Extracting Kerberos Credentials from PCAP blog post.

Even more NetBIOS and CIFS Artifacts!

NetworkMiner is a popular tool for extracting files transferred over SMB and SMB2 from capture files. It can also extract a great deal of information about the communicating hosts from protocols like NetBIOS and SMB/CIFS, but earlier this year Chris Raiter notified us about an important piece of information that was missing in NetworkMiner: NetBIOS Name Service (NBNS) lookups and responses!

Detection and export of NBNS packets request on twitter

A couple of months later Dan Gunter sent us another great feature request for another protocol that runs on top of NetBIOS: the CIFS Browser Protocol (aka MS-BRWS).

We’re happy to announce that NBNS queries and responses are now shown in NetworkMiner’s Parameters tab, and details like hostnames, domain names, Windows versions and uptime us extracted from the MS-BRWS protocol. See the screenshots below, which were created by loading the capture file “case09.pcap” from Richard Bejtlich’s TCP/IP Weapons School 2.0 Sample Lab into NetworkMiner 2.5. Thanks for sharing Richard!

Hostname, domain and Windows version extracted from MS-BRWS traffic

Image: Hostname, domain and Windows version extracted from MS-BRWS traffic

NBNS queries and responses in NetworkMiner’s Parameters tab

Image: NBNS queries and responses in NetworkMiner’s Parameters tab

Mono 5 Required for Linux and MacOS

Linux and MacOS users, who run NetworkMiner with help of Mono, will need to ensure they have Mono 5 (or later) installed in order to run NetworkMiner 2.5. We recommend using at least Mono 5.18.

Instructions for installing NetworkMiner on Linux can be found in our blog post ”HowTo install NetworkMiner in Ubuntu Fedora and Arch Linux”.

MacOS users can refer to our “Running NetworkMiner on Mac OS X” blog post.

Users who are unable to install Mono 5 are recommended to use the old NetworkMiner 2.4 release, which can be downloaded here: https://www.netresec.com/?download=NetworkMiner_2-4

NetworkMiner Professional

Apart from the features mentioned so far, our commercial tool NetworkMiner Professional now comes with a few additional new features. One of these features is port independent identification of RDP traffic, so that mstshash credentials can be extracted from RDP sessions even if the service doesn’t run on port 3389. The OSINT lookup context menus in NetworkMiner Professional have also been enriched with the following online services:

Several new features have also been included in the command line tool NetworkMinerCLI, including:

  • Recursive loading of PCAP files with the "-R” switch.
  • Configurable export types (hosts, files, DNS etc) with the “-x” switch.
  • Relative paths in CSV, XML and JSON/CASE exports unless the “-- absolutePaths” switch is used.

Credits

I’d like to thank Dan Gunter, Chris Raiter, Chris Sistrunk and a few more (who I cannot mention here) for contributing with feature requests and bug reports that have helped improve NetworkMiner.

Upgrading to Version 2.5

Users who have purchased a license for NetworkMiner Professional 2.x can download a free update to version 2.5 from our customer portal, or use the “Help > Check for Updates” feature. Those who instead prefer to use the free and open source version can grab the latest version of NetworkMiner from the official NetworkMiner page.

Posted by Erik Hjelmvik on Thursday, 07 November 2019 11:45:00 (UTC/GMT)

Tags: #NetworkMiner #HTTP/2 #http2 #DoH #Kerberos #NetBIOS #PCAP #hashcat #John #NetworkMinerCLI #OSINT

More... Share  |  Facebook   Twitter   Reddit   Hacker News Short URL: https://netresec.com/?b=19B4a1a


Raspberry PI WiFi Access Point with TLS Inspection

This is a how-to guide for setting up a Raspberry Pi as a WiFi Access Point, which acts as a transparent TLS proxy and saves the decrypted traffic in PCAP files.

Raspberry Pi 4 Model B running PolarProxy
Image: Raspberry Pi 4 Model B running PolarProxy

Step 1: Install PolarProxy for Linux ARM

We will start with installing PolarProxy, which will be used for the TLS decryption and re-encryption. The steps are almost identical to those in the official PolarProxy installation guide, except here we will download the "linux-arm" build of PolarProxy instead of the x64 version.

sudo adduser --system --shell /bin/bash proxyuser
sudo mkdir /var/log/PolarProxy
sudo chown proxyuser:root /var/log/PolarProxy/
sudo chmod 0775 /var/log/PolarProxy/
sudo su - proxyuser
mkdir ~/PolarProxy
cd ~/PolarProxy/
curl https://www.netresec.com/?download=PolarProxy_linux-arm | tar -xzf -
exit
sudo cp /home/proxyuser/PolarProxy/PolarProxy.service /etc/systemd/system/PolarProxy.service
sudo systemctl enable PolarProxy.service
sudo systemctl start PolarProxy.service

Note: The installation instructions above will fail on 64-bit ARM Linux OS's since the downloaded PolarProxy tarball is compiled for 32-bit ARM Linux. Luckily, we do have a linux-arm64 build as well, which is available here:
https://www.netresec.com/?download=PolarProxy_linux-arm64

Verify that the PolarProxy service is running as expected with these commands:

systemctl status PolarProxy.service
journalctl -t PolarProxy

Step 2: Set up your Pi as a WiFi AP

The Raspberry Pi Foundation have a great guide for "Setting up a Raspberry Pi as a Wireless Access Point". Follow the instructions in their guide for the NAT mode setup (first section), but replace the iptables config with this:

sudo iptables -t nat -A POSTROUTING -o eth0 -j MASQUERADE
sudo iptables -A INPUT -i wlan0 -p tcp --dport 10443 -m state --state NEW -j ACCEPT
sudo iptables -t nat -A PREROUTING -i wlan0 -p tcp --dport 443 -j REDIRECT --to 10443
Then save the iptables rules with:
sudo sh -c "iptables-save > /etc/iptables.ipv4.nat"
Finally, edit /etc/rc.local and add this iptables-restore command just above "exit 0" to install the rules on boot.
iptables-restore < /etc/iptables.ipv4.nat

Step 3: Configure the Clients

The final step is to connect the clients (phones, tablets or computers) to the Raspberry Pi WiFi Access Point and install the root CA from PolarProxy.

Follow the instructions for "Trusting the PolarProxy root CA" in the official PolarProxy setup guide to install the public certificate from the TLS proxy in your clients. The certificate can be downloaded from the Raspberry Pi by browsing to http://192.168.4.1:10080/polarproxy.cer.

PCAP PCAP PCAP

Your Raspberry Pi WiFi AP will now intercept all HTTPS traffic going to tcp/443 and save the decrypted traffic in PCAP files, one per hour. The PCAP files with decrypted TLS traffic can be found in the /var/log/PolarProxy/ directory of your Raspberry Pi.

pi@raspberrypi:/var/log/PolarProxy $ ls *.pcap
proxy-190925-075704.pcap proxy-190925-152902.pcap
proxy-190925-085704.pcap proxy-190925-162902.pcap
proxy-190925-095704.pcap proxy-190925-172902.pcap
proxy-190925-105704.pcap proxy-190925-182902.pcap
proxy-190925-115704.pcap proxy-190926-062902.pcap
proxy-190925-125704.pcap proxy-190926-072902.pcap
proxy-190925-132704.pcap proxy-190926-082902.pcap
proxy-190925-132902.pcap proxy-190926-092902.pcap
proxy-190925-142902.pcap proxy-190926-102902.pcap

HTTP/2 traffic to Facebook opened in Wireshark
Image: Decrypted HTTP/2 traffic to Facebook opened in Wireshark

Posted by Erik Hjelmvik on Thursday, 26 September 2019 11:37:00 (UTC/GMT)

Tags: #PolarProxy #PCAP #WiFi #TLS #SSL #HTTPS #Wireshark #http2

More... Share  |  Facebook   Twitter   Reddit   Hacker News Short URL: https://netresec.com/?b=199dc9a


Packet Injection Attacks in the Wild

I have previously blogged about packet injection attacks, such as the Chinese DDoS of GitHub and Covert Man-on-the-Side Attacks. However, this time I've decided to share some intelligence on real-world packet injection attacks that have been running for several months and that are still active today.


Packet Injection by Network Operators

Gabi Nakibly, Jaime Schcolnik and Yossi Rubin recently released a very interesting research paper titled “Website-Targeted False Content Injection by Network Operators”, where they analyzed packet injection attacks in the wild. Here's a snippet from the paper's abstract:

It is known that some network operators inject false content into users’ network traffic. Yet all previous works that investigate this practice focus on edge ISPs (Internet Service Providers), namely, those that provide Internet access to end users. Edge ISPs that inject false content affect their customers only. However, in this work we show that not only edge ISPs may inject false content, but also core network operators. These operators can potentially alter the traffic of all Internet users who visit predetermined websites.

The researchers analyzed 1.4 petabits of HTTP traffic, captured at four different locations; three universities and one corporation. Some of their findings have been made available as anonymized PCAP files here:
http://www.cs.technion.ac.il/~gnakibly/TCPInjections/samples.zip

We have attempted to recreate these packet injections by visiting the same URLs again. Unfortunately most of our attempts didn't generate any injected responses, but we did manage to trigger injections for two of the groups listed by Nakibly et al. (“hao” and “GPWA”).


Redirect Race between hao.360.cn and hao123.com

We managed to get very reliable packet injections when visiting the website www.02995.com. We have decided to share one such PCAP file containing a packet injection attack here:
https://www.netresec.com/files/hao123-com_packet-injection.pcap

This is what it looks like when loading that PCAP file into CapLoader and doing a “Flow Transcript” on the first TCP session:

CapLoader Flow Transcript of race between hao.360.cn and hao123.com
Image: CapLoader Flow Transcript (looks a bit like Wireshark's Follow-TCP-Stream)

We can see in the screenshot above that the client requests http://www.02995.com/ and receives two different responses with the same sequence number (3820080905):

  • The first response is a “302 Found”, forwarding the client to:
    http://www.hao123.com/?tn=93803173_s_hao_pg
  • The second response is a “302 Moved Temporarily”, that attempts a redirect to:
    http://hao.360.cn/?src=lm&ls=n4a2f6f3a91

Judging from the IP Time-To-Live (TTL) values we assume that the first response (hao123.com) was an injected packet, while the second response (hao.360.cn) was coming from the real webserver for www.02995.com.

If you have an eye for details, then you might notice that the injected packet doesn't use the standard CR-LF (0x0d 0x0a) line breaks in the HTTP response. The injected packet only uses LF (0x0a) as line feed in the HTTP header.

Since the injected response arrived before the real response the client followed the injected redirect to www.hao123.com. This is what the browser showed after trying to load www.02995.com:

Browser showing www.hao123.com when trying to visit www.02995.com

SSL encryption is an effective protection against packet injection attacks. So if the user instead enters https://www.02995.com then the browser follows the real redirect to hao.360.cn

Browser showing hao.360.cn when using SSL to visit www.02995.com


id1.cn redirected to batit.aliyun.com

Prior to the release of Gabi's packet injection paper, the only publicly available PCAP file showing a real-world packet injection was this one:
https://github.com/fox-it/quantuminsert/blob/master/presentations/brocon2015/pcaps/id1.cn-inject.pcap

That PCAP file was released after Yun Zheng Hu (of Fox-IT) gave a presentation titled “Detecting Quantum Insert” at BroCon 2015. A video recording of Yun Zheng's talk is available online, including a live demo of the packet injection.

We have managed to re-trigger this packet injection attack as well, simply by visiting http://id1.cn. Doing so triggers two injected HTTP responses that attempts to do a redirect to http://batit.aliyun.com/alww.html. The target page of the injected responses has a message from the Alibaba Group (aliyun.com) saying that the page has been blocked.

Website blocked message from Alibaba Group

We have decided to also share a PCAP file containing a packet injection attack for id1.cn here:
https://www.netresec.com/files/id1-cn_packet-injection.pcap

This is what it looks like when that PCAP file is loaded into NetworkMiner Professional, and the Browsers tab is opened in order to analyze the various HTTP redirections:

Browsers tab in NeworkMiner Professional 2.0
Image: Browsers tab in NetworkMiner Professional 2.0

Here's a short recap of what is happening in our shared PCAP file for id1.cn:

  • Frame 13 : http://id1.cn is opened
  • Frame 18 : Real server responds with an HTML refresh leading to http://id1.cn/rd.s/Btc5n4unOP4UrIfE?url=http://id1.cn/
  • Frame 20 : The client also receives two injected packets trying to do a “403 Forbidden” that redirects to http://batit.aliyun.com/alww.html. However, these injected packets arrived too late.
  • Frame 24 : The client proceeds by loading http://id1.cn/rd.s/Btc5n4unOP4UrIfE?url=http://id1.cn/
  • Frame 25 : Two new injected responses are sent, this time successfully redirecting the client to the Alibaba page.
  • Frame 28 : The real response arrives too late.
  • Frame 43 : The client opens the Alibaba page with message about the site being blocked


Protecting against Packet Injection Attacks

The best way to protect against TCP packet injection attacks is to use SSL encryption. Relying on HTTP websites to do a redirect to an HTTPS url isn't enough, since that redirect could be targeted by packet injection. So make sure to actually type “https://” (or use a browser plug-in) in order to avoid being affected by injected TCP packets.


Referenced Capture Files

The following PCAP files have been referenced in this blog post:

For more PCAP files, please visit our list of publicly available PCAP files here: https://www.netresec.com/?page=PcapFiles

Posted by Erik Hjelmvik on Tuesday, 01 March 2016 13:37:00 (UTC/GMT)

Tags: #NetworkMiner #HTTP #browser #CapLoader #MOTS #HTTPS #TCP #PCAP

More... Share  |  Facebook   Twitter   Reddit   Hacker News Short URL: https://netresec.com/?b=163e02b


Detecting TOR Communication in Network Traffic

The anonymity network Tor is often misused by hackers and criminals in order to remotely control hacked computers. In this blog post we explain why Tor is so well suited for such malicious purposes, but also how incident responders can detect Tor traffic in their networks.

Yellow onions with cross section. Photo taken by Andrew c

The privacy network Tor (originally short for The Onion Router) is often used by activists and whistleblowers, who wish to preserve their anonymity online. Tor is also used by citizens of countries with censored Internet (like in China, Saudi Arabia and Belarus), in order to evade the online censorship and surveillance systems. Authorities in repressive regimes are therefore actively trying to detect and block Tor traffic, which makes research on Tor protocol detection a sensitive subject.

Tor is, however, not only used for good; a great deal of the traffic in the Tor networks is in fact port scans, hacking attempts, exfiltration of stolen data and other forms of online criminality. Additionally, in December last year researchers at Rapid7 revealed a botnet called “SkyNet” that used Tor for its Command-and-Control (C2) communication. Here is what they wrote about the choice of running the C2 over Tor:

"Common botnets generally host their Command & Control (C&C) infrastructure on hacked, bought or rented servers, possibly registering domains to resolve the IP addresses of their servers. This approach exposes the botnet from being taken down or hijacked. The security industry generally will try to take the C&C servers offline and/or takeover the associated domains
[...]
What the Skynet botnet creator realized, is that he could build a much stronger infrastructure at no cost just by utilizing Tor as the internal communication protocol, and by using the Hidden Services functionality that Tor provides."


Tor disguised as HTTPS

Tor doesn't just provide encryption, it is also designed to look like normal HTTPS traffic. This makes Tor channels blend in quite well with normal web surfing traffic, which makes Tor communication difficult to identify even for experienced incident responders. As an example, here is how tshark interprets a Tor session to port TCP 443:

$ tshark -nr tbot_2E1814CCCF0.218EB916.pcap | head
1 0.000000 172.16.253.130 -> 86.59.21.38 TCP 62 1565 > 443 [SYN] Seq=0 Win=64240 Len=0 MSS=1460 SACK_PERM=1
2 0.126186 86.59.21.38 -> 172.16.253.130 TCP 60 443 > 1565 [SYN, ACK] Seq=0 Ack=1 Win=64240 Len=0 MSS=1460
3 0.126212 172.16.253.130 -> 86.59.21.38 TCP 54 1565 > 443 [ACK] Seq=1 Ack=1 Win=64240 Len=0
4 0.127964 172.16.253.130 -> 86.59.21.38 SSL 256 Client Hello
5 0.128304 86.59.21.38 -> 172.16.253.130 TCP 60 443 > 1565 [ACK] Seq=1 Ack=203 Win=64240 Len=0
6 0.253035 86.59.21.38 -> 172.16.253.130 TLSv1 990 Server Hello, Certificate, Server Key Exchange, Server Hello Done
7 0.259231 172.16.253.130 -> 86.59.21.38 TLSv1 252 Client Key Exchange, Change Cipher Spec, Encrypted Handshake Message
8 0.259408 86.59.21.38 -> 172.16.253.130 TCP 60 443 > 1565 [ACK] Seq=937 Ack=401 Win=64240 Len=0
9 0.379712 86.59.21.38 -> 172.16.253.130 TLSv1 113 Change Cipher Spec, Encrypted Handshake Message
10 0.380009 172.16.253.130 -> 86.59.21.38 TLSv1 251 Encrypted Handshake Message

A Tor session to TCP port 443, decoded by tshark as if it was HTTPS

The thsark output above looks no different from when a real HTTPS session is being analyzed. So in order to detect Tor traffic one will need to apply some sort of traffic classification or application identification. However, most implementations for protocol identification rely on either port number inspection or protocol specification validation. But Tor often communicate over TCP 443 and it also follows the TLS protocol spec (RFC 2246), because of this most products for intrusion detection and deep packet inspection actually fail at identifying Tor traffic. A successful method for detecting Tor traffic is to instead utilize statistical analysis of the communication protocol in order to tell different SSL implementations apart. One of the very few tools that has support for protocol identification via statistical analysis is CapLoader.

CapLoader provides the ability to differentiate between different types of SSL traffic without relying on port numbers. This means that Tor sessions can easily be identified in a network full of HTTPS traffic.


Analyzing the tbot PCAPs from Contagio

@snowfl0w provides some nice analysis of the SkyNet botnet (a.k.a. Trojan.Tbot) at the Contagio malware dump, where she also provides PCAP files with the network traffic generated by the botnet.

The following six PCAP files are provided via Contagio:

  1. tbot_191B26BAFDF58397088C88A1B3BAC5A6.pcap (7.55 MB)
  2. tbot_23AAB9C1C462F3FDFDDD98181E963230.pcap (3.24 MB)
  3. tbot_2E1814CCCF0C3BB2CC32E0A0671C0891.pcap (4.08 MB)
  4. tbot_5375FB5E867680FFB8E72D29DB9ABBD5.pcap (5.19 MB)
  5. tbot_A0552D1BC1A4897141CFA56F75C04857.pcap (3.97 MB) [only outgoing packets]
  6. tbot_FC7C3E087789824F34A9309DA2388CE5.pcap (7.43 MB)

Unfortunately the file “tbot_A055[...]” only contains outgoing network traffic. This was likely caused by an incorrect sniffer setup, such as a misconfigured switch monitor port (aka SPAN port) or failure to capture the traffic from both monitor ports on a non-aggregating network tap (we recommend using aggregation taps in order to avoid these types of problems, see our sniffing tutorial for more details). The analysis provided here is therefore based on the other five pcap files provided by Contagio.

Here is a timeline with relative timestamps (the frame timestamps in the provided PCAP files were way of anyway, we noticed an offset of over 2 months!):

  • 0 seconds : Victim boots up and requests an IP via DHCP
  • 5 seconds : Victim perform a DNS query for time.windows.com
  • 6 seconds : Victim gets time via NTP
    ---{malware most likely gets executed here somewhere}---
  • 22 seconds : Victim performs DNS query for checkip.dyndns.org
  • 22 seconds : Victim gets its external IP via an HTTP GET request to checkip.dyndns.org
  • 23 seconds : Victim connects to the Tor network, typically on port TCP 9001 or 443
    ---{lots of Tor traffic from here on}---

This is what it looks like when one of the tbot pcap files has been loaded into CapLoader with the “Identify protocols” feature activated:
CapLoader detecting Tor protocol
CapLoader with protocol detection in action - see “TOR” in the “Sub_Protocol” column

Notice how the flows to TCP ports 80, 9101 and 443 are classified as Tor? The statistical method for protocol detection in CapLoader is so effective that CapLoader actually ignores port numbers altogether when identifying the protocol. The speed with which CapLoader parses PCAP files also enables analysis of very large capture files. A simple way to detect Tor traffic in large volumes of network traffic is therefore to load a capture file into CapLoader (with “Identify protocols” activated), sort the flows on the “Sub_Protocol” column, and scroll down to the flows classified as Tor protocol.


Beware of more Tor backdoors

Most companies and organizations allow traffic on TCP 443 to pass through their firewalls without content inspection. The privacy provided by Tor additionally makes it easy for a botnet herder to control infected machines without risking his identity to be revealed. These two factors make Tor a perfect fit for hackers and online criminals who need to control infected machines remotely.

Here is what Claudio Guarnieri says about the future use of Tor for botnets in his Rapid7 blog post:

“The most important factor is certainly the adoption of Tor as the main communication channel and the use of Hidden Services for protecting the backend infrastructure. While it’s surprising that not more botnets adopt the same design, we can likely expect more to follow the lead in the future.”

Incident responders will therefore need to learn how to detect Tor traffic in their networks, not just in order to deal with insiders or rogue users, but also in order to counter malware using it as part of their command-and-control infrastructure. However, as I've shown in this blog post, telling Tor apart from normal SSL traffic is difficult. But making use of statistical protocol detection, such as the Port Independent Protocol Identification (PIPI) feature provided with CapLoader, is in fact an effective method to detect Tor traffic in your networks.

Posted by Erik Hjelmvik on Saturday, 06 April 2013 20:55:00 (UTC/GMT)

Tags: #CapLoader #TOR #Protocol Identification #SSL #TLS #HTTPS #PCAP #PIPI

More... Share  |  Facebook   Twitter   Reddit   Hacker News Short URL: https://netresec.com/?b=13400BE


Webmail Information Leakage

Switching protocol from unencrypted HTTP to encrypted HTTPS is a good move in order to help the users of a website to protect their privacy online. Many webmail providers have therefore started rolling out their encrypted services during the past few years. Google announced their optional “always use https” setting back in 2008 and also provided some guidance as to why it was important to use HTTPS:

Https keeps your mail encrypted as it travels between your web browser and our servers, so someone sharing your favorite coffee shop's public wifi can't read it.

It took Microsoft Hotmail until November 2010 to announce their optional support for HTTPS encryption, which users could activate by visiting https://account.live.com/ManageSSL.

Adding the option to manually turn on encryption seems to satisfy most people in the security community, probably since it enables us geeks to protect our privacy online through encryption. But the majority of the webmail users online are not aware of the risk of getting their traffic sniffed and do also not know how to turn the encryption feature on. The encryption must therefore be turned on by default in order to protect the broad mass of webmail users.

An open letter written by some well known security profiles, such as Jacob Appelbaum, Richard Clayton, Roger Dingledine, RSnake, Jeff Moss, Ronald L. Rivest and Bruce Schneier, was sent to Google in June 2009. In the letter the authors requested that Google should turn on encryption as part of the default settings:

Rather than forcing users of Gmail, Docs and Calendar to “opt-in” to adequate security, Google should make security and privacy the default.

The letter also mentioned that other competing webmail providers had even worse security since they didn't even provide any “opt-in” encryption at that time:

Google is not the only Web 2.0 firm which leaves its customers vulnerable to data theft and account hijacking. Users of Microsoft Hotmail, Yahoo Mail, Facebook and MySpace are also vulnerable to these attacks. Worst of all – these firms do not offer their customers any form of protection.

But how many of the major webmail providers do actually provide HTTPS as the default protocol today? We will in this blog post look closer at two major webmail services online: Gmail and Hotmail.

GMAIL

Just like most other webmail services Google's Gmail use HTTPS to encrypt the username and password while logging in. But Gmail now also provide encryption by default also after the user has logged in. This prevents hackers as well as investigators/analysts from extracting sent emails by sniffing network traffic.

Hackers have on the other hand been able to take over other users' logged in Gmail sessions for some time by sniffing the GX cookie and using it to fool Gmail that they are logged into the victim's user account. Google have now mitigated this issue by adding encryption and setting the GX cookie to “Secure connections only”, which means it will only be sent in HTTPS sessions.

There is, however, another cookie parameter used by Gmail that is allowed to be sent across an unencrypted HTTP session. This cookie is called “gmailchat” and is typically submitted when visiting http://mail.google.com/mail. This cookie parameter is picked up by NetworkMiner and displayed on both the Credentials tab as well as the Parameters tab.

gmailchat parameter

The client IP address, login time and Gmail account of a gmailchat cookie can be used as evidence by an analyst in order to determine which person that was using a particular computer at a particular time.

HOTMAIL

The security in Hotmail is much worse than that of Gmail. With default settings only the login is protected with encryption, everything after that is sent in cleartext HTTP. This makes it possible to extract emails sent with Hotmail just by passively sniffing the network traffic from a logged in Hotmail user. In our recent “TCP/IP Weapons School” blog post we showed how NetworkMiner displays extracted emails in the Messages tab, this feature works just as well also with Hotmail traffic. By loading a pcap with Hotmail web traffic into NetworkMiner you would get something like this:

Extracted Hotmail message

You can also use the Parameters tab to look for parameter names “fFrom”, “fTo”, “fSubject” and “fMessageBody” and thereby manually extract who sent and received the email as well as read the subject and message of the email. These parameters are all sent in an unencrypted HTTP POST to mail.live.com.

So if you're using Hotmail or Windows Live Mail, make sure to visit https://account.live.com/ManageSSL and enable the encryption functionality!

Posted by Erik Hjelmvik on Saturday, 12 February 2011 15:01:00 (UTC/GMT)

Tags: #Hotmail #SSL #HTTPS #NetworkMiner

More... Share  |  Facebook   Twitter   Reddit   Hacker News Short URL: https://netresec.com/?b=112B58E

twitter

NETRESEC on Twitter

Follow @netresec on twitter:
» twitter.com/netresec


book

Recommended Books

» The Practice of Network Security Monitoring, Richard Bejtlich (2013)

» Applied Network Security Monitoring, Chris Sanders and Jason Smith (2013)

» Network Forensics, Sherri Davidoff and Jonathan Ham (2012)

» The Tao of Network Security Monitoring, Richard Bejtlich (2004)

» Practical Packet Analysis, Chris Sanders (2017)

» Windows Forensic Analysis, Harlan Carvey (2009)

» TCP/IP Illustrated, Volume 1, Kevin Fall and Richard Stevens (2011)

» Industrial Network Security, Eric D. Knapp and Joel Langill (2014)