NETRESEC Network Security Blog - Tag : NetworkMiner

rss Google News

Hunting for C2 Traffic

In this video I look for C2 traffic by doing something I call Rinse-Repeat Threat Hunting, which is a method for removing "normal" traffic in order to look closer at what isn't normal.

The video was recorded in a Windows Sandbox in order to avoid accidentally infecting my Windows PC with malware.

The PCAP files analyzed in the video are:

Thank you for sharing these capture files Brad!

IOC List

  • QBot source: 23.29.125.210
  • QBot md5: 2b55988c0d236edd5ea1a631ccd37b76
  • QBot sha1: 033a22c3bb2b0dd1677973e1ae6280e5466e771c
  • QBot sha256: 2d68755335776e3de28fcd1757b7dcc07688b31c37205ce2324d92c2f419c6f0
  • Qbot proxy protocol server: 23.111.114.52:65400
  • QBot C2: 45.46.53.140:2222
  • QBot C2 JA3: 51c64c77e60f3980eea90869b68c58a8
  • QBot C2 JA3S : 7c02dbae662670040c7af9bd15fb7e2f
  • QBot X.509 domain: thdoot.info
  • QBot X.509 thumbprint: 5a8ee4be30bd5da709385940a1a6e386e66c20b6
  • IcedID BackConnect server: 78.31.67.7:443
  • IcedID BackConnect server: 91.238.50.80:8080

References and Links

Posted by Erik Hjelmvik on Friday, 30 September 2022 12:37:00 (UTC/GMT)

Tags: #Threat Hunting#PCAP#CapLoader#NetworkMiner#NetworkMiner Professional#Video#51c64c77e60f3980eea90869b68c58a8#IcedID

Share: Facebook   Twitter   Reddit   Hacker News Short URL: https://netresec.com/?b=2296553


What is PCAP over IP?

PCAP over IP

PCAP-over-IP is a method for reading a PCAP stream, which contains captured network traffic, through a TCP socket instead of reading the packets from a PCAP file.

A simple way to create a PCAP-over-IP server is to simply read a PCAP file into a netcat listener, like this:

nc -l 57012 < sniffed.pcap

The packets in “sniffed.pcap” can then be read remotely using PCAP-over-IP, for example with tshark like this (replace 192.168.1.2 with the IP of the netcat listener):

nc 192.168.1.2 57012 | tshark -r -

But there’s an even simpler way to read PCAP-over-IP with Wireshark and tshark, which doesn’t require netcat.

wireshark -k -i TCP@192.168.1.2:57012
tshark -i TCP@192.168.1.2:57012

The Wireshark name for this input method is “TCP socket” pipe interface, which is available in Linux, Windows and macOS builds of Wireshark as well as tshark.

Live Remote Sniffing

Sniffed traffic can be read remotely over PCAP-over-IP in real-time simply by forwarding a PCAP stream with captured packets to netcat like this:

tcpdump -U -w - not tcp port 57012 | nc -l 57012
dumpcap -P -f "not tcp port 57012" -w - | nc -l 57012
PCAP-over-IP with tcpdump, netcat and tshark

Tcpdump is not available for Windows, but dumpcap is since it is included with Wireshark.

Note how TCP port 57012 is purposely filtered out using BPF when capturing in order to avoid a snowball effect, where the PCAP-over-IP traffic otherwise gets sniffed and re-transmitted through the PCAP-over-IP stream, which again gets sniffed etc.

Reading PCAP-over-IP with NetworkMiner

We added PCAP-over-IP support to NetworkMiner in 2011 as part of NetworkMiner 1.1, which was actually one year before the TCP socket sniffing feature was included in Wireshark.

Live remote sniffing with NetworkMiner 2.7.3 using PCAP-over-IP

Image: Live remote sniffing with NetworkMiner 2.7.3 using PCAP-over-IP

NetworkMiner can also be configured to listen for incoming PCAP-over-IP connections, in which case the sniffer must connect to the machine running NetworkMiner like this:
tcpdump -U -w - not tcp port 57012 | nc 192.168.1.3 57012

This PCAP-over-IP feature is actually the recommended method for doing real-time analysis of live network traffic when running NetworkMiner in Linux or macOS, because NetworkMiner’s regular sniffing methods are not available on those platforms.

Reading Decrypted TLS Traffic from PolarProxy

PolarProxy

One of the most powerful use-cases for PCAP-over-IP is to read decrypted TLS traffic from PolarProxy. When PolarProxy is launched with the argument “--pcapoverip 57012” it starts a listener on TCP port 57012, which listens for incoming connections and pushes a real-time PCAP stream of decrypted TLS traffic to each client that connects. PolarProxy can also make active outgoing PCAP-over-IP connections to a specific IP address and port if the “--pcapoveripconnect <host>:<port>” argument is provided.

In the video PolarProxy in Windows Sandbox I demonstrate how decrypted TLS traffic can be viewed in NetworkMiner in real-time with help of PCAP-over-IP. PolarProxy’s PCAP-over-IP feature can also be used to read decrypted TLS traffic from PolarProxy with Wireshark as well as to send decrypted TLS traffic from PolarProxy to Arkime (aka Moloch).

Replaying PCAP-over-IP to an Interface

There are lots of great network monitoring products and intrusion detection systems that don’t come with a built-in PCAP-over-IP implementation, such as Suricata, Zeek, Security Onion and Packetbeat, just to mention a few. These products would greatly benefit from having access to the decrypted TLS traffic that PolarProxy can provide. Luckily we can use netcat and tcpreplay to replay packets from a PCAP-over-IP stream to a network interface like this:

nc localhost 57012 | tcpreplay -i eth0 -t -

But for permanent installations we recommend creating a dedicated dummy interface, to which the traffic gets replayed and sniffed, and then deploy a systemd service that performs the replay operation. See our blog post Sniffing Decrypted TLS Traffic with Security Onion for an example on how to deploy such a systemd service. In that blog post we show how decrypted TLS traffic from PolarProxy can be replayed to a local interface on a Security Onion machine, which is being monitored by Suricata and Zeek.

Nils Hanke has also compiled a detailed documentation on how decrypted TLS packets from PolarProxy can be replayed to Packetbeat and Suricata with help of tcpreplay.

In these setups netcat and tcpreplay act as a generic glue between a PCAP-over-IP service and tools that can sniff packets on a network interface, but there are a few drawbacks with this approach. One drawback is that tcpreplay requires root privileges in order to replay packets to an interface. Another drawback is that extra complexity is added to the solution and two additional single point of failures are introduced (i.e. netcat and tcpreplay). Finally, replaying packets to a network interface increases the risk of packet drops. We therefore hope to see built-in PCAP-over-IP implementations in more network monitoring solutions in the future!

FAQ for PCAP-over-IP

Q: Why is it called “PCAP-over-IP” and not “PCAP-over-TCP”?

Good question, we actually don’t know since we didn’t come up with the name. But in theory it would probably be feasible to read a PCAP stream over UDP or SCTP as well.

Q: What is the standard port for PCAP-over-IP?

There is no official port registered with IANA for PCAP-over-IP, but we’ve been using TCP 57012 as the default port for PCAP-over-IP since 2011. The Wireshark implementation, on the other hand, uses TCP port 19000 as the default value.

Q: Which software comes with built-in PCAP-over-IP servers or clients?

The ones we know of are: Arkime, NetworkMiner, PolarProxy, tshark and Wireshark.

Q: Is there some way to encrypt the PCAP-over-IP transmissions?

Yes, we recommend encrypting PCAP-over-IP sessions with TLS when they are transmitted across a non-trusted network. NetworkMiner’s PCAP-over-IP implementation comes with a “Use SSL” checkbox, which can be used to receive “PCAP-over-TLS”. You can replace netcat with socat or ncat in order to establish a TLS encrypted connection to NetworkMiner.

Q: Is there a tool that can aggregate multiple PCAP-over-IP streams into one?

No, none that we’re aware of. However, multiple PCAP-over-IP streams can be merged into one by specifying multiple PCAP-over-IP interfaces in dumpcap and then forwarding that output to a netcat listener, like this:

dumpcap -i TCP@10.1.2.3:57012 -i TCP@10.4.5.6:57012 -w - | editcap -F pcap - - | nc -l 57012

Posted by Erik Hjelmvik on Monday, 15 August 2022 08:05:00 (UTC/GMT)

Tags: #PCAP-over-IP#PCAP#tcpdump#Wireshark#tshark#NetworkMiner#PolarProxy#Suricata#Zeek#Arkime#tcpreplay#netcat#ASCII-art

Share: Facebook   Twitter   Reddit   Hacker News Short URL: https://netresec.com/?b=228fddf


NetworkMiner 2.7.3 Released

NetworkMiner 2.7.3

NetworkMiner now extracts meterpreter payloads from reverse shells and performs offline lookups of JA3 hashes and TLS certificates. Our commercial tool, NetworkMiner Professional, additionally comes with a packet carver that extracts network packets from memory dumps.

Extraction of Meterpreter Payloads

NetworkMiner 2.7.3 supports extraction of meterpreter DLL payloads from reverse shell TCP sessions deployed with Metasploit. The free version of NetworkMiner will try to extract the meterpreter DLL from TCP sessions going to "poker-hand ports" commonly used for meterpreter sessions, such as 3333, 4444, 5555, etc. The port-independent protocol detection feature available in NetworkMiner Professional additionally enables extraction of meterpreter DLLs regardless which LPORT the attacker specifies when deploying the reverse shell.

Meterpreter DLL extracted from PCAP file in NetworkMiner Professional

Image: Meterpreter DLL extracted from DFIR Madness' case001.pcap

Packet Carving in NetworkMiner Professional

If you try to open anything other than a PCAP, PcapNG or ETL file in NetworkMiner Professional, then you'll be presented with an option to carve packets from the opened file as of this release.

NetworkMiner Unknown Capture File Format

The packet carver can extract packets from any structured or unstructured data, such as memory dumps and proprietary packet capture formats. NetworkMiner Pro's carver is a simplified version of the packet carving feature in CapLoader.

Loading the 1GB "memdump.mem" from Ali Hadi's Challenge #1 - Web Server Case into NetworkMiner Professional takes roughly five seconds, during which 612 packets get extracted.

NetworkMiner Professional with packets extracted from memory dump

Image: Information about network hosts carved from memory dump

In this scenario the memory was dumped on the 192.168.56.101 host, which NetworkMiner identifies as "WIN-L0ZZQ76PMUF". The carved packets also indicate that this computer had an outgoing TCP connection to 192.168.56.102, which appears to be a Linux machine called "kali". As you can see in the screenshot, the packets carved from the memory dump also reveal a great deal about other hosts on the network, such as the 192.168.56.1 host, which seems to be a Windows 7 machine called "IT104-00".

Offline Matching of JA3 and X.509 hashes

NetworkMiner 2.7.3 comes with a local copy of the SSL Certificate and JA3 Fingerprint Blacklists from the awesome abuse.ch project. JA3 hashes and extracted X.509 certificates are matched against these lists in order to see if they are associated with any piece of malware or botnet.

Here's one example showing the default Cobalt Strike certificate being identified as "AKBuilder C&C", since that's how it is listed in abuse.ch's SSL certificate database.

CobaltStrike default X.509 certificate

Image: Cobalt Strike's default certificate identified as "AKBuilder C&C"
PCAP: Cobalt Strike PCAP from malware-traffic-analysis.net

The port-independent protocol detection feature in NetworkMiner Professional additionally enables X.509 certificates to be extracted even from non-standard TLS ports, such as this certificate, which is identified as "BitRAT" with help of the abuse.ch certificate block-list.

NetworkMiner Professional with BitRAT TLS traffic

Image: Both X.509 certificate and JA3 hash identified as BitRAT
PCAP: BitRAT PCAP from Joe Sandbox

The client's JA3 hash 8515076cbbca9dce33151b798f782456 is also associated with BitRAT according to abuse.ch.

DBSBL Lookup Detection

DNSBL services are used by servers handling incoming email to verify that the sender's IP address isn't a known SPAM sender and that it isn't from a network that shouldn't be sending emails.

But DNSBL services can also be used by malware and botnets, such as TrickBot and Emotet, to verify that the public IP of a victim is allowed to send emails and that it hasn't already been blacklisted for sending SPAM. We have therefore decided to add DNSBL lookups to the Host Details section in NetworkMiner 2.7.3.

DNSBL lookups in NetworkMiner

Image: TrickBot victim checks if its public IP is blocked by DNSBL services
PCAP: TrickBot PCAP from malware-traffic-analysis.net

DNSBL lookups are also logged to the "Parameters" tab of NetworkMiner.

NetworkMiner with DNSBL parameters

Image: NetworkMiner's Parameters tab with "DNSBL" filter
PCAP: TrickBot PCAP from malware-traffic-analysis.net

Additional Features and Updates

We'd also like to mention some additional new features, bug fixes and improvements that have been included in this new release.

  • Support for HTTP CONNECT request method to extract artifacts like X.509 certificates and JA3 hashes from HTTPS traffic passing through a web proxy.
  • Traffic to TCP ports 3000 and 8000 are now configured to be parsed as HTTP by default in order to handle WEBrick traffic.
  • Improved extraction of SMTP credentials.
  • JA3 hashes were previously incorrect for clients that supported more than one EC point format (RFC 8422). This has now been fixed.
  • Support for SLL2 (Linux cooked capture v2) frames.
  • Improved handling of concurrent GUI events, for example when poking around in the "Hosts" tab while loading a PCAP file or doing live sniffing.
  • NetworkMiner's GUI no longer reloads between each PCAP file when multiple files are loaded at once.

New Features in NetworkMiner Professional

We have also added a few new features exclusively to NetworkMiner Professional, which is the commercial version of NetworkMiner. Apart from the packet carver feature, mentioned earlier in this blog post, we've also updated the collection of OSINT lookup services available in the GUI. One of the newly added services is Ryan Benson's unfurl, which picks apart URLs to reveal data that might have been encoded into a complex URL. The unfurl lookup can be found by right-clicking an URL in NetworkMiner Professional's "Browsers" tab and selecting the "Lookup URL" sub menu.

Other OSINT services that we've added are FileScan.IO and JoeSandbox lookups of extracted files. These lookups can be performed by right clicking a file in the "Files" tab and opening the sub-menu called "Lookup Hash".

Lookup of file hash on JoeSandbox

Image: OSINT lookup of an EXE file extracted from network traffic

The command-line version of NetworkMiner Professional, NetworkMinerCLI, has also been updated to allow extracted information to be printed directly on standard output instead of logging everything to files. Here is an example showing this feature while running NetworkMinerCLI in Linux (with help of Mono):

mono /opt/NetworkMinerProfessional_2-7-3/NetworkMinerCLI.exe -r 2022-03-14-Qakbot-with-Cobalt-Strike-and-VNC-module.pcap -w /tmp/malware -X FileInfos | cut -d, -f 5,9
"s2Fmok83x.zip.html","ba2ef33c7aef593f95d261b6f4406b39"
"nexus.officeapps.live.com.cer","373ccffe30d3477867642abab723a351"
"Microsoft RSA TLS CA 01.cer","806f1c72f6d67c9c114eff43d3d84100"
"nexusrules.officeapps.live.c.cer","4c08442740cb020d457a5df16be406ff"
"Microsoft RSA TLS CA 02.cer","65d17ecae5798c79db8e840fe98a53b9"
"6537991.dat.exe","124207bc9c64e20e114bcaeabde12a4e"
"6537991.dat.exe","ca7ef367c935182a40a95b9ad8b95f42"
"6537991.dat.exe","a9a8366fa6be54b45ca04192ca217b75"
[...]

The command above extracts files from a PCAP file, which contains traffic from a Windows PC infected with Qbot. The "-w" switch specifies the output directory for the files extracted from network traffic, and the "-X FileInfos" specifies that metadata for these files should be sent to STDOUT instead of being written to log files. The cut utility was used to show only the filename (column 5) and MD5 hash (column 9) of the file info output.

The MD5 hashes of the extracted files confirm that this is indeed a Qbot infection:

  • 124207bc9c64e20e114bcaeabde12a4e (VT)
  • ca7ef367c935182a40a95b9ad8b95f42 (VT)
  • a9a8366fa6be54b45ca04192ca217b75 (VT)

NetworkMinerCLI previously printed some information about the parsing process to STDOUT. That output has now been moved to STDERR in order to provide the "-X [type]" output with exclusive access to STDOUT.

Credits

We'd like to thank Michael Taggart for noticing that NetworkMiner previously failed to parse HTTP traffic to ports 3000 and 8000.

Upgrading to Version 2.7.3

Users who have purchased NetworkMiner Professional can download a free update to version 2.7.3 from our customer portal, or use the “Help > Check for Updates” feature. Those who instead prefer to use the free and open source version can grab the latest version of NetworkMiner from the official NetworkMiner page.

Posted by Erik Hjelmvik on Monday, 04 April 2022 06:52:00 (UTC/GMT)

Tags: #NetworkMiner#carve#JA3#X.509#CobaltStrike#Cobalt Strike#TrickBot#Emotet#PIPI#Protocol Detection#OSINT#NetworkMinerCLI

Share: Facebook   Twitter   Reddit   Hacker News Short URL: https://netresec.com/?b=22479d5


PolarProxy in Windows Sandbox

In this video I demonstrate how PolarProxy can be run in a Windows Sandbox to intercept and decrypt outgoing TLS communication. This setup can be used to inspect otherwise encrypted traffic from malware or suspicious Windows applications, which communicate over HTTPS or some other TLS encrypted protocol.

The Windows Sandbox WSB file used in the demo can be downloaded from here: https://www.netresec.com/?download=PolarProxySandbox

Note: Windows Pro or Enterprise is required to run WSB files

Parsing Decrypted TLS Traffic with NetworkMiner

This sandbox also includes NetworkMiner, primarily because it can be used to read a real-time PCAP-over-IP stream with decrypted traffic from PolarProxy. As shown in the video, this feature can be used in order to extract files, images or parameters from the decrypted TLS traffic in near real-time.

Images extracted from decrypted HTTP/2 traffic shown in NetworkMiner

For more info about how to run NetworkMiner in Windows Sandbox, please see our blog post Running NetworkMiner in Windows Sandbox.

Configuring a Proxy Server in Windows Sandbox

Windows’ built-in proxy settings are unfortunately not available in Windows Sandbox, which is why I installed a third-party proxy client that redirects all outgoing network traffic to PolarProxy’s SOCKS server. I used Proxifier in the video, which has the additional benefit of being able to redirect all traffic to the proxy, even from applications that aren’t proxy aware. This feature is crucial when attempting to intercept and decrypt TLS traffic from malware that doesn’t respect the proxy settings configured in the operating system.

Command Log

Start PolarProxy with a PCAP-over-IP listener on TCP 57012, SOCKS server on TCP 1080, HTTP proxy on 8080 and a transparent TLS proxy on port 443:

PolarProxy --pcapoverip 57012 -x ..\proxyroot.cer --socks 1080 --httpconnect 8080 --allownontls -p 443,80

Test PolarProxy’s SOCKS server by sending an unencrypted HTTP request through the proxy:

curl --socks4 localhost http://www.netresec.com

Test PolarProxy’s SOCKS server by sending an HTTPS request through the proxy:

curl --insecure --socks4 localhost https://www.netresec.com

Test PolarProxy’s HTTP CONNECT proxy server by sending an HTTPS request through the proxy:

curl --insecure --proxy localhost:8080 https://www.netresec.com

Start Menu Search

As shown in the video, text typed into Windows’ start menu gets sent to Microsoft. For more information about this behavior, and how it can be disabled, check out our Start Menu Search video and blog post.

Posted by Erik Hjelmvik on Monday, 31 January 2022 09:50:00 (UTC/GMT)

Tags: #PolarProxy#NetworkMiner#SOCKS#proxy#Windows Sandbox#Sandbox#PCAP-over-IP#pcapoverip#Windows#TLS#HTTPS

Share: Facebook   Twitter   Reddit   Hacker News Short URL: https://netresec.com/?b=221d46b


Open .ETL Files with NetworkMiner and CapLoader

NetTrace.ETL in CapLoader 1.9.3 and NetworkMiner 2.7.2

Windows event tracing .etl files can now be read by NetworkMiner and CapLoader without having to first convert them to .pcap or .pcapng. The ETL support is included in NetworkMiner 2.7.2 and CapLoader 1.9.3, which were both released this morning.

What is an ETL Trace File?

ETL is short for Event Trace Log, which is ETW session data that has been logged to a file. You can, for example, extract EVTX logs from ETL files. But in this blog post we're gonna focus on network traffic that has been captured to an ETL file with a command like:

netsh trace start capture=yes report=no tracefile=packets.etl
...wait while packets are being captured...
netsh trace stop

Pro-tip: You can specify a capture NIC explicitly with "CaptureInterface=<GUID>"

NetworkMiner and CapLoader can also read packets in Pktmon ETL files, which actually are different from those created with netsh. Capturing packets to an ETL file with Pktmon is very simple:

pktmon start --capture --pkt-size 0 -f packets.etl
...wait while packets are being captured...
pktmon stop

Pro-tip: You can specify capture filters with "pktmon filter add"

You can also capture packets to ETL files with PowerShell:

New-NetEventSession -Name sniffer -LocalFilePath C:\packets.etl
Add-NetEventPacketCaptureProvider -SessionName sniffer -TruncationLength 2000
Start-NetEventSession -Name sniffer
...wait while packets are being captured...
Stop-NetEventSession -Name sniffer
Remove-NetEventSession -Name sniffer

Pro-tip: You capture packets on a remote PC by specifying a CimSession

Advantages

The built-in support for ETL files in NetworkMiner and CapLoader makes it easy to work with ETL files. Not only will you no longer need to go through the extra step of converting the ETL file to PCAP using etl2pcapng or Microsoft Message Analyzer (which was retired in 2019), the analysis will also be faster because both CapLoader and NetworkMiner read ETL files faster compared to etl2pcapng and MMA.

Limitations

The primary limitation with NetworkMiner and CapLoader's ETL support is that it only works in Windows. This means that you will not be able to open ETL files when running NetworkMiner in Linux or macOS.

Another limitation is that both NetworkMiner and CapLoader might fail to parse logged packets if the event trace was created on an OS version with an event manifest that is incompatible with the OS version on which the ETL file is opened.

Under the Hood

Both NetworkMiner and CapLoader leverage Windows specific API calls to read packets from ETL files. An ETL file opened in CapLoader first get converted to PcapNG, then CapLoader parses that PcapNG file. NetworkMiner, on the other hand, parses the packets in the ETL file directly to extract artifacts like files, images and parameters. NetworkMiner's approach is both simpler and quicker, but by converting the ETL file to PcapNG CapLoader can utilize its packet indexing feature to rapidly extract any subset of the captured traffic upon request by the user.

CapLoader's approach is also useful for users who are wondering how to open ETL files in Wireshark, since the packets from an ETL file can be opened in Wireshark by dragging the PcapNG file from the CapLoader GUI onto Wireshark.

Drag-and-drop NetTrace.pcapng from CapLoader to Wireshark
Image: NetTrace.etl converted to PcapNG in CapLoader can be drag-and-dropped onto Wireshark.

Additional Updates in NetworkMiner

The ETL support is not the only new feature in NetworkMiner 2.7.2 though. We have also added support for the ERSPAN protocol. The FTP parser has also been improved to support additional commands, such as AUTH (RFC2228).

We've also added a useful little feature to the context menu of the Parameter's tab, which allows users to send extracted parameters to CyberChef (on gchq.github.io) for decoding.

Submit Parameter value from NetworkMiner to CyberChef
Image: Right-clicking a parameter brings up a context menu with "Submit to CyberChef" option.

Additional Updates in CapLoader

The only major improvement in CapLoader 1.9.3, apart from the built-in ETL-to-PcapNG converter, is that the protocol identification speed and precision has been improved. We've also separated the identification of SSL (version 2.0 to 3.0) and TLS (SSL 3.1 and later) as two separate protocols in this version, whereas they previously both were fingerprinted as "SSL".

Credits

We'd like to thank Dick Svensson and Glenn Larsson for their input on reading ETL files. We also want to thank Markus Schewe for recommending us to add ERSPAN support to NetworkMiner!

Posted by Erik Hjelmvik on Tuesday, 02 November 2021 07:15:00 (UTC/GMT)

Tags: #PowerShell#CapLoader#NetworkMiner#PcapNG#Windows#Wireshark#PCAP#CyberChef

Share: Facebook   Twitter   Reddit   Hacker News Short URL: https://netresec.com/?b=21B0d0e


Start Menu Search Video

In this video I demonstrate that text typed into the Windows 10 start menu gets sent to Microsoft and how that traffic can be intercepted, decrypted and parsed.

What Was Sent?

The XML files shown in the video were sent by Cortana's "SmartSearch" app to https://www.bing.com/threshold/xls.aspx in HTTP/2 POST requests. As shown in the video, the POST'ed keystrokes can be found inside requestInfo XML tags that have a "RawQuery" key.

The following tcpdump and grep commands can be used to list the RawQuery data sent to Bing in these HTTP/2 requests:

tcpdump -A -r proxy-210927-134557.pcap | grep -a -o 'key="RawQuery" value="[^"]*"'

Running that command on the PolarProxy PCAP file from the video gives the following output:

key="RawQuery" value="n"
key="RawQuery" value="no"
key="RawQuery" value="not"
key="RawQuery" value="note"
key="RawQuery" value="notep"
key="RawQuery" value="notepa"
key="RawQuery" value="notepad"
key="RawQuery" value="s"
key="RawQuery" value="se"
key="RawQuery" value="sea"
key="RawQuery" value="sear"
key="RawQuery" value="searc"
key="RawQuery" value="search"
key="RawQuery" value="search .."
key="RawQuery" value="search ..e"
key="RawQuery" value="search ..er"
key="RawQuery" value="search ..e"
key="RawQuery" value="search .."
key="RawQuery" value="search"
key="RawQuery" value="search p"
key="RawQuery" value="search per"
key="RawQuery" value="search perm"
key="RawQuery" value="search permi"
key="RawQuery" value="p"
key="RawQuery" value="pr"
key="RawQuery" value="pri"
key="RawQuery" value="priv"
key="RawQuery" value="priva"
key="RawQuery" value="privac"
key="RawQuery" value="privacy"

The same data also gets sent in the query string variable "qry" of GET requests for https://www.bing.com/AS/API/WindowsCortanaPane/V2/Suggestions, as shown in this NetworkMiner screenshot.

Parameters tab in NetworkMiner
Image: NetworkMiner's Parameters tab with filter "qry" on "Parameter name" column

How to Intercept, Decrypt and Decode HTTPS Traffic

The following section presents the technical details regarding my setup, so that others can reproduce and verify these findings.

My first step was to install PolarProxy on a Linux machine on the local network. PolarProxy is a TLS proxy, which can intercept and decrypt TLS traffic. This TLS proxy is primarily designed to decrypt traffic from malware and hackers, but can also be used to decrypt legitimate traffic when needed.

PolarProxy was configured to listen for incoming TLS connections on TCP port 443 and output PCAP data with the decrypted traffic as if it had been transmitted over TCP 80. The decrypted traffic was accessible as a real-time stream through a PCAP-over-IP service running on port 57012. Here's the full command that was used to start PolarProxy:

sudo ./PolarProxy -p 443,80 --pcapoverip 0.0.0.0:57012 --certhttp 10080

In the video I showed the Windows 10 client's modified hosts file, which included an entry for www.bing.com pointing to the PolarProxy machine. What was not shown in the video though, is that PolarProxy's own CA certificate had been added to the Win10 machine's list of trusted root CA's, as explained in the "Trusting the PolarProxy root CA" section of the PolarProxy installation instructions. With these two changes in place all HTTPS requests for www.bing.com from the Win10 PC got diverted through the PolarProxy TLS inspection service, which then decrypted and re-encrypted the traffic before forwarding it to Bing.

The decrypted Bing requests could be accessed either locally on the Linux machine, or remotely using the PCAP-over-IP service on TCP port 57012. I used NetworkMiner to read the live PCAP stream with decrypted traffic from port 57012 and extract all files being sent and received in real-time.

Is it Possible to Disable the Cortana Search?

When Ars Technica reporters asked Microsoft back in 2015 if there was any way to disable this communication, Microsoft replied with the following statement:

As part of delivering Windows 10 as a service, updates may be delivered to provide ongoing new features to Bing search, such as new visual layouts, styles and search code. No query or search usage data is sent to Microsoft, in accordance with the customer's chosen privacy settings.

There are plenty of how-to guides online with instructions on how the Cortana search feature can be disabled. Most of these guides suggest disabling the AllowCortana setting in group policies or in the registry. We've tried several of the settings suggested in these how-to guides, but none of them seem to prevent Windows from sending keystrokes to Bing.

If you know how to successfully disable Cortana's Bing searches, then please feel free to reach out to us so that we can update this blog post.

UPDATE 210928 - How to Actually Disable Cortana Search

Twitter user @GeorgeProfonde3 reached out to suggest a fix that might prevent the start menu from sending data to Bing. We have now verified this fix and we're happy to announce that it works (at least for us).

  1. Start regedit.exe
  2. Open the following registry key:
    HKEY_CURRENT_USER\SOFTWARE\Microsoft\Windows\CurrentVersion\Search
  3. Ensure that the value for CortanaConsent is set to 0
  4. Create a new DWORD registry entry called "BingSearchEnabled" with value 0

You should no longer see any connections to www.bing.com when interacting with the start menu after implementing this fix.

UPDATE 211015 - Another way to Disable Cortana Search

You may need to use a different method to disable the start meny search, depending on your Windows version and build. Kimberly (@StopMalvertisin) suggested the following method, which seems to work on Windows 11:

  1. Start regedit.exe
  2. Create a registry key for:
    HKEY_CURRENT_USER\SOFTWARE\Policies\Microsoft\Windows\Explorer
  3. Create a new DWORD registry entry called "DisableSearchBoxSuggestions" with value 1

Disabling Start Menu Search from Group Policy

There are also a few different methods for disabling start menu searches using GPO. However, please note that your success will vary depending on your Windows version and build.

GPO Method #1

  1. Start gpedit.msc
  2. Open the following branch:
    User configuration\Administrative templates\Windows components\File Explorer
  3. Enable the following group policy:
    "Turn off display of recent search entries in the File Explorer search box"

GPO Method #2

  • Start gpedit.msc
  • Open the following branch:
    User Configuration\Administrative Templates\Start Menu and Taskbar
  • Enable the following group policy: "Do not search communications"

Posted by Erik Hjelmvik on Tuesday, 28 September 2021 08:24:00 (UTC/GMT)

Tags: #PCAP#NetworkMiner#PolarProxy#Microsoft#video#videotutorial#pcapoverip#PCAP-over-IP#HTTP/2#http2

Share: Facebook   Twitter   Reddit   Hacker News Short URL: https://netresec.com/?b=2199fe7


Walkthrough of DFIR Madness PCAP

I recently came across a fantastic digital forensics dataset at dfirmadness.com, which was created by James Smith. There is a case called The Stolen Szechuan Sauce on this website that includes forensic artifacts like disk images, memory dumps and a PCAP file (well, pcap-ng actually). In this video I demonstrate how I analyzed the capture file case001.pcap from this case.

Follow Along in the Analysis

Please feel free to follow along in the analysis performed in the video. You should be able to use the free trial version of CapLoader and the free open source version of NetworkMiner to perform most of the tasks I did in the video.

Here are some of the BPF and Column Criteria filters that I used in the video, so that you can copy/paste them into CapLoader.

  • net 10.0.0.0/8
  • Umbrella_Domain =
  • not ip6 and not net 224.0.0.0/4
  • host 194.61.24.102 or host 203.78.103.109 or port 3389

ASCII Network Flow Chart

References and Links

Timeline

All events in this timeline take place on September 19, 2020. Timestamps are in UTC.

  • 02:19:26 194.61.24.102 performs RDP brute force password attack against DC01.
  • 02:21:47 RDP password brute force successful.
  • 02:22:08 194.61.24.102 connects to DC01's RDP service as Administrator. Duration: 9 sec.
  • 02:22:36 194.61.24.102 connects to DC01's RDP service as Administrator again. Duration: 30 min.
  • 02:24:06 DC01 downloads coreupdater.exe from 194.61.24.102 using IE11.
  • 02:25:18 DC01 establishes Metrepreter reverse_tcp connection to 203.78.103.109. Duration: 4 min.
  • 02:29:49 DC01 re-establishes Metrepreter reverse_tcp connection to 203.78.103.109. Duration: 23 min.
  • 02:35:55 DC01 connects to DESKTOP's RDP service Administrator (username in Kerberos traffic). Duration 16 min.
  • 02:39:58 DESKTOP download coreupdater.exe from 194.61.24.102 using MS Edge.
  • 02:40:49 DESKTOP establishes Metrepreter reverse_tcp connection to 203.78.103.109. Duration: 2h 58 min.
  • 02:56:03 194.61.24.102 connects to DC01's RDP service as Administrator one last time. Duration: 1 min 38 sec.
  • 02:56:38 DC01 re-establishes Metrepreter reverse_tcp connection to 203.78.103.109. Duration: 2h 42 min.

IOC's

  • IP : 194.61.24.102 (Attacker)
  • IP : 203.78.103.109 (C2 server)
  • MD5 : eed41b4500e473f97c50c7385ef5e374 (coreupdater.exe)
  • JA3 Hash : 84fef6113e562e7cc7e3f8b1f62c469b (RDP scan/brute force)
  • JA3 Hash : 6dc99de941a8f76cad308d9089e793d7 (RDP scan/brute force)
  • JA3 Hash : e26ff759048e07b164d8faf6c2a19f53 (RDP scan/brute force)
  • JA3 Hash : 3bdfb64d53404bacd8a47056c6a756be (RDP scan/brute force)

Wanna learn more network forensic analysis techniques? Then check out our upcoming network forensics classes in September and October.

Posted by Erik Hjelmvik on Friday, 09 July 2021 13:20:00 (UTC/GMT)

Tags: #PCAP#NetworkMiner#CapLoader#video#videotutorial

Share: Facebook   Twitter   Reddit   Hacker News Short URL: https://netresec.com/?b=217dfc7


NetworkMiner 2.7 Released

NetworkMiner 2.7 Logo

We are happy to announce the release of NetworkMiner 2.7 today! The new version extracts documents from print traffic and pulls out even more files and parameters from HTTP as well as SMB2 traffic. We have also updated our JA3 implementation to fingerprint the server side in TLS sessions using JA3S hashes and added a few tweaks to the user interface to better identify the extension of extracted files.

Extraction of Printed Data

NetworkMiner 2.7 can extract documents from LPR/LPD print traffic on TCP 515 (RFC1179). The extracted print data is saved to disk as .prn files, which can be analyzed with tools like PCL Paraphernalia. The professional version of NetworkMiner also comes with a carver that attempts to extract PostScript and PDF files from print traffic.

Improved File Extraction from PCAP

One of the premier features of NetworkMiner is its ability to extract transferred files from network traffic. We have fine tuned NetworkMiner’s file extraction code for SMB2 as well as HTTP POST in this release, in order to retrieve as much information as possible from these protocols. We’ve also added more granular logging of SMB2 requests and responses to the Parameters tab.

More DNS Types Supported

NetworkMiner 2.7 now parses DNS TXT and SRV resource records, which are displayed in NetworkMiner’s DNS tab. The TXT records can be used for almost anything, but the SRV records are used to map service types to the hostnames that provide that service. SRV lookups are often used in order to locate the domain controller on a network by querying for “_ldap._tcp.dc._msdcs.<DOMAIN>”.

DNS SRV and TXT records in NetworkMiner

DNS SRV of lookups are performed by malware and attackers as well as for legitimate reasons, even though attackers sometimes make mistakes that can be used for detection or threat hunting.

TLS Server Fingerprinting with JA3S

We introduced TLS client fingerprinting using JA3 hashes in NetworkMiner 2.5. We have now also added support for JA3S hashes, which is a method for fingerprinting the server side of a TLS connection. The JA3S hashes are extracted from the “Server Hello” TLS packets and shown on NetworkMiner’s Parameters tab as well as in the Host Details of the server. We have also improved how NetworkMiner displays the JA3 hashes in the Host Details view.

JA3S hashes in NetworkMiner

Additional User Interface Improvements

Double clicking on an extracted file in NetworkMiner's Files tab now brings up the File Details window. We’ve extended this window to also include a simple hex viewer and a feature that attempts to identify the file type based on the reassembled file’s header.

NetworkMiner's File Details window with hex viewer

The file type identification feature is also used in order to provide more accurate file extensions to extracted files, such as “.exe” or “.zip”, instead of the “.octet-stream” that you’d often see in previous versions of NetworkMiner. We have added a warning dialogue to NetworkMiner 2.7 that shows up if a user tries to run an executable file directly from the NetworkMiner GUI.

Warning dialogue in NetworkMiner when opening executable file

NetworkMiner Professional

Our commercial tool NetworkMiner Professional has received a few additional updates. It can, for example, carve PDF and PostScript files from extracted LPR print data. We have also added several OSINT services, such as ANY.RUN, MalwareBazaar, URLHaus and ThreatFox, for performing lookups of file hashes. The OSINT context menu is opened by right-clicking an extracted file in NetworkMiner Professional.

GPS data stored in pcap-ng option fields, typically by Kismet, is now extracted as capture file metadata. Right-click a capture file and select "Show Metadata" to show the coordinates from Kismet. We have also re-implemented the support for a PCAP-over-IP listener in NetworkMinerCLI, which is the command line version of NetworkMiner Pro. This feature allows the command line tool to receive PCAP data over a TCP socket instead of reading from a capture file. The PCAP-over-IP listener feature was previously broken in NetworkMinerCLI.

Credits

We’d like to thank Hayo Brouwer (of Ricoh) for requesting the LPR extraction feature and providing capture files for testing, Jeff Rivett for reporting a 64 bit issue with WinPcap/Npcap and Ali Mohd for reporting the broken PCAP-over-IP listener feature.

Upgrading to Version 2.7

Users who have purchased NetworkMiner Professional can download a free update to version 2.7 from our customer portal, or use the “Help > Check for Updates” feature. Those who instead prefer to use the free and open source version can grab the latest version of NetworkMiner from the official NetworkMiner page.

Posted by Erik Hjelmvik on Tuesday, 15 June 2021 11:55:00 (UTC/GMT)

Tags: #NetworkMiner#PCAP#SMB2#JA3#JA3S#OSINT

Share: Facebook   Twitter   Reddit   Hacker News Short URL: https://netresec.com/?b=21644b7

2021 May

Running NetworkMiner in Windows Sandbox

2021 April

Analysing a malware PCAP with IcedID and Cobalt Strike traffic

2020 October

Honeypot Network Forensics

2020 September

NetworkMiner 2.6 Released

2020 March

Discovered Artifacts in Decrypted HTTPS

2020 January

Sharing a PCAP with Decrypted HTTPS

2019 November

The NSA HSTS Security Feature Mystery

NetworkMiner 2.5 Released

2019 January

Video: TrickBot and ETERNALCHAMPION

NetworkMiner 2.4 Released

2018 December

TorPCAP - Tor Network Forensics

2018 August

NetworkMiner 2.3.2 Released!

2018 April

NetworkMiner 2.3 Released!

2018 February

Examining Malware Redirects with NetworkMiner Professional

Analyzing Kelihos SPAM in CapLoader and NetworkMiner

Antivirus Scanning of a PCAP File

Examining an x509 Covert Channel

Zyklon Malware Network Forensics Video Tutorial

2017 September

Hunting AdwindRAT with SSL Heuristics

2017 August

NetworkMiner 2.2 Released

2017 March

Enable file extraction from PCAP with NetworkMiner in six steps

2017 February

10 Years of NetworkMiner

2017 January

Network Forensics Training at TROOPERS 2017

NetworkMiner 2.1 Released

2016 September

Bug Bounty PCAP T-shirts

2016 March

Packet Injection Attacks in the Wild

2016 February

Analyzing Web Browsing Activity

NetworkMiner 2.0 Released

2015 December

Network Forensics Training at TROOPERS

2015 October

Port Independent Protocol Detection

2014 October

Chinese MITM Attack on iCloud

2014 June

Running NetworkMiner on Mac OS X

NetworkMiner 1.6 Released

2014 February

HowTo install NetworkMiner in Ubuntu Fedora and Arch Linux

2013 October

Command-line Forensics of hacked PHP.net

2013 September

New features in NetworkMiner 1.5

2013 August

Security Advisory: Two Vulnerabilities in NetworkMiner

2012 December

HowTo handle PcapNG files

twitter

NETRESEC on Twitter

Follow @netresec on twitter:
» twitter.com/netresec