NetworkMiner 2.8 Released

I am happy to announce the release of NetworkMiner 2.8 today! This new version comes with an improved user interface, better parsing of IEC-104 traffic and decapsulation of CAPWAP traffic. The professional edition of NetworkMiner additionally adds port-independent detection of SMTP and SOCKS traffic, which enables extraction of emails and tunneled traffic even when non-standard ports are used.

User Interface Improvements

The first thing you see when starting NetworkMiner is the Hosts tab, which now has been updated to include a filter text box. This text box can be used to filter the displayed hosts based on the property fields they contain. By entering “Android” into the filter box NetworkMiner will show only the hosts having a property containing the string “Android”, for example in the OS classification or User-Agent string. Other properties you might find useful to filter on are hostname, JA3 hash and MAC address. If you’re running NetworkMiner Professional then you’ll also be able to filter on Country thanks to the MaxMind GeoLite2 feature included in the Pro edition.

It’s now also possible to copy text from most tabs in NetworkMiner with Ctrl+C or by right-clicking and selecting “Copy selected rows”. A maximum of 10 rows can be copied at a time using the free version of NetworkMiner, while the Professional version allows all rows to be copied in one go.

The content based file type identification introduced in NetworkMiner 2.7 has been improved to also differentiate between EXE and DLL files as of version 2.8.

• Matanbuchus malware DLL disguised as PNG
• MD5: d733d96a25fd14f4494f6d76f9ff9892
• Source: 2022-06-17-Matanbuchus-with-Cobalt-Strike.pcap

• AutoIt EXE disguised as ZIP file
• MD5: c56b5f0201a3b3de53e561fe76912bfd
• Source: 2022-08-19-Astaroth-Guildma-infection-traffic.pcap

IEC 60870-5-104

NetworkMiner’s parser for the SCADA protocol IEC 60870-5-104 (IEC-104) has been significantly improved in version 2.8. NetworkMiner now supports more IEC-104 commands and the commands are presented on the Parameters tab in a clearer way than before.

Image: IEC-104 commands sent by the Industroyer2 malware

I’m also proud to announce that NetworkMiner 2.8 now extracts files transferred over the IEC-104 protocol. More details about that particular feature is available in our IEC-104 File Transfer Extraction blog post.

CAPWAP Decapsulation

NetworkMiner 2.8 can read IEEE 802.11 packets inside CAPWAP tunnels between WLAN Controllers and Access Points. This feature allows WiFi traffic to be analyzed without having to capture packets in the air.

Reading PCAP from a Named Pipe

NetworkMiner previously allowed packets to be read from PacketCache over a named pipe. This feature has been upgraded to allow a PCAP stream to be read from any named pipe, not just from PacketCache. Here’s an example showing how to capture packets from localhost for 10 seconds with RawCap and make those packets available via a named pipe called “RawCap”:

RawCap will start capturing packets once a PCAP reader connects to the “RawCap” named pipe, which now can be done with NetworkMiner by clicking “Read from Named Pipe” on the File menu.

Bug Fixes

NetworkMiner previously produced incorrect JA3S signatures for TLS servers if they sent Session ID values in Server Hello messages or listed only one supported TLS version using the Supported Versions extension. These bugs have now been fixed in NetworkMiner 2.8.

NetworkMiner’s live sniffing feature has been improved to better handle huge packets caused by Large Send Offload (LSO). NetworkMiner previously crashed with an error message saying that the received packet was “larger than the internal message buffer” when attempting to capture a too large packet.

TCP sessions occasionally didn’t show up in NetworkMiner’s Sessions tab previously if the application layer protocol was unknown. This bug has now been fixed in version 2.8.

New Features in NetworkMiner Professional

NetworkMiner Professional includes a feature for port independent protocol detection of protocols like FTP, HTTP, IRC, Meterpreter, SSH and TLS, which enables extraction of artifacts from those protocols even though the service is running on a non-standard port. This new release adds two additional protocols to the collection of identified protocols, namely SMTP and SOCKS. This allows analysts to extract emails from spam runs sent to ports other than 25 or 587, as well as to see what goes on inside covert SOCKS tunnels running on non-standard ports.

Image: SMTP credentials extracted from spam run to non-standard SMTP port

In addition to allowing hosts to be filtered using string and regex matching, NetworkMiner Professional also allows the discovered hosts to be filtered on IP address using CIDR notation, such as “192.168.1.0/24” or “10.0.0.0/8”.

Image: NetworkMiner Professional with CIDR filter 192.168.88.0/24

Here are some IPv4 and IPv6 CIDR filters that you might find useful:

• 224.0.0.0/4 = IPv4 multicast (224/4 is also supported)
• 127.0.0.0/8 = IPv4 loopback (127/8 is also supported)
• fe80::/10 = IPv6 link-local addresses
• ff00::/8 = IPv6 multicast
• 0.0.0.0/0 = IPv4 hosts (0/0 is also supported)
• 0::/0 = IPv6 hosts

Credits

We’d like to thank René Perraux, Matt Smith and Anand Kumar Singh for reporting bugs that have been fixed in this new release.

Upgrading to Version 2.8

Users who have purchased NetworkMiner Professional can download a free update to version 2.8 from our customer portal, or use the “Check for Updates” feature from NetworkMiner's Help menu. Those who instead prefer to use the free and open source version can grab the latest version of NetworkMiner from the official NetworkMiner page.

Posted by Erik Hjelmvik on Monday, 02 January 2023 08:00:00 (UTC/GMT)

Tags: #NetworkMiner​ #IEC-104​ #SMTP​ #SOCKS​ #PIPI​ #CIDR​

Emotet C2 and Spam Traffic Video

This video covers a life cycle of an Emotet infection, including initial infection, command-and-control traffic, and spambot activity sending emails with malicious spreadsheet attachments to infect new victims.

The video was recorded in a Windows Sandbox in order to avoid accidentally infecting my Windows PC with malware.

Initial Infection

Palo Alto's Unit 42 sent out a tweet with screenshots and IOCs from an Emotet infection in early March. A follow-up tweet by Brad Duncan linked to a PCAP file containing network traffic from the infection on Malware-Traffic-Analysis.net.

Image: Screenshot of original infection email from Unit 42

• Attachment MD5: 825e8ea8a9936eb9459344b941df741a

Emotet Download

The PCAP from Malware-Traffic-Analysis.net shows that the Excel spreadsheet attachment caused the download of a DLL file classified as Emotet.

Image: CapLoader transcript of Emotet download

• DNS: diacrestgroup.com
• MD5: 99f59e6f3fa993ba594a3d7077cc884d

Emotet Command-and-Control

Just seconds after the Emotet DLL download completes the victim machine starts communicating with an IP address classified as a botnet command-and-control server.

Image: Emotet C2 sessions in CapLoader

• C2 IP: 209.15.236.39
• C2 IP: 147.139.134.226
• C2 IP: 134.209.156.68
• JA3: 51c64c77e60f3980eea90869b68c58a8
• JA3S: ec74a5c51106f0419184d0dd08fb05bc
• JA3S: fd4bc6cea4877646ccd62f0792ec0b62

Emotet Spambot

The victim PC eventually started sending out spam emails. The spam bot used TLS encryption when possible, either through SMTPS (implicit TLS) or with help of STARTTLS (explicit TLS).

Image: Emotet spambot JA3 hash in NetworkMiner Professional

• SMTPS JA3: 37cdab6ff1bd1c195bacb776c5213bf2
• STARTTLS JA3: 37cdab6ff1bd1c195bacb776c5213bf2

Transmitted Spam

Below is a spam email sent from the victim PC without TLS encryption. The attached zip file contains a malicious Excel spreadsheet, which is designed to infect new victims with Emotet.

Image: Spam email extracted from Emotet PCAP with NetworkMiner

• .zip Attachment MD5: 5df1c719f5458035f6be2a071ea831db
• .xlsm Attachment MD5: 79cb3df6c0b7ed6431db76f990c68b5b

Network Forensics Training

If you want to learn additional techniques for analyzing network traffic, then take a look at our upcoming network forensic trainings.

Posted by Erik Hjelmvik on Monday, 09 May 2022 06:50:00 (UTC/GMT)

Tags: #Emotet​ #C2​ #video​ #pcap​ #JA3​ #JA3S​ #51c64c77e60f3980eea90869b68c58a8​ #ec74a5c51106f0419184d0dd08fb05bc​ #fd4bc6cea4877646ccd62f0792ec0b62​ #SMTP​ #SMTPS​ #Windows Sandbox​

NetworkMiner 2.6 Released

We are happy to announce the release of NetworkMiner 2.6 today! The network forensic tool is now even better at extracting emails, password hashes, FTP transfers and artifacts from HTTP and HTTP/2 traffic than before.

Some of the major improvements in this new release are related to extraction and presentation of emails from SMTP, POP3 and IMAP traffic. On that note, we’d like to thank Mandy van Oosterhout for reporting a bug in our email parser!

I have previously blogged about how to extract John-the-Ripper hashes from Kerberos network traffic with NetworkMiner. We have now added support for presenting LANMAN and NTLM credentials as JtR hashes as well.

We have also improved NetworkMiner’s Linux support. Files, images and folders can now be opened in external tools directly from the NetworkMiner GUI also when running NetworkMiner in Linux using Mono 6 (or later). Linux users previously got a “System.ComponentModel.Win32Exception” error message saying something like “Cannot find the specified file” or “Access denied” due to a breaking change introduced in Mono version 6.

The new release also comes with several updates of how HTTP and HTTP/2 traffic is handled and presented. We have, for example, added better extraction of data sent in HTTP (or HTTP/2) POST requests. Posted JSON formatted parameters are also extracted even if the JSON data has been gzip compressed. The “Accept-Language” header values in HTTP and HTTP/2 are extracted as “Host Details” in order to support forensic analysis of user language settings, as shown by Fox-IT in their “Operation Wocao - Shining a light on one of China’s hidden hacking groups” report.

NetworkMiner has supported decapsulation of tunneling protocols and protocols for network virtualization, like 802.1Q, GRE, PPPoE, VXLAN, OpenFlow, MPLS and EoMPLS, since version 2.1. We have now improved our GRE parser to also support NVGRE (RFC 7637) by adding support for Transparent Ethernet Bridging.

Jan Hesse sent us a feature request on Twitter earlier this year, where asked about support for FritzBox captures. We are happy to announce that NetworkMiner now supports the modified pcap format you get when sniffing network traffic with a FritzBox gateway.

NetworkMiner 2.6 can now also parse and extract SIP chat messages (RFC 3428) to the “Messages” tab. Audio extraction of VoIP calls is still a feature that is exclusively available only in NetworkMiner Professional though.

NetworkMiner Professional

Our commercial tool NetworkMiner Professional has received a few additional updates, such as support for analysis of HTTP/2 traffic in the “Browsers tab”. However, please note that NetworkMiner does not perform TLS decryption, so the HTTP/2 traffic will have to be decrypted by a TLS proxy like PolarProxy prior to being saved to a PCAP file.

We have added a few new great online services to NetworkMiner Pro’s OSINT lookup as well, such as shouldiclick.org, Browserling, MalwareDomainList and VirusTotal lookups of URL’s in the “Browsers” tab. We have also added some additional external OSINT sources for lookups of IP addresses and domain names, such as MalwareDomainList and mnemonic ACT. The JA3 hash lookup menu in NetworkMiner Professional’s “Hosts” tab has also been extended to include GreyNoise.

NetworkMiner Pro previously played back G.722 VoIP audio at half speed. This issue has now been fixed, so that G.722 RTP audio is extracted and played back in 16k samples/s. The bug was due to an error in RFC 1890 that was later corrected in RFC 3551. Thanks to Michael "MiKa" Kafka for teaching us about this!

Excerpt from RFC 3551:

Even though the actual sampling rate for G.722 audio is 16,000 Hz, the RTP clock rate for the G722 payload format is 8,000 Hz because that value was erroneously assigned in RFC 1890 and must remain unchanged for backward compatibility. The octet rate or sample-pair rate is 8,000 Hz.

We’d also like to mention that NetworkMiner Professional now comes with improved analytical support to help investigators detect Tor traffic.

Upgrading to Version 2.6

Users who have purchased a license for NetworkMiner Professional 2.x can download a free update to version 2.6 from our customer portal, or use the “Help > Check for Updates” feature. Those who instead prefer to use the free and open source version can grab the latest version of NetworkMiner from the official NetworkMiner page.

Posted by Erik Hjelmvik on Wednesday, 23 September 2020 09:10:00 (UTC/GMT)

Tags: #NetworkMiner​ #SMTP​ #POP3​ #IMAP​ #email​ #FTP​ #JtR​ #John​ #Mono​ #Linux​ #HTTP​ #HTTP/2​ #GRE​ #SIP​ #VoIP​ #Tor​ #PCAP​

Installing a Fake Internet with INetSim and PolarProxy

This is a tutorial on how to set up an environment for dynamic malware analysis, which can be used to analyze otherwise encrypted HTTPS and SMTPS traffic without allowing the malware to connect to the Internet. Dynamic malware analysis (or behavioral analysis) is performed by observing the behavior of a malware while it is running. The victim machine, which executes the malware, is usually a virtual machine that can be rolled back to a clean state when the analysis is complete. The safest way to prevent the malware from infecting other machines, or doing other bad things like sending SPAM or taking part in DDoS attacks, is to run the victim machine in an offline environment. However, network traffic analysis of malware is a central part of dynamic malware analysis, which is is why a “fake Internet” is needed in most malware labs.

INetSim and PolarProxy

INetSim is a software suite that simulates common internet services like HTTP, DNS and SMTP, which useful when analyzing the network behavior of malware samples without connecting them to the Internet. INetSim also has basic support for TLS encrypted protocols, like HTTPS, SMTPS, POP3S and FTPS, but requires a pre-defined X.509-certificate to be loaded at startup. This can cause malware to terminate because the Common Names (CN) in the presented certificates don’t match the requested server names. The victim machine will actually get the exact same certificate regardless of which web site it visits. INetSim’s TLS encryption also inhibits analysis of the network traffic captured in the malware lab, such as C2 traffic or SPAM runs, because the application layer traffic is encrypted. PolarProxy can solve both these issues because it generates certificates on the fly, where the CN value is dynamically set to the requested host name, and saves the network traffic in decrypted form to PCAP files. It is therefore a good idea to replace the TLS services in INetSim with PolarProxy, which will be used as a TLS termination proxy that forwards the decrypted traffic to INetSim’s cleartext services.

Install Linux

The first step is to install a Linux VM, which will act as a fake Internet to the victim machine(s). I'm using Ubuntu Server 18.04.3 LTS in this tutorial, but you can use any 64-bit linux distro. I'm adding two network interfaces to the Linux VM, one interface with Internet access and one that connects to an isolated offline network to which the victim VM's will be connected. The offline interface is configured to use the static IP 192.168.53.19.

Important: Do not bridge, bond or enable IP forwarding between the two interfaces!

Install INetSim

INetSim is available in Ubuntu's repo, so it is possible to install it with "apt install inetsim". However, I recommend installing INetSim as described in the official documentation to get the latest packaged version of INetSim.

INetSim listens on 127.0.0.1 by default, change this to INetSim's offline IP address by un-commenting and editing the service_bind_address variable in /etc/inetsim/inetsim.conf.

Also configure INetSim's fake DNS server to resolve all domain names to the IP of INetSim with the dns_default_ip setting:

Finally, disable the "start_service https" and "start_service smtps" lines, because these services will be replaced with PolarProxy:

Restart the INetSim service after changing the config.

Verify that you can access INetSim's HTTP server with curl:

It looks like INetSim's web server can be accessed alright.

Install PolarProxy

Next step is to install PolarProxy as a systemd service (as instructed here):

We will need to modify the PolarProxy service config file a bit before we start it. Edit the ExecStart setting in /etc/systemd/system/PolarProxy.service to configure PolarProxy to terminate the TLS encryption for HTTPS and SMTPS (implicitly encrypted email submission). The HTTPS traffic should be redirected to INetSim's web server on tcp/80 and the SMTPS to tcp/25.

Here's a break-down of the arguments sent to PolarProxy through the ExecStart setting above:

• -v : verbose output in syslog (not required)
• -p 10443,80,80 : listen for TLS connections on tcp/10443, save decrypted traffic in PCAP as tcp/80, forward traffic to tcp/80
• -p 10465,25,25 : listen for TLS connections on tcp/10465, save decrypted traffic in PCAP as tcp/25, forward traffic to tcp/25
• -x /var/log/PolarProxy/polarproxy.cer : Save certificate to be imported to clients in /var/log/PolarProxy/polarproxy.cer (not required)
• -f /var/log/PolarProxy/proxyflows.log : Log flow meta data in /var/log/PolarProxy/proxyflows.log (not required)
• -o /var/log/PolarProxy/ : Save PCAP files with decrypted traffic in /var/log/PolarProxy/
• --certhttp 10080 : Make the X.509 certificate available to clients over http on tcp/10080
• --terminate : Run PolarProxy as a TLS termination proxy, i.e. data forwarded from the proxy is decrypted
• --connect 192.168.53.19 : forward all connections to the IP of INetSim
• --nosni nosni.inetsim.org : Accept incoming TLS connections without SNI, behave as if server name was "nosni.inetsim.org".

Finally, start the PolarProxy systemd service:

Verify that you can reach INetSim through PolarProxy's TLS termination proxy using curl:

Yay, it is working! Do the same thing again, but also verify the certificate against PolarProxy's root CA this time. The root certificate is downloaded from PolarProxy via the HTTP service running on tcp/10080 and then converted from DER to PEM format using openssl, so that it can be used with curl's "--cacert" option.

Yay #2!

Now let's set up routing to forward all HTTPS traffic to PolarProxy's service on tcp/10443 and SMTPS traffic to tcp/10465. I'm also adding a firewall rule to redirect ALL other incoming traffic to INetSim, regardless of which IP it is destined to, with the final REDIRECT rule. Make sure to replace "enp0s8" with the name of your interface.

Verify that the iptables port redirection rule is working from another machine connected to the offline 192.168.53.0/24 network:

Yay #3!

Yay #4!

It is now time to save the firewall rules, so that they will survive reboots.

Install the Victim Windows PC

Configure a static IP address on the victim Windows host by manually setting the IP address. Set the INetSim machine (192.168.53.19) as the default gateway and DNS server.

Download the X.509 root CA certificate from your PolarProxy installation here:
http://192.168.53.19:10080/polarproxy.cer

1. Double-click on "polarproxy.cer"
2. Click [Install Certificate...]
3. Select 🔘 Local Machine and press [Next]
4. Select 🔘 Place all certificates in the following store and press [Browse...]
5. Choose "Trusted Root Certification Authorities" and press [OK], then [Next]
6. Press [Finish]

You might also want to install the PolarProxy certificate in your browser. This is how you install it to Firefox:

1. Options / Preferences
2. Press [Privacy & Security]
3. Scroll down to "Certificates" and press [View Certificates...]
4. In the "Authorities" tab, press [Import...]
5. Open "polarproxy.cer"
6. ☑ Trust this CA to identify websites. (check the box)
7. Press [OK]

Now, open a browser and try visiting some websites over HTTP or HTTPS. If you get the following message regardless of what domain you try to visit, then you've managed to set everything up correctly:

Accessing the Decrypted Traffic

PCAP files with decrypted HTTPS and SMTPS traffic are now available in /var/log/PolarProxy/

PolarProxy will start writing to a new capture file every 60 minutes. However, the captured packets are not written to disk instantly because PolarProxy uses buffered file writing in order to improve performance. You can restart the proxy service if you wish to flush the buffered packets to disk and have PolarProxy rotate to a new capture file.

I also recommend capturing all network traffic sent to INetSim with a sniffer like netsniff-ng. This way you’ll get PCAP files with traffic from INetSim’s cleartext services (like DNS and HTTP) as well.

PCAP or it didn’t happen!

Credits

I'd like to thank Thomas Hungenberg and Patrick Desnoyers for providing valuable feedback for this blog post!

Posted by Erik Hjelmvik on Monday, 09 December 2019 08:40:00 (UTC/GMT)

Tags: #PolarProxy​ #HTTPS​ #SMTPS​ #HTTP​ #SMTP​ #DNS​ #Malware​ #Sandbox​ #TLS​ #PCAP​ #proxy​ #tutorial​ #ASCII-art​

NetworkMiner 2.3 Released!

The free and open source network forensics tool NetworkMiner now comes with improved extraction of files and metadata from several protocols as well as a few GUI updates. But the biggest improvements for version 2.3 are in the commercial tool NetworkMiner Professional, which now supports VoIP call audio extraction and playback as well as OSINT lookups of file hashes, IP addresses, domain names and URLs.

I’m happy to announce that NetworkMiner 2.3 now does an even better job than before at extracting files and metadata from several protocols. Improvements have been made in the parsers for the following protocols: HTTP, IEC-104, IPv4, Modbus, SIP, SMB, SMB2, SMTP and SSL/TLS.

We have also added support for the SNMP protocol in NetworkMiner 2.3, so that SNMP community strings can be extracted and displayed on the Parameters and Credentials tabs.

Another change is that timestamps are now displayed using the UTC time zone instead of using the local time zone. We have also fixed a few GUI quirks in order to further improve the usability of the tool.

NetworkMiner Professional

The commercial version of NetworkMiner, i.e. NetworkMiner Professional, comes with several additional improvements which are presented below.

VoIP Call Playback

NetworkMiner Professional has received a new tab called “VoIP”, which enables audio playback of VoIP calls that are using SIP and RTP with G.711 μ-law or A-law encoding (u-Law is primarily used in North America and Japan while A-law is used in Europe and most other parts of the world).

Video: Audio playback and extraction to WAV from the “SIP_CALL_RTP_G711” PCAP file in the Wireshark Sample Captures.

The audio streams from the VoIP calls are also extracted to disk as .WAV files when codecs G.729 or G.711 (u-Law and A-Law) is used. NetworkMiner Professional also attempts to reassemble RTP streams encoded with G.722 to .au files.

OSINT Lookups of IP Addresses, Domains, URLs and File Hashes

Right-clicking a network host in NetworkMiner Professional’s Hosts tab brings up a context menu with options for performing lookups of IP and domain names using external sources. We refer to this method as open-source intelligence (OSINT) because the accessed data resides at publicly available sources.

Clicking on an OSINT provider brings up a webpage with more detailed information about the selected IP address, such as IBM X-Force, mnemonic Passive DNS, Shodan, UrlQuery or VT. However, if you’re lazy like me, then you’ll probably click the “All above!” option instead, which will bring up all of the sources in separate tabs in your browser.

The full list of OSINT providers available for IP lookups includes APNIC Whois, BFK Passive DNS, Censys, Cymon, DNSTrails, ExoneraTor, Google Public DNS, GreenSnow.co, Hurricane Electric, IBM X-Force, Internet Storm Center, mnemonic Passive DNS, PacketTotal, Shodan, ThreatCrowd, ThreatMiner, UrlQuery and VirusTotal.

The domain name lookup menu contains a similar set of providers: BFK Passive DNS, Cymon, DNSTrails, Google Public DNS, Google Safe Browsing, Hybrid Analysis, IBM X-Force Exchange, mnemonic Passive DNS, MXToolBox, MyWOT, Norton Safe Web, PacketTotal, ThreatCrowd, ThreatMiner, URL Void, UrlQuery, VirusTotal, Website Informer, Webutation and Whoisology.

Right-clicking a URL in the Browsers tab brings up a similar context menu, which additionally includes the following services for URL lookups: Google Safe Browsing, IBM X-Force, ThreatMiner, URLhaus and UrlQuery.

Finally, right-clicking on one of the files that NetworkMiner has extracted from a PCAP file brings up a menu for doing OSINT lookups based on the MD5 or SHA256 hash of the file. The sources used for lookups of hashes include IBM X-Force, PacketTotal, ThreatCrowd, TotalHash, UrlQuery, VirScan.org, Comodo Valkyrie, AlienVault OTX, Hybrid Analysis, ThreatMiner and VirusTotal.

Hybrid Analysis API Integration

Did you know that the malware analysis service Hybrid Analysis provides free API keys to people in the IT security community?

This is a great move by the Hybrid Analysis team, and we’re happy to announce that we have leveraged their API in NetworkMiner Professional in order to submit files for analysis directly from within the NetworkMiner GUI. The API integration also enables you to query for an IP on Hybrid Analysis to see which previously submitted samples has communicated with that particular IP address.

Here are the steps required to enable the Hybrid Analysis API integration:

• Create a HybridAnaysis account
• Create an API key
• Upgrade the API key by completing a vetting process
• Start NetworkMiner Pro, open the Tools > Settings menu and input your API key

Credits

I would like to thank Chris Sistrunk, Mats Karlsson and Michael Nilsson for suggesting several of the protocol and GUI improvements that have been incorporated into this new release. I’d also like to thank Doug Green and Ahmad Nawawi for discovering and reporting bugs in the IP and SSL parser respectively.

Upgrading to Version 2.3

Users who have purchased a license for NetworkMiner Professional 2.x can download a free update to version 2.3 from our customer portal.

Those who instead prefer to use the free and open source version can grab the latest version of NetworkMiner from the official NetworkMiner page.

⛏ FOR GREAT JUSTICE! ⛏

Posted by Erik Hjelmvik on Tuesday, 03 April 2018 06:27:00 (UTC/GMT)

Tags: #NetworkMiner​ #PCAP​ #OSINT​ #SMTP​ #SIP​ #RTP​ #VoIP​ #Network Forensics​ #extract​ #Netresec​

NetworkMiner 2.1 Released

We are releasing a new version of NetworkMiner today. The latest and greatest version of NetworkMiner is now 2.1.

Yay! /throws confetti in the air

Better Email Parsing

I have spent some time during 2016 talking to digital forensics experts at various law enforcement agencies. I learned that from time to time criminals still fail to use encryption when reading their email. This new release of NetworkMiner therefore comes with parsers for POP3 and IMAP as well as an improved SMTP parser. These are the de facto protocols used for sending and receiving emails, and have been so since way back in the 90’s.

Not only does NetworkMiner show the contents of emails within the tool, it also extracts all attachments to disk and even saves each email as an .eml file that can be opened in an external email reader in order to view it as the suspect would.

Encapsulation Protocols

There are several protocols that can be used to provide logical separation of network traffic, in order to avoid using multiple physical networks to keep various network segments, security domains or users apart. Some of these techniques for logical separation rely on tagging or labeling, while others are tunneling the encapsulated traffic. Nevertheless, it’s all pretty much the same thing; an encapsulation protocol is used in order to wrap protocol X inside protocol Y, usually while adding some metadata in the process.

NetworkMiner has been able to parse the classic encapsulation protocols 802.1Q, GRE and PPPoE since 2011, but we now see an increased use of protocols that provide logical separation for virtualization and cloud computing environments. We have therefore added parsers in NetworkMiner 2.1 for VXLAN and OpenFlow, which are the two most widely used protocols for logical separation of traffic in virtualized environments. We have also added decapsulation of MPLS and EoMPLS (Ethernet-over-MPLS) to NetworkMiner 2.1.

The new release additionally comes with support for the SOCKS protocol, which is an old school encapsulation protocol used by administrators as well as hackers in order to bypass firewalls or provide anonymous Internet access. The SOCKS parser in NetworkMiner can even be used to read network traffic from Tor in cleartext before it enters the Tor network. However, in order to capture Tor’s SOCKS traffic you’ll have to sniff traffic from the Tor client’s localhost interface on TCP port 9150.

PacketCache

NetworkMiner 2.1 can read packets directly from a local PacketCache service by clicking ”File > Read from PacketCache”. This eliminates the need to run a PowerShell script in order to dump a PCAP file with packets recently captured by PacketCache.

HTTP Partial Content / Range Requests

Byte serving is a feature in HTTP that makes it possible to retrieve only a segment of a file, rather than the complete file, by using the “Range” HTTP header. This feature is often used by BITS in order to download updates for Windows. But we have also seen malware use byte serving, for example malware droppers that attempt to download malicious payloads in a stealthy manner. See Ursnif and Dridex for examples of malware that utilize this technique.

NetworkMiner has previously only reassembled the individual segments of a partial content download. But as of version 2.1 NetworkMiner has the ability to piece together the respective parts into a complete file.

SSL/TLS and X.509 Certificates

NetworkMiner has been able to extract X.509 certificates to disk for many years now, simply by opening a PCAP file with SSL traffic. However, in the 2.1 release we’ve added support for parsing out the SSL details also from FTP’s “AUTH TLS” (a.k.a explicit TLS or explicit SSL) and STARTTLS in SMTP.

NetworkMiner now also extracts details from the SSL handshake and X.509 certificate to the Parameters tab, such as the requested SNI hostname and the Subject CN from the certificate.

NetworkMiner Professional

The new features mentioned so far are all part of the free open source version of NetworkMiner. But we have also added a few additional features to the Professional edition of NetworkMiner as part of the 2.1 release.

The “Browsers” tab of NetworkMiner Professional has been extended with a feature for tracking online ads and web trackers. We are using EasyList and EasyPrivacy from easylist.to in order to provide an up-to-date tracking of ads and trackers. HTTP requests related to ads are colored red, while web tracker requests are blue. These colors also apply to the Files tab and can be modified in the Settings menu (Tools > Settings).

The reason why NetworkMiner Pro now tracks ads and trackers is because these types of requests can make up almost half of the HTTP requests that a normal user makes while surfing the web today. Doing forensics on network traffic from a suspect criminal can be a very time consuming task, we therefore hope that being able to differentiate between what traffic that is initiated by the user rather than being triggered by an online advertisement service or internet tracker can save time for investigators.

The RIPE database previously contained a bug that prevented NetworkMiner Professional from properly leveraging netname info from RIPE. This bug has now been fixed, so that the Host Details can be enriched with details from RIPE for IP addresses in Europe. To enable the RIPE database you’ll first have to download the raw data by clicking Tools > Download RIPE DB.

We have also extended the exported details about the hosts in the CSV and XML files from NetworkMiner Professional and the command line tool NetworkMinerCLI. The exported information now contains details such as IP Time-to-Live and open ports.

Upgrading to version 2.1

Users who have purchased a license for NetworkMiner Professional 2.0 can download a free update to version 2.1 from our customer portal.

Those who instead prefer to use the free and open source version can grab the latest version of NetworkMiner from the official NetworkMiner page.

Credits

There are several persons I would like to thank for contributing with feature requests and bug reports that have been used to improve NetworkMiner. I would like to thank Dietrich Hasselhorn, Christian Reusch, Jasper Bongertz, Eddi Blenkers and Daniel Spiekermann for their feedback that have helped improve the SMB, SMB2 and HTTP parsers as well as implementing various encapsulation protocols. I would also like to thank several investigators at the Swedish, German and Dutch police as well as EUROPOL for the valuable feedback provided by them.

Posted by Erik Hjelmvik on Wednesday, 11 January 2017 14:30:00 (UTC/GMT)

Tags: #NetworkMiner​ #POP3​ #SMTP​ #IMAP​ #VXLAN​ #X.509​ #PCAP​

NetworkMiner 2.0 Released

I'm proud to announce the release of NetworkMiner 2.0 today! There are several longed-for features that are part of this major release, such as:

• SMB/CIFS parser now supports file extraction from SMB write operations.
• Added parser for SMB2 protocol (read and write).
• Additional IEC-104 commands implemented.
• Added Modbus/TCP parser (as requested by attendees at 4SICS 2014).
• Improved SMTP parser.
• Improved FTP parser.
• Improved DNS parser.
• GUI flickering is heavily reduced when loading PCAP files or doing live sniffing.
• Extraction of web server favicon images (shown in Hosts tab).
• Added "Keyword filter" to several tabs (see more details below).

For those not familiar with NetworkMiner, here's a short summary:

NetworkMiner is a Network Forensic Analysis Tool (NFAT) for Windows (but also works in Linux / Mac OS X / FreeBSD). NetworkMiner can be used as a passive network sniffer/packet capturing tool in order to detect operating systems, sessions, hostnames, open ports etc. without putting any traffic on the network. NetworkMiner can also parse PCAP files for off-line analysis and to regenerate/reassemble transmitted files and certificates from PCAP files.

NetworkMiner has, since the first release in 2007, become a popular tool among incident response teams as well as law enforcement. NetworkMiner is today used by companies and organizations all over the world.

Keyword Filtering

Users of NetworkMiner sometimes run into a needle-in-the-haystack problem when trying to find some specific entry in the Files, Parameters or DNS tab. The technique most analysts have been using so far is to sort the data based a column of interest (by clicking the column header) and then scroll down to the row they are looking for. This method isn't optimal, which is why several users have requested support for keyword filtering. I've personally received several of these requests when teaching my Networks Forensics class.

I'm therefore happy to say that version 2.0 of NetworkMiner has a built-in filtering capability for the data displayed in the following tabs:

• Files
• Parameters
• DNS
• Messages
• Sessions

Image: Parameters tab with filter “user-agent” (case insensitive)

The filter allows rows to be filtered based on one or several keywords. The entered keywords are matched against all text in all columns of the tab. A drop-down menu in the filter bar allows the analyst to chose “Exact Phrase”, “All Words” or “Any Word” as search criteria for the entered keywords.

More Data in the Parameters Tab

We have previously held back on what data we add to the Parameters tab. However, now with the filtering feature in place, we decided to add a lot more information to the Parameters list. Some of the new parameter types available in version 2.0 are:

• HTTP request methods
• HTTP URI's
• HTTP response status codes
• HTTP headers
• SMB Tree Connect AndX Request (attempts to connect to a named file share)
• SMB NT Create AndX Request (mapping of filename to file handle ID)
• SMB2 Connect Requests (attempts to connect to a named file share)
• SMB2 File ID (mapping of filename to file handle ID)
• SMB2 file timestamps (Created, Modified and Accessed).

Image: SMB2 file timestamps shown in Parameters tab

A Warning to Malware Analysts

NetworkMiner has previously appended the “.octet-stream” or “x-msdos-program” extension to all binary files being downloaded over HTTP (since those are the MIME types used for Windows executables). As of version 2.0, however, files named {something}.exe will not be renamed this way. This means that there is now a risk of accidental execution of such files, for example if the user right-clicks an .exe file in NetworkMiner and selects “Open File”.

If you analyze PCAP files that might contain malware, then our recommendation is to perform the analysis on some other operating system than Windows. NetworkMiner runs fine on Linux as well as Mac OS X.

NetworkMiner Professional

On top of the updates provided in the free version of NetworkMiner we have added a few additional useful features to NetworkMiner Professional.

The new features in the Pro version include:

• Advanced OS fingerprinting. Identifies a great range of operating systems and device types (including Apple iOS, Android and many others) based on DHCP traffic.
• Web Browsing Analysis. A new GUI tab called "Browsers" has been added, which shows what URLs each unique browser has visited. More details on this feature will be released shortly in a separate blog post.
• User Settings. Settings in GUI can now be saved to make them persistent between executions.
• Better Export Logs. The CSV format used for exporting data has been improved, we have also added support for XML formatted data export.

Defang Executables

Remember the warning about .exe files downloaded over HTTP no longer get the “.octet-stream” extension? To counter the risk of accidental execution of malware we've added a defang feature to NetworkMiner Professional. When enabled, this feature will rename files like “malware.exe” to “malware.exe_” in order to prevent execution. It is not only .exe files being renamed. At the moment the following file extensions are defanged by appending “_” to the extension:

exe, bat, msi, vb, vbe, vbs, pif, com, scr, jar, cmd, js, jse, ps1 and psc1

The defang feature renames files regardless if they were transmitted over HTTP, FTP, TFTP, SMB, SMB2, SMTP or any other protocol supported by NetworkMiner.

Please note that the defang feature is turned off by default. So if you wanna protect yourself from accidental execution, make sure to enable this feature in the NetworkMiner Settings (available under Tools > Settings).

Upgrade Path for Professional Users

We always provide free minor version upgrades of our software. However, please note that version 2.0 is a major version release, which require a new license to be purchased. Customers with a license for NetworkMiner Professional 1.x can get a 20% discount when purchasing version 2.0 by entering their current license number into the “Additional Information” field of the NetworkMiner Professional order form.

Credits

There are many people who have provided feedback and ideas for improvements that are now part of NetworkMiner 2.0. I would especially like to thank Eric Kollmann (author of Satori) for his impressive ability to find bugs in beta releases. I would also like to thank Ulf Skoglund, Dmitry Shchemelev, @xredumb, Sabin and Andrew Brant for their feedback on improvements in DNS, TCP, HTTP and SMTP parsing. Finally, I wanna give a shout-out to Steffen Thorkildsen for spawning the idea for a browser tracking feature.

Posted by Erik Hjelmvik on Tuesday, 09 February 2016 11:38:00 (UTC/GMT)

Tags: #NetworkMiner​ #SMB​ #SMB2​ #Modbus​ #filter​ #favicon​ #SMTP​ #email​

NetworkMiner 1.6 Released

We've released version 1.6 of NetworkMiner today!

The new features in NetworkMiner 1.6 include:

• Drag-and-Drop
  Reassembled files and images can be opened with external tools by drag-and-dropping items from NetworkMiner's Files or Images tabs onto your favorite editor or viewer.


• Email extraction
  Improved extraction of emails and attachments sent over SMTP.


• DNS analysis
  Failed DNS lookups that result in NXDOMAIN and SERVFAIL are displayed in the DNS tab along with the flags in the DNS response.


• Live sniffing
  Improved live sniffing performance.


• PCAP-over-IP
  Remote live sniffing enabled by bringing the PCAP-over-IP feature into the free open source version of NetworkMiner.

Identifying Malware DNS lookups

DNS traffic from the Kuluoz-Asprox botnet (PCAP file available via Contagio)

Note the NXDOMAIN responses and “No” in Alexa top 1 million column in the screenshot above; these domains are probably generated by a domain generation algorithm (DGA).

Live Sniffing with Pcap-over-IP

The PCAP-over-IP functionality enables live sniffing also on non-Windows machines, simply by running tcpdump (or dumpcap) and netcat like this:

# tcpdump -i eth0 -s0 -U -w - | nc localhost 57012

To receive the Pcap-over-IP stream in NetworkMiner, simply press Ctrl+R and select a TCP port.

For more information about this feature please see our previous blog post about the PCAP‑over‑IP feature.

NetworkMiner Professional

The professional version of NetworkMiner additionally contains the following improvements of the command line tool NetworkMinerCLI:

• Enabled reading of PCAP and PcapNG data from standard input (STDIN)
• Full support for PCAP-over-IP
• More detailed DNS logging in NetworkMinerCLI's CSV export of DNS responses

The ability to read PCAP data from STDIN with NetworkMinerCLI makes it really simple to do live extraction of emails and email attachments. Here's an example showing how to do live SMTP extraction in Linux:

# tcpdump -i eth0 -s0 -w - port 25 or 587 | mono NetworkMinerCLI.exe -r - -w /var/log/smtp_extraction/

The syntax for extracting emails and attachments in Windows is very similar:

C:\>dumpcap.exe -i 1 -f "port 25 or 587" -w - | NetworkMinerCLI.exe -r -

The TCP ports 25 and 587, which are used in the capture filter above, are the standard port numbers for SMTP. In order to do live extraction of files sent over HTTP, simply use “port 80” as capture filter instead. Likewise, X.509 certificates can also be extracted from HTTPS sessions simply by using “port 443” as capture filter.

Download NetworkMiner 1.6

The most recent release of the free (open source) version of NetworkMiner can be downloaded from SourceForge or our NetworkMiner product page. Paying customers can download an update for NetworkMiner Professional from our customer portal.

Credits

We would like to thank Dan Eriksson (FM CERT) and Lenny Hansson (Danish GovCERT) for submitting bug reports and feature requests.

Posted by Erik Hjelmvik on Monday, 16 June 2014 11:00:00 (UTC/GMT)

Tags: #Netresec​ #NetworkMiner​ #Professional​ #SMTP​ #Extract​ #DNS​ #PCAP-over-IP​