NETRESEC Network Security Blog - Tag : OSINT


NetworkMiner 2.4 Released

NetworkMiner 2.4

We are proud to announce the release of NetworkMiner 2.4 today! The new version comes with several improvements, such as username extraction from Kerberos traffic, better OS fingerprinting and even better Linux support.


Protocol Updates

The Kerberos v5 implementation in NetworkMiner 2.4 can be used to to extract usernames, hostnames and realms (domains) from unencrypted Kerberos requests/responses on port 88. NetworkMiner also parses and extracts usernames etc. from HTTP auth headers and SMB security blobs when they use Kerberos for authentication.

Kerberos username (Administrator) and realm (DENYDC.COM) in NetworkMiner's Host tab
Image: NetworkMiner showing extracted username (Administrator) and realm (DENYDC.COM) from the Wireshark sample capture file “Krb-contrained-delegation.cap”.

NetworkMiner also automatically attempts to parse traffic to TCP port 11371 as HTTP in order to extract GPG keys sent using the HKP protocol.


MAC Address Magic

We’ve added two new features related to MAC addresses to this release. One of them is the “MAC Age” field (showing “2000-11-09” in the previous screenshot), which is a guesstimate of how hold a device/host is based on its MAC address. This functionality uses HD Moore’s mac-ages database, which contains approximate dates for when hardware address ranges were allocated by IEEE (original concept from DeepMac).

The second MAC feature is a simple yet useful feature that adds links between hosts that share the same MAC address. This feature is useful for linking a host's IPv6 and IPv4 addresses with each other, but it can also be used to track if a physical host has changed its IP address. The MAC address links can be accessed by expanding the MAC address node in NetworkMiner’s Hosts tab.

IPv4 and IPv6 address with the same MAC address
Image: NetworkMiner with a PCAP file from ISTS 2012

ICS Asset Inventory

Hard Hat

We’ve put in some ground work in order to create OS fingerprinting signatures for several Industrial Control System (ICS) devices. Our signatures have been submitted and merged into Eric Kollmann’s Satori TCP database, which NetworkMiner uses to passively fingerprint hosts by examining various TCP and IP fields in the initial SYN/SYN+ACK packets of TCP sessions. The ICS devices we’ve added include PLCs, RTUs as well as rugged network equipment from vendors like ABB, Allen-Bradley, Modicon, Moxa, Phoenix Contact and Siemens. Some ICS vendors even got an icon showing their logo in the Hosts tab (see the Siemens/RUGGEDCOM device in the screenshot below) while the others got a yellow hard hat.

Asset inventory list with ICS devices
Image: Asset inventory list generated by NetworkMiner using PCAP files from the 4SICS 2015 ICS Lab.

EternalBlue

NetworkMiner isn’t designed to be used as an IDS. Nevertheless we decided to add detection for the EternalBlue exploit to NetworkMiner 2.4. The fact that NetworkMiner parses NetBIOS and SMB makes it pretty straightforward to identify when an attacker is attempting to allocate a large non-paged pool in srvnet.sys by using a vulnerability in Microsoft’s SMB implementation (see MS17-010 for reference). This type of detection is difficult to perform using a standard IDS solution that cannot parse the NetBIOS and SMB protocols. Detected EternalBlue exploit attempts are listed in NetworkMiner's “Anomalies” tab. Example PCAP files with attackers/malware using the EternalBlue exploit can be found here:


NetworkMiner in Linux

NetworkMiner Loves Linux

NetworkMiner is a Windows tool, but it actually runs just fine also in other operating systems with help of the Mono Framework (see our guide “HowTo install NetworkMiner in Ubuntu Fedora and Arch Linux”). However, there are a few pitfalls that must be avoided to get the software running smoothly using Mono. With this release we’ve implemented workarounds for two bugs in Mono’s GUI implementation (System.Windows.Forms).

The first workaround handles a Mono bug that sometimes could be triggered by Drag-and-Dropping a file or image from NetworkMiner to another application, such as a browser, text editor or image viewer. Doing so would previously trigger a NullReferenceException in System.Windows.Forms.X11Dnd+TextConverter.SetData under certain conditions. We’re happy to report that you can now reliably drag and drop files extracted by NetworkMiner to other tools, even when running Linux.

The second workaround handles a bug in Mono’s GDIPlus implementation related to rendering of Unicode characters. We were unfortunately not able to reliably get Mono to render Unicode characters, NetworkMiner will therefore convert all Unicode MIME data to ASCII when using Mono (typically in Linux). Windows users will still get the proper Unicode representations of exotic characters and emojis in NetworkMiner though. ☺


NetworkMiner Professional

The commercial version of NetworkMiner, i.e. NetworkMiner Professional, comes with a few additional improvements. One of them is is that the following additional online sources have been added to the OSINT lookup feature:

OSINT lookup of file hash in NetworkMiner Professional
Image: OSINT lookup menu for .exe file extracted from Malware-Traffic-Analysis.net’s 2018-10-16-trickbot.pcap.

The CSV export from NetworkMinerCLI has been updated to use the ISO 8601 format with explicit time zone for timestamps. An exported timestamp now look something like this:

2019-01-08T13:37:00.4711000+02:00

NetworkMiner Professional 2.4 also identifies application layer protocols regardless of port number (a.k.a. PIPI) with much better precision than earlier versions. It also extracts audio from VoIP calls (SIP) more reliably than before.


Credits

I would like to thank Chris Sistrunk for requesting GUI support to link IPv4 and IPv6 hosts with the same MAC address and Jonas Lejon for the HKP GPG key extraction idea. I would also like to thank Phil Hagen for notifying us about the issue with Unicode in emails when running NetworkMiner under Mono and Ahmad Nawawi for notifying us about the protocol identification shortages in the previous version.


Upgrading to Version 2.4

Users who have purchased a license for NetworkMiner Professional 2.x can download a free update to version 2.4 from our customer portal. Those who instead prefer to use the free and open source version can grab the latest version of NetworkMiner from the official NetworkMiner page.

⛏ FOR GREAT JUSTICE! ⛏

Posted by Erik Hjelmvik on Thursday, 10 January 2019 14:20:00 (UTC/GMT)

Tags: #NetworkMiner #ICS #SIP #VoIP #IPv6 #Mono #Linux #Satori #OSINT #PIPI

More... Share  |  Facebook   Twitter   Reddit   Hacker News Short URL: https://netresec.com/?b=191BD02


CapLoader 1.7 Released

We are happy to announce the release of CapLoader 1.7! CapLoader 1.7 logo

Here’s an overview of what’s new in this release:

  • Regular expression searching
  • Lookup of IP addresses using online services
  • Lookup of domain names using online services
  • Improved protocol fingerprinting speed and precision
  • Support for GRE, IGMP and ICMPv6 flows
  • More precise period estimation of “periodic services


Regular Expressions Search

CapLoader’s “Find Keyword” window has been extended with an option to search flows using regular expressions (regex) as an alternative to searching for strings or byte sequences. With help of the powerful regex syntax built into .NET this new search option enables very flexible searching.

Searching for Alfa Ransomware flows in CapLoader using regex
Image: Searching for Alfa Ransomware flows using regex

The Find Keyword window can be opened by clicking Edit > Find Keyword, or by pressing Ctrl+F after having loaded a PCAP file.

You can learn more about regex searching with CapLoader in our short video called "Detecting the Pony Trojan with RegEx using CapLoader".


OSINT Lookups of IP Addresses and Domains

We added support for querying online services for IPs and domain names to the latest release of NetworkMiner, now it’s time to add this very handy feature to CapLoader as well.

Right-clicking a Flow, Service or Host in CapLoader brings up a context menu with links to various online resources that might have more details regarding the clicked IP address or domain name.

CapLoader OSINT lookups

The services available for IP address OSINT lookup include:
APNIC Whois, Censys, Cymon, ExoneraTor, Google Public DNS, GreenSnow.co, Hurricane Electric, IBM X-Force, Internet Storm Center, mnemonic Passive DNS, PacketTotal, SecurityTrails, Shodan, ThreatCrowd, ThreatMiner, UrlQuery and VirusTotal.

The domain name lookup menu contains a similar set of providers:
Cymon, Google Public DNS, Google Safe Browsing, Hybrid Analysis, IBM X-Force Exchange, mnemonic Passive DNS, MXToolBox, MyWOT, Norton Safe Web, PacketTotal, SecurityTrails, ThreatCrowd, ThreatMiner, URL Void, UrlQuery, VirusTotal, Website Informer, Webutation and Whoisology.


Protocol Identification

The dynamic protocol detection (or Port Independent Protocol Detection, aka “PIPI”) in CapLoader has been improved to support even more protocols than before. We have also fine-tuned the protocol identification algorithm to be both faster and more accurate.

CapLoader 1.7 identifying SSL on non-standard port
Image: Traffic to TCP 8777 identified as SSL (PCAP file from Stratosphere IPS)


Updating to the Latest Release

Users who have previously purchased a license for CapLoader can download a free update to version 1.7 from our customer portal. All others can download a free 30 day trial from the CapLoader product page (no registration required).


Credits

We’d like to thank Michael Nilsson for suggesting the IP and domain name lookup feature and Tohar Braun for suggesting RegEx search support. We’d also like to thank Ralf Alvarsson and Jarmo Lahtiranta for reporting bugs that have been resolved in this release.

Posted by Erik Hjelmvik on Tuesday, 03 July 2018 11:37:00 (UTC/GMT)

Tags: #CapLoader #regex #keyword #pcap #OSINT #IP

More... Share  |  Facebook   Twitter   Reddit   Hacker News Short URL: https://netresec.com/?b=1878305


NetworkMiner 2.3 Released!

The free and open source network forensics tool NetworkMiner now comes with improved extraction of files and metadata from several protocols as well as a few GUI updates. But the biggest improvements for version 2.3 are in the commercial tool NetworkMiner Professional, which now supports VoIP call audio extraction and playback as well as OSINT lookups of file hashes, IP addresses, domain names and URLs.

I’m happy to announce that NetworkMiner 2.3 now does an even better job than before at extracting files and metadata from several protocols. Improvements have been made in the parsers for the following protocols: HTTP, IEC-104, IPv4, Modbus, SIP, SMB, SMB2, SMTP and SSL/TLS.

We have also added support for the SNMP protocol in NetworkMiner 2.3, so that SNMP community strings can be extracted and displayed on the Parameters and Credentials tabs.

SNMP Community Strings in NetworkMiner's Credential tab

Another change is that timestamps are now displayed using the UTC time zone instead of using the local time zone. We have also fixed a few GUI quirks in order to further improve the usability of the tool.


NetworkMiner Professional

The commercial version of NetworkMiner, i.e. NetworkMiner Professional, comes with several additional improvements which are presented below.

VoIP Call Playback

NetworkMiner Professional has received a new tab called “VoIP”, which enables audio playback of VoIP calls that are using SIP and RTP with G.711 μ-law or A-law encoding (u-Law is primarily used in North America and Japan while A-law is used in Europe and most other parts of the world).

Video: Audio playback and extraction to WAV from the “SIP_CALL_RTP_G711” PCAP file in the Wireshark Sample Captures.

The audio streams from the VoIP calls are also extracted to disk as .WAV files when codecs G.729 or G.711 (u-Law and A-Law) is used. NetworkMiner Professional also attempts to reassemble RTP streams encoded with G.722 to .au files.

OSINT Lookups of IP Addresses, Domains, URLs and File Hashes

Right-clicking a network host in NetworkMiner Professional’s Hosts tab brings up a context menu with options for performing lookups of IP and domain names using external sources. We refer to this method as open-source intelligence (OSINT) because the accessed data resides at publicly available sources.

OSINT IP lookup in NetworkMiner Professional 2.3

Clicking on an OSINT provider brings up a webpage with more detailed information about the selected IP address, such as IBM X-Force, mnemonic Passive DNS, Shodan, UrlQuery or VT. However, if you’re lazy like me, then you’ll probably click the “All above!” option instead, which will bring up all of the sources in separate tabs in your browser.

The full list of OSINT providers available for IP lookups includes APNIC Whois, BFK Passive DNS, Censys, Cymon, DNSTrails, ExoneraTor, Google Public DNS, GreenSnow.co, Hurricane Electric, IBM X-Force, Internet Storm Center, mnemonic Passive DNS, PacketTotal, Shodan, ThreatCrowd, ThreatMiner, UrlQuery and VirusTotal.

The domain name lookup menu contains a similar set of providers: BFK Passive DNS, Cymon, DNSTrails, Google Public DNS, Google Safe Browsing, Hybrid Analysis, IBM X-Force Exchange, mnemonic Passive DNS, MXToolBox, MyWOT, Norton Safe Web, PacketTotal, ThreatCrowd, ThreatMiner, URL Void, UrlQuery, VirusTotal, Website Informer, Webutation and Whoisology.


OSINT URL lookup in NetworkMiner Professional 2.3

Right-clicking a URL in the Browsers tab brings up a similar context menu, which additionally includes the following services for URL lookups: Google Safe Browsing, IBM X-Force, ThreatMiner, URLhaus and UrlQuery.


OSINT file hash lookup in NetworkMiner Professional 2.3

Finally, right-clicking on one of the files that NetworkMiner has extracted from a PCAP file brings up a menu for doing OSINT lookups based on the MD5 or SHA256 hash of the file. The sources used for lookups of hashes include IBM X-Force, PacketTotal, ThreatCrowd, TotalHash, UrlQuery, VirScan.org, Comodo Valkyrie, AlienVault OTX, Hybrid Analysis, ThreatMiner and VirusTotal.

Hybrid Analysis API Integration

Did you know that the malware analysis service Hybrid Analysis provides free API keys to people in the IT security community?

@HybridAnalysis: We are excited to announce that full API keys for file submissions are now available to everyone of the IT security community

This is a great move by the Hybrid Analysis team, and we’re happy to announce that we have leveraged their API in NetworkMiner Professional in order to submit files for analysis directly from within the NetworkMiner GUI. The API integration also enables you to query for an IP on Hybrid Analysis to see which previously submitted samples has communicated with that particular IP address.

Here are the steps required to enable the Hybrid Analysis API integration:


Credits

I would like to thank Chris Sistrunk, Mats Karlsson and Michael Nilsson for suggesting several of the protocol and GUI improvements that have been incorporated into this new release. I’d also like to thank Doug Green and Ahmad Nawawi for discovering and reporting bugs in the IP and SSL parser respectively.


Upgrading to Version 2.3

Users who have purchased a license for NetworkMiner Professional 2.x can download a free update to version 2.3 from our customer portal.

Those who instead prefer to use the free and open source version can grab the latest version of NetworkMiner from the official NetworkMiner page.

FOR GREAT JUSTICE!

Posted by Erik Hjelmvik on Tuesday, 03 April 2018 06:27:00 (UTC/GMT)

Tags: #NetworkMiner #PCAP #OSINT #SMTP #SIP #RTP #VoIP #Network Forensics #extract #Netresec

More... Share  |  Facebook   Twitter   Reddit   Hacker News Short URL: https://netresec.com/?b=1846A54

twitter

NETRESEC on Twitter

Follow @netresec on twitter:
» twitter.com/netresec


book

Recommended Books

» The Practice of Network Security Monitoring, Richard Bejtlich (2013)

» Applied Network Security Monitoring, Chris Sanders and Jason Smith (2013)

» Network Forensics, Sherri Davidoff and Jonathan Ham (2012)

» The Tao of Network Security Monitoring, Richard Bejtlich (2004)

» Practical Packet Analysis, Chris Sanders (2017)

» Windows Forensic Analysis, Harlan Carvey (2009)

» TCP/IP Illustrated, Volume 1, Kevin Fall and Richard Stevens (2011)

» Industrial Network Security, Eric D. Knapp and Joel Langill (2014)