SCADA Network Forensics with IEC-104

A great way to enable digital forensics of control system networks is to implement network security monitoring. Captured network traffic is a great source for evidence when analyzing an attackers steps as he attempts to hack a SCADA system. The newly added support for the IEC-104 protocol in NetworkMiner also allows investigators and incident responders to see what commands the attacker sent to the control system.

We at Netresec recently announced the release of NetworkMiner 1.4, which comes with a parser for the SCADA protocol IEC 60870-5-104 (aka IEC-104). Bringing this Industrial Control System (ICS) protocol into NetworkMiner is a first step to support forensics of compromised ICS networks. The traffic from ICS networks does, of course, need to be captured (sniffed) in order to support network forensics; we are strong supporters of such network monitoring for ICS networks (read our “Monitor those Control System Networks” blog post for more details).

Why monitor ICS networks?

Computer forensics typically involves performing forensic analysis of hard disks. Disk forensics is very effective when analyzing a hard drive from a PC (like an operator workstation), but far more complicated when it is an embedded device like a PLC or RTU that is to be analyzed.

In regard to what was believed to be a hacked SCADA system at a water facility in Illinois, David Marcus from McAfee said:

“My gut tells me that there is greater targeting and wider compromise than we know about. Why? Again, my instincts tell me that there is a lack of cyber forensics and response procedures at most of these facilities. If you do not have cyber forensic capabilities, it is hard to know if you have a cyber intrusion.”

Even though the hack was later shown to just be just a false alarm, David’s point about lacking capabilities for digital forensics and incident response for this type of critical infrastructure still holds true.

Joe Weiss also commented on the same story saying:

“We don't know how many other SCADA systems have been compromised because they don't really have cyber forensics.”

As Joe and David say, the ability to perform digital forensics in SCADA systems is truly lacking today. Our propose with this blog post is to inform control system operators that forensic data/evidence can be easily collected from ICS / SCADA systems by implementing a simple solution for network monitoring with full packet capture.

How to monitor ICS networks

The SCADA network diagram below has been sectioned into multiple security zones according to the zoning principle published by Jens Z, Iiro and me at CIRED 2009 (our zones align nicely with ISA-99 security Levels by the way).

The purple octagons represent interconnections between zones. Each such interconnection should be secured with perimeter protection, typically by a firewall, but we additionally argue that all network traffic passing through should be captured and stored as pcap files. Storing all network traffic this way makes it possible to perform network forensics on the network traffic after an intrusion is believed to have taken place.

We recommend a very simple setup, where a network tap is used to provide a copy of all traffic to a sniffer. An acceptable alternative to buying a network tap is to configure a monitor / SPAN port on a switch (see our sniffing tutorial “Intercepting Network Traffic” for more details on how to choose sniffing hardware).

Our recommended solution for the sniffer is to install FreeBSD with dumpcap (part of the net/tshark ports package). An even easier solution is to install Doug Burks’ Security Onion, which is a Linux distro built especially for network security monitoring. More about configuring a sniffer can be found in our second sniffing tutorial titled “Dumping Network Traffic to Disk”.

Analyzing captured IEC 104 traffic

Let’s assume the file 090813_diverse.pcap from pcapr contains network traffic from a suspected security breach at a hydro-power plant. Let’s also assume that parameter 4821 (i.e. IOA 4821 in IEC-104 language) controls the floodgates of the plant’s dam, where setting a value greater than 0% for this parameter would mean opening the floodgates.

By loading the pcap file into NetworkMiner and selecting the “parameters” tab we can see a nice log of all IEC-104 communication.

NOTE: We’ve hidden several fields (like IP, port, time etc) in the screenshot above in order to make it fit.

The following timeline can be extracted from the list of events provided by NetworkMiner:

• Frame 154 - The attacker sends command to set IOA 4821 to 50.354%
  
• Frame 156 - The RTU confirms the request
  
• Frame 162 - The RTU reports that the requested command has been successfully completed, i.e. floodgates are now open!
  

More ICS protocols

Would you like to see more ICS protocols in NetworkMiner? We’d be happy to implement protocols like DNP3, MODBUS, ICCP, Siemens S7, IEC 61850, etc. if you can provide us with captured network traffic! Please send an email to info[at]netresec.com if you are interested!

Posted by Erik Hjelmvik on Thursday, 30 August 2012 12:03:00 (UTC/GMT)

Tags: #Forensics #ICS #SCADA #control system #Network #Sniff #Capture #Monitor #pcap

Herr Langner advises against Intrusion Detection

The Industrial Control Systems Cyber Emergency Response Team (ICS-CERT) issued a security advisory for Siemens' SIMATIC Step 7 PLCs a couple of weeks ago. I've previously recommended asset owners to monitor the network traffic in their Industrial Control Systems (ICS), and ICS-CERT have followed my line of thinking by suggesting the following defensive measures:

"Configure an intrusion detection system (IDS) to monitor traffic for unusual or unauthorized activity.

• Monitor traffic on the ISO-TSAP protocol, Port 102/TCP.
• Monitor traffic being unexpectedly sent outside the automation network.
• Monitor traffic between workstations. This traffic may be indicative of attacker pivoting through your network"

The German ICS security cowboy Ralph Langner has written a somewhat confused blog post where he is critisizing ICS-CERT's advisory. In this blog post Langner says the following about ICS-CERT's recommendation to monitor the ISO-TSAP traffic:

"It would be interesting to learn how the authors of the advisory suggest this should actually be done. We wonder if they have ever peeked into the data traffic of a Siemens PLC’s port 102 in a real installation [...] In order to make any sense out of TCP port 102 traffic it is required to do deep packet inspection. Unfortunately, the details of the layer seven protocol that needs to be analyzed, along with certain peculiarities at layer four such as pre-defined binary TSAPs, are not documented by the vendor. So in essence what ICS-CERT suggests is that asset owners start reverse analyzing the S7 protocol in order to configure their intrusion detection systems, which seems like a far stretch."

So, is Langner saying that the Siemens S7 protocol is too complicated to be reverse engineered? If encrypted and strongly obfuscated protocols like Skype can be reversed, then the S7 protocol should be a piece of cake. I've manually reverse engineered multiple protocols when building protocol parsers for NetworkMiner, and I can testify that most unencrypted and non-obfuscated protocols can be reversed in just a few hours. It would therefore be quite simple for IDS vendors to add support for the S7 protocol to their software. I also believe that even a very rudimentary IDS functionality, which just checks which IP addresses that are communicating over TCP port 102, would provide value. Such a simple feature doesn't even require the IDS vendor to implement a parser for the S7 protocol or even the ISO-TSAP protocol.

Ralph also criticizes ICS-CERT's recommendation to "Monitor traffic being unexpectedly sent outside the automation network" by saying:

"While the advice per se might not be completely wrong, we don’t see any relation to the Beresford vulns which highlight the risk of process manipulation, not the risk of industrial espionage and exfiltration of trade secrets."

A machine on an ICS network trying to contact an external IP address is typical Indicator of Compromise, but Langner fails to understand this basic principle of network security monitoring and incident response. Malware very often use outbound connections to access Command-and-Control servers as well as to download additional software to maintain its foothold on the infected machine. I'm certain that this is why ICS-CERT recommend asset owners to monitor for outgoing traffic, especially since ICS systems normally don't communicate with external systems and typically don't host any confidential data or "trade secrets".

A point that ICS-CERT failed to stress, however, is the need for asset owners to also store the full content network traffic (pcap files) from their network monitoring installations. This is an absolute necessity when investigating an alert from an IDS in order to better determine if an event is a security incident or just a false positive.

More on capturing network traffic can be read in my blog post Sniffing Tutorial part 2 - Dumping Network Traffic to Disk.

Posted by Erik Hjelmvik on Wednesday, 24 August 2011 14:47:00 (UTC/GMT)

Tags: #SCADA #PLC #ICS #control system

Monitor those Control System Networks!

Network security monitoring is an ideal security feature to apply to industrial control system networks. Owners of the IT-systems that control our critical infrastructure have unfortunately not yet understood the usefulness of monitoring their own network traffic.

SCADA security has in the past few years become a hot topic at mainstream hacker conferences like BlackHat and DEFCON. Stuxnet has also increased the interest for SCADA security even more in the “traditional” IT security and hacking community. This interest has caused security researchers to find and publicly disclose  multiple  vulnerabilities in SCADA and Industrial Control Systems (ICS).

Having worked with IT security for a major electric utility company (in the pre-Stuxnet era) I know from my own experience that the resilience against network based hacking attacks varies greatly between different brands and models of PLCs and RTUs. But an attacker with access to a control system network don't need to use any vulnerability to control a PLC. The reason for this is that the communication protocols used by these embedded devices don't use authentication. The attacker can therefore simply send any commands he wishes to the PLCs to make them open a dam gate, blow a generator or spin a centrifuge out of control; no vulnerabilities needed!

This morning I read a blog post titled “PLC’s: Insecure By Design v. Vulnerabilities” written by Dale G Peterson (an old friend from my SCADA security days). In this blog post Dale stresses the fact that many control system devices are “Insecure By Design”. He also mentions Secure DNP3 (an encrypted SCADA protocol that basically is an American fork from the IEC 60870-5 standard), which can increase security by introducing authentication and encryption.

I've always been against introducing any form of encryption in control system environments since availability is what's needed in these environments, not confidentiality! Adding encryption is also against the KISS principle, which should always be a foundation when designing control systems.

Instead I see two major efforts that the ICS community need to carry out in order to achieve better security. The first effort is to segment the ICS networks into different security zones and apply appropriate perimeter protection between the zones. The second effort is to establish proper network security monitoring of the control system networks.

Segmentation and perimeter protection are nowadays widely accepted measures in the ICS community. There are even special ICS firewall vendors, such as Tofino, RuggedCom and Moxa. Even crazy concepts such as “unidirectional gateways” are successfully used to protect critical ICS networks.

But the concept of network security monitoring has, on the other hand, not really been grasped by the ICS community yet. It actually seems as if they don't even understand the value provided by monitoring the network traffic in control systems. The ANSI/ISA-TR99 standard does, for example, mention “sniffing” several times, but only in the context of sniffing being a threat rather than treating it as a security control. The DHS document “Cyber Security Procurement Language for Control Systems” even contains this somewhat absurd statement:

“Scanning is an effective tool to identify vulnerabilities. Use caution, however, because active scanning of live control system networks has been known to disable the networks during operations. FAT and SAT provide critical opportunities for active scanning tests without an impact to production. Even passive scanning is not recommended on production systems until the impact to operations is fully understood.”

(emphasis added)

I therefore take it upon myself to educate the authors of these misguiding standards as well as control system owners as to why they should monitor their networks. Here are six good reasons for why process control system networks should be monitored:

1. Embedded devices used in control systems do often have poor or none-existent host based security logging. Event logs as well as security logs can be built simply by sniffing and analyzing sniffed network traffic, without having to introduce any additional complexity to the embedded devices.
2. There is usually no centralized administration of the devices on a control system network, and network diagrams often differ significantly from the reality. Performing an NMAP scan of a control system network isn't suitable since that actually can cause some devices to crash (trust me!), but an inventory of the devices on a network can easily be created simply by sniffing network traffic. See my article “Passive Network Security Analysis with NetworkMiner” for more details.
3. Viruses and worms can get to even isolated/air gapped networks, either through USB flash drives or through infected laptops that get connected to the isolated network. Many viruses can be detected simply by looking at the network traffic they generate when they attempt to establish a connectionto a command-and-control server. In the case with Stuxnet, for example, the infected machines would try to establish connections to the domains mypremierfutbol [dot] com and todaysfutbol [dot] com. Any attempts to lookup external DNS names from within an isolated network are always worth looking closer at!
4. Prevention eventually fails, i.e. no matter how secure you think your network is someone or something will eventually penetrate your perimeter protection. So you'll better be prepared!
5. Assume your network security perimeter has already been breached. In a recent report from McAfee Dmitri Alperovitch (VP Threat Research at McAfee) writes:

   “Having investigated intrusions such as Operation Aurora and Night Dragon (systemic long-term compromise of Western oil and gas industry), as well as numerous others that have not been disclosed publicly, I am convinced that every company in every conceivable industry with significant size and valuable intellectual property and trade secrets has been compromised (or will be shortly), with the great majority of the victims rarely discovering the intrusion or its impact. In fact, I divide the entire set of Fortune Global 2000 firms into two categories: those that know they’ve been compromised and those that don’t yet know.”

   The best way to find out if you are infected is to monitor your network for suspicious traffic.
6. Network security monitoring is simple and doesn't affect the network being monitored! Check out my sniffing tutorials “Intercepting Network Traffic” and “Dumping Network Traffic to Disk” to get an introduction.

Now, go out there and sniff those process control networks! ;)

Posted by Erik Hjelmvik on Wednesday, 03 August 2011 18:56:00 (UTC/GMT)

Tags: #SCADA #NSM #ICS