[Analysis] Full Duplex Capture in SCADA and Industrial Control Networks (by Thomas Tannhäuser and Al
Why SPAN Ports Should Not be Used in Security Solutions
The convergence of IT and OT (Operational Networks) in the context of Industry 4.0 has led to a crowded market of security solutions targeting the shop floor on different levels. While the security of the legacy IT systems was part of the initial planning of those systems, the industry now faces the challenge to integrate security solutions in legacy OT systems.
Figure 1: Layers of a typical automation system using Profinet
At least for the lower levels of the Industry 4.0 infrastructure, (control and device level) security solution vendors tend to use the mirror/span ports offered by the network switches to integrate their solutions into the infrastructure.
Here I will explore why those mirror ports shouldn't be used to build security solutions. Based on the hardware commonly used in Industrial Networks, we will specifically look at:
How it all works
Why a mirror port will drop frames even if a link is not saturated
We came across some research data Garland posted from Packet Pioneer that said a mirror port will drop up to 8% of good frames and could be a lot more. Not Good!
I took a deep breath because an 8% loss is a lot, especially if you are going to build security applications on top of the mirrored network traffic from systems using Profinet in different flavors (configuration, real-time and alarm IO).
I continued reading and calmed down, in their research, they used iperf to saturate a link that was mirrored at the same time. Saturation means the system is in an overload condition, so drops are acceptable. And every network engineer will agree, that using more than 80% of the bandwidth of an Ethernet link will result in weird behavior at some point.
But I couldn’t get past these questions, ‘What happens, if only small frames are transmitted?’ The average frame size in industrial networks is ~130 bytes, (calculated from a sampling of shop floor traces containing mostly Profinet) but the aforementioned test was done using 1500 byte frames (iperf usually uses TCP and that will send the data in chunks of MTU size).
‘What if we use more than one link?’ On the process level usually one controller talks to a number of IO-devices via one switch.
Figure 2: Process level - Profinet master and IO-Devices
In the context of industrial automation systems a test setup using a one-to-one connection running 1500 byte frames does not reflect a common scenario. And regarding automation systems, what if the traffic on those multiple links shows different bandwidth patterns?
Keeping those questions in mind I built a small test setup based on the open source network traffic generator and analyzer Ostinato. This tool offers a nice interface to create well defined streams of Ethernet frames for multiple network interfaces while intercepting traffic at the same time.
This is not a hardware frame generator, but capable of handling multiple 100 Mbps streams with relative low-tech network interface cards. 100 Mbps is absolutely sufficient for running tests against network switches commonly used on the lower levels in a shop floor automation environment, as this is the usual network speed used there.
Want to see the test details and results?
Authors - Thomas Tannhäuser and Alexander Pirogov are regular writers for the www.garlandtechnology.com .They are both Technologists and Network Experts.