• Salman Zahid

How to build monitoring and visibility networks with changing application architectures!

One of the most important factors that drives the monitoring and visibility architecture is the underlying application architecture and its placement.

Today, applications are designed to be mobile and can live in on-prem clouds or in public clouds, in VMs, containers or bare-metal hosts. Since applications are distributed and mobile, the threat vector is distributed as well and constantly evolving. Monitoring only one part of the network may leave other parts of the network vulnerable to security gaps. It is important for security professionals to take a comprehensive view when designing a monitoring and visibility regimen for a given environment. Visibility into East-West traffic that stays within the servers or otherwise cannot be captured by placing physical taps, need to be part of the thought process. Similarly, security professionals need to pay close attention to monitoring traffic in the public cloud as well.

Big Switch has brought to market a comprehensive visibility architecture, Big Monitoring Fabric (Big Mon) that allows customers to do monitoring in any type of data center architecture (on-prem or in-cloud), any application hosting architecture (VM, containers, bare-metal) and do so with simplicity and scale. In this paper, I will describe what makes up Big Switch’s Big Mon architecture and how customers can take advantage of this architecture for their visibility needs. Before we get into the details, here are the 2 versions for Big Monitoring Fabric:.

Before we get into the details, there are 2 variations :

BMF-EC (BMF-Enterprise Cloud): Designed to monitor applications/traffic on prem.

BMF-PC (BMF Public Cloud): Designed to monitor applications in public cloud

The following diagram shows the distinction between BMF-EC and BMF-PC -

As customers moved workloads to the cloud, while still having workloads running on-prem, it became even more important to provide a simple, and consistent way to manage these hybrid deployments. This is why Big Switch developed its newest offering: Multi-Cloud Director.

Multi-Cloud Director (MCD): allows customers to manage on-prem and in-cloud visibility deployments from a single pane of glass. The following diagram shows four Big Mon deployments in different locations being managed and monitored from the same console.

Big Monitoring Fabric delivers these key set of functions.

Traffic Acquisition for monitoring purposes

Big Mon’s simple and scale-out architecture enables traffic collection for

  • Traditional, non-virtualized environments using Taps and mirror (SPAN) ports

  • Fully virtualized private clouds based on VMware or Openstack

  • Public clouds such as AWS or Azure

Acquiring traffic is the first step in being able to deliver insights into any customer’s environment and detect traffic anomalies. These insights range from aiding capacity planning to helping customers design a robust security policy by mapping out application dependencies or finding other anomalies in the network.

Integrations with vCenter, NSX and OpenStack

As I mentioned at the start of this article, one of the keys to building modern visibility systems is to not just monitor north-south traffic, but also vm-to-vm traffic. A common solution is to place an agent inside a host for monitoring purposes. Big Mon takes a different approach. Instead of placing an agent inside of a host or a hypervisor, Big Mon takes advantage of an API-based integration (where available) with a number of virtualization platforms (vCenter, NSX) and focuses on making east-west traffic monitoring simple by automating the configuration of SPAN or ERSPAN or GRE based mirroring. Big Mon switches act as tunnel aggregation or span [KL1] aggregation for the traffic originating from the VMs and rest of the workflow is same as monitoring any other traffic.

Filtering, Replication and Load balancing of traffic

Once Big Mon has traffic from the production network, it needs to be filtered down to traffic of interest, replicated to a set of tools or load balanced to multiple instances of the same tool.

Advanced Packet Services

Leveraging industry-standard x86 servers, Big Mon additionally provides high-performance, advanced packet services like de-duplication, packet slicing, header-stripping, regex matching, packet masking, GTP correlation, UDP replication and NetFlow generation in a scale-out architecture.

Analytics & packet recording

Big Monitoring Fabric is unique in the industry that not only it solves the problem of traffic acquisition and delivery to tools, but provides analytics right out of the box with its unique architecture of combing analytics within the network packet broker solution. Not only can Big Mon deliver traffic to any set of security & monitoring tools, but it can also consume that same traffic & provide analytics based on traffic meta-data such as sFlow, NetFlow , DHCP, DNS and TCP. In addition to providing traffic analytics, Big Mon can also store the traffic using scale out packet recording capabilities. This can help customers identify network issues using on-demand or continuous recording.


Controller Based Architecture

BMF was designed with a few goals in mind. One of the primary goal was to make it simple to design and operate a monitoring network. Controller is a center piece of the architecture. BMF controller is a piece of software that can be deployed as a virtual machine or an appliance and has a few functions.

  • For BMF-EC, controller can be deployed as a VM or an appliance.

  • Controllers are a central point of control and management for all components within a visibility solution such as tap aggregation switches, flow collectors known as BMF-Analytic Node or packet capture devices known as BMF-recorder node.

  • Controllers are also the central point of policy definition – defining traffic of interest and sending it to one or many tools, load balancing of traffic. All of the policy is described at the controller.

Big Monitoring Fabric Switches:

BMF switches are open networking switches (Dell or Edgecore) that allows customer to aggregate traffic from SPAN or Tap ports at very high speed. These switches can range in port speeds from 1GE to 100GE. The primary function of these switches to aggregate the traffic, filter the traffic and send the traffic of interest to one or many tools. The switch fabric could consist of one switch or 10s of switches. Size of the fabric can vary depending on the density of SPAN or Tap ports. All the switches are managed from the controller & does not require customers to do any box-by-box management. Using lower cost white box switches for traffic aggregation can open up possibilities for customers to tap/span traffic not just at the data center edge, but inside the data center in a leaf-spine fabric.

Big Monitoring Fabric Integration with vCenter , NSX and Openstack

As I mentioned at the start of this paper, one of the key to building modern visibility systems is to not just monitor north-south traffic , but also vm-to-vm traffic. One of the common solutions is to place an agent inside a host for monitoring purposes. BMF takes a different approach. Instead of placing an agent inside of a host or a hypervisor, BMF takes advantage of an API based integration with a number of virtualization platforms (vcenter, openstack, nsx) and focuses on making e-w traffic monitoring simple by automating the configuration of SPAN or ERSPAN or GRE based mirroring in Openstack. BMF tap switches then act as tunnel aggregation or span aggregation for the traffic originating from the VMs and rest of the workflow is same as monitoring any other traffic.

Big Monitoring Fabric Service Nodes:

BMF Service nodes are special x86 appliances (4X10GE or 16X10GE) that provide additional functionality such as packet slicing, masking, netflow generation and deduplication. High speed switches are typically not well suited for these functions. Service nodes complement the architecture by taking on the role of providing these additional optional capabilities within a visibility architecture. If traffic needs to be deduplicated or customers want to leverage a monitoring fabric to generate netflow records, traffic can be service chained through service nodes before being sent to the monitoring and security tools. Multiple services can be enabled concurrently on the service node.

Big Monitoring Fabric Analytics Node:

Big Monitoring analytics node is a 1 RU commodity server that can be deployed in a scale-out architecture and collects metadata from multiple sources including: sFlow, Netflow, and Big Monitoring Fabric; and creates a correlated dashboard of data that a user can search on and receive alerts. It is based on a general-purpose open source analytics platform which means the data answers a broad set of questions across NPM, APM & Security use cases. Dashboarding is customizable, modern and shareable.

Big Mon Analytics Node Benefits
  • Supports various Health / Capacity Planning / Troubleshooting dashboards

  • Supports Performance views like Top Talkers, Top Apps, TCP connection/latency tracking

  • Supports Security views displaying Rogue DHCP/ DNS servers, identifies IP / MAC Spoofing

  • Support various Host views like New Hosts seen, DHCP OS fingerprinting.

  • Supports Automatic alerting on exceeding various thresholds like link utilization.

  • Supports sFlow/NetFlow collection to provide real-time application level visibility, including tunneled or encapsulated traffic, enable detection of security attacks like DoS/DDoS and support sub-second triggering.

  • Easy to use, Scale-out, High-Performance

  • Integrated / centralized configuration and operational workflows via Big Mon Controller

  • Machine learning to learn about any anomalous traffic behaviors (unusual spikes of traffic)

Big Mon Recorder Node:

The Big Mon Recorder node provides the ability to capture raw packets at a very cost-effective price point by leveraging commodity x86 scale-out architecture. Each packet recorder is a 2RU appliance with 160TB of recording capacity. Packet recorder allows customers to go back to a point in time, and narrow down traffic of interest and receive a local copy. This is very useful for identifying network issues and security threats. Network Packet recording is useful for incident management, fraud detection, troubleshooting, and security analytics. Think of a security surveillance system where there are recording triggers based on motion detection events, ingress/egress building activities, time of day events, guard activities, heightened security concerns etc. Network recording is analogous where abnormal network behaviors can be detected by the analytics node, and based on these behaviors, traffic/packets related to these behaviors can be recorded for analyzing and troubleshooting the issues. Examples include odd spikes in the traffic patterns, traffic from unknown hosts, traffic for unknown ports, traffic that is exhibiting a lot of errors.

Traditional, closed solutions require touch points on different devices to identify and send packets to the recorder. Secondly, having the packet recorder capture packets at line rate and at the same time, be able to retrieve packets efficiently, is not something traditional recorders do well. Finally, there is the question of how easy is it to get packets out of the recorder as pcap or as a packet replay function. Big Mon Recorder node built on an industry-standard x86 server appliance, integrates with the Big Mon Fabric (controller and analytics), to provide immense simplicity and high capture / query performance, even at scale.

This integrated approach tremendously simplifies user workflows:

  • Auto-discover recorders connected to the fabric.

  • Auto-cluster multiple recorders to present a larger, logical Recorder node.

  • Optionally, filter traffic via the Big Mon Fabric before sending to the recorder.

  • Construct and retrieve / replay packets from recorder based on the user selected “event” in the analytics. via the user-configured.

  • REST API based architecture enables the user to programmatically and automatically begin / end packet capture based on user-specified event triggers. This next-gen architecture enables rapid anomaly detection and security issue identification and improves time to resolution.

Big Monitoring Fabric-EC Inline:

Big Monitoring Fabric Inline is the Next-Generation Security Services Broker aka a packet flow switching fabric, enabling Scale-out DMZ Protection and Rapid Attack Mitigation.

Big Monitoring Fabric (Big Mon) Inline offers a simple, scale-out method for deploying security tools in the DMZ and creating on-demand service chains. Its controller-based, software defined networking (SDN) design accelerates high-performance attack mitigation and enables organizations to deploy countermeasures in response to cyber threats.

Big Mon Inline provides the centralizing fabric needed for organizations to rollout a consistent, organization-wide DMZ security posture. Security teams now have a single pane interface to build and manage scale-out security tool chains. Multiple active, inline tools can be deployed logically inline, in defined sequence, and receive only the traffic of interest to each. Other non-security tools, such as web-proxies, can also take advantage of Big Mon Inline for rapid, non-intrusive inline deployment. Furthermore, multiple sites can be managed by leveraging Big Switch Multi Cloud Director (MCD).

Big Monitoring Fabric – Public Cloud (BMF-PC):

Big Monitoring Fabric-Public Cloud is a fully virtualized version of Big Monitoring Fabric-Enterprise cloud & it helps customers in providing analytics and monitoring for workloads deployed in public cloud such as AWS. Lack of visibility and security is one of the biggest challenges of customers being able to move the workloads from on-prem private clouds into public clouds. BMF-PC helps customers with providing analytics and visibility for traffic within the public cloud by setting up virtual taps within a public cloud environment. It then aggregates that traffic, filters it & can deliver the traffic to the tools deployed within a public cloud or back to on-prem security and monitoring tools. Multi Cloud Director helps customers retain single pane of glass management across monitoring environments within a private and public cloud.

In summary, Big Switch Network’s BMF provides a comprehensive security and visibility capabilities across a broad set of environments while providing simplified management and scalable architecture.

Author - Salman Zahid leads the systems engineering team at Big Switch Networks. Salman has been in the IP networking field for 18 years having started at Cisco Systems and spending 9 years in customer support and technical marketing. He also spent 5 years at Juniper networks as a product manager for data center networking portfolio. As a systems engineering leader at Big Switch, Salman is responsible for technical engagements with the customers.

Big Switch Networks was founded in 2010, with roots in the original Stanford research team that invented software-defined networking (SDN), and the company is widely considered one of the original pioneers of the technology. In 2013 the company made available its first commercial product, Big Monitoring Fabric™ for network visibility and security and in 2014 the company released Big Cloud Fabric™, a data center switching fabric that brings the simplicity of public cloud to on-prem data center networks.

#BigSwitch #NetworkVisibility #DatavisibilityintheVM

328 views0 comments

Recent Posts

See All