Understanding the "Blind Spots in the Cloud" Network Visibility Architecture

Understanding the "Blind Spots in the Cloud" Network Visibility Architecture

Network visibility is becoming increasingly blurred. Businesses are investing in tools to strengthen security and increase the productivity of IT professionals to get through the fog.

According to Shamus McGillicudy, vice president of research at Enterprise Management Associates (EMA), a whopping 78% of enterprises plan to increase their spending on network visibility tools over the next two years. The main factor is the increase in traffic due to the adoption of hybrid and multi-cloud architectures. In addition, other factors that require increased visibility include increased east-west traffic and increased use of encryption by malicious actors to hide malicious traffic. 

McGillicudy says networks are generating more and more data that needs to be analyzed. He said, 'I think something happened. You shouldn't say, 'I'll go ahead and physically connect the network, do a packet dump, and then do a forensic analysis'. To always have a perfect view, you need to have the equipment and keep the lights on. You can't turn the lights off to save money and turn them on when you need to see what happened.” 

What is Network Visibility Architecture?

EMA defines Network Visibility Architecture as an overlay of traffic mirroring, aggregation and distribution tools that provide network traffic data to other systems. In other words, it captures packet data from the cloud and on-premises networks and provides it to security tools and performance analysis systems, such as intrusion detection or application performance management software. 

The key components of a network visibility architecture are TAP and SPAN ports that mirror the traffic data of the production network, and network packet broker equipment and integrations. Enterprise-grade visibility architectures typically also incorporate software-based probes and packet brokers for virtual infrastructure and cloud-based probes and packet brokers for cloud systems. In the last one or two years, traffic mirroring services from cloud vendors have appeared, and are also included in some enterprises' network visibility architectures

Network Visibility Challenges

Most organizations agree that the existing network visibility landscape has room for improvement. According to EMA's survey, only 34% of all respondents said they were successful in using a network visibility architecture (though this is down from 2020 (40%)). Challenges are scalability issues (27%), architectural complexity (26%), data quality (23%), skills gaps (19%), budgets (19%), and limited cloud visibility (17%).

"The biggest challenges are 'scalability' and 'architecture complexity'," McGillicudy said. "Scaling a visibility architecture is sometimes difficult. Enterprises are spending a lot of money on these architectures as they struggle to keep up with traffic growth. It's a chase and chase fight."

A problem in terms of architectural complexity is the inability to fully understand the state of the network from start to finish. McGillicudy asks, “Where should traffic be mirrored with analytics tools? Do you know all the parts of your network that need to do this? A lot of them don't.”

Cloud Disrupts Visibility Tools

Overall, the effectiveness of network visibility systems is declining for a variety of reasons, the biggest being the cloud. Migrating applications to the cloud created blind spots, and multi-cloud made visibility even worse. “The network operations team is not satisfied with the level of network visibility in the cloud, and is struggling to extend the solution to the cloud,” McGillicudy said. 

Network blind spots caused by the cloud can lead to policy violations (49%), IT service issues or downtime (46%), security breaches (45%), and cloud overruns (44%). 

According to the EMA, building an end-to-end visibility architecture that spans on-premises infrastructure and the public cloud can eliminate these blind spots. "The cloud is increasing the relevance of these products, not reducing them," McCullidi added.

In addition, when asked about their primary method of providing cloud-related network packet data to security and performance analysis tools, the majority (60%) said they are using third-party software, such as virtual network packet brokers or virtual TAPs. 38% said they use a native packet mirroring service provided by a cloud vendor. The remaining 2% said they did not use alternative methods or analyze packet data in the cloud.

The benefits of third-party visibility software in the cloud include: 

Data collection reliability (54%)

Administrative Security (36%)

Manageability/Automation (34%)

Advanced packet filtering and alteration capabilities (32%)

Integration with private infrastructure visibility technologies (30%)

TAP vs. SPAN Ports

The EMA examines the rate of port mirroring on a network performed through a Switched Port Analyzer (SPAN) port or a Test Access Port (TAP) every two years. With a SPAN port, one of the ports on a network switch becomes a traffic mirroring service that can copy traffic and forward it to other systems. A TAP is a dedicated device that copies network traffic from the production network and offloads that task on the switch. 

In the past, most enterprises performed port mirroring through a TAP rather than a SPAN port. However, recently, it is changing to a SPAN port rather than a TAP port. "Many enterprises are using SPAN ports rather than TAPs for traffic mirroring," McGillicudy said. It has implications,” he said.

As network complexity increases, enterprises can mirror more points on the network to improve overall visibility, and SPAN ports are a less expensive approach in terms of capital expenditure (CAPEX). But there are also benefits of TAP. For example, TAPs are typically provided by solution vendors specializing in visibility, and software is supported to manage TAPs, especially when network configuration changes. “It reduces operational complexity,” he added.

“Conversely, if you use a SPAN port, you may not have a central view of the SPAN ports configured on multiple switches in the network,” McGillicudy said. This means that it is very difficult to manage changes to the visibility fabric at the traffic mirroring layer and prevent unauthorized changes.” He also noted that using TAP also improves data quality. TAPs are optimized to carry mirrored traffic to the visibility architecture, while SPAN ports are best-effort. 

“When a network switch is highly utilized, it relinquishes resources on the SPAN port to perform its primary mission,” McGillicudy said. For example, a SPAN port will start dropping packets, which will affect the data quality. That's why companies are investing in TAP, and I think there's a bit of a problem with a lot of companies recently using more SPAN ports."

Encrypted traffic interferes with network visibility According to

McGillicudy, network visibility architectures can play a key role in inspecting encrypted traffic and detecting malicious activity, but many organizations fail to properly identify malicious traffic. .

When asked how many times the EMA had detected malicious activity hidden inside encrypted packets on their networks over the past year, the average response was 27%. However, this percentage varied depending on how successful the enterprise was with its network visibility solution. Encrypted traffic accounted for 34% of all malicious activity on their networks, compared with 23% of those reporting somewhat successful use of it. 

"It's a pretty big difference," McGillicudy said. This shows that a network visibility architecture is essential for detecting malicious activity hidden in encrypted traffic. But a lot of companies aren't doing that."

When asked about the resources they use to decrypt TLS/SSL traffic, the most popular answers were security and performance analysis tools (43%). However, using a security analysis tool for decryption may consume the tool's resources, which affects the ability to actually analyze decrypted traffic. "Too many companies are decrypting traffic in analytics tools," McGillicudy said. "It's not efficient." 

The second most popular approach was to decrypt traffic (23%) at a network packet broker, which McGillicudy said "personally sees the ideal spot." In addition, there were dedicated decryption devices (12%), packet collection devices (11%), and application-provided controls (7%). 

Visibility Increases IT Efficiency

According to the report, the benefits of using a network visibility architecture include:  

Increased IT/Security Team Productivity (36%)

Reduced security risk (33%)

Improved capacity management (25%)

Cloud Migration Optimization (23%)

Network/application performance and resiliency (22%)

Improved cross-team collaboration/decision making (19%)

Reduced compliance risk (18%)

Extend the life of security and performance analytics tools (14%)

While it's hard to quantify security risk reduction, McGillicudy said, it's easy to quantify the productivity gains. There is,” he explained. 

In many enterprises, critical IT staff spend hundreds of hours providing network traffic data to analytics tools. A network visibility architecture allows this to be automated. IT pros don't have to do the hard work to get the data and pass it to the tool.