When we tested next-generation firewalls last May, at least one important security vendor wasn’t there: Cisco, because they weren’t ready to be tested. Now that the ASA CX next-generation firewall has had a year to mature, we put the product through its paces, using the same methodology as our last NGFW test.
We found that Cisco has an outstanding product, with good coverage and strong application identification and control features. Enterprise security managers who have upgraded to the “-X” versions of the ASA firewalls (announced at the RSA Conference in March 2012) can add next-generation features to the hardware in their data centers and branch offices and gain immediate benefits.
Network managers who haven’t upgraded their hardware or who are considering a switch from a different vendor should make a competitive scan before deciding on the Cisco ASA. We found the ASA CX to be a solid “version 1” effort, but Cisco still has significant work to do in improving the management, integration, threat mitigation and application controls, leaving the ASA CX a work in progress.
Introducing the ASA CX
When Cisco decided to add next-generation features to its ASA firewall, it must have faced a daunting task: how to take a mature firewall architecture and add the next-generation features, especially application identification and control, that security managers were asking for. And by next-generation features, we mean application identification and control. Cisco took a stab at this in 2009, when it added the Modular Policy Framework, which brought many application-layer controls to the ASA. Rather than touch the delicately constructed NAT and policy rules of existing ASA firewalls, the MPF layered on top of existing security policies.
The ASA 5515-X is a standard ASA firewall with an additional processing module, called “CX” (for “context”) that handles application identification and control. In the ASA 5512-X through 5555-X, the CX next-generation firewall runs as a software module. In the high-end ASA 5585-X, Cisco has two hardware accelerators available today (the SSP10 and SSP20) with two additional models (the SSP40 and SSP60) targeted for end-of-year release, designed to take ASA throughput to 10Gbps and beyond.
Running CX does come with a performance penalty. For example, the ASA 5515-X we tested is rated for 1.2Gbps of raw firewall throughput, but only 350Mbps of next-generation throughput. With a list price of $5,600, the 5515-X delivers very competitive price/performance compared to other next-gen firewalls.
For Cisco engineers, adding the CX set of next-generation features meant either going back to the drawing board on the ASA or wedging the next-generation feature set in without tipping the boat too much. Cisco took a little of each option: the next-generation features are glued on the side of the ASA in a way that leaves the core firewall completely undisturbed. This is the approach Cisco has taken when adding other security features to the ASA, such as IPS and anti-malware scanning, and will continue to take, as add-ons like web security make their way into the ASA.
But Cisco also promised us that it was serious about a unified management system that would bring ASA and next-gen features together in a single GUI before the end of 2013. Even if the security features are run by very separate policy engines, good policy management tools can give a unified experience to security managers building firewall policies.
But in the current Version 9.1 we tested, network managers will be very aware that there are two distinct policy engines at work. The ASA’s next-generation features don’t even share an IP address with the base ASA firewall — next-gen policies are configured using Cisco Prime Security Manager (PRSM), a completely different management system from the ASA firewall’s Adaptive Security Device Manager (ASDM).
The basic ASA firewall is still handling access control, NAT and VPN. To enable next-generation features, an entry is made in Service Rules, part of the Modular Policy Framework, that defines which traffic is sent over to the CX part of the firewall. This means that any traffic has to be passed first by the normal access control rules, and then is subject to additional checks and controls based on application and user identification information.
As each connection passes through the CX engine, three different policies come into play. First, the CX engine decodes SSL. Next, it ties user authentication information to the connection. And finally, the access control policies are applied, blocking or allowing the connection based on user identification and application-layer information (including application id, application type, and URL category) and user identification.
Although most application identification and controls are in the new CX policy set, they’re not all there — everything added to the ASA before CX as part of the Modular Policy Framework is still down in the core ASA. This leads to some overlap and confusion, because you have to look in two places to do very similar application controls.
In some places, the CX and the ASA MPF completely overlap; in other areas, the division of labor is more intuitive. Cisco told us that their engineers are working on an 18-month road map to push application-layer features into the CX code, and move common services, such as identity-based access controls, into the ASA base, with progress expected at each release.
The current release of the ASA 5515-X hardware has a choice of running IPS or next-generation firewall (CX), but can’t run both. Cisco told us that IPS will be integrated with the CX code by the end of 2013, with a separate license to enable the IPS feature set. As for anti-malware, Cisco couldn’t give us a definite answer. Like many security companies, they are shying away from traditional anti-virus scanners as being ineffective against many new threats. With reputation services beautifully integrated into the ASA CX policies, along with botnet detection at the ASA level, Cisco thinks it has the luxury of sitting back and looking at alternative approaches to provide anti-malware protection rather than rushing into yet another anti-virus engine.
How’s that firewall looking?
We started our testing by reviewing the ASA CX’s basic firewall capabilities. As the great-grandchild of Cisco’s PIX and VPN Concentrator 3000 series, the ASA remains a significant presence in many enterprises. When we tested the ASA as an end-user VPN concentrator with the AnyConnect Secure Mobility Solution v3.0 two years ago, we knew enterprise network managers would be happy -- Cisco delivered solid client support across multiple platforms, end-point posture assessment, integration with their WSA web security gateways and solid performance.
Although we didn’t dive deep into this part of the ASA, things haven’t changed, and the ASA still looks great as a VPN concentrator for remote access.
For this review, we tested firewall features in 10 areas and found a mixed bag with strong plusses and a few minuses.
Unfortunately, our biggest complaint is in the most important part of any basic firewall: policy management. The core ASA firewall has to filter traffic before it gets to the next-generation application controls, making policy management important. We found this part of the ASA policy, called ACLs (access control lists), to be problematic.
The ASA is unusual among enterprise firewalls because it’s not zone-based (although Cisco IOS firewall is), which means that any deployment with more than two interfaces (or security zones) can get very complicated, very quickly. For example, to build our policy which differentiated between three types of trusted users, servers and the Internet, we had to write more rules than one would use in a zone-based firewall, in some cases defining two rules in different directions on different interfaces to cover the same traffic. This is rules to cover both traffic going into an interface and traffic going out of the same interface — a strange way of thinking that takes a while to get used to and, more importantly, leads to larger rule sets.
The larger the rule set, the more likely it is to have an error, and the more difficult it is to understand. Since the default behavior for the core firewall is “drop everything” and the next-generation firewall is “permit everything,” you don’t want anything to percolate up to the next-generation part that you’re not comfortable with.
The ASA’s lack of integration of VPN access controls with other ACLs is also a complicating factor that can lead to human error in networks where the same ASA is used both for VPN and standard firewalling. Our advice: don’t do that, at least not in any complicated network. These devices are inexpensive enough that you can get one for remote access and a another one for firewalling and avoid potential security problems at very little cost.
Areas of the base firewall that we take for granted, such as NAT, VLAN support, dynamic routing, joint Layer 2/Layer 3 support, and IPv6 capabilities are all done very well in the ASA.
We were disappointed that Cisco still hasn’t pushed BGP routing into the ASA, especially since they’ve done a really incredible job with the routing protocols (OSPF, OSPFv3, EIGRP, RIP and several multicast routing protocols) which come already inside the ASA. It took a while for the outstanding routing feature set we all know and love from IOS to migrate into the ASA, but it was worth the wait, and network managers looking for the ASA to be a strong participant in their dynamic routing will be happy with its capabilities.
Since we couldn’t test BGP according to our standard NGFW test methodology, we used OSPFv3 instead to integrate the ASA into our existing IPv4 and IPv6 network, a very painless experience. The ASA is also missing full integration between the VPN and dynamic routing. Network managers hoping to use dynamic routing to build large VPN-based WANs will need to stick with IOS for their site-to-site tunnels. We did not test high availability, but Cisco told us that the ASA CX currently supports active/passive failover and will support active/active clustering next year.
Aside from the firewall ACLs, we found two other areas lacking: QoS enforcement, and central management. We’ll cover central management separately, but our testing of QoS enforcement found that the ASA has a very weak feature set. Most firewalls have some ability to help prioritize and control traffic during periods of congestion, but not the ASA. We found that the ASA has only simple policing (limiting bandwidth of a particular application, even when there is plenty available) and queuing without any attempt to manage traffic. QoS is one big area where the ASA could pick up a lot of great features from Cisco IOS.
We think that security managers who have grown up with other firewalls and are not used to the ASA’s quirks will find it a more difficult product to integrate into complicated networks. However, in simpler topologies and especially in environments with a lot of remote-access VPN, the ASA fits in with the rest of the basic firewall marketplace and remains a competitive solution.
Next generation application identification and control
If anything defines next-generation firewalls, it’s application identification and control, and the ASA CX next-generation features aim squarely at that target. To evaluate how well the ASA CX could identify and control applications, we used the same set of 41 test scenarios in nine categories that we tried in our next-gen firewall test last year.
In the ASA CX, application control is straightforward and simple: “block” or “allow.” There are no other options, such as redirecting to a warning page or sending an alert (although this could be done by scanning the logs). With that in mind, we dove in and started testing the ability of the ASA CX to dive deep into applications.
Compared to our test of other NGFW appliances a year ago, the ASA CX did surprisingly well with a 60% identification and block rate, essentially tying with our top performer (SonicWALL) and narrowly edging our second-best performer, Check Point.
The ASA CX has nice granularity for many applications. For example, with LinkedIn, the ASA CX lets you allow most of LinkedIn, but block job searches or posts. In some cases, the ASA CX divides applications into multiple categories -- for example, there are five LinkedIn applications and 10 Facebook ones. In popular applications, the ASA CX let us focus on more specific application behaviors, such as posting vs. reading or uploading vs. downloading. The design is well thought-out from a security manager’s perspective trying to map a security policy to a firewall rule set.
Failure to identify and block applications can come from two sources: bugs, or just not supporting the application in the first place. With the ASA CX, we found a little of each. For example, with Skype, the ASA CX just didn’t work if we engaged in any type of evasion.
In our case, we evaded the Skype filter using the oh-so-stealthy “wait for a few minutes” uber-hacker technique. While the ASA CX blocked Skype initially, if we simply waited a few minutes, calls would go through. In other cases, such as H.323 conferencing, Microsoft Exchange, and Sophos anti-virus updates, the ASA CX didn’t work at all even though these applications were part of its repertoire.
Our testing from a year ago was all based on laptop clients, both Mac and Windows. With the ASA CX, though, we added a new twist by trying handheld devices for two commonly mentioned applications: Facebook and LinkedIn. The ASA CX didn’t do its job when we used the native applications to connect to these services.
We did see that the ASA CX team learned from the mistakes of their peers. For example, in our testing last year we were able to work around all the other next-generation firewalls attempts to block Google Mail by simply using the basic HTML interface. That trick didn’t work with the ASA CX, which stubbornly blocked us no matter what we tried.
The ASA CX also was fully IPv6 aware, and if it blocked an application on IPv4, the same block worked with IPv6, whether we used a laptop or an iPad.
We also tested the ASA CX’s ability to use user- and group-based identification as part of application controls. Anyone who has tried to deploy identity-based networking or NAC knows that one of the hardest parts is gathering the identity from the user or device.
The ASA CX has two mechanisms available to it: one is a captive portal approach, which works in certain guest user environments where people don’t have much opportunity to complain. The other is called “passive authentication” and it depends on installing a Cisco agent on Windows domain controllers in a network. When a Windows user logs onto their domain-connected PC, this generates an incidental log entry in the Windows Event Logs which happens to have the apparent IP address of the PC as well as the user name used to login. We installed this agent on our lab’s domain controllers very easily and were able to link the ASA CX authentication controls to user identity. Sort-of.
It happens that our lab has some internal NAT going on to simplify routing when we build networks quickly for tests like this and this confounded the ASA CX, as the user-to-IP mappings were not correct. The same issue would happen in any network with NAT between the PC and the domain controllers. Of course, there are also scalability issues — if one domain is all you need, that’s great, but some enterprises have dozens or even hundreds of domains running around on their wide area network. Of course, this only works for domain-attached Windows PCs — still a dominant force in enterprise networks, but not the only way to deliver computing nowadays. And let’s not even mention that there’s no real tracking of when someone disconnects from the LAN.
None of this is Cisco’s fault -- any vendor who tries the hokey pokey of sniffing Windows logins or capturing the information from the logs is going to have the same problem. There are other tricks and techniques various NAC vendors have created to work around this fundamental problem, but the only 100% real solution is having a client tool on the connecting PC which collects authentication information and interacts actively with enforcement devices.
And this brings us to a serious problem in the ASA CX: all of Cisco’s products for NAC, and all of Cisco’s products for Secure Mobility, don’t interact with the ASA CX. The problem is especially ludicrous when it comes to remote access VPN -- even if the remote access tunnel is connected to the ASA CX, it stubbornly won’t pass user identity information up to the CX.
Failing to make this obvious link even in a v1.0 product like the ASA CX is poor product management.
SSL decryption
If one of the main advantages of a next generation firewall is application and protocol identification and control, then SSL decryption is a basic requirement. In the ASA CX, SSL Decryption is handled by a separate set of policy rules, essentially defining which traffic is decrypted and which is not. From a management point of view, configuration and control of the SSL Decryption part of the ASA CX is outstanding. We started by installing our own certification authority, which is trusted by all systems in our lab, into the ASA CX, a simple matter. Then, we enabled decryption for all traffic and let ‘er rip.
At first glance, the ASA CX worked great, catching SSL on non-standard ports, and extending application identification even into encrypted sessions. And for the most part, the ASA CX SSL decryption is just fine and didn’t create any performance problems in our limited lab testing. But it’s not perfect.
The biggest issue we ran into was application identification of encrypted traffic. Although the ASA CX does a good job of decrypting and re-encrypting SSL traffic, the application identification engine doesn’t work properly for many protocols. It’s not a complete failure -- the most important protocol that most people care about, HTTP, is successfully unpacked and identified. But security managers who want to identify and control non-HTTP other protocols which might be blocked by policy can’t see into the encrypted session. For example, the ASA CX was great at identifying unencrypted IMAP on standard and non-standard ports. But if we opened up a connection to an IMAP server on the standard IMAP-over-SSL port, the ASA CX didn’t identify the traffic as IMAP, but just as SSL. We had the same issue with SMTP using standard ports. Once the traffic was encrypted, the ASA CX didn’t identify these protocols. For HTTP traffic, there wasn’t a problem -- the ASA CX unpacked encrypted HTTP just fine and identified applications, such as Google Mail and Facebook inside of the encrypted session.
Early on, we also had smaller issues with some SSL negotiation that the ASA CX didn’t like, and hence inappropriately blocked. This first showed up in our lab’s iPhone and iPad, which started behaving poorly and couldn’t communicate back to the iCloud mother ship. The problem was easy to see in the ASA CX logs, but there was no simple solution other than to exempt certain IP addresses at iCloud from SSL decryption. Problem solved, but maintenance and documentation headache begun. And we didn’t know how that would affect other controls. For example, we tried blocking iTunes installation of applications, which didn’t work. Was that because the SSL wasn’t being properly decrypted, or was it just a bug in the iTunes blocking?
A different issue that slowed us is related to the list of trusted root certification authorities pre-loaded by Cisco. The ASA CX will engage in man-in-the-middle SSL decryption only if the server presents a certificate trusted by the ASA CX. Cisco doesn’t release the list of CAs that it will trust, leaving you to guess and debug. Most of our tests were fine, but we did run into a few perfectly legitimate servers that the ASA CX wouldn’t trust until we tracked down their CA and intermediate CA certificates. The lack of a list of CAs makes debugging hard, and there is no legitimate reason to keep this list secret -- every other product with similar trust settings, such as Windows and Mac OS X, is happy to let you see and edit the list of trusted CAs.
The ASA CX did not fare well in our test of invalid certificates, potentially hiding revoked or incorrect certificates from the end user. When a server hands a certificate to the ASA CX as part of the SSL handshake, the ASA does not do full checks on that certificate. Then, it replaces the certificate with one that it creates — and any revocation information is lost. In simpler terms, the main mechanisms in place in the world of digital certificates and X.509 are not correctly implemented by the ASA CX, leading to a small but very significant vulnerability, especially in the world of spear-phishing attacks.
Next generation visibility and management
We found the ASA CX with Cisco Prime Security Manager (PRSM) provides outstanding visibility into application type and flow statistics, with a strong drill-down capabilities and a well-designed interface. However, ASA overall management is in rapid motion, and we found it difficult to evaluate the rest of the management interface.
The ASA’s standard management interface, for those who don’t want to use the command line interface, is ASDM (Adaptive Security Device Manager), a Java-based GUI that is used to handle most aspects of ASA configuration and management. ASDM has evolved into a stable and powerful product. ASDM isn’t the most elegant interface for managing a firewall and is tightly tied to the command-line configurations it generates, but it’s solid and gets the job done fairly quickly.
Prime is Cisco’s new management interface, designed to work across security, switching and routing product lines. When managing firewalls, Cisco calls it PRSM, a web-based GUI that can be run either on-box directly on the ASA or on a separate management server. Although on-box operation is supported, we think that any manager with more than a single ASA CX should elect for a separate management appliance, even though there is an additional modest cost ($3,000 list price for five devices).
Without a separate management server, PRSM presents a risk to the firewall by running hosting reporting, log storage, and management all on the same CPU that is handling packets. On-box PRSM makes for a fast demo of PRSM capabilities, but we prefer to ship logs to a separate server so that any analysis and debugging won’t risk slowing down a production device.
The ASA CX is only managed by PRSM, which creates a disconnect between the next-generation firewall rules and the rest of the ASA management. However strange this interface is, it’s a bit unfair to grade the ASA down because of it -- Cisco told us that migration of firewall configuration to PRSM (from ASDM) is their No.1 priority for 2013 for the ASA line.
So, yes, today, network managers need to go through contortions to manage a firewall, but we are looking at a product in transition. Even though PRSM’s visibility capabilities are excellent, network managers will find PRSM’s configuration capabilities to be clumsy and disappointing compared to ASDM. Firewall rules are spread sparsely across a web page without critical information, making policy analysis and editing difficult; policy objects are named with the least helpful terms possible, and redundant and inconsistent terminology pervades the interface. We hope that this poor configuration interface is cleaned up as part of the migration of ASDM functionality into PRSM.
How We Tested
We used the methodology from our 2012 Next Generation firewall test so that readers could evaluate the Cisco ASA CX against competition from Barracuda, Check Point, Fortinet, Palo Alto Networks, and SonicWALL.
Because the Cisco ASA CX does not support anti-malware and traditional IPS, we did not repeat those tests.
We also changed our application identification testing slightly to include a wider range of clients, including smart phone and tablet devices. We highlighted the performance of those clients so that readers can more fairly compare coverage and accuracy of the ASA CX against competitive products.
