FRAMINGHAM, Mass. — Anyone running multiple firewalls in a complex, enterprise environment knows how difficult it can be to catch misconfigurations, avoid conflicting rules, identify vulnerabilities and meet auditing and compliance mandates.
In this test, Network World U.S. looks at five firewall operations management products: AlgoSec Inc.’s Firewall Analyzer, RedSeal System Inc’s. Network Advisor and Vulnerability Advisor, Secure Passage’s FireMon, Skybox Security Inc.’s View Assure and View Secure and Tufin Software Technologies Ltd.’s SecureTrack.
We found that these products perform similar core functions: they retrieve configuration files of firewalls (and other network devices), store the data and analyze it. They can look at change history, analyze existing rules, perform rules-based queries, re-order rules, and send out alerts, if policies are violated. They can also create automated compliance audit analysis and reports.
In addition, they can do modeling and wargame analysis based on a snapshot-in-time version of the real network. Plus, Algosec, RedSeal and Skybox can provide network diagrams and topology views of the underlying networks.
Overall, we were most impressed with RedSeal and Skybox, which cover all the basics, plus have the added benefits of being able to support multiple vendor vulnerability scanning products, which can calculate the network’s risk scores and run vulnerability analyses on your whole network. However, we were impressed with all of the products.
Algosec’s Firewall Analyzer had an intuitive interface and came with predefined standard audit and analysis reports. Installation was simple and the program offered a wizard for easy data collection.
Network Advisor and Vulnerability Advisor from RedSeal answered questions on how well the network is configured to protect from Internet threats. The programs generate vulnerability reports showing weaknesses in the network, and contain pre-configured compliance management reports in pdf and xml formats.
FireMon from Secure Passage performs real-time analysis on device configuration and stays current by using an automated analysis of compliance guidelines. There is a wizard to import device information en mass for large networks.
Skybox View Assure and Skybox View Secure can automate the collection schedule of configuration files by the hour, day, week, month or year. A built-in ticketing system supports access change tickets and policy violation tickets.
SecureTrack from Tufin has a What-If analysis feature to test changes to policies before they are implemented. Pre-defined analysis and reporting options are based on industry best practices.
AlgoSec Firewall Analyzer
This Linux-based software package consists of an analysis engine, collection engine, Web server, administrative GUI for local and remote administration, and user, policy storage, and syslog databases.
The analyzer engine runs queries on the data collected, based on predefined or custom rules, and then generates a detailed report. The Web server sends e-mail alerts to the firewall manager.
Installer kits are available for 32-bit Red Hat Enterprise Linux 4&5 and Centos 4&5. We installed it as a VMware appliance on our Dell 600SC server. Once the VMware player is loaded onto the Firewall Analyzer, it boots up, and logging in as root will bring up the Firewall Analyzer browser application. With the browser path set to https://hostaddress/, the Algosec management screen appears, and the management application client is launched by clicking on the login.
There are three methods for data collection – a wizard accessed from the Administration tab, semi-automated scripts provided by AlgoSec, or doing it manually, which is time consuming and could result in errors.
Once files are retrieved and stored, Firewall Analyzer runs a risk analysis based on PCI compliance, NIST, SANS Top 20 and vendor best practices. In addition, we found that we could create custom analysis reports. Selecting the Firewall Reports option displays charts and a connectivity diagram summarizing changes, findings, policy optimization, rule reordering, firewall information and a firewall connectivity diagram. Choosing the Risks option displays the findings with risk codes and details about the risk with suggestions and diagrams on how to deal with it.
The Optimization Policy feature provides the Rules Cleanup and Reordering tools. The Cleanup Report lists any rules that need correction and their number of instances. Some rule types flagged in a Cleanup Report are labeled as unused, covered, redundant, disabled, and rules with a non-compliant name. A similar list is provided for Object Cleanup. The Rule Reordering Report gave us information on how to improve a rule and how much the rule can be improved. You can access a detailed report that tells you how to make the changes.
The AlgoSec Firewall Analyzer client application dashboard is well organized and multi-tiered, making it easy to find features and wizards. A useful wizard, Optimize Policy, could specifically identify rules to cleanup. There are pre-defined compliance audits such as PCI-DSS, ISO/IEC 27001, Sarbanes-Oxley and others. In addition, the compliance reports are well organized and available in PDF, HTML and XML. A drawback was the lack of integration with a vulnerability scanner, but AlgoSec is an excellent product for compliance auditing and compliance and rule optimization.
RedSeal Network Advisor 4.1 and Vulnerability Advisor
You can automate the process of analyzing, identifying, quantifying and mitigating risk and vulnerabilities in complex networks with these applications. Network Advisor uses plugins to import configuration files from each supported device. We liked that we could create a unified network topology map with a best practices analysis and solutions for remediation after we imported risk and vulnerability analyses.
We installed Red Seal software on our server running Windows XP. Once the server is installed and started, the client is installed. After we logged in with the client application, we could access the server that had a feature-rich GUI dashboard.
Both Network Advisor and Vulnerability Advisor require importing router, switch and firewall configuration files to the database. The analytical engine processes information that includes host names, IP addresses, subnet masks and device interfaces. Analysis results appear in the form of graphical displays, reports, maps and charts detailing the current status and configuration of the network. Plugins are available for a wide range of products from Cisco, Check Point, Juniper and dozens of others.
After device configuration files are imported into the RedSeal Advisor, the files were checked against RedSeal’s best practices database. We could drill down to locate the offending policy by double-clicking on a selected row. Any changes to hosts and devices could be analyzed and reported with the View Changes application.
We accomplished rule usage analysis and reordering by using RedSeal’s Custom Best Practice Check feature. Using a regular expression tool, we could search the configuration files and use the available plugin associated with the device. Since configuration files can be edited, we performed what-if analysis to determine if changes to a rule would adversely affect the network.
We liked how RedSeal’s interface for running vulnerability analysis presents a topology map of the network, offering a graphical method for analyzing network vulnerabilities. Arrows point from the source of the threat to the assets at risk. The map states highly detailed information quantifying the risk, based on the Common Vulnerability Scoring System (CVSS). This is an important feature for saving time and preventing attacks on valuable assets. We were impressed with seeing the threats at a high level and drilling down into the report to explore the details. The topology map feature provides a similar method for running the pre-defined PCI-DSS analysis on targeted network segments. We could select a network segment and run an analysis report on it with one mouse click.
RedSeal integrates their product with several well known vulnerability scanners, such as Qualys, nCircle and McAfee, to provide vulnerability and risk metrics. We recommend this product for quantifying risk and vulnerabilities and to allocate resources based on asset value.
FireMon from Secure Passage
FireMon manages firewalls by reporting on changes to the firewall policy, checking unused rules and reporting how traffic flows through rules. Compliance is safe guarded by the program’s automated analysis of compliance guidelines such as Payment Card Industry (PCI) and National Security Agency (NSA).
Its architecture includes an application server, data collector and a graphical user interface (GUI). The application server tracked the data collected, performed real-time analysis on transactions and device configuration and generated scheduled reports. The data collector is a Firemon application running on an appliance or PC to monitor and collect data from firewalls, switches and routers, and any other security devices on the network.
FireMon offers a wizard for importing Check Point, Cisco Systems Inc., F5, Juniper Networks, Nokia and McAfee/Secure Computing devices. Once the entries are made to the wizard, all the associated firewalls, management servers and log servers are auto-discovered and added automatically in sequence.
FireMon provides several tools for analyzing firewall, router and switch rules and policies. We used the Firewall Traffic Flow Analysis tool to produce a report that zeros in on “Any” rules configured on firewalls in a large network. We could fine tune the firewall rules by reducing or eliminating overly permissive “Any” rules and large complicated ones.
We looked at some of the reports for rule policy management. We generated FireMon’s Rule Recommendation Report that offers analyzing issues, such as a request for https traffic from source and destination addresses. The report showed if a policy already existed for the requested access. At the bottom, the report listed a table of each policy tested and the source and destination routes involved. You can get the report in http, pdf and xml format.
Secure Passage has an interface that is well organized with features that are easy to navigate. We saw that some of the analysis and report wizards, such as the Rule Recommendation Report, displayed helpful examples showing how to set parameters. The FireMon traffic flow analysis feature is a handy tool for determining how to eliminate audit-triggering firewall ANY rules. We could print a logically organized report detailing the traffic flow from source to destination that revealed the ports and services actually used. A firewall administrator can create a more secure rule to eliminate the ANY rule using this report.
Although the FireMon Rule Comparison Analysis Report was confusing at first with its color-coded parameters that indicated changes, we feel that FireMon has excellent analysis features for optimizing rules and creating audit trails. This product should be considered a good firewall management solution for the enterprise environment.
Skybox View Assure and Skybox View Secure
This platform is comprised of two products: the Skybox Secure 4.5 for risk exposure and security profile analysis, and threat alert management, and Skybox Assure that manages the firewall and performs network compliance auditing. The platform application is scalable and is made of the Skybox View Server, Skybox View Collector, Skybox View Manager and Skybox View Dictionary. The dictionary is the database for definitions and profiles for vulnerabilities, threats, worms and network security policies.
Skybox uses vulnerability scanners and analysis to categorize, quantify and prioritize threats to the network. Using the Skybox Assure software suite, we could manage network policy validations, regulatory compliance audits and network device changes. With the automation features provided, we could run audit checks on thousands of firewall rule-bases.
We found that the install documentation for Skybox was excellent. The user manuals and tutorials are automatically loaded onto the C: drive.
Skybox provides several methods to import device configuration files into the Skybox View database. You can use the Add Device wizard application that has a Collect feature to import the configuration files directly from the device. There are also several ways to automate the configuration collection process. If configuration data is located in a database or file repository, the data can be directly imported into Skybox View. You need additional Skybox View Collectors if you want to directly import configuration files on segmented networks.
We used the Operational Console to create tasks using the New Task wizard and selecting a Task Type. There is a convenient option for scheduling collection that can be set for a specific hour, or to be run daily, weekly, monthly or yearly. We could also program the Task Wizard to schedule data import from file repositories with configuration files.
We could create task sequences to run the tasks at a scheduled time. Task sequences have exit codes so that if a task fails, any other tasks set to import configurations, run audits and change management will not be blocked.
Once the configuration files are loaded into Skybox View, the compliance auditor in Skybox View Assure uses its predefined best-practice access policy to analyze the firewall policies. The best practice policies are compared with the device configuration rules and policies to display security violations and configuration errors. We used the Policy Compliance Report table to view Violated Rules, Access Compliance and Rule Compliance. In the case of an Access Compliance report failure, the rule violation is highlighted and detailed information about the violation is presented.
We tested the Risk Exposure Analyzer that simulates potential attack and access scenarios. After Skybox Secure builds a virtual map of the security model, a business impact analysis is created for what-if attack scenarios. These scenarios are based on malicious code and human attackers. Using the analyzer, we saw a graphical flow chart diagram displaying the step-by-step process taken by the attacker and the network access path available for the attack.
Results of the attack are used to calculate the business impact of a security breach in terms of confidentiality, integrity and availability. Skybox Secure can import business-impact rules and regulations to classify assets and determine an accurate risk assessment metric.
Rule-usage analysis requires three to 12 months of information to obtain a valid rule use analysis report. Shadowing and redundancy analysis can be run as soon as the configuration information for the network devices is imported and the network model is built.
For tracking changes, we used the Change Tracking option in Skybox View Assure by selecting it under the expanded device object icon in the GUI. When data is collected periodically to update network models, you can display and analyze comparisons between ACL rules, routing rules and network interface changes. We saw that you could keep records of network and firewall changes for compliance recordkeeping. What-If modeling changes can be made as firewall rules in the model and then compared with the actual firewall rules.
Skybox View Assure offers change control and workflow with a ticketing system. While the Firewall Compliance Auditor supports Access Change tickets, the Network Compliance Auditor supports both Access Change and Policy Violation tickets.
We were impressed with the modeling capabilities of the SkyBox View Firewall Assurance product. We could simultaneously store three models of the network for running comparison analyses. A side-by-side analysis report makes it effortless to see the changes between two versions of the same network model.
Skybox View Risk Exposure Analyzer presents features to organize the network based on business units and assets. We obtained network vulnerability data from second party vulnerability scanners such as Nessus and Qualys. Using attack scenario options, we generated detailed reports on vulnerabilities uncovered by the simulation. Although we did not see a predefined vulnerability test suite for running attack situations, the Risk Exposure Analyzer is a valuable asset when combined with the modeling capabilities of View Firewall Assurance. Vulnerabilities could be tested on a network model before deploying any equipment.
Tufin SecureTrack
You can manage and audit firewalls, routers and switches, plus access an incorporated view of firewalls and other devices in your network. SecureTrack supplies automated reporting of risk and audit status, monitors firewall operating systems and supports security compliance standards.
Since the Tufin T-500 appliance has the TufinOS and SecureTrack pre-installed, the install process was conducted on a VMware appliance. Installation was quick, with no problem. After we saved the settings, the login screen appeared and we could access the Tufin SecureTrack server.
The screen has icons for Policy Change Reports, Rule Usage Statistics, Security Risk Reports and Best Practices Audit. Users can choose to be notified immediately of policy changes and to receive weekly reports.
Tufin SecureTrack categorizes the devices it can monitor as Devices, Plugins and Firewall OS Monitoring. Plugins are preinstalled for Blue Coat ProxySG, F5 Big IP and Linux iptables We also could select plugins for devices from Check Point, Cisco, Juniper, Fortinet, Blue Coat, F5 and others. The tab for Firewall OS Monitoring is a separately licensed feature for extending SecureTrack to use SNMP for device changes, in addition to monitoring.
Optimization and cleanup is a big part of SecureTrack’s capabilities. With the goal of ensuring the rule base is not in violation of corporate and regulatory compliance, SecureTrack continually monitors firewalls, routers and switches. The SecureTrack Compare feature lists the number of recent revisions next to the device name. New revision alerts appear when revisions are generated. The Revision List can be filtered based on 10 attributes.
We used SecureTrack Analyzer to identify overlapping and redundant rules. To access predefined best practice policies that are stored in the SecureTrack database, we used the Audit and Compliance option. There are best practice checks for all firewalls and specific firewalls such as Check Point. SecureTrack also offers predefined policy analysis audits for PCI-DSS compliance. You can also set up alerts to be sent when security policy rule changes are made.
We found the browser dashboard to be crisp and well laid out. We liked the Compare Analysis option for comparing firewall revisions and maintaining the audit trail. Users familiar with the interfaces and screen presentations of major firewall vendors will appreciate this feature.
Custom firewall audits were created with the SecureTrack Audit wizard for detailed answers on compliance policies. An impressive list of predefined audit templates can be selected with a wizard, thereby saving time. There is also a predefined PCI-DSS audit analysis feature used to create reports for audit policy with a summary detailing the compliance verification.
We liked the Security Trend analysis reports with charts, graphs and a summary table displaying risk scoring. Tufin does not base the scores on the CVSS as is common practice with similar products. We did find SecureTrack to be a good product for auditing and maintaining compliance with best practices based on industry and corporate policies.
Smithers is a Network World Test Alliance Partner and CEO of Miercom, a testing lab and network consultancy. He can be reached at rsmithers@miercom.com.