Developing quick and effective security metrics seems to be a challenging task, but it can in fact be fairly straightforward provided a few important considerations are kept in mind. Effective security metrics are important, as they can help ascribe quantifiable ROI to your security investments.
Security metrics can be categorized into implementation metrics,
1. What are the objectives?
Effective security metrics are invariably customized to the needs of each organization. The first step is to identify what the objectives of the security initiative are and map the security measures to these objectives. Establishing what needs to be protected will help in defining key objectives.
While it is possible to measure various parameters, unless these are linked to the organization’s security objectives, such security metrics are irrelevant. On the other hand, a single metric might not always be sufficient for measuring a particular security control. For example, when measuring vulnerability to Internet-based threats, a combination of metrics such as port scans detected; phishing attacks detected; and, anti-virus/firewall breaches detected, would all have to be integrated into a comprehensive indicator to provide actionable data.
2. What does the security policy say?
Unless there is a security policy in place, it is not a good idea to go in for security metrics. Measuring logs and statistics will not make sense unless a well-defined security policy exists. Once the security policy has been signed off by the management, it is possible to establish effective security metrics from the security policy.
The challenge is reducing the standard policy statement into measurable terms. Key risks must be identified first before a measure of how you can counteract those risks can be reached. Following the risk assessment, controls for these risks can be identified.
3. How can risk be measured?
The next challenge in developing effective security metrics is measuring risk. Every organization has some formal or informal risk assessment in place; is broadly aware of the key risks; and, has some controls in place. Thus, you need to first identify the key risks and measure how they are being countered presently. Once the risks are identified, you can directly measure the effectiveness of the relevant controls.
When measuring risk, you must define both; the risks for which security policies do not exist, as well as the risks for which there is an improper implementation of the policies. For example, a call center facing the internal risk of employees stealing sensitive client data will have measures in place to prevent such incidents occurring, such as barring cameras and phones. You must therefore measure the effectiveness of these controls to get a good idea about your security posture.
4. What measures are in place?
For effective security metrics, one of the things an organization can do is look at what is already being measured. Most organizations have considerable data that is not proactively being used for security metrics, such as antivirus logs, firewall logs and similar data. Instead of looking to collect additional data, first take stock of what is already available.
This could include data such as the percentage of machines on which antivirus software is not installed, the number of virus incidents reported, the number of technical vulnerabilities on the Web server, number of phishing attacks reported by tools, and so on.
5. How does it aid decision making?
The organization must evaluate the actionable outcomes obtained from the security metrics information being collected. For example, if antivirus logs list the number of virus infections thwarted, what actions does this precipitate? Does this information help in the decision-making process? Does filing an exception report of personal mail achieve any objective as far as effective security metrics are concerned?
Before you invest time, money and manpower to collect specific security metrics, it is best to have an idea of what the results can be used for. The data must bring value to the decision-making process; else it will not contribute to the goal of developing effective security metrics.
To sum it up
In the final analysis, establishing appropriate security metrics boils down to culling from the hundreds of metrics available to the ones that are best suited to your purposes. One of the key factors to avoid when creating metrics is the temptation to have metrics for everything. Many organizations make the mistake of having a metric associated with every security checkpoint. While an organization can measure anything given the right inclination and resources, what needs to be measured is whether the security controls in place meet your objectives.
About the author: Chaitanya Kunthe is presently a vice president at Risk Quotient solutions. His area of focus is risk management and compliance. Prior to this, Kunthe headed consultancy services at Miel e-Security.
This was first published in November 2011