Why Measurement Is Important?

My decade-long auditing and somewhat longer consultancy experience shows that the importance of information security measurement is underestimated by the management. In many cases it is merely considered a standard requirement, not as a tool to improve security. Yet managers who are not interested in or don’t receive the proper measurement results, miss the opportunity of evidence-based decision-making. Risk assessment  is important to estimate the level of risks, measurement provides feedback on the correctness of these estimations. The importance of effectiveness measurement is underlined by the fact, that key performance indicator (KPI) values are compulsory subjects to the management review.    

What to Measure?

All previous versions of the information security standard necessitated measurement of information security, yet none of them defined, what shall be measured. There were stories that auditors at the old times expected metrics for every Annex A control (there are currently 114 of them). Yet the measurement itself, and also the evaluation of the results would require huge resources and the result would be useless for the management – too much information is practically the same as none. It is worth the effort to evaluate measurements that the organization can make use of. While for a small organization 6-8 somewhat general KPI’s can be satisfactory, a larger organization may require two dozen or even more.

Metrics shall be aligned to the strategic goals of the organization and why the ISMS was set up. It is crucial to ask for input from different levels and roles of the management on what they are interested in and supply them with specific results. It is also important to define information security measurement frequency and the method to remain efficient.

It is not easy to measure the effectiveness of information security controls. Unfortunately, it is easy however to define KPI’s, that look good, but measure not explicitly what is expected. The ISO have published the ISO 27004 standard to support identification of metrics: the 2009 issue was not much of a help, being too theoretical (and not fit for the purpose in some cases), but since the 2014 issue the standard is more usable.

How to Measure and Report?

Where feasible, it is a good idea to support data gathering with a software tool. The CISO will typically create an easily comprehensible report from the measurement data and forward them to the recipients. It is a good idea to immediately forward suggestions for corrective/preventive action to the top management, should data imply. There is no guidance on measurement frequency in the standard. It may differ by metric, but typically a monthly measurement period is too frequent, but an annual may be too rare. The ideal KPI is monitored longer term, so measurement reports can contain 2-3 previous measurement results to reveal tendencies. ISMS objectives may be tied to desired KPI values.

15/10/2020