indicators and metrics

  • MARKET - CYBER SECURITY - Indicators and Metrics

     

    In the past, a great variety of indicators have been developed to measure cyber-security related aspects. 

    Three fields can be differentiated:

    • Firm-level cyber-security indicators
    • Industry-level cyber-security indicators
    • Economy-level cyber-security indicators

     

    Firm-level Cyber-security Indicators

    These differ according to the organization’s goals. Some indicators are supposed to measure return on security investments, others have the primary intention to measure security risk or policy compliance. It is observable, however, that the tendency is to find quantitative measures in order to be better able to track security policies in organizations.
    There are hundreds of metrics to choose from and an organization’s mission, industry, and size will affect the nature and scope of the task as well as the metrics and combinations of metrics appropriate to accomplish it.
    There is an abundance of such indicators and for organizations it is typically difficult to judge on what indicators to use once security policies are to be evaluated. In the area of measurement comprehensive overviews already exist, Table 1 below lists some of them. Overviews are also presented by Mateski at al. (2012) and Swanson et al. (2003), among many others.


    Table 1 Some References to Measurement of Cyber-security Aspects

    Author Explanation
    Herrmann (2007) The author lists more than 900 privacy and security metrics, that measure compliance, resilience and return on investment.The metrics are also scaled by information sensitivity, asset criticality, and risk.
    Brotby and Hinson (2013)

    In Brotby and Hinson (2013) more than 150 metrics are listed, ranging from risk management metrics to IT security metrics to compliance and assurance metrics. The authors have made the list accessible over the Internet by putting it on their website 

    or directly as an XLS on the IPACSO website.

     

     

    Notes: This literature overview is not meant to be complete, but to provide a starting point of research for the interested reader.

    This is an area, where we will witness an expansion of consultancy services in the cyber-security market in future, because the need for improved information security policies is increasing.

     

    IPACSO overview on risk metric lists

    While there are areas of overlap (for example with respect to data breaches), privacy metrics are more focused on the subject matter of compliance with data protection laws and the protection of personal data.

    More information on privacy metrics.

     

    Industry-level Cyber-security Indicators


    At this stage, there are more indicators to choose from at the level of the firm compared to the level of industries. There are, however, now a number of reports on the costs of cyber-crime and data breaches.

    IPACSO overview
    These reports typically differ in terms of coverage of firms, methodology and region covered (information is given in the Excel file). Most of the reports are surveys of firms with respect to data breaches (e.g. Verizon, Javelin Strategy & Research). Others are using information delivered by threat surveillance networks owned by the publisher (e.g. McAfee, Kaspersky Labs).

    More of this type of industry-level data will come from the CERTs.

     

    Economy-level Cyber-security Indicators 

    Maybe least researched is the area of economy-level indicators that are supposed to map cyber-security preparedness or resilience of different countries.

    Figure 1 Global Cybersecurity Index of ITU-ABIresearch

    globalcyberindex

     

    There is only a small number of institutions providing this kind of information. In the following a short list with references to some organizations that compile such indices is provided.


    Examples of Country Ratings:

    1. Global Cybersecurity Index: This index (developed by ITU-ABIresearch) measures the cybersecurity capacities of countries. It uses five categories for its rating: legal measures, technical and organizational measures, capacity building and cooperation. It then ranks countries according to their cybersecurity capabilities (not vulnerabilities). The index is available for a rounded 190 countries (2014).

    2. Cyber Power Index: The Cyber Power Index (developed by Booz Allen Hamilton / EUI) is supposed to map the ability of countries to withstand cyber-attacks (and to deploy secure critical infrastructure). The index uses indicators in four areas, including legal and regulatory framework, economic and social context, technology infrastructure and industry application. It is available for 19 leading economies.

    3. Cyber-security readiness:   This index (published by McAfee and Security & Defence Agenda (SDA) ranks 23 countries on their readiness. This indicator is based upon leading experts' subjective perceptions of a nation's defense system. While no country gets the highest mark (five stars), Israel, Sweden and Finland lead the list of countries being prepared.

     

    There are also other sources that use metrics at the country level. For example in the BSA (The Software Alliance) EU Cybersecurity Dashboard countries are given a status (“Yes”, “No”, “Partial”, or “Not Applicable”) in each criteria used. There is no overall ranking, but policy makers can judge their country on weaknesses.

    This is not to be confused with the European Commission’s Digital Agenda Scoreboard, which is primarily devoted to map how advanced in digitalization the European Member States are.

     

    Quoted References:
    Brotby, W.K. and G. Hinson (2013). Pragmatic Security Metrics: Applying Metametrics to Information Security, CRC Press, http://www.securitymetametrics.com/html/book.html
    Herrmann, D. (2007). Complete Guide to Security and Privacy Metrics Measuring regulatory compliance, operational resilience, and ROI, Auerbach Publications, http://www.crcpress.com/product/isbn/9780849354021
    Mateski, M. C.M. Trevino, C.K. Veitch, J. Michalski, J.M. Harris, S. Maruoka, J. Frye (2012) Cyber Threat Metrics, SANDIA REPORT, SAND2012-2427, https://www.fas.org/irp/eprint/metrics.pdf
    Swanson, M. Nadya Bartol, John Sabato, Joan Hash, and Laurie Graffo (2003). Security Metrics Guide for Information Technology Systems, National Institute for Standards and Technology (NIST), http://www.rootsecure.net/content/downloads/pdf/nist_security_metrics_guide.pdf

     

    Back to MARKET

     

    IPACSO Publications and further links:

    Jentzsch, N. (2015) State-of-the-Art of the Economics of Cyber-Security and Privacy, IPACSO - Innovation Framework for ICT Security Deliverable 4.1.

  • MARKET - Economics - Privacy - Privacy Metrics

    MARKET - Economics - Privacy - Privacy Metrics

     

    Privacy Metrics

    The recent development of privacy metrics – metrics that quantitatively capture privacy-related aspects in a firm – challenge the general assumption among legal scholars that privacy and privacy measures are not quantifiable. The drive for quantitative measures is partially due to the increased pressure of data protection officers in firms to justify their budgets, but also due to the need for a measurement of effectiveness of specific measures. Within the research field privacy metrics are a methodological advancement.The main goal of quantification is to make privacy (aspects) in firms measurable and comparable. Quantification also allows inter-temporal comparisons and trend analysis. 

     

    Privacy metrics are related to two main areas:

    • Key performance indicators used by firms or by policy makers; and
    • Algorithms that are related to the sensitivity of data in a given set

     

    This section discusses privacy metrics as performance indicator in a “return on investment” context. The selection of relevant metrics must be based upon the strategic goal of the firm (such as effectiveness measurement).


    There are by now a number of examples of key performance indicators to capture of privacy-relevant matters, i.e. number of data security incidences, the number of privacy impact assessments conducted in a company, the number of lost or stolen records, etc. Two examples are the privacy risk exposure as well as the return on privacy investment indicator (see also Jentzsch 2015).


    Privacy Risk Exposure:Privacy risk exposure can be best described as potential loss resulting from the compromising of personal data sets held by a firm. This indicator is often the outcome of a Privacy Impact Assessment. Important is the probability with which a data breach can occur (based upon past experience in the firm or in similar firms).The input of such a calculation is often not more than informed guessing; therefore the indicator is more qualitative than quantitative in nature.


    Return on Privacy Investment: This indicator consists of the return of avoided potential losses because of data breaches, Annual Loss Expectancy (ALE), where ALE = single loss expectancy (SLE) * Annual Rate of Occurrence (ARO), see below. SLE describes potential losses, ARO the frequencies of such losses. Red, in the formula below, denotes the reduction in frequencies of breaches occurring (say from 10 cases 8 can be avoided, 0.8). Finally, cost of measure indicates the costs for the implementation of the protective measure. Thus,

     

    ROPI Formula 1

    If the outcome is greater than 1, the protective measure can be regarded as cost efficient by the investor. Again, the inputs into this formula are rather indicative and often subject to informed guesswork. Most of the outputs in privacy metrics are subject to this problem. Therefore, the outcome of this calculation should be accompanied by a confidence estimate regarding the quality of the outcome.

     

    Further IPACSO Reading

    Jentzsch, N. (2015) State-of-the-Art of the Economics of Cyber-Security and Privacy, IPACSO - Innovation Framework for ICT Security Deliverable 4.1. (download)

Getting Started

Which type of company are you? Choose one of the options below and get a head-start.

Framework Overview

Navigate through the different parts of the Framework

leeg

Joomla! Debug Console

Session

Profile Information

Memory Usage

Database Queries