[ISM3 Users] Tuesday Insight: Correct vs Useful

[Vicente Aceituno]

A Metric is a quantitative measurement that can be interpreted in the
context of a series of previous or
equivalent measurements. In ISM3, metrics are used to:
- Determine whether security objectives are met;
- Show how security objectives contribute to business objectives;
- Measure how changes in a process improve the ISM system;
- Detect significant anomalies;
- Inform decisions to fix or improve the ISM processes.

Sometimes Metrics design and choice seem to be guided by what are the
things we can measure easily, but not everything that can be measured
is worth measuring.

Some metrics give data; other metrics give information.

Last week I cited Shannon about information; his definition of
information says that there is more information the more:
- Quantity of data.
- Utility of the data.
- Surprise of the data.
- Improbability of the data.

Ask yourself; How important is it that data is correct? Correct is
another way of saying "accurate". In my opinion, accuracy is important
for engineering, but metrics need to be only as accurate as necessary
to be useful

Metrics are most useful when they lead to detect abnormal conditions,
understand trends help finding correlation and causation relationships
between events, as this discoveries lead directly to decisions to fixe
and improve security processes. If we had to choose between two
options, data with values between 0 and 100, were the decision would
change at 50, is no more useful than data with values 0 and 1

For this reason metrics measurements don't need to be more accurate or
granular than necessary to distinguish when it reaches the thresholds
were decisions are taken or investigations started.

So don't go crazy about the third decimal place, an please never never
use it in presentations! :)

My best

Vicente