[ISM3 Users] Last Blast from the Past - Three Questions About Risk Asessment That Start Flames

[Vicente Aceituno]

****Risk Assessment****originally posted May 1, 2007 10:31 am***
Question 1: Do you limit yourself to RA&Audit?

Most ISMS standards emphasize Risk Assessment and Audit, but those are not
the only management activities of information security management?

ISM3 considers the following management activities:
- Risk Assessment (part of GP-3) - Considers assets, threats,
vulnerabilities, impacts to get a picture of security and prioritize
design and improvements.
- Audit (part of GP-2) - Compares the actual management system with
the documented management system.
- Monitor - Use metrics to watch processes outputs, detect abnormal
conditions, assess the effect of changes in the process.
- Test - Check if inputs to the process produce the expected outputs.
- Compliance Audit - Compares the actual management system with a
externally defined management system (for example ISO27001)
- Design&Improvement - Find ways to produce outputs better fit to
their purpose, fewer false positives and false negatives (including
faster outpus).
- Optimization - Find ways to produce the same outputs with less
resources.

Question 2: What risk assessment standards you know meet the following criteria?
1- Reproducibility. This means two different independent practitioners
should get virtually the same work products and results.
2- Productivity (Added value) This means the work products should
serve as inputs for:
--Gauge how safe is the organization;
--Identify threats and weaknesses;
--Choosing what processes are appropriate for fulfilling the security
objectives;
--Prioritizing investment in security processes;
--Quantifying investment in security processes.
3- Cost-effectiveness. Setting up a ISM system should be cheaper than
operating it, just like the cost of choosing a security tool should be
small in comparison with the cost of purchasing and using the tool.
4- Added value. This means the result of the process selection should
be learnt from the process selection itself. If the process selection
result is known beforehand, and the process selection is just a
justification for a previously taken decision, the added value is nil,
which negates any cost-effectiveness.

Question 3: Do you use a Risk Assessment method you made up yourself?
Creating Risk Assessment methods you can make the following choices:
- The scope (what's in, what's out)
- The depth (think OSI levels and above to business processes)
- The way you model the parts/objects of the organization, their
relationships, and the states of their lifecycles.
- Your threat taxonomy (there is not a single one widely accepted one
at all depth levels)
- The way you score the impact on assets (dollars, high-medium-low or
1-5 Confidentiality, Integrity,
Availability scales and expansions or combinations thereof)
- Controls taxonomy (there is not a single one widely accepted one at
all detail levels. Many use the ISO17799 list)
- How you combine threats, their probability, controls, their quality
and impact to reach a Risk figure.

This makes exceedingly difficult to reuse of compare risk assessments,
as any change in the method design or even the way it is used makes
risk assessments results inherently different. This make very
difficult to compare this year's RA with the last years one,
and comparing RA from different companies becomes an unattainable Saint Grial.

My best

Vicente

************
About Question 2 Gary said:

Repeatably implies some objectivity and rigor behind the method, which seems
like a good idea ... Except that I personally do not put much faith in any
purely scientific risk assessment method which would be the ultimate
expression of a truly repeatable method. It seems to me the current RA
methods all depend on sensible application and interpretation by one or more
experienced practitioners because there are simply too many variables and
unknowns for pure science to provide meaningful answers. Information
security risks cannot not (yet) be treated like insurance using detailed
actuarial tables based on decades of life data, or automated stock dealing
using mathematical risk modelling based on millions of trades. It's not
pure art but neither is it pure science. RA methods are guides for the well
informed, not maps for the lost.

Repeatedly implies it has to be done several times, fair enough ... Except
that even a one-off risk assessment can have value. A few months ago, for
example, I was involved in a superb RA workshop looking at "extreme, low
probability, high impact events", focusing on IT-related risks. We dreamed
up some pretty obscure, unlikely but seriously worrying risks (e.g. "the
arrival of quantum computing making all current forms of cryptography
obsolescent overnight"!). The point of the exercise was not so much to
figure out specific control options for these bizarre risks so much as to
assess the general level of risks (in relation to extreme market risks,
personnel risks etc. etc. being assessed by parallel workshops) and the
suitability of general controls in our domain such as infrastructure
security and contingency planning. It might be good to repeat this every
year or three but it's not exactly a priority.

So, the bottom line is that I too am ambiguous on this point.

Sorry to take up so much valuable screen estate on just one little word!

Kind regards,
Gary

Dr Gary Hinson PhD MBA CISSP CISM CISA
CEO IsecT Ltd.
Phone: +64 634 22922
www.NoticeBored.com Creative security awareness
www.ISO27001security.com ISO 27000-series standards

***********

About Question 2 Vicente said:

Gary,

> it - should it be 'repeatably' (capable of being repeated)
or 'repeatedly'
> (repeated several times)? There is an argument for both.
"Repeatable" means that it should be independent of the observer,
objective not subjective.

> like a good idea ... Except that I personally do not put much faith
in any
> purely scientific risk assessment method which would be the ultimate
Modern Medicine is quite repeatable. Go to different doctors, and
will very often get the same diagnosis for the same condition. I
wouldn't put the security of my organization in the hands of faith...

> Repeatedly implies it has to be done several times, fair enough ...
Except
> that even a one-off risk assessment can have value. A few months
RA can have value...but that value is less if you can't trust the
assessment. I wouldn't personally trust medicine if every doctor
diagnosed differently the same person with the same common condition.

Classifying RA methods could help to choose the method with the most
objective and useful assesment results...

Regards

Vicente

*****

About Question 2 Gary said:

Dear all,

We've barely touched on a load of separate issues regarding RA methods and
their utility:
- objectivity vs subjectivity
- scientific method vs art and experience
- definition of scope
- actuarial models vs inadequacy of reliable historical data
- consistency of repeated RAs with similar inputs
- comparability of RAs by different people
- complexity of the inputs and 'unknowns'
- need for standards/choice of standards
- and more I'm sure (Vicente then said: "the lack of any widely
accepted taxonomy of threats")


There's plenty of scope here for PhD/Masters research projects and a good
article to explore the topic in more depth. Any volunteers?

Kind regards,
Gary

Dr Gary Hinson PhD MBA CISSP CISM CISA
CEO IsecT Ltd.
Phone: +64 634 22922
www.NoticeBored.com Creative security awareness
www.ISO27001security.com ISO 27000-series standards

************
About Question 2 Anup said:

My replies marked with **

Gary Hinson wrote:
>
> Well actually, gentlemen, you are both right, and it's fascinating that you
> both picked up on that one word. I also thought about it as I was writing
> it - should it be 'repeatably' (capable of being repeated) or 'repeatedly'
> (repeated several times)? There is an argument for both.
>
> Repeatably implies some objectivity and rigor behind the method, which
seems
> like a good idea ... Except that I personally do not put much faith in any
> purely scientific risk assessment method which would be the ultimate
> expression of a truly repeatable method. It seems to me the current RA
> methods all depend on sensible application and interpretation by one or
more
> experienced practitioners because there are simply too many variables and
> unknowns for pure science to provide meaningful answers.

** I agree. The only catch being that organizations may not have the
resources to employ multiple RA practitioners to understand all possible
angles and dimensions of a risk.


> Information
> security risks cannot not (yet) be treated like insurance using detailed
> actuarial tables based on decades of life data, or automated stock dealing
> using mathematical risk modelling based on millions of trades. It's not
> pure art but neither is it pure science. RA methods are guides for the well
> informed, not maps for the lost.

** Nice sentence :-), In this context, what is your take on ISO 27001
that insists that RA be the core for designing the ISMS. I am not
questioning the validity of the standard here, but the possible
interpretation of the standard. When the standard says that RA must be
used to identify the risks to information assets and then treat the
same, isn't there a possibility that without a standard RA methodology,
different practitioners may come with different reports.

*************