The folks at Intel posted a very interesting paper on ROSI (Return on Security Investment). As I mentioned in my blog on metrics, trying to determine your value for your security investment is very difficult. Some may argue its not even possible. What is so interesting with the Intel report is its both easy to read and its based on past implementation. In other words, they talk about what they did. I think this paper is a wonderful start. However, I'm not sure just how effective the method would be. Overall their approach is you measure all the incidents that happened in the past (they used two years of data), estimate the average cost per incident and then total up the total cost. Then implement your security controls and measure how many incidents you have. The delta is your savings. While a very effective starting point, I have several questions I can't figure out.
- What happens with this method when your new security program mitigates incidents you never detected in the first place? For example, lets say you counted 400 incidents in your organization last year, but there were really 500. When you implemented your new security measures your incidents drop to zero. Your delta is off by 100 incidents. I'm nit picking here, but your security program actually has far greater ROI. The reason I'm concerned about this is because a good security program mitigates threats/vulnerabilities you did not know about.
- However, what I'm even more concerned about is good security includes good detection. Now, what happens if you start with 400 incidents, then implement security controls which includes good detection. Now all of the sudden you are detecting many more incidents you never would have detected before. Even though the total incidents could have gone down, because of your improved detection capabilities management perceives they have gone up.
I commend Intel for what appears to be a great start. However, I just can't believe ROSI is as simple as counting incidents and measuring deltas. Just look at TJX, all they have had is just one security incident in the past 3 years (that we know about).