Letter Drop: Shady Stats; Playing Up the Positive - InformationWeek

InformationWeek is part of the Informa Tech Division of Informa PLC

This site is operated by a business or businesses owned by Informa PLC and all copyright resides with them.Informa PLC's registered office is 5 Howick Place, London SW1P 1WG. Registered in England and Wales. Number 8860726.

Software // Information Management

Letter Drop: Shady Stats; Playing Up the Positive

Richard Starnes challenges, "Anyone with an elementary understanding of statistics and surveys knows the CSI-FBI survey is statistically questionable."

Shady Stats

I am an economist with a strong background in statistical analysis, now working full time in computer security. I am appalled that the CSI and FBI continue to circulate these kinds of results without properly qualifying them (Dashboard: "Bucking the Hype, IT Security Losses Decline"), but it's even more frightening to see the media continue to consume it. The headline should have emphasized that companies' reporting behavior is changing and we don't have very good data on what's really going on.

The big story is the lack of useful data to determine if things are getting better or worse. If we got rid of the police and the means for reporting crime, there would seem to be a drop in crime, but it would be the reporting process, not the underlying reality. If we want to know about cybercrime and cyberexploits, government, industry, law enforcement and educational institutions would do well to address our common ignorance on what's really happening out there.

The reason there is no basis for making judgments about trends over time from this data is that it is not a time-series sample, in which the same people in the same companies using the same methods and definitions report each year. It is a completely voluntary sample with different participants, different definitions and different perspectives each year. So we don't know if the change is the result of a different sample or a real-world shift. This is pretend science and is bad for the industry. It should not be reported this way without clarification, and it certainly gives no support for the arguments on the level or nature of malicious activities over time.

Ken Kousky
Saginaw, Mich.
[email protected]

You talk about the hype surrounding information security and cite the latest CSI-FBI study that claims financial losses are dropping along with information security budgets. Anyone with an elementary understanding of statistics and surveys knows the CSI-FBI survey is statistically questionable. They do not publish the methodology of the survey, but the response size alone will tell you the error rate is above acceptable limits. I see little change to indicate this is no longer the case.

Richard Starnes
[email protected]

Play Up the Positive

There is a paradox of speed (Dashboard: "Should You Speed Up BI? Not So Fast!" September 2006, ): You can speed up the steps yet still go no faster. The Concorde, for example, cut flying time from New York to Paris in half, yet door-to-door travel time decreased only 15 percent due to airport time and traffic congestion. However, downplaying the positive impact real-time BI could have by fretting over OLTP response times and Ferraris towing boats misses the point. Analytics embedded into operational processes are the real payoff for BI. The problem there is data warehouses and bloated BI tools--business as usual.

Neil Raden
Santa Barbara, Calif.
[email protected]

We welcome your comments on this topic on our social media channels, or [contact us directly] with questions about the site.
Comment  | 
Print  | 
More Insights
InformationWeek Is Getting an Upgrade!

Find out more about our plans to improve the look, functionality, and performance of the InformationWeek site in the coming months.

11 Things IT Professionals Wish They Knew Earlier in Their Careers
Lisa Morgan, Freelance Writer,  4/6/2021
Time to Shift Your Job Search Out of Neutral
Jessica Davis, Senior Editor, Enterprise Apps,  3/31/2021
Does Identity Hinder Hybrid-Cloud and Multi-Cloud Adoption?
Joao-Pierre S. Ruth, Senior Writer,  4/1/2021
White Papers
Register for InformationWeek Newsletters
Current Issue
Successful Strategies for Digital Transformation
Download this report to learn about the latest technologies and best practices or ensuring a successful transition from outdated business transformation tactics.
Flash Poll