IT Analytics: Measuring Up Is Hard To Do - InformationWeek

InformationWeek is part of the Informa Tech Division of Informa PLC

This site is operated by a business or businesses owned by Informa PLC and all copyright resides with them.Informa PLC's registered office is 5 Howick Place, London SW1P 1WG. Registered in England and Wales. Number 8860726.

Data Management // Big Data Analytics
02:58 PM

IT Analytics: Measuring Up Is Hard To Do

IT groups are struggling to measure their own performance, especially around cloud services, new research says.

Vendor surveys can be a snooze, but they can also offer interesting insights, sometimes unintentionally.

Case in point: The first ever IT Operations Analytics Benchmark Survey, from Continuity Software, has a few things worth noting.

For starters, analytics may be a big buzzword, but it doesn't yet pervade corporate IT. Continuity asked whether companies were using analytics to measure the performance of internal IT, and a majority (56%) do not. Its survey did find that 57% of large companies (those with more than 2,500 employees) use analytic tools to measure performance. But only 29% of companies with fewer than 2,500 employees use such tools.

At their worst, vendor surveys reinforce company talking points or remind investors just how much upside still exists in a market. The Continuity survey does some of that, since the company's product involves helping companies analyze their systems for risk. But Continuity is not a monitoring company. Executives told Information Week they thought the survey was useful for understanding what the market wants, but that no one tool would be likely to fill those needs.

[What's the difference between the three key types of data analytics? Read Big Data Analytics: Predictive Vs. Prescriptive.]

What is surprising is that very few companies appear to measure their cloud services -- only 14% of respondents reported doing this. As for other parts of IT infrastructure, the numbers range from 49% to 75%. Storage and network tied at the top, followed by applications and databases. Clusters were cited by only 49%.

Doron Pinhas, Continuity's CTO, believes the reason many companies aren't monitoring their cloud services is because the technology is still maturing, and it's hard to measure how these services should be performing. For clusters, Pinhas explained, "The rate of change over the last five years has been spectacular, with [the] rise of virtualization across the board." Such rapid change has created a far more complex IT environment that changes rapidly, making it hard to measure.

Measurement is a mantra of management, and the survey found that uptime was the top metric tracked among organizations that are tracking. Here's a breakdown showing the percentage of respondents that track each metric (note that the survey allows for multiple responses):

  • Uptime: 92%
  • Performance/response time: 80%
  • Data loss: 56%
  • Number of open issues: 52%
  • Average time to fix: 51%
  • Security breaches: 49%
  • Mean time between failures: 38%

Less than half of the companies track in real time. When they do, uptime remains the most tracked metric, at 43% of respondents, and security breaches jumps to number two, at 33%.

Respondents said the hardest metrics to meet were performance (29% rated it number one) and uptime (17% rated it number one). Performance was ranked in the top three by 74% of respondents.

Analytics consultants are happy to tell us that many companies are not data-driven. The survey suggests less than half of IT departments are. Of respondents, 48% said that most or all critical decisions are informed by operational analytics, meaning more than half use analytics to inform operational decisions rarely or only some of the time.

The survey, conducted online from companies in Continuity's database, received 90 responses. The majority of respondents were not Continuity customers, according to Eran Livneh, the company's VP of marketing.

Of responding companies, 44% had more than 5,000 employees, 7% had between 2,500 and 5,000, 18% had from 501 to 2,500, and 31% had between 1 and 500 employees. More than half the respondents had more than 500 servers in their datacenters, and of those 25% had more than 2,500 servers. Only 19% of respondents had fewer than 50 servers.

Private clouds are moving rapidly from concept to production. But some fears about expertise and integration still linger. Also in the Private Clouds Step Up issue of InformationWeek: The public cloud and the steam engine have more in common than you might think. (Free registration required.)

We welcome your comments on this topic on our social media channels, or [contact us directly] with questions about the site.
Comment  | 
Print  | 
More Insights
Newest First  |  Oldest First  |  Threaded View
User Rank: Strategist
1/15/2014 | 4:42:50 AM
Deploy Data Solutions in Cloud
Big data analytics is going to be mainstream with increased adoption among every industry and forma virtuous cycle with more people wanting access to even bigger data.

However, often the requirements for big data analysis are really not well understood by the developers and business owners, thus creating an undesirable product.

For organizations to not waste precious time and money and manpower over these issues, there is a need to develop expertise and process of creating small scale prototypes quickly and test them to demonstrate its correctness, matching with business goals.

Following up on this, I came across and registered for a webinar on Deploy Big Data solutions Rapidly in Cloud through Harbinger's ABC model (Agile-Big Data-Cloud), it looks a promising one
2021 Outlook: Tackling Cloud Transformation Choices
Joao-Pierre S. Ruth, Senior Writer,  1/4/2021
Enterprise IT Leaders Face Two Paths to AI
Jessica Davis, Senior Editor, Enterprise Apps,  12/23/2020
10 IT Trends to Watch for in 2021
Cynthia Harvey, Freelance Journalist, InformationWeek,  12/22/2020
White Papers
Register for InformationWeek Newsletters
Current Issue
2021 Top Enterprise IT Trends
We've identified the key trends that are poised to impact the IT landscape in 2021. Find out why they're important and how they will affect you.
Flash Poll