Ventana Research performed this research and analysis independently to determine the compliance of software products to the Ventana Research Performance Management methodology, PerformanceCycle™. We charged no fees for this research and invited all IT vendors that are related to this research to participate. The original research took more than four months to conduct and analyze, and encompassed more than 30 vendors and 70 products. The November update was undertaken to include new products released and made generally available between November 2004 and October 2005. This update took an additional two months to complete and was done in a fair and objective manner.

Our explicit purpose was to research the maturity of software vendors and products and their appropriateness for the methodology and process of Performance Management. This research and report is not intended to imply that one vendor or product is the right choice for any particular organization. Rather, it provides a baseline of knowledge that can be used to determine the right level of requirements to align business and IT to improve performance. Unlike other analyst reports that use subjective influences to score vendors, our research is based exclusively on scoring vendors’ ability to provide specific capabilities in their software products and packages.

We certify that Ventana Research performed the research to the best of our ability; that the analysis is a faithful representation of our study and knowledge of vendors and products; and that the final analysis and scoring that were made are our own.

Introduction

The Ventana Research Scorecard is a tool that organizations can use to improve their understanding of how well different software and solutions may meet their business and IT requirements for Performance Management. This unbiased and objective research is the first industry report to assess software designed specifically for enabling Performance Management.

Only through applying information technology efficiently can organizations efficiently connect their people and processes to manage performance. By helping them gauge their existing management processes and IT investments, this report can lead to evaluation of the quality of an organization’s Performance Management or become a framework for assessing new investments and strategic direction.

Despite our ongoing intensive work in this market area, we have found little substantive research on how to apply information technology to the business requirements of managing performance. While there are dozens of reports that evaluate products and their features, none focus on how to apply information technology to the people and processes of an organization for the express goal of improving performance. In addition, a company’s purchasing history and personnel’s vendor biases can cloud the objectivity of assessments. Making the wrong decision could have significant negative impacts, raising the total cost of ownership (TCO) and lowering the return on investment (ROI).

The most important single caveat we can offer in this area is that information technology alone cannot help an organization improve its business performance. Ventana Research defined the first methodology that links the business process of managing performance to the organization’s information technology. The methodology, called PerformanceCycle™, is the foundation of this research and will enable organizations to ensure they direct their IT investments to manage performance most effectively. We have used PerformanceCycle™ to assess the maturity of software vendors and products for Performance Management.

We have conducted this research over the last four years. The research for the 2005 report, which covers more than 30 vendors and 70 products, was conducted from September 2004 through January 2005. The November Update was prepared between September 2005 and October 2005. Seven vendors completed the update in the time allotted and followed the rules originally set out in 2004. Products submitted for the update had to be generally available in October 2005. Ventana Research collected the information, validating the results both independently and with the technology suppliers to ensure the highest level of accuracy. We are publishing the findings in two major research publications, this summary report and an in-depth research report that can be used to conduct a full business and IT assessment.

Companies that do not utilize a methodology for evaluating their business processes and IT investments for Performance Management could waste years and many dollars by trying to take short cuts and selecting approaches not fine-tuned to their organization – and perhaps never get the full benefits they expect. Any organization looking to assess, validate and improve its Performance Management processes and systems therefore should utilize this research.

It can be especially useful for two critical activities in your organization. The first is assessing existing processes and IT systems for managing performance to identify gaps and other issues in need of correction. The second is to build a business and technology framework or blueprint for evaluating new IT investments and processes. Using one framework that links business and technology projects together for Performance Management can ensure an effective plan and a program built according to it. While your entire organization can benefit from consulting this report, it is specifically applicable for finance, operations, supply chain, sales, contact center, marketing, manufacturing, customer operations and other similar departments.

The PerformanceCycle™ methodology approach is the fastest path for ensuring that your people, processes and systems are connected together for Performance Management. Use the information in this report within the context outlined in the methodology and in the appropriate steps for your program, and you will ensure the alignment of your IT investments and resources with the business requirements of managing performance.

Overview of Research and Assessment Methodology

Ventana Research believes that an objective review of specific vendors and products is critical to the methodology that enables Performance Management. Your review should include a thorough analysis of what’s possible and what’s relevant. Our research will help you understand how vendors and products fit into a Performance Management framework. Rather than try to assess every detail of every product, we researched characteristics and features that relate to the three steps of Performance Management – Understand, Optimize and Align.

We made every effort to explore the supporting functional requirements and capabilities for PerformanceCycle™. Even so, there may be additional areas or weightings of importance that could affect which vendor and products best fit your particular requirements. Utilizing this research in your own proper context is critical to ensure that products deliver the highest level of compliance and quality for supporting your projects in Performance Management.

Ventana Research invited participation by all vendors whose products are developed, marketed and sold to enable Performance Management. We charged no fee for this research and took every precaution to ensure it was carried out in a fair and objective manner. After sending invitations, Ventana Research followed up with each vendor, reviewed the criteria for participation in the research program and interviewed each vendor to assess the compliance of its products to Performance Management. The interviews were the baseline for our research. We used a questionnaire for each product to determine whether it meets requirements and capabilities outlined in the Performance Management methodology, PerformanceCycle™. Answers to the questions were validated and then presented to the vendors in scored form to ensure that they accepted the results.

We asked more than 290 questions, broken down into 13 sections about functional requirements and 43 sections about functional capabilities. Each question had a “yes” or “no” answer. This hierarchy of functional requirements and capabilities that support PerformanceCycle™ is the foundation for determining the level of compliance for each product. To qualify for assessment, every product had to be packaged separately, available for sale and listed on the vendor’s standard price list. Each product was scored by the results of the questions.

Next we aggregated, evaluated and validated the scores to determine the product’s score overall and for each of the three steps of the PerformanceCycle™. If a particular vendor submitted more than one product for evaluation, we included the product with the best score for each of the three steps in the vendor’s final score. The result is the vendor’s best product fit for each of three steps and then for overall compliance. In each of these four categories – Overall, Align, Optimize and Understand – the products and vendors are calculated as a percentage of a possible perfect score of 100 and then ranked.

Summary of Benchmark and Ratings

The Ventana Research Scorecard provides a rating for each vendor’s and product’s compliance to the Ventana Research PerformanceCycle™ methodology. Based on the research and assessment methodology, we ranked the vendors to determine the best fit for enabling Performance Management. Here is a summary of the findings for the overall ranking as well as rankings for the Align, Optimize and Understand steps.

Overall – Summary of Results
The overall winner was Geac MPC Corporate Performance Management bundle 7.0, which had 97.3% compliance to the questions. It was followed by SymphonyRPM 4.5 at 90.1% and Cognos 8 BI Professional at 89.8%. The following vendors’ products round out the top 10 positions (including ties) by rank: Business Objects EPM Enterprise Premium bundle, 85.3%; Oracle Business Intelligence 10g and OutlookSoft Everest 4 both 82.9%; Actuate 8 Enterprise Reporting Platform, 79.8%; Corporate Radar 7.0 and arcplan dynaSight, both 79.1%; Epistemic, 78.4%; Pilot Software PilotWorks and Siebel Systems Business analytics, 78.1%, and Hyperion Essbase 7, 77.4%. The remaining vendors are ranked in the Participating Vendors and Compliance Percentage section of this report.

Align – Summary of Results
This step makes sure that actions and plans are directly related to reaching the goals, objectives and initiatives set out by executives and management. It enables you to align the workforce to performance goals and ensure they clearly define, communicate and coordinate targets, score performance, and provide appropriate rewards and incentives. This step includes defining targets and rewards while setting up automated notifications of the scores to the workforce.

The overall winner for this category was Geac MPC Corporate Performance Management bundle 7.0, which had 96.6% compliance to the questions. It was followed by Cognos 8 BI Professional at 88.6% and SymphonyRPM 4.5 at 86.4%. The following vendors round out the top 10 (including ties) by rank: Pilot Software PilotWorks Winter 2005, 84.1%; Business Objects EPM Enterprise Premium Bundle, 80.7%; arcplan dynaSight, 78.4%; Corporate Radar 7 and OutlookSoft Everest 4, both 77.3%; Siebel Systems Business Analytics Performance Scorecard, 76.1%; Longview Khalix 3.4.3, 73.9%; and Information Builders WebFocus Reporting Server and Tools 5.3, 72.7%. The remaining vendors are ranked in the Participating Vendors and Compliance Percentage section of this report.

Optimize – Summary of Results
This step prepares for future actions and decisions to define the goals, objectives and initiatives as set out in the Align step. It enables an organization to determine through planning and collaborating how the workforce and operations can improve their value to the organization. This step includes projecting the potential through forecasting and planning, collaborating on actions, and integrating recommended actions to improve performance.

The overall winner for this category was Geac MPC Corporate Performance Management bundle 7.0, which had 95.4% compliance to the questions. It was followed by SymphonyRPM 4.5 at 90.8% and Oracle Business Intelligence 10g at 87.7%. The following vendors round out the top 10 (including ties) by rank: arcplan dynaSight 4, 83.1%; Cognos 8 BI, 78.5%; Business Objects EPM Enterprise Premium Bundle, 76.9%; Hyperion Essbase 7X, 73.8%; Epistemic Analytic Visibility Suite 2.0, 72.3%; OutlookSoft Everest 4, ProClarity Power User Client bundle and QlikView Enterprise 6, all at 70.8%; and Cartesis ES Magnitude, 69.2%. The remaining vendors are ranked in the Participating Vendors and Compliance Percentage section of this report.

Understand – Summary of Results
This step reviews the historical performance of people and processes. It enables you to understand how results impact the organization. This step enables management to access information and analysts to iterate through modeling, discovery and interactions on the information, the measurement of performance and ongoing monitoring.

The overall winner for this category was Geac MPC Corporate Performance Management Bundle 7.0, which had 98.6% compliance to the questions. It was followed by Cognos 8 BI Professional and Oracle Business Intelligence 10g, both at 95.7%, and Actuate Enterprise Reporting Platform 8.0 at 94.2%. The following vendors round out the top 10 (including ties) by rank: Corporate Radar 7, 92.8%; SymphonyRPM 4.5 and OutlookSoft Everest 4, both 92.1%; CA CleverPath Forest & Trees r7.1, Hyperion Essbase 7X and QlikView Enterprise 6, all at 88.5%; Applix TM1 Planning v8.3 and ,Vanguard Solutions GPS Power User, 87.8%; Epistemic Analytic Visibility Suite 2.0, 87.1%; Cartesis ES Magnitude and Siebel Business Analytics – Siebel Answers 7.7, 842%; and Silvon Stratum Suite 4.1, 83.5%. The remaining vendors are ranked in the Participating Vendors and Compliance Percentage section of this report.

Participating Vendors and Compliance Percentages

The following vendors participated in the research and complied fully with the validation rules and procedures. Vendors are listed in alphabetical order along with their ranking and percentage of compliance overall and for each step of the PerformanceCycle™. Silvon participated in the process for the first time and Microsoft and SAP did not respond to the invitation. Databeacon was not included because it was acquired by Cognos in September 2005. The following vendors provided updates based on new product releases to the market: Applix, Business Objects, Cartesis, Cognos, Geac, Pilot Software and SymphonyRPM. Hyperion submitted an update for Hyperion System 9 BI+ but did not complete the validation step in the time allotted, so the previous submission remains.

About Ventana Research

Ventana Research, the leading Performance Management research and advisory services firm, works with thousands of organizations worldwide. We conduct research and analyze market trends, best practices and technologies, with the aim of helping our clients improve the efficiency and effectiveness of their organizations. We approach the market from the management layer down, rather than from the technology stack up. We look first at people, processes, information and then technology. Our approach enables us to help clients solve real business problems, with an emphasis on business outcomes, rather than having a preoccupation with technology features and functions.
We offer a variety of customizable workshops, assessments and advisory services to address your specific Performance Management challenges:

Performance Management Focus
Ventana Research focuses on Performance Management – the methodology and process that align business and IT to improve business results. As well as our focus on Performance Management, what makes Ventana Research different from other analyst firms is a foundation of research, our market reach and an emphasis on results.

Research Ventana
Research recommendations are based on research we conduct regularly on Performance Management business issues, technologies and best practices. We share this with our clients.

Reach Ventana
Research reaches more than 2 million business executives through extensive media partnerships, providing education about and increased market awareness of Performance Management.
Results Using a high-value, low-risk approach, Ventana Research delivers education and knowledge transfer to empower you to make the right decisions for your business.

Please contact Ventana Research at [email protected] or (650) 931.0880 to learn more about the Ventana Research Scorecard and our services.