Measuring the Performance of Enterprise Architecture - InformationWeek

InformationWeek is part of the Informa Tech Division of Informa PLC

This site is operated by a business or businesses owned by Informa PLC and all copyright resides with them.Informa PLC's registered office is 5 Howick Place, London SW1P 1WG. Registered in England and Wales. Number 8860726.

10:30 AM
Imre Kabai
Imre Kabai

Measuring the Performance of Enterprise Architecture

Enterprise architecture teams often struggle with measuring and communicating the value of their function. We have implemented a set of simple and straight-forward performance indicators to address this problem.

Do you remember the TV ad from BASF? “We don’t make a lot of the products you buy, we make a lot of the products you buy better”. The predicament of enterprise architecture is similar: We do not make products -- business value generating information technology solutions -- but rather make them better.

In order to create good performance indicators (KPIs) the purpose of the function has to be defined. Herein lies the first challenge.

The architecture function differs from company to company based on culture, funding levels, the role information technology plays in the enterprise, and several other factors. Here is a definition that should would work for many organizations:

As-Is State: The EA function makes the current state of business capabilities, related information, and underpinning technologies visible. This is important so companies do not have to re-discover what they have every time something needs to be changed, interfaced with, or fixed. Understanding as-is states to a sufficient degree is the foundation for alignment, agility, and simplification.

Governance: The function sets rules about the ways the architecture is allowed to change. This is often accomplished by creating principles, standards, and guidelines.

To-Be State: The function creates plans to guide the long-term evolution of the architecture. Business strategy, constraints, current state, and technological opportunities are combined into three-year roadmaps. This is where business strategy meets information technology based innovation.

Assurance: The function makes sure that architecture related decisions follow the rules and roadmaps laid out in Governance and To-B State. This can be accomplished using design reviews.

We can call the four interrelated components the Enterprise Architecture Lifecycle and set out to establish our KPIs:

Good KPIs should be SMART and easy to understand for everybody involved.

As-Is State: The goal is to maintain sufficient coverage of business and technology assets, their important attributes, and relationships.

We have built a lightweight collaboration tool for this purpose. It is populated by periodic application and infrastructure scans. The differences between the scans and the data in the tool trigger workflows and subject matter experts establish the relationships and maintain the attributes.

Our KPI is the percentage of documented assets versus scanned assets and our goal is to keep this number above 90%. 

Governance: Our goal is to have a clear set of agreed upon rules governing the changes of our technology assets. At the beginning of each quarter we assemble a list of principles, standards, and guidelines to be created or updated. Then we prioritize them and select those that we can tackle given the available resources. 

Our KPI measures the percentage that gets created. We strive to reach at least 90% by the end of each quarter.

To-Be State: The goal is to maintain a near complete set of plans for our enterprise technology assets. We plan three years ahead and indicate the desired quarterly state of the assets. We also include the technologies we are planning to introduce in our environments and include detailed rationale for our decisions.

Our KPI measures the percentage of assets with roadmaps and we keep this number above 90%

Assurance: The goal is to make sure that our rules and roadmaps are followed when we design and implement solutions. We use design reviews to evaluate design quality and generate quarterly aggregated reports based on the scores.

Our KPI measures the aggregated design quality and should reach at least 85%, which is B+ level.

 Bonus KPI: We want to see how we measure up to other enterprise architecture shops, so we run annual Gartner benchmarks. We plan on reaching a certain level that makes sense for us.

In Summary

Dear reader, you have probably noticed that the holy grail of the enterprise architecture KPIs is missing. How much business value does the enterprise architecture function generate? This KPI can be decomposed in many ways:

  1. How much value do we generate because we have the necessary business insight through well integrated systems and clean, properly governed data?
  2. What is the value of speed, due to our solutions being built with agility in mind?
  3. What is the value of a clean, well-managed technology portfolio?
  4. What is the value of critical business capabilities relying on high quality technology components?
  5. How much is the worth of solutions that are simple, intuitive, and a pleasure for our users?
  6. What is the value of technology based innovation?

Well, we haven’t figured out the way to measure the business value of enterprise architecture yet, but one thing is for sure, if we wish to get there, it has to be a joint business-IT undertaking. I would love to hear from you if you have good ideas or solutions.

Imre Kabai is director and chief architect at Granite, a $2.5B heavy construction company. Previously he worked as the enterprise architect of Stanford Healthcare, and chief architect of the SLAC National Accelerator Laboratory. His interests include enterprise architecture, ... View Full Bio
We welcome your comments on this topic on our social media channels, or [contact us directly] with questions about the site.
Comment  | 
Print  | 
More Insights
Newest First  |  Oldest First  |  Threaded View
Charlie Babcock
Charlie Babcock,
User Rank: Author
1/11/2017 | 6:44:07 PM
Achieving value in enterprise architecture
Imre Kabai has come up with a clear way to think about a complicated subject: enterprise architecture. It's hard to assign value to an individual application process within that architecture, harder yet to assign value to the architecture as a whole. But any experienced operational manager can sense how much value lies in sticking to these guides and rules. They lean toward combating spot fixes, ad hoc cures and freelance work-arounds, all of which contribute to ongoing complexity. 
11 Ways DevOps Is Evolving
Lisa Morgan, Freelance Writer,  2/18/2021
Graph-Based AI Enters the Enterprise Mainstream
James Kobielus, Tech Analyst, Consultant and Author,  2/16/2021
What Comes Next for AWS with Jassy to Become Amazon CEO
Joao-Pierre S. Ruth, Senior Writer,  2/4/2021
White Papers
Register for InformationWeek Newsletters
The State of Cloud Computing - Fall 2020
The State of Cloud Computing - Fall 2020
Download this report to compare how cloud usage and spending patterns have changed in 2020, and how respondents think they'll evolve over the next two years.
Current Issue
2021 Top Enterprise IT Trends
We've identified the key trends that are poised to impact the IT landscape in 2021. Find out why they're important and how they will affect you.
Flash Poll