The New Deal 2

The alternative is a model-driven architecture. With this approach, the models are not data models; rather, they are expressive business models designed not to just arrange data neatly in a drawer but to solve end-to-end problems. Such models are expressed in terms that are meaningful to stakeholders in the sales, marketing, finance, procurement, product design, engineering, actuarial, auditing, and risk management.

Software tools can capture these models and manage versions, iterations, and other changes directly, also managing the design and redesign of physical objects in the data warehouse. Similarly, a model-driven architecture specifies ETL processes at a level of abstraction rather than to tables and columns, allowing the automated schema managers free range to reconfigure data stores in the most efficient manner (see Figure 1). The data itself may not even be physically mapped from the source systems, as powerful algorithms decide where and when to stage data based on requirements.

Figure 1 Characteristics of a model-driven architecture.

To be relevant, data warehousing has to shift its focus from excruciating design and architecture to rapid deployment, flexibility and, above all else, connectedness. In the pipeline to support the model-driven approach is a fascinating array of products, some of which are already available. Here are some examples:

Rapid deployment. Two major challenges to delivering BI rapidly are the data modeling process and the integration of typically five or more software tools. An alternative is a single server with all of these services bundled into one product and operating as a unit. Certive's toolset allows businesspeople to build declarative business models in a graphical environment removed from the limitations and arcane concepts of information technology; in turn, the tool builds and modifies the underlying data structures. Another interface leverages the metadata created in the modeling process and provides ETL services. The BI component presents metamodels — not physical structures — to users, and gracefully adapts as the models evolve. Certive's concept is to place modeling in the hands of users, not relational database technicians.

Flexibility. Like Certive, Kalido also employs the concept of declarative modeling and a rich semantic layer to build and modify the physical structures of the data warehouse. Compared to Certive, Kalido's approach is more closely aligned with existing data warehouse technology (especially the underlying relational database). But Kalido's large, multinational customers apply the tool to very large and dispersed data warehouses, offering a single point of change and control. Kalido renders the painful process of modifying the data warehouse's data models into a simple set of manipulations from an administrator's console, imbuing even traditional data warehouses with flexibility and change control not possible previously.

Connectedness. No longer can a data warehouse can be an island unto itself. Older approaches left the data warehouse standing off to the side of enterprise computing: the final resting place of secondhand data. Today, data warehouses must participate with other operational systems, even external ones, as a peer. Designing and maintaining the architecture for that kind of connectedness and interoperability, with the throughput and performance needed, is beyond the current capabilities of most IT departments and data warehouse practitioners. SAP's Business Information Warehouse, as part of SAP BI and the overall NetWeaver architecture, offers a blueprint and set of tools and applications to envelop the data warehouse within the total computing architecture.

The emerging model-driven approach will vastly simplify the design, development and maintenance of data warehouses. Traditional approaches create a partitioned universe — you have either operational or analytical data — that is too rigid for rapid innovation and incapable of providing blended operations and processes that many companies now require.

Model-driven approaches will bring a measure of business sense to data warehousing. In addition, three key improvements will be enabled:

Pushing through the status quo will be slow because so many industry players have a vested interest. However, two forces for change are already at work. Forward-looking vendors, such as the ones mentioned here, will apply their influence in what has always been a vendor-driven market. Second and most important, leading companies will demand better solutions, especially to provide greater flexibility and connectedness. There's an ocean of unmet need: It's become obvious that current practices cannot address it.

Guest columnist Neil Raden, [[email protected]] is founder of Hired Brains Inc., a consulting, systems integration, and implementation services firm. Based in Santa Barbara, Calif., Raden is a widely published author and speaker on data warehousing, BI, and IT strategy.