Bill Inmon’s versions of the truth

Recently Bill Inmon made a post on his B-eye channel about what the data warehouse is not. It immediately led to a storm of discussion on Twitter between several analysts, among them Colin White, Claudia Imhoff and Seth Grimes. Bottom-line of their reaction: Bill’s definition is a traditional one and does not reflect current industry practices. Moreover, Inmon initially stated that SAP was the only company offering a true data warehouse solution (see Colin White’s tweets) but in the current version of his article this statement has been left out. Another version of his truth?

With the risk of initiating another fierce discussion, I react to Bill’s following statement: “Data warehouses exist for the purpose of supporting management, not operations”. Five years ago I would have concurred but now I simply disagree. It’s like having a word processor but restricting its use to only writing memos. The roots of ERP date back to MRP, Material Requirements Planning. Obviously, its current scope is much broader. I used to dislike the idea of using a data warehouse to support operations, but have come to a different insight. What is the use of a sound data warehouse when it’s use is restricted to only one business function? Data (or information or knowledge for that matter) is to share, not to sit on top it. As Colin White points out: there’s theory and there’s practice and the industry simply evolves.

Ok, that said, I need to say that I’m also allergic to people redefining and misinterpreting originally sound defined concepts. To that end, I support Bill’s plea. Take for example the concept of a core competence, defined by Prahalad & Hamel. Does anybody still knows the three distinguishing characteristics of a core competence? Guess not (it’s: provide entry to new markets, difficult to imitate, support customer’s needs). Take any declared core competence and I bet hardly any instance complies with these criteria.

But, I’m not going into the discussion regarding the (sound) definition of a data warehouse (again). Instead, I like to approach the issue of operational use from a different perspective, yep the architectural one. One key question we need to ask is whether there’s a fundamental architectural principle that inhibits the use of a data warehouse for operational purposes. Or rather, a universal principle. My answer to that question would be no, there’s not. There’s no principle like ‘Thou shall not use a data warehouse operationally, ever”. Architectural principles that do guide this trade-off are for example reducing dependencies, reusability, business responsiveness etc. But it is up to each invididual organization how they want to trade this off and what its implications are for the data warehouse. Second, we need to bear in mind that the way the application architecture is realized (i.e. physically implemented), is driven to a large extent by the capabilities of the technology architecture (i.e. infrastructure). Nowadays, new data warehouse-technologies are capable of dealing with operational service levels (real-time integration, split-second response, failover etc.) at an affordable cost level making it practically and economically feasible to use a data warehouse for operational purposes. Again, this should be a trade-off driven by architecture principles and constraints. Then there’s the question whether the data warehouse is a logical or a physical construct. If it is seen logically, operational use of a data warehouse at the physical level may imply for example a seperate datastore in order to avoid direct physical access to the (physical) data warehouse.

It’s a difficult discussion and not one that will likely be settled after this post. What I wanted to make clear is that there are multiple considerations and viewpoints – principles, logical vs. physical, application vs. infrastructure – that need to be considered when operational use of a data warehouse is being discussed.

Very much like to hear your comments on this one!

wouter

A basic architecture framework & some first implications for BI

In order to start our discussion on enterprise architecture and its implications on BI and data warehousing, we’ll need a basic framework. Let’s start with that. Below is a picture of such a basic framework. It is adapted from the content framework of TOGAF (The Open Group Architecture Framework) and Capgemini’s Integrated Architecture Framework (IAF). Other frames of reference like Archimate and DyA will show a similar structure. I won’t go into all the similarities and differences; they are important to a certain extent but I will save that kind of detail until later. So, let’s look at the framework:

This basic framework distinguishes 4 layers or aspects (I deliberately do not use the term domain or view). The business architecture describes business concepts like business processes, organisational structures and business goals. The information architecture describes the information the business uses and their interrelationships. The information architecture typically is NOT a (conceptual) data model. The application architecture describes the information systems that will be used to deliver the information defined in the layer above. Finally, the technology architecture (NOT technical architecture) describes infrastructural components, like hardware, DBMS’s, networking and so on.

If we use this framework to look at an entire enterprise, the business layer would describe the enterprise’s entire business (i.e. all its business processes, roles etcetera), the information layer all the information that is used within the enterprise, the application layer would describe all the applications and the technology layer the entire infrastructure. For example, the application layer would typically be visualized or represented like an application landscape, denoting families or clusters of systems.

As I said, such an enterprise perspective would include everything, including for example at the application layer BI-systems like a data warehouse, an analysis application or a dashboard. This whole notion of a BI-system (or rather family of systems) and how typical BI-components like a data warehouse, staging areas, cubes, reports etcetera fit into this conceptualization also needs elaboration but that’s for later. For now, I propose to adopt a information system view. At the application layer – when adopting an enterprise-wide perspective – such BI-systems would be modeled next to all other systems within the enterprise, most notably primary transactional systems like order processing, inventory management, pricing systems etc. Similarly, the technology layer would show all the infrastructure used within the company: the data warehouse appliance next to ‘operational’ infrastructure like RDBMS’s and an enterprise service bus. Note that if we look from an enterprise perspective, we don’t see the distinction between BI and non-BI (or OLTP for that matter anymore): we simply see everything!

So, what’s the importance of all this? I would like to hear your answers to this question. To give you a heads up: adopting this layered and integrated views support discussions around ‘operational BI’, the use of your BI-tool as an enterprise-wide reporting platform (from a technology architecture point of view) and the role of the data warehouse in the enterprise application landscape to name a few. More to come up in my next post!

Enterprise architecture and BI: the prequel

For long I’ve been wanting to start this post and finally got to it! You may have noticed the tag line I’ve added to my site: it emphasizes my desire to blog and write more and deeply about enterprise architecture and BI. To get to the bottom-line of my argument right away: BI and enterprise architecture are not 2 seperate worlds. When enterprise architecture is viewed in a broad sense as a holistic approach to conceptualize and (re)design an enterprise, BI is simply part of that. And to throw in another one: architecture is not the same as data modeling. Rather, they are two completely different things. So, in a sense this post is a sequel to the “BI is the same, not different” post. My mind is exploding with ideas and insights I want to share with you, so the next blogs will be used by me to deepdive into this matter (hopefully on a more frequent basis). To give you some idea, I will blog about the different type of architectures (enterprise, domain, reference, solution), architectural frameworks (IAF, TOGAF, DyA), architecture modeling language (Archimate) and denote the relevance and application for the BI-field.

By deepdiving into some of these theories and providing practical examples, the application of these concepts for BI can be made clear. This adds to understanding between BI-professionals and enterprise architects. Unfortunately, this understanding is often obscured in practice because of misunderstanding, misconception or misinterpretation of each other’s profession and area of interest, hence my motivation to try to improve this understanding a bit.

I’ll try to follow-up on short notice with a next post, which will start the deepdive. It’s my intention to discuss one topic a time, so each blogpost will address one particular issue. Stay tuned!

Next generation front-end architecture published in DB/M

DB/M 5 features my article on next generation front-end architecture. It uses TOGAF as a frame of reference to introduce the BI Triangle, distinghuishing between a business, application and technology layer for defining the front-end architecture.

Comment on ‘EIS revisited’

In a recent post on BeyeNetwork Dan Power made an argument to revisit the concept of an Executive Information System (EIS). This post builds upon that argument and adds to the discussion. For people who don’t know Dan Power, Dan is a professor of IS and one of the few scholars who actively participates in the BI practioners community, trying to bridge the gap between science and practice. As you know, I’m a strong advocate of this kind of interaction, so all the praise to Dan.

Now, let’s get to the article. An argument is made to readdress our attention to executive users and renew the EIS-concept. Next to that, Dan argues that system concepts like EIS, BI, datawarehousing etc. are converging. Here’s my response.

First of all, it’s good to know that EIS’s have been covered widely in scientific research which has resulted in a significant body of knowledge. Next to that, the EIS-concept resembles our current and most common understanding of BI (i.e. management reporting in its broadest sense). BI on the other hand, has hardly been investigated yet; datawarehousing has, but BI has not. So EIS research is our most recent resource of scientific knowledge. Therefore it’s logical to revert to these theories.

I agree with Dan’s final premise that the executive is an important target group and we should focus special attention to them. In order to do so, I’d like to add two refinements. To begin with, we need a clear definition  of who the executive is. Is it limited to the CxO’s, is the 2^nd management level included etc.? What characterizes his role and his way of working? I personally experienced a situation were the board of directors insisted on receiving a printed version of the monthly performance review. No strong reporting and drill-down capabilities here… So, to target this user group, we need to define it more precisely.

Secondly, we need to separate te executive user (support) from executive process (support). Executive processes resembles processes like strategic decision making, negotiating and monitoring market trends. These kind of processes require different BI-support than tactical management processes for example. Executive usage defines usability requirements (e.g. no training required, PDA-support) whereas the required process support determines required functionality.

Current BI-platforms provide powerful capabilities to both consolidate the different kind of systems and technologies and at the same time provide the targeted support the executive needs.

The need for rigourness

This post is a comment to Ronald Damhof’s latest blog in which he argues the need for a stronger theoretical foundation in the field. By now, you should know that’s also one of my hobby horses. Let me illustrate the need for this kind of rigourness by two examples.

First, we all know the success stories of successful BI-implementations, especially those that were put on the stand by Davenport’s book “Competing on analytics”. Harrah’s,  a casino and hotel chain is one of those examples; the CEO wrote the foreword of Davenport’s book. However, Harrah’s is also used as an example of a successful implementation of IT Portfolio Management (MIT Sloan Management Review, spring 2004, Vol.45, no.3) with corresponding net effects on Harrah’s performance. I won’t deny a priori the positive effects of both a portfolio management approach and the extensive use of analytics. But available publications gives us to little insight into the real net-effects and the causal relationships (instead of positive correlations).

This leads me to the second example. Even in scientific research we often read about the same succesful companies (sorry, no reference here). Amazon, Google, Dell you name them are always in the spotlights. In the ’80’s it was Sabre, American Hospital Supply (AHS) and Otis Elevator to name some earlier examples.

Why is that, I ask myself…

International Journal of Business Intelligence Research

I’ve argued several times that we lack fundamental insights into our own playing field. Fortunately, it seems that change is on the way! Recently, the International Journal of Business Intelligence Research (IJBIR) has been launched. According to the journal’s mission statement, it aims to fill the gap between practioner journals and the ‘hard core’ academic ones. A great initiative! Check out the journal’s home page and a comment by the new editor-in-chief.

BI is the same, not different

BI people – and I am one of them! – tend to see the world differently, sometimes very differently from the rest of the IT-world. For long, BI was a neglected area of IT, often seen as byproduct of custom systems development (oops, forgotten, we also need some management reports). First triggered by huge advances in technology and more recently by Davenport’s seminal article ‘Competing on analytics’ in HBR, BI got a business face and a new, much more appealing connotation: the underpinnings were there to view BI as a whole new area of business with programme managers, architects and consultants lobbying for dedicated attention, trying to convince the CEO the company needs to follow an analytics strategy (instead of…?) and transform the organization. As a result, a whole new ecosystem within the IT-world has evolved, with its own vocabulary and the urge to create its own methods, tools etc. For BI people, BI is a whole different ball-game than traditional IT (or transactional IT, or operational IT or….).

I’m increasingly confused and less convinced that this distinct view of the world is justifiable. True, technological developments have created opportunities that didn’t exist before. We’re now able to store tons of data, apply advanced analytics and use the results in operational processess on a real-time, transactional scale. But does this change the way we formulate an IT-strategy, develop solutions, run IT-operations, govern developments? More and more, I tend to answer no, instead of yes. Sure, some things are different but only to a certain extent. It certainly does not justifies a whole separate approach. Why come up with a proprietary architecture / requirements / development method when there are industry standards available? No doubt common standards will lack some specifics that are required for BI-initiatives but isn’t it more effective to provide some kind of add-on to a common standard instead of developing one’s own? Data Vault project management? I don’t know…

More and more, I tend to take a common standard or method as a start, as I’m doing in my current assignment which deals with enterprise architecture, instead of staying with my “BI is unique and the rest of the world should change”-paradigm and trying to convince any other they should adhere to it. In a short time, it has offered me with many valuable insights and allowed to me better (!) govern and architect BI instead of less. It seems to me that this proprietary approach to BI rather impedes success and performance impact than boosts it.

Book review: The profit impact of Business Intelligence

In search of relevant literature that adressess the business side of BI, I encountered this book written by Steve Williams and Nancy Williams. Steve and Nancy are instructors at the TDWI and run their own firm, DecisionPath. If there’s one message that remains after reading the book, it’s that BI should be used to impact profits and business performance. This message is repeated so often that it becomes somewhat annoying but I guess that’s a kind of (American) style of writing. Nevertheless, that message is most valuable: all too often the emphasis is on better data, sexier tools or new analytics but less on how this is supposed to impact processes that drive profits and performance (I’m beginning to sound like them…).

Let’s have a brief look at the outline of the book. The first chapter stresses the key message but for me the most important content concerns two other key points: the need for process engineering and the need for change management.  This viewpoint was also brought forward in their 2003 article in Business Intelligence Journal. Especially the process engineering-perspective is important: business processes need to be re-engineered as a result of other (more sophisticated) use of information. It would have been nice if this viewpoint had been elaborated more in the book. Change management deals with changing the business practice and thus human behaviour as a result of changed information provision and business processes. Common change management-practices – like Kotter’s Leading Change approach – apply to this one, but the value is in emphasizing the imperative that BI-initiatives require change management to be succesful.

The next two chapters deal with opportunity analysis and risk- and readiness assessment. Once one has adopted the paradigm set forth in the first chapter, the risk and opportunity-assessment become rather straightforward. The key to do a proper assessment is in understanding the paradigm. Chapter 4 deals with their development methodology, the BI Pathway, and to be honest, this was a somewhat disappointing chapter. To me, the methodology didn’t bring any new fundamental insights and rather than providing some new in-depth knowledge or skills, the reader is encouraged to take a course at the TDWI. Moreover, I would sincerely refrain from presenting an overview of the method (Fig. 4-2 p. 70) to a business audience. For a non-IT person it contains a lot of detail and techno terminology.

BI people tend to see the world differently from the rest (guilty, I used to do that too) and the BI Pathway is just another example. It’s content is good but I see little value in wrapping it into  another methodology. In my opinion, BI professionals should strive to adhere to common standards and frameworks, like TOGAF and RUP. Such an approach does not inhibit or exclude the special treatment of specific BI-aspects.

Chapter 5 is about leading a change progam and here a need for changing the culture of information usage is stressed. Next to that, required core competencies are listed and these provide a valuable checklist for a BI-program. The next chapter deals with the IT that is specific to BI and should have no secrets for any experienced BI professional. In a way, the 7th chapter extends the opportunity analysis of chapter 2 and provides several more specific examples of how BI can impact profits and performance. Basically, chapter 8 is about the don’ts and illustrates common pitfalls, like lack of program management, insufficient information requirements and using an OLTP infrastructure. The final chapter concludes with a view over the horizon.

As explained at the start, the key message is to focus on performance improvements and the primary value of the book is that that message has been written down. However, for a book that intends to convey this thinking, it still contains a lot of BI-technical details, which is a bit odd to me considering that the target audience are managers and executives. Apart from that, as a BI-professional, I’m curious about the HOW. On several occasions, some very BI-specific issues are illustrated. A good example is the observation that end-users tend to define their requirements in terms of reports because that is what they are used to. This observation is a step-stone for describing ways to improve the requirements process but the book provides this kind of deepening too little. A good step-stone to write another book…

Fuzzy Gartner

Gartner’s been here and it hasn’t been left unnoticed. The annual BI Summit was used to announce 5 predictions for the future which triggered several discussions  (e.g. in Automatiseringgids and on the Computable blog) both about the predictions themselves and the course Gartner seems to be taking (less focused on technology but more on the management side).

Gartner’s message gave me an awkward feeling. Just a few weeks earlier, a Gartner study was published stating that BI was still on top of mind in 2009 and now they warn us to be careful. What is Gartner trying to tell us?  I can’t help the feeling that for me Gartner’s value to our profession and industry is declining. Some of their research has been very helpful in tracking movements on the (vendor) market, most notably their Magic Quadrants. But more and more, I get disappointed in their message to the market: either they are too obvious (‘BI-governance will shift from IT to the Business’) or way to broad (‘SaaS will gain bigger share’).

I have argued before that to my opinion we lack deep understanding of our own playing field. I’d rather see Gartner provide that in-depth insight we need so desperately. For example, how does the application of BI differ among industries; where, how, when and why does real-time information really makes a difference, what are the fundamental criteria to differentiate the different types of BI, just to name a few. But this kind of research probably doesn’t make a sexy summit…

wouter