Probabilistic Analysis Predicts IT Systems Problems Before Costly Applications Outages

Edited transcript of BriefingsDirect[TM] podcast on probabilistic IT systems analysis and management, recorded Aug. 16, 2007.

Listen to the podcast here. Sponsor: Integrien Corp.

Dana Gardner: Hi, this is Dana Gardner, principal analyst at Interarbor Solutions, and you're listening to BriefingsDirect. Today our sponsored podcast focuses on the operational integrity of data centers, the high cost of IT operations, and the extremely high cost of application downtime and performance degradation.

Rather than losing control to ever-increasing complexity, and gaining less and less insight into the root causes of problematic applications and services, enterprises and on-demand application providers alike need to predict how systems will behave under a variety of conditions.

By adding real-time analytics to their systems management practices, operators can fully determine the normal state of how systems should be performing. Then, by measuring the characteristics of systems under many conditions over time, datacenter administrators and systems operators gain the ability to predict and prevent threats to the performance of their applications and services.

As a result they can stay ahead of complexity, and contain the costs of ongoing high-performance applications delivery.

This ability to maintain operational integrity through predictive analytics means IT operators can significantly reduce costs while delivering high levels of service.

Here to help us understand how to manage complexity by leveraging probabilistic systems management and remediation, we are joined by Mazda Marvasti, the CTO of Integrien Corp. Welcome to the show, Mazda.

Mazda Marvasti: Thank you, Dana.

Gardner: Why don’t we take a look at the problem set? Most people are now aware that their IT budgets are strained just by ongoing costs. Whether they are in a medium-sized company, large enterprise, or service-hosting environment, some 70 percent to 80 percent of budgets are going to ongoing operations.

That leaves very little left over for discretionary spending. If you have constant change or a dynamic environment, you're left without much resources to tap in order to meet a dynamic market shift. Can you explain how we got to this position? Why are we spending so much just to keep our heads above water in IT?

Marvasti: When we started in the IT environment, if you remember the mainframe days, it was pretty well defined. You had couple of big boxes. They ran a couple of large applications. It was well understood. You could collect some data from it, so you knew what was going on within it.

We graduated to the client-server era, where we had more flexibility in terms of deployment -- but with that came increasing complexity. Then we moved ahead to n-tier Web applications, and we had yet another increase in complexity. A lot of products came in to try to alleviate that complexity for deep-data collection. And management systems grew out, covering an entire enterprise for data collection, but the complexity was still there.

Now, with service-oriented architecture (SOA) and virtualization moving into application-development and data-center automation, there is a tremendous amount of complexity in the operations arena. You can’t have the people who used to have the "tribal knowledge" in their head determining where the problems are coming from or what the issues are.

The problems and the complexity have gone beyond the capability of people just sitting there in front of screens of data, trying to make sense out of it. So, as we gained efficiency from application development, we need consistency of performance and availability, but all of this added to the complexity of managing the data center.

That’s how the evolution of the data center went from being totally deterministic, meaning that you knew every variable, could measure it, and had very specific rules telling you if certain things happened, and what they were and what they meant -- all the way to a non-deterministic era, which we are in right now.

Now, you can't possibly know all the variables, and the rules that you come up with today may be invalid tomorrow, all just because of change that has gone on in your environment. So, you cannot use the same techniques that you used 10 or 15 years ago to manage your operations today. Yet that’s what the current tools are doing. They are just more of the same, and that’s not meeting the requirements of the operations center anymore.

Gardner: At the same time, we are seeing that a company’s applications are increasingly the primary way that it reaches out to its sell side, to customers -- as well as its buy side, to its supply chain, its partners, and ecology. So applications are growing more important. The environment is growing more complex, and the ability to know what’s going on is completely out of hand.

Marvasti: That’s right. You used to know exactly where your application was, what systems it touched, and what it was doing. Now, because of the demand of the customers and the demands of the business to develop applications more rapidly, you’ve gone into an SOA era or an n-tier application era, where you have a lot of reusability of components for faster development and better quality of applications -- but also a lot more complexity in the operations arena.

What that has led to is that you no longer even know in a deterministic fashion where your applications might be touching or into what arenas they might be going. There's no sense of, "This is it. These are the bounds of my application." Now it’s getting cloudier, especially with SOA coming in.

Gardner: We’ve seen some attempts in the conventional management space to keep up with this. We’ve been generating more agents, putting in more sniffers, applying different kinds of management. And yet we still seem to be suffering the problems. What do you think is a next step in terms of coming to grips with this -- perhaps on an holistic basis -- so that we can get as much of the picture as possible?

Marvasti: The "business service" is the thing that the organization offers to its customers. It runs through their data center, IT operations, and the business center. It goes across multiple technology layers and stacks. So having data collection at a specific stack or for a specific technology silo, in and of itself, is insufficient to tell you the problems with the business service, which is what you are ultimately trying to get to. You really need to do a holistic analysis of the data from all of the silos that the business service runs through.

You may have some networking silos, where people are using specific tools to do network management -- and that’s perfectly fine. But then the business service may go through some Web tier, application tier, database tier, or storage -- and then all of those devices may be virtualized. There may be some calls to a SOA.

There are deep-dive tools to collect data and report upon specifics of what maybe going on within silos, but you really need to do an analysis across all the silos to tell you where the problems of the business service may be coming from. The interesting thing is that there is a lot of information locked into these metrics. Once correlated across the silos, they paint a pretty good picture as to the impending problem or the root cause of what a problem may be.

By looking at individual metrics collected via silos you don’t get as full a picture as if you were to correlate that individual metric with another metric in another silo. That paints a much larger picture as to what may be going on within your business service.

Gardner: So if we want to gather insights and even predictability into the business service level -- that higher abstraction of what is productive -- we need to go in and mine this data in this context. But it seems to me that it’s in many different formats. From that "Tower of Babel" how do you create a unified view? Or you are creating metadata? What’s the secret sauce that gets you from raw data to analysis?

Marvasti: One misperception is that, "I need to have every piece of metric that I collect go into a magical box that then tells me everything I need to know." In fact, you don’t need to have every piece of metrics. There is much information locked between the correlation of the metrics. We’ve seen at our customers that a gap in monitoring in one silo can often be compensated by data collection in other silos.

So, if you have a monitoring system already -- IBM Tivoli, as an example -- and you are collecting operating-system metrics, you may have one or two other application-specific metrics that you are also collecting. That may be enough to tell you everything that is going on within your business service. You don't need to go to the nth degree of data collection and harmonization of that data into one data repository to get a clear picture.

Even starting with what you’ve got now, without having to go very deep, what we’ve seen in our customers is that it actually lights up a pretty good volume of information in terms of what may be going on across the silos. They couldn't achieve that by just looking at individual metrics.

Gardner: It’s a matter of getting to the right information that’s going to tell you the most across the context of a service?

Marvasti: To a certain degree, but a lot of times you don’t even know what the right metrics are. Basically I go to our customers and say, "What do you have?" Let’s just start with that, and then the application will determine whether you actually have gaps in your monitoring or whether these metrics that you are collecting are the right ones to solve those specific problems.

If not, we can figure out where the gaps may be. A lot of times, customers don’t even know what the right metrics are. And that’s one of the mental shifts of thinking deterministically versus probabilistically.

Deterministically is, "What are the right metrics that I need to collect to be able to identify this problem?" In fact, what we’ve found out is that a particular problem in a business service can be modeled by a group or a set of metric event conditions that are seemingly unrelated to that problem, but are pretty good indicators of the occurrence of that problem.

When we start with what they have, we often point out that there is a lot more information within that data set. They don’t really need to ask, "Do I have the right metrics or not?"

Gardner: Once you’ve established a pretty good sense of the right metrics and the right data, then I suppose you need to apply the right analysis and tools. Maybe this would be a good time for you to explain about the heritage of Integrien, how it came about, and how you get from this deterministic to more probabilistic or probability-oriented approach?

Marvasti: I’ve been working on these types of problems for the past 18 years. Since graduate school, I’ve been analyzing data extraction of information from disparate data. I went to work for Ford and General Motors -- really large environments. Back then, it was client-servers and how those environments were being managed. I could see the impending complexity, because I saw the level of pressure that there was on application developers to develop more reusable code and to develop faster with higher quality.

All that led to the Web application era. Back then, I was the CTO of a company called LowerMyBills.com here in the Los Angeles area. One problem I had was that I had a few people with the tribal knowledge to manage and run the systems, but that was very scary to me. I couldn't rely on these people to be able to have a continuous business going on.

So I started looking at management systems, because I thought it was probably a solved problem. I looked at a lot of management tools out there, and saw that it was mainly centered on data collection, manual rule writing, and better way of presenting the same data over and over.

I didn’t see any way of doing a deep analysis of the data to bring out insights. That’s when I and my business partner Al Eisaian, who is our CEO, formed a company to specifically attack this problem. That was in 2001. We spent a couple of years developing the product, got our first set of customers in 2003, and really started proving the model.

One of the interesting things is that if you have a small environment, your tendency is to think that it's small enough that you can manage it, and that actually may be true. You develop some specific technical knowledge about your systems and you can move from there. But in the larger environments where there is so much change happening in the environment it becomes impossible to manage it that way.

A product like ours almost becomes a necessity, because we’ve transitioned from people knowing in their heads what to do, to not being able to comprehend all of the things happening in the data center. The technology we developed was meant to address this problem of not being able to make sense of the data coming through, so that you could make an intelligent decision about problems occurring in the environment.

Gardner: Clearly a tacit knowledge approach is not sufficient, and just throwing more people at it is not going to solve the problem. What’s the next step? How do we get to a position where we can gather and then analyze data in such a way that we get to that Holy Grail, which is predictive, rather than reactive, response.

Marvasti: Obviously, the first step is collecting the data. Without the data, you can’t really do much. A lot of investment has already gone into data collection mechanisms, be it agent-based or agent-less. So there is data being collected right now.

The missing piece is the utilization of that data and the extraction of information from that data. Right now, as you said at the beginning of your introduction, a lot of cost is going toward keeping the lights on at the operations center. That’s typically people cost, where people are deployed 24/7, looking at monitors, looking at failures, and then trying to do postmortem on the problem.

This does require a little bit of mind shift from deterministic to probabilistic. The reason for that is that a lot of things have been built to make the operations center do a really good job of cleaning up after an accident, but not a lot of thought has been put into place of what to do if you're forewarned of an accident, before it actually happens.

Gardner: How do I intercede? How do I do something?

Marvasti: How do I intercede? What do I do? What does it mean? For example, one of the outputs from our product is a predictive alert that says, "With 80 percent confidence, this particular problem will occur within the next 15 minutes." Well, nothing has happened yet, so what does my run book say I should do? The run book is missing that information. The run book only has the information on how to clean it up after an accident happens.

That’s the missing piece in the operations arena. Part of the challenge for our company is getting the operations folks to start thinking in a different fashion. You can do it a little at a time. It doesn’t have to be a complete shift in one fell swoop, but it does require that change in mentality. Now that I am actually forewarned about something, how do I prevent it, as opposed to cleaning up after it happens.

Gardner: When we talk about operational efficiency, are we talking about one or two percent here and there? Is this a rounding error? Are we talking about some really wasteful practices that we can address? What’s the typical return on investment that you are pursuing?

Marvasti: It’s not one or two percent. We're talking about a completely different way of managing operations. After a problem occurs, you typically have a lot of people on a bridge call, and then you go through a process of elimination to determine where the problem is coming from, or what might have caused it. Once the specific technology silo has been determined, then they go to the experts for that particular silo to figure out what’s going on. That actually has a lot of time and manpower associated with it.

What we're talking about is being proactive, so that you know something is about to happen, and we can tell you to a certain probability where it’s going to be. Now you have a list of low-hanging fruits to go after, as opposed to just covering everybody in the operations center, trying to get the problem fixed.

The first order of business is, "Okay, this problem is about to occur, and this is where it may occur. So, that’s the guy I’m going to engage first." Basically, you have a way of following down from the most probable to the least probable, and not involving all the people that typically get involved in a bridge call to try to resolve the issues.

One gain is the reduction in mean time to identify where the problem is coming from. The other one is not having all of those people on these calls. This reduces the man-hours associated with root-cause determination and source identification of the problem. In different environments you're going to see different percentages, but in one environment that I saw first hand, one of the largest health-care organizations, it is like 20-30 percent of cost, just associated with people being on bridge calls, on a continuous basis.

Gardner: Now, this notion of "management forensics," can you explain that a little bit?

Marvasti: One of the larger problems in IT is actually getting to the root cause of problems. What do you know? How do you know what the root cause is? Often times, something happens and the necessity of getting the business service back up forces people to reboot the servers and worry later about figuring out what happened. But, when you do that, you lose a lot of information that would have been very helpful in determining what the root cause was.

The forensic side of it is: The data is collected already, so we already know what it is. If you have the state when a problem occurred, that’s a captured environment in the data base that you can always go back to.

What we offer is the ability to walk back in time, without having the server down, while you are doing your investigation. You can bring the server back up, come back to our product, and then walk back in time to see exactly what were the leading indicators to the problems you experienced. Using those leading indicators, you can get to the root causes very quickly. That eliminates the guess work of where to start, reduces the time to get to the root cause, and maybe even prevent it.

Sometimes you only have so much time to work on something. If you can’t solve it by that time, you move on, and then the problem occurs again. That's the forensic side.

Gardner: We talked earlier about this notion of capturing the normal state, and now you've got this opportunity to capture an abnormal state. You can compare and contrast. Is that something that you use on an ongoing basis to come up with these probabilities? Or is the probability analysis something different?

Marvasti: No, that’s very much part and parcel of it. What we do is look to see what is the normal operating state of an environment. Then it is the abnormalities from that normal that become your trigger points of potential issues. Those are your first indicators that there might be a problem growing. We also do a cross-event analysis. That’s another probability analysis that we do. We look at patterns of events, as opposed to a single event, indicating a potential problem. One thing we've found is that events in seemingly unrelated silos are very good indicators of a potential problem that may brew some place else.

Doing that kind of analysis, looking at what’s normal, then abnormal becomes your first indicator. Then, doing a cross-event analysis to see what patterns indicate a particular problem becomes total normal to problem-prevention scenario.

Gardner: There has to be a cause-and-effect. As much as we would like to imagine ghosts in the machines, that’s really not the case. It's simply a matter of tracking it down.

Marvasti: Exactly. The interesting thing is that you may be measuring a specific metric that is a clear indicator of a problem, but it is oftentimes some other metric on another machine that gets to be out of normal first, before the cause of the problem surfaces in the machine in question. So early indicators to a problem become events that occur some place else, and that’s really important to capture.

When I was talking about the cross-silo analysis, that’s the information that it brings out. It gives you lot more "heads-up" time to a potential problem than if you were just looking at a specific silo.

Gardner: Of course, each data center is unique, each company has its own history and legacy, and its IT department has evolved on its own terms. But is there any general crossover analysis? That is to say, is there a way of having an aggregate view of things and predicting things based on some assumptions, because of the particular systems that are in use? Or, is it site by site on a detailed level?

Marvasti: Every customer that I have seen is totally different. We developed our applications specifically to be learning-based, not rules-based. And by rules I mean not having any preconceived notion of what an environment may look like. Because if you have that, and the environment doesn’t look like that, you're going to be sending a lot of false positives -- which we definitely did not want to do.

Ours is a purely learning-based system. That means that we install our product, it starts gathering the metrics, and then it starts learning what your systems look like and behave like. Then based on that behavior it starts formulating the out-of-normal conditions that can lead to problems. That becomes unique to the customer environment. That is an advantage, because when you get something, it actually adapts itself to an environment.

For example, it learns your change management patterns. If you have a change windows occurring, it learns that change window. It knows that those change windows occur without anybody having to enter anything into the application. When you are doing wholesale upgrade of devices, it knows that change is coming about, because it has learned your patterns.

The downside of that is that it does take two to three weeks of gathering your data and learning what has been happening for it to become useful. The good side of it is that you get something that completely maps to your business, as opposed to having to map your business through a product. The downside is that it takes two or three weeks of learning time, before it starts producing some results for you.

Gardner: The name of your product set is Alive, is that correct?

Marvasti: That’s correct.

Gardner: I understand you are going to have a release coming out later this year, Alive 6.0?

Marvasti: That’s correct.

Gardner: I don’t expect you to pre-release, but perhaps you can give us some sense of the direction that the major new offerings within the product set will take. What they are directed toward? Can you give us a sneak peek on that?

Marvasti: Basically, we have three pillars that the product is based on. First is usability. That's a particular pet peeve of mine. I didn't find any of the applications out there very usable. We have spent a lot of time working with customers and working with different operations groups, trying to make sure that our product is actually usable for the people that we are designing for.

The second piece is interoperability. The majority of the organizations that we go to already have a whole bunch of systems, whether it be data collection systems, event management systems, or configuration management databases, etc. Our product absolutely needs to leverage those investments -- and they are leveragable. But even those investments in their silos don’t produce as much benefit to the customer as a product like ours going in there and utilizing all of that data that they have in there, and bringing out the information that’s locked within it.

The third piece is analytics. What we have in the product coming out is scalability to 100,000 servers. We've kind of gone wild on the scalability side, because we are designing for the future. Nobody that I know of right now has that kind of a scale, except maybe Google, but theirs is basically the same thing replicated thousands of times over, which is different than the enterprises we deal with, like banks or health-care organizations.

A single four-processor Xeon box, with Alive installed on it, can run real-time analytics for up to 100,000 devices. That’s the level of scale we're talking about. In terms of analytics, we've got three new pieces coming out, and basically every event we send out is a predictive event. It’s going to tell you this event occurred, and then this other set of events have a certain probability within a certain timeframe to occur.

Not only that, but then we can match it to what we call our "finger printing." Our finger printing is a pattern-matching technology that allows us to look at patterns of events and formulate a particular problem. It indicates particular problems and those become the predictive alerts to other problems.

What’s coming out in the product is really a lot of usability, reporting capabilities, and easier configurations. Tens of thousands of devices can be configured very quickly. We have interoperability -- Tivoli, OpenView, Hyperic -- and an open API that allows you to connect to our product and pump in any kind of data, even if it’s business data.

Our technology is context agnostic. What that means is that it does not have any understanding of applications, databases, etc. You can even put in business-type data and have it correlated with your IT data, and extract information that way as well.

Gardner: You mentioned usability. Who are the typical users and buyers of a product like Integrien Alive? Who is your target audience?

Marvasti: The typical user would be at the operations center. The interesting thing is that we have seen a lot of different users come in after the product is deployed. I've seen database administrators use our product, because they like to see what is normal behavior of their databases. So they run the analytics under database type metrics and get information that way.

I've seen application folks who want to have more visibility in terms of how this particular application is impacting the database. They become users. But the majority of users are going to be at the operations center -- people doing day-to-day event management and who are responsible for reducing the mean time to identify where the problems come from.

The typical buyers are directors of IT operations or VP of IT operations. We are really on the operation side, as opposed to the application development side.

Gardner: Do you suppose that in the future, when we get more deeply into SOA and virtualization, that some of the analysis that is derived through Integrien Alive becomes something that’s fed into a business dashboard, or something that’s used in governance around how services are going to be provisioned, or how service level agreements are going to be met?

Can we extrapolate as to how the dynamics of the data center and then the job of IT itself changes, on how your value might shift as well?

Marvasti: That link between IT and the business is starting to occur. I definitely believe that our product can play a major part in illuminating what in the business side gets impacted by IT. Because we are completely data agnostic, you can put in IT-type data, business-type data, or customer data -- and have all of it be correlated.

You then have one big holistic view as to what may impact what. ... If this happens, what else might happen? If I want to increase this, what are the other parameters that may be impacted?

So, you know what you want to play from the business side in terms of growth. Having that, we project how IT needs to change in order to support that growth. The information is there within the data and the very fact that we are completely data agnostic allows us to do that kind of a multi-function analysis within an enterprise.

Gardner: It sounds like you can move from an operational efficiency value to a business efficiency value pretty quickly?

Marvasti: Absolutely. Our initial target is the operations arena, because of the tremendous amount of inefficiencies there. But as we move into the future, that’s something we are going to look into.

Gardner: We mentioned Alive 6.0. Do you have a ball-park figure on when that’s due? Is it Q4 of 2007?

Marvasti: We are going to come out with it in 2007, and it will be available in Q4.

Gardner: Well, I think that covers it, and we are just about out of time. I want to thank Mazda Marvasti, the CTO of Integrien, for helping us understand more about the notion of management forensics and probabilistic- rather than deterministic-based analysis.

We have been seeking to understand better how to address high costs, and inefficiencies in data centers, as well as managing application performance -- perhaps in quite a different way than many companies have been accustomed to. Is there anything else you would like to add before we end, Mazda?

Marvasti: No, I appreciate your time, Dana, and thank you for your good questions.

Gardner: This is Dana Gardner, principal analyst at Interarbor Solutions. You have been listening to a sponsored BriefingsDirect podcast. Thanks, and come back next time.

Listen to the podcast here. Sponsor: Integrien Corp.

Transcript of BriefingsDirect podcast on systems management efficiencies and analytics. Copyright Interarbor Solutions, LLC, 2005-2007. All rights reserved.

Transcript of B2B Search Trends Podcast Based on Enquiro's Survey Findings

Edited transcript of BriefingsDirect[TM] podcast on B2B search usage, trends and future direction with host Dana Gardner, recorded June 26, 2007.

Listen to the podcast here. Watch the video here. Sponsor: ZoomInfo.

Dana Gardner: Hi, this is Dana Gardner, principal analyst at Interarbor Solutions, and you're listening to a sponsored BriefingsDirect podcast. Today, a discussion about online search and its role in business-to-business (B2B) activities, particularly the research and acquisitions process for people who are in businesses trying to find goods and services. They seem to be using search more than ever.

We're going to look at a survey conducted last March, a fairly recent look at B2B users, their relationship to search, and how search is shifting based on the findings. Joining us to discuss this we have the originator of the survey, President and CEO of Enquiro Search Solutions, Gord Hotchkiss. Welcome to the show, Gord.

Gord Hotchkiss: Thank you.

Gardner: Also joining us to add some analysis and perspective on where search for B2B activities is going is Bryan Burdick, COO of ZoomInfo. Welcome, Bryan.

Bryan Burdick: Dana, thanks for having me.

Gardner: It seems that people, whether they're buying consumer goods, small business supplies -- anything from Gorilla Glue to guided missiles -- are using search at some point in this process. Some people start and end with search. They actually buy the products through a search process. I want to start by understanding a little bit about the survey itself and the fact that it’s the second survey that’s been done on this B2B search activity, the first being in 2004.

So, let’s go to Enquiro with Gord. Tell us a little bit about the history and some of the major bullet points of this particular survey?

Hotchkiss: As you said, we did the original survey in 2004 and, at the time, there wasn't a lot of research out there about search in general, even on the consumer side. There was virtually nothing on the B2B side. We knew search was important, just from what our clients were saying and the results we had had, but we hadn’t done anything extensive enough outside our client base to start to quantify how important it was.

The first survey was an attempt to do that. It certainly proved that search was important. We found that online activity, in particular that connected with search activity, was consistent in a large percentage of purchases. In 2007, we added more insight to the methodology. We wanted to understand the different roles that are typical in B2B purchases -- economic buyers versus technical buyers versus user buyers. We also wanted to get more understanding of the different phases of the buying cycle.

We structured the survey so that we could slice the data based on those parameters and get more insight into those areas. As far as the main takeaways from the study, obviously online activity is more important than ever. In fact, we asked respondents to indicate from a list of over 30 influencers what was most important to them in making the purchase decision. Online factors, such as interaction with the vendor Website and interaction with the search engine were right up there with the traditional winner, word of mouth. What we see is a real link between those and looking for objective information and specific detail.

A lot of that search activity happens on specific properties, and we’ll be diving into that a little bit later in the podcast. We did notice an evolution of behavior as you move through the funnel, and the nature of the interactions with the different online resources changes how you navigate to them and how you go to different sites for information. But, online research was consistent through the entire process, from awareness right through to purchase.

There’s a lot of back and forth. Offline factors influence online activity and vice-versa. So, we saw a merging of the online and the offline worlds in making these decisions and trying to come to what’s the right decision for your company or what’s the right product or service.

Gardner: Tell us a little bit about Enquiro. You are a marketing, consulting, and research firm. Is that correct?

Hotchkiss: Yes. We work with clients in putting together their search campaigns in the B2B space, but we also have an active research arm. So, we're continually doing research primarily on the usability and qualitative analysis side, but we do survey work as well. The purpose of that is to provide more insight into how consumers use search and how businesses use search to make buying decisions.

Gardner: Three years between these surveys is probably not a lot for many businesses, but it’s a huge amount of time in the search industry. What would you say was the biggest difference in your results and findings over this three-year period?

Hotchkiss: Surprisingly, we didn’t notice huge trend differences in the three-year period. If anything, we saw increased reliance on online factors and probably just more activity online and more interactions with sites, but it was the continuation of a trend we saw in 2004. We didn’t see any big shifts. We just found increased reliance on online to do that research.

When we say "increased reliance," we're probably talking 10 percentage points up over the three years. So, if 65 percent of the people were doing it in 2004, 75 percent of the people are doing it now. That’s primarily where we saw the trends going.

Gardner: And, you say that that you're also seeing search applied to this process in different ways and different facets. For example, word of mouth would tip someone off to go look for something, and the first way that they look for it is by using search.

Hotchkiss: Yes. When we looked at the different phases of the buying cycle, it starts with awareness. You become aware that you need something. There was a high percentage of people -- in the high 60-percent range -- who said, "Once I become aware that I need something, the first place I'm going to go is the search engine to start looking for it."

A lot of that traffic is going to end up on Google. It was the overwhelming choice among general search engines for B2B buyers.

But, as you move through the process, you start doing what we call a "landscape search." The first search is to get the lay of the land to figure out the information sites that have the information you are looking for. Who are the main vendors playing in this space? Where are the best bets to go and get more information to help make this purchase decision?

So, those first searches tend to be fairly generic -- shorter key phrases -- just to get the lay of the land to figure out where to go. As you progress, search tends to become more of a navigational shortcut, and we’ve seen this activity increase over the last two to three years. Increasingly, we're using search engines to get us from point A to point B online.

As you get into the buying process, you’re familiar with the vendor site. You’ve been on the site. You’ve checked different product information pages. As you come back to that research process, you say, "I really want to find that product spec sheet that I saw on this vendor site." A lot of that navigation to those specific pages happens through a search engine. So, there are multiple touch points through the process.

Gardner: Now, you did this search in March, and you surveyed 1,086 people, North Americans, mostly women -- 63 percent -- average age 43 years old, with 67 percent of them having at least attended university.

Hotchkiss: Right.

Gardner: Can you tell us little bit more about these people? Are these people that you acquired through strictly business activities? How did you know that they were in a purchasing mode?

Hotchkiss: When we structured the study, we used our sampling partner, Survey Sampling International, for access to their B2B decision-maker panel. In two different parts of the survey, we asked, “Are you currently considering a purchase in excess of a thousand dollars?” That was a qualifying question. If they answered yes, they got to continue the survey.

That’s how we determined what role they were playing in this purchase that was happening right now. What were they considering purchasing? What was influencing them? We wanted to use a purchase process they were in the middle of, because it would obviously be fresh in their minds and they could really tell us what they were going through, as far as what influenced them.

We also wanted to get a retroactive view of a successful transaction. So, in the second part of the survey, we asked them to recall a transaction they had made in the past 12 months. We wanted to see whether that initial search led to a successful purchase down the road, and, at the end of the road, how the different factors influenced them. So, we actually approached them from a couple of different angles.

Gardner: Now, 85 percent of these people say they're using online search for some aspect of this purchasing process. It strikes me that this involves trillions of dollars worth of goods. These are big companies and, in some cases, buying lots of goods at over a hundred thousand dollars a whack. Do you concur that we're talking about trillions of dollars of B2B goods now being impacted significantly by the search process?

Hotchkiss: Absolutely. The importance of this is maybe the most mind-numbing fact to contemplate. Traditionally, the B2B space has been a little slow to move into the search arena. Traditionally, in the search arena, the big advertisers tend to be travel or financial products.

B2B is just starting to understand how integral search is to all this activity. When you think of the nature of the B2B purchase, risk avoidance is a huge issue. You want to make sure that whatever decisions you make are well-researched and well-considered purchases. That naturally leads to a lot of online interaction.

Gardner: I suppose if I am making a $100,000 purchases, and I make a mistake, I am not going to be around for long. Right?

Hotchkiss: Exactly. The other thing is that we don’t tend to be as emotionally involved with the B2B purchase. Things like branding play different roles than when you're doing consumer purchases. The brand affinity is something you might have if it’s an area where you don’t have a lot of experience. It may be a new need that’s coming on the horizon for your business. You are really starting from Square One, and that’s the perfect place for search to plug in and be the solution when you start researching those purchases.

Gardner: Right. These folks are looking for practical approaches and real information. Let’s go to Bryan Burdick at ZoomInfo.

This is growing quickly as an overall trend, but ZoomInfo, which is focused on business search, is growing much more rapidly. What’s driving your growth at ZoomInfo, and how does that relate to this B2B search activity?

Burdick: The business information search is a primary factor driving our growth. Our company right now is growing on two fronts. One is our traditional paid-search model, where we have subscription services focused on people information that is targeted at salespeople and recruiters as a source for candidates and prospects.

The more rapidly growing piece of our business is the advertising-driven business information search engine, which I think is a really interesting trend related to the concept you guys were just talking about. Not only does the B2B advertiser spend lots of money today trying to reach out, but the B2B searcher has new tools, services, and capabilities that provide a richer, better, more efficient search than they’ve had through the traditional search engines.

Gardner: By far, the largest player in this is Google with, according to the Enquiro survey, 78 percent use by this group of respondents. Way down the line was Yahoo!, and then far below that was MSN. It strikes me that Google is a general search engine, and yet we are asking for very specific business information.

Bryan, do you expect that we are going to see some sort of a specialization or a cleaving between general search and more vertical specialized search?

Burdick: Absolutely, and, in fact, that’s really ZoomInfo’s mission -- to do for the business-information search world what the Expedia or Travelocity did for travel search. When you think about it, you can actually go to Google and find an airplane ticket, but why would you?

It’s so much more efficient to go to one of these vertical search engines that are looking at the databases, looking at the data, and indexing it in a much more efficient way. You're starting to see that in a lot of other verticals. Travel has been the quickest to adopt that, but everything from business information, like ZoomInfo, to podcasts with Podzinger, and other types of vertical searches, have been focusing on a niche and organizing the content more efficiently.

Hotchkiss: One thing we found in the survey is that there's a natural evolution through the process. Although you might start on Google as you are trying to find those information sites, quite often it’s the verticals that people work into as they are starting to look for specific, more granular information on the companies they're thinking of doing business with. That’s where ZoomInfo and other vertical players fit a need.

Gardner: I suppose another thing that seems the same from 2004 to 2007 is the importance of a supplier’s Website. According to your survey findings, people are very interested in word of mouth. They use the search engine to move from that point to gather more information, but they're very quickly interested in solid, simple, straightforward, text-based information from the suppliers themselves. I suppose that underscores the need for sellers to have a very strong Website.

Hotchkiss: That was a really strong finding, and not really surprising. It made sense, but I think how important the straightforward information was to the people doing the research was somewhat surprising. It’s one of those things where you get findings in research and then afterwards, when you apply common sense to it, you say, "Yeah, that just makes sense."

But, remember these searchers are out to gather information for an organizational buying model. They are gathering information that will, in a lot of cases, be passed on to other individuals to help them make the decision as a group.

You don’t necessarily want to sit through a linear presentation of information, like an online video, or even a podcast, if your intent is just to pickup specific data and pass it along. Now, if you are the user, and this is that something you are going to be using, you may be a lot more open to a linear how-to demo. But, it’s important to match the content on your site to the types of buyers and individuals who are gathering the information.

The takeaway we got from this was to make sure you're covering the basics first. Make sure that you're getting the clear concise product information out. In a lot of cases, you know you are going to be compared to the competition. Why not enable some of that to happen on your site and make the buyer's job easier by arming them with the necessary equipment.

The number one thing that came across as desirable was pricing information, which is really tough for B2B marketers to put on the site, because in a lot of cases these are complex solutions. But, what the buyers are looking for is qualifying it in a budget range.

Is this a $10,000 purchase, a $100,000 purchase or a million dollar purchase? I need to know that to qualify, so I can move on. Please give me that information. It can be ranges. It doesn’t have to be specific, but I need to be able to qualify it.

Burdick: Dana, I would add that some of the typical mistakes that a B2B marketer will make from a search-engine marketing perspective is jumping too quickly or focusing too much on the actual advertising piece. They need to do that, but sometimes they forget about the search engine optimization. Very often, they leave that up to the technical team, which may optimize the search or the site in ways that aren't optimal from a marketing perspective.

Then, as Gord was saying, they get the user, or the potential customer, in there and the customer gets lost on their own site, searching for the type of information that they're looking for. It’s not like a consumer model, where the consumer already knows, in most cases that they are looking for a DVD player or whatever it is. They may even have a model number, and they're looking for the best price online. It’s much more of an information-gathering process in the B2B case.

Gardner: Perhaps the takeaway here would be that people want to get just the facts upfront and they want a ballpark figure, but they also want to be able to use search to get to that information fairly quickly.

So, if you’re going to optimize your site, your key information can’t be 18 pages deep into the search process, but you also have to consider that factual information needs to be as accessible as your main branding page.

Burdick: Absolutely.

Hotchkiss: One more point on that. A lot of B2B marketers like to capture as much information about a lead as early in the process as possible, so it can be handed over to the sales department, which can close the sale.

But, what happens in a B-to-B purchase is that the first touch point with your vendor Website is typically not the decision maker, the ultimate decision maker? It tends to be somebody who is gathering information to help arm the company to make that eventual purchase decision.

So, if you push to establish contact with that person, they're not ready to establish contact with the vendor, because they don’t have the buying power. Even if you do push to get it, your salespeople are spending a lot of time following up on leads that aren’t qualified buyers. They have to retrench down the road and try to re-establish connections with the person who has the economic power.

So, it’s really a "date," and, in a lot of cases, it’s a long series of dates. If you push too fast you are just going to push the prospect away.

Gardner: You don’t want to propose on the first date.

Hotchkiss: Yeah.

Gardner: On the other hand, one of the findings from the survey was that 50 percent convert to a sale online. So, that means that when the research, winnowing, triage, and the comparative shopping are over, the economic buyer, who is empowered to make a decision, will go back online and consummate the deal. Does that make sense?

Hotchkiss: Yeah. Here's some further insight into that, because we saw that number surprising when we did the overall data. When we pulled the data apart, we found that a lot of those purchases tended to be things like computer systems, where they might have been buying through a Dell or someone like that.

Gardner: The direct model.

Hotchkiss: We thought that was a really high online conversion rate. As we looked at the data a little more, we saw that in a lot of cases it was smaller software purchases or system-based hardware purchases. That made a little more sense, as we went down that road. There was a fairly strong manufacturing component, where people were buying parts and different things. In those cases, a lot of those purchases are made through an electronic marketplace. We're seeing that as an increasing trend too, e-commerce-enabled market places.

Gardner: I suppose it's also logical that when the price or the total purchase price is less than $50,000 or closer to $10,000, they’ll be more likely to do that online confirmation and make a purchase. To me, this says that small- to medium-sized businesses selling small types of goods should be very focused on search and B2B online activities. Does that follow?

Hotchkiss: Everybody needs to be focused on search. I can’t see an exception. You mentioned the percentage that said they would go online. We segmented out the group that didn’t indicate they go online to see what was unique about them.

The only thing unique about them was their age. They tended to be older buyers and tended to be with smaller organizations, where the CEO was more actively involved in the purchase decision. That was really the only variants we saw. If it’s a generational thing, then obviously that percentage is going to get smaller every year.

Burdick: Dana, could I ask Gord a follow-up question to that?

Gardner: Of course.

Burdick: I'm curious whether, as you dug into the data, you saw any differences between online follow-throughs to purchase on hard goods versus services. I'd think that people buy computers online, but if they're buying services from a B2B company, that tends to be offline.

Hotchkiss: When we were looking at influencing factors, B2B services was the one where word of mouth edged out online factors by a significant margin. When you're trying to retain a service, word of mouth is still very influential. In pretty much every other category, online was right up there with word of mouth, in some cases edging it out as an influencing factor.

Gardner: Okay, another number from this was that 95 percent said they use search to find what they want at some point or another, but 37 percent were still seeking other sources. There seems to be a recognition that search is very powerful, that it’s a tool that shouldn’t be ignored under any circumstances, but that it's not getting the entire job done. Bryan, I wonder if you could respond to that. What else needs to happen in order for these people to get what they need?

Burdick: The short answer is they just need to come to ZoomInfo.com. Seriously, I don’t think it’s a matter of needing more information, but, in some cases, finding better information. When you think about the traditional search engines -- the Googles and the Yahoos! of the world -- there are so many consumer-oriented search transactions on a day-to-day basis that they have optimized their experience with the consumer in mind.

Search engines like ZoomInfo and some of the other business-information search engines are taking a different approach and optimizing the search experience, and therefore the relevance of the results, with the business-information searcher in mind. You can much more efficiently and quickly get to the information you're looking for.

The simple example that I like to use is that if you search for "enterprise routers" on Google, you are going to get 32 million results that will include everything from Enterprise Rent-A-Car to the latest episodes of Star Trek. Search for that on a business search engine like ZoomInfo and you'll get 150 companies that sell enterprise routers or manufacture enterprise routers. It just becomes a much more efficient process.

Gardner: Well, even the alternatives cited in this survey are still very general. There was Business.com and Thomas Register. This is every good under the sun. It might as well be a general search. KnowledgeStorm was also mentioned, but it's very specific to IT. It seems like there’s a whole category that needs to be filled here around vertical business search.

Burdick: The original survey that Enquiro did for us in 2004 was a key factor in our decision to move to more of a search-engine model, because what we see happening is the same kind of evolution that happened with the big search engines way back. It’s happening now in the vertical-search categories, where search engines started out as directories or, if you think back to Overture in their early days, totally paid listings.

Eventually those two forces came together to provide a best-of-both-worlds situation, where you’ve got not only great relevance on the results, but also great relevance on the targeted ads, and now that’s starting to happen in the verticals as well.

So, you’re starting out with some of the business-directory sites or the business paid-listing sites, because those are easy to do. As the technology gets more sophisticated and you can provide more relevant results for the business information seeker, you are delivering the value that the information seeker is looking for. Plus, you can target the ads better and provide an overall better experience.

Gardner: Let's go back to Gord on that. The survey found a larger percentage of people looked at the organic results on the left, but they were pretty much limited to the top four. A smaller percentage, about 12 or 13 percent, said that they look over to the right-hand side for the ads. That 13 percent might sound small, but compared to a click-through rate in a Web advertising model of usually less than 1 or 2 percent, 13 percent is still a pretty large chunk of people. What’s your impression of the impact of the advertising model in search for B2B activities?

Hotchkiss: Those percentages, by the way, aren’t that different from what we've seen in consumer-based research. Those breaks between organic and sponsored tend to remain fairly consistent across a number of different channels. One thing that’s just golden about search is it will connect a motivated and engaged user with the content they are looking for.

If that content is provided by a relevant ad, then they're open to that. In fact depending on where they are in the buying cycle, they may even be biased toward that, because they are ready to get information from somebody who is trying to provide them what they need to decide whether this is the thing they need to buy.

There's a totally different interaction when you're on a search engine actively engaged in a task and actively looking for information. You're far more receptive to messaging at that point. You're actively looking for it. And this seems to be slowly breaking into the consciousness of most advertisers. They're getting it, but they're getting it slowly. Anyone who moves into the search space, if they do search in a smart way and they capitalize on the potential of it, is just amazed by the return they get on this.

Gardner: Let’s beg a little more from these results than was intended. I started to see in the findings some patterns about typical processes, about how people would go about this activity -- the awareness, the research, the purchasing, and so forth. It seemed to me that there was, on one hand, a pattern of word of mouth that led to a search, that led to a Website, that then led to a refined search based on what they found, which then led to a hand-off to a purchasing activity by maybe a different department or individual.

There also seem to be offline influences, perhaps trades shows, perhaps traditional sales calls and activities, but that also took into consideration word of mouth that then identified what to search on, and so on. Am I reading too much into this, Gord, or were there patterns of process around the use of search in the purchasing activity?

Hotchkiss: It goes back to what I said before. Search tends to be the thing that connects you, as you move through the buying process, and it’s used in different ways and places as you progress through that. As far as identifiable patterns and usage behavior, if you did an end-up study of a large enough dataset, patterns would emerge. They always do emerge, but I'm not sure we would be confident enough diving into this particular dataset to try to tease that out of it.

What we did see is that B2B buying decisions are tremendously complex when they are compared to an individual consumer buying decision, where you have one person going through all the phases. You have multiple individuals influenced by different factors going back and forth.

What is consistent in that is whatever is influencing you -- whether it’s online or offline, a discussion with a colleague or a recommendation by a paid consultant -- there tends to be a mirror activity, in which whatever happens offline generates some kind of online activity that typically is initiated through a search engine. Then, you pull that information back, and it dances back and forth between the online and the offline world.

Gardner: So, there’s a barrier here in some senses. I'm sure most companies, especially the larger ones, have a standard operating procedure about how purchases will happen. It will be form x, y, z, and it will go through process review a, b, c, and then we’ll have to get signatures from individual g, b, h. How can we bridge this value that people see in search, and somehow bake that into a procedural process inside of an enterprise. Or are we asking too much here?

Hotchkiss: One of the interesting things, being both a researcher and a vendor, is we get to see both sides of it. We have access to more information than ever before, and buyers out there are armed with better information before they ever initiate contact with a vendor. They are gathering a lot of information and then they are trying to cram it into an existing buying process, whether that’s an RFP process or whatever.

So, like most things with the Internet, the traditional systems are being challenged by this new access to information that we never had before.

Over the next two to three years, what we're going to see is organizational buying processes being streamlined and being able to incorporate the fact that you have better informed buyers than you may have had before. The whole RFP process was to eliminate risk. The reason was make sure that you are considering alternatives and to make sure that you are almost forced to gather the information you need to make a dispassionate judgment about what was the best choice.

Now, in a lot of cases, the decision is already 80 percent made before the RFP process ever begins. Somebody has researched a vendor, has a very strong feeling about the vendor but now has to try to fit that into the established procedure, and they say, "Okay, now we've got to go out and find two or three more alternatives."

Heaven help the other two or three alternatives that are getting involved in that process. They have to go through the whole dance, but usually the preferred vendor on the front end gets the business on the back end. The other two or three players are just used as negotiating chips to try to get the price down. It’s interesting to watch how information from the Internet is changing virtually everything out there. This is no exception.

Gardner: This might be a little bit out in the future, but the role of search could morph into the role of auction and brokering activities. Does that make sense?

Hotchkiss: Yes, and for players in the space -- I suppose ZoomInfo as well -- if we can streamline the marketplace, if we can take this access to the information and make the buyer’s job easier, that’s a tremendous saving. I would hate to think of the number of man-hours that are invested internally in an organization for a fairly major purchase decision. How much more efficient you can make that process by simply empowering them with the information that they are going out to look for anyway?

Gardner: Bryan, you said that the Enquiro survey in 2004 made an impact on ZoomInfo in terms of its direction. Have the findings from 2007 had a similar influence? What new directions might you be heading in?

Burdick: As Gord had said earlier, the findings in 2007 reinforced what they had already learned three years ago. So, in one sense, it confirmed our own strategic direction as well. We re-launched the ZoomInfo.com site back on April 1, and moved into more of a traditional search-engine model, where all of the content, all of the search capabilities on ZoomInfo.com are free and subsidized by advertising.

We’ve seen that piece of our business take off like crazy in the last couple of months. Search traffic has grown dramatically. We’re up to close to 5 million unique visitors a month doing about 16 or 17 million searches a month on our site. All that is really driven by this need, this desire, among B2B buyers to find a more efficient process to get at this type of buying information.

Gardner: What advice would you have for those folks that are on the selling side of this? What should they do to position themselves in order to take better advantage of what’s occurring on the buyer side, particularly, in their use of search?

Burdick: There are a couple of landmines or traps to avoid. The first is to try to avoid competing with the consumer brands, either on the traditional engines or with the same types of keywords. If you are buying the same keyword as a consumer brand is buying on one of the traditional engines, you are typically going to get drowned out by the noise.

The other thing is to make sure that your own search marketing is coordinated with channel partners. I’ve seen lots of examples where the manufacturer is buying a set of key words and their value-added resellers are buying the same keywords. They end up bidding up the same keywords just to attract the same eyeballs. At the end of the day, the manufacturer is going to direct them to one of the resellers anyway.

Gardner: So that leads to confusion, rather than streamlining that particular process.

Burdick: Exactly. The other key thing, which we already touched on, is leveraging not only on the marketing side, but the search-engine optimization of your site in general, and optimizing the search-for-information experience that the buyer has once they get to your site.

Hotchkiss: One thing I would add on a more fundamental note is to make sure that the perspective you are using to evaluate your search strategy is the customer’s perspective and not your internal one. One of the common pitfalls we see is companies get into a bad case of "internal think." They look at everything from their internal perspective, and they are not shifting the looking glass 180 degrees and looking at it from their prospect’s perspective.

It’s amazing how enlightening it can be, when you start looking at what types of sites they are going to. You need to catch their attention, and know what kind of messaging you have to present and what kind of onsite experience you have to present, once you are successful in capturing the click. If you can force yourself to see from that perspective, you're going to do more to improve the effectiveness of your campaign.

Gardner: Well, thanks very much. We’ve been discussing the recent survey by Enquiro Search Solutions. It was Enquiro’s 2007 B2B search survey, and to help understand it better we’ve been joined by Gord Hotchkiss, the president and CEO of Enquiro. Thanks, Gord. Is there anything meaningful in the survey that we didn’t cover and that you think we should?

Hotchkiss: The original release covers the high-level findings. What we're working on now are three follow-up white papers that will be available on the site through the next three to four months. We're going to take each of the three buying roles that a lot of people within the survey fell within -- the economic buyer, the technical buyer, and the user buyer.

We're going to do insight from that particular buyer on how to search more effectively, plod through the process a little bit more, and how those hand-offs happen from role to role. I imagine there are going to be new insights out of that. We're taking different slices of the data. So, for those of you who are interested in that, keep checking out on Enquiro.com and we’ll ping you as those white papers become available.

Gardner: Terrific. And we’ve also been joined by Bryan Burdick, the chief operating officer at ZoomInfo. Is there anything that you’d like to cap the discussion with, Bryan?

Burdick: Only that I think from a vertical business information search perspective, that we’re really in the first inning here. A lot of interesting trends and enhancements are going to be coming down the road. One in particular that may have an influence in the next year or two is the community aspect within the search.

Gord, talked earlier about how there are multiple people that influence the B2B buying decision. I think that you’ll start to see a marriage of, not only B2B search, but also online community and a factoring into that whole process. Then, who knows where we’ll go from there? But I appreciate you having us on.

Gardner: I suppose that this notion of word of mouth being so important in shaping people’s direction that you might use search to find the word of mouth.

Burdick: Right, the word of community.

Gardner: There it is. Okay, well thank you, Bryan. This is Dana Gardner, principal analyst at Interarbor Solutions. You've been listening to a sponsored BriefingsDirect podcast. Thanks for joining.

Listen to the podcast here. Watch the video here. Sponsor: ZoomInfo.

Transcript of Dana Gardner’s BriefingsDirect podcast on B2B search usage, trends and future direction. Copyright Interarbor Solutions, LLC, 2005-2007. All rights reserved.

Book Review Discussion: 'Total Architecture' Elevates SOA to its Business Benefits Potential

Edited transcript of BriefingsDirect[TM] podcast on the book "Succeeding with SOA: Realizing Business Value Through Total Architecture," with author Dr. Paul Brown, recorded Aug. 24, 2007.

Listen to the podcast here. Sponsor: TIBCO Software, Inc.

Dana Gardner: Hi, this is Dana Gardner, principal analyst at Interarbor Solutions, and you're listening to BriefingsDirect. Today, a sponsored podcast discussion about an interesting book called "Succeeding with SOA: Realizing Business Value Through Total Architecture."

We're going to be reviewing the book and discussing some of its implications for businesses and IT departments -- that Total Architecture impacts the very notion of enterprises and IT. With us is the author of the book, Dr. Paul Brown, a principal software architect at TIBCO Software. Welcome to the show, Paul.

Paul Brown: Hello, glad to be here.

Gardner: Also joining us to discuss the implications of, and gather some insights from, the book is Todd Biske, enterprise architect at MomentumSI, an Austin, TX consultancy. Welcome to the show, Todd.

Todd Biske: Thanks, Dana.

Gardner: I really took a lot away from this book, and one of the big things that it reaffirmed to me is the impact that Services Oriented Architecture (SOA) has across an organization. We talk about the technology implications and the need for alignment between business and IT issues, but it seems to me that we really need to address a total enterprise situation.

I think that’s what you drive at with this concept of "Total Architecture." Am I getting that right, Paul?

Brown: Yes, definitely. Our enterprise business processes and our enterprise systems have become so hopelessly intertwined that we can’t really talk about designing one without designing the other. We’re moving lots of activities that used to be done manually either completely or partially into IT systems.

Yet we still have to have people involved for the exception handling, even when we try to do a fully automated system. So the idea is that you have to explore the design of both together -- and you can’t separate them.

Gardner: I suppose we’ve gotten to this position because there are new realities that enterprises face. Many of them are 20 or 30 years into managing and producing IT and systems, and deriving productivity from those systems.

But it seems that things are not progressing, that there are scaling and complexity issues. You say in the book that there we are "stressing" existing architectures. Can you elaborate on that?

Brown: There are a couple of things going on. The structure and organization of IT is very often not completely aligned with the business. The care and feeding of the IT infrastructure has become a focal point of a lot of the IT investment.

What we’ve lost is the connection between what’s going on in IT and what’s going on in the business. It’s somewhat analogous to when we started exploring object oriented design (OOD). One of the benefits there was that the structure of the problem now became recognizable in the structure of the solution.

We’re faced with a similar situation today. The structure of our IT solutions are difficult to map onto the structure of the business processes. It’s that realignment that we’re trying to address with SOA.

Ideally, the services that we’re building -- that have technical implementations -- are building blocks of business processes. And so we see that tie-in once again.

Gardner: I expect we’re going to have some listeners that aren’t all that technical. So, just for the edification of our audience, could you please give us your "elevator definition" of SOA?

Brown: SOA is an architecture in which the functions that you need in your business processes become recognizable services of your IT information structure. At a lower level, we can also build services that the IT community consumes. Those are technical services. They become the lower-level building blocks for ultimately building the business services.

The real business value comes out of having a portfolio of business functionality in the form of services that you can quickly reorganize and re-orchestrate to build new business processes or to modify your existing processes. That’s where the big business win comes in.

Gardner: I have had people come to me and say they don’t understand the difference between an application and a service. Why is it that applications can’t be used in a similar way?

Brown: You almost can. Most of the functionality that we need is in those applications. The problem is that the functionality wasn’t designed for reuse. Applications tend to be somewhat specialized for use in a particular situation. Their functionality is hard to get at.

It’s a COBOL mainframe transaction, and you have to be on the mainframe to invoke it. Or it’s a piece of Java code, and you need to be in Java to invoke it. One of the things we’re trying to get to here is just to make the stuff more accessible and to standardize it in such a way that it reasonably fits into different business situations.

IT Misalignment

Gardner: Todd Biske, let’s go to you for a moment. As we try to define the issues that IT organizations are grappling with, you and I have discussed in the past the notion that IT can’t keep up with the pace of business, and that business would like IT to be a lot more agile and responsive.

Where do you see the root causes here? What is the main problem that is preventing IT from responding today?

Biske: Every organization is going to have its own distinct challenges, but as with a lot of things you have to look at the whole notion of the enterprise -- not just IT, not just business -- the whole piece of it.

Dr. Brown points out that a lot of times IT may be misaligned and not be able to see the business processes that are spanning the silos. It may also be the case that the business isn’t either. So, you’re really setting up challenges, when you’re trying to view things from more of an enterprise perspective.

I really see things like this as an organic growth of technology over 20, 30, or 40 years. That’s what happens as you slowly add things. You’re adopting things as you go along, and you eventually get to a state where you have to step back and view it from a higher level -- to really understand what you've got, and then to start breaking it apart.

Trying to do that at the same time that you’re trying to keep the business growing and keep up with new competitors who are nimble and don’t have all of the boat anchors tying them down, is always going to be a challenge -- whether it’s SOA, business process management (BPM), OOD, or the Web.

It’s the same pattern repeating itself, but the pressures of globalization make viewing your entire enterprise and your business as a whole even more important. It’s hard to continue to operate in a niche and continue to sustain that organization for many years, whether it’s from the IT aspect or the business aspect.

Gardner: Now, Paul, you finished this book earlier this year. It was published by Addison-Wesley in April. You have another related book in the works: This was the first book that came out, and it’s geared toward a business or a higher-level audience.

The second book, which is going to be coming out later this year, is called "Implementing SOA: Total Architecture in Practice." It's more of a hands-on and how-to book, tailored toward developers, designers, and the architects themselves.

Why do you think that SOA needed to be broken up as a subject in this way?

Brown: To build on a point that Todd just made, we have to look at our business intent, which is to deliver the results of the business processes, and tie that back together with IT.

It’s distressing, when I go into a company, how infrequently I find anybody who can articulate what the end-to-end business process is that produces the key results or can tell me what the entire order-to-cash cycle looks like. There are lots of experts for fragments of it. The IT focus has traditionally been on fragments of the processes.

The pressure these days is to improve the overall business process, the response time to the customers and partners, and the overall quality of what’s going through. So we need this focus on end-to-end business process and the focus on the end-to-end system interaction that helps bring that business process to life.

The pair of books focuses on that issue. This first book is essentially a business argument, and this perspective is required. There is a needed role to carry that perspective forward. That role is the role of the architect, whether the business process architect or the systems architect.

This first book tries to make the case that we need to start taking an end-to-end perspective, and it has business implications. It has implications for how we organize and conduct projects and how we manage the business processes themselves, because very often they’re not being managed.

The second book gets down into the technical aspects of what exactly it is that the architect is doing at the business-processes and systems level. What kind of information do they need? How do they get the stakeholders involved? Essentially, it’s oriented to the execution of the architect’s role.

Evolution, Not Revolution

Gardner: We’ve talked about a progression over decades in IT. We’ve also had, of course, transformation and evolution in how businesses are organized and managed. There have been plenty of buzzwords and schools of thought over the years. I’m wondering if we also need to address that organizational side.

It seems to me that many of the points you make in the book point out the foibles of decentralization. When things are too fragmented, when the systems as well as the processes are not being addressed holistically, and that leads to eventual breakdown.

Are we really talking about a change from decentralization to command-and-control? Do we need to address all of this organizationally pretty much from the outset?

Brown: I think it's an evolution. It’s inevitable that the bulk of the resources are going to remain in the organizational silos that they’re in. They’re organized that way, because they’re grouped around a particular piece of functionality, and that functionality is not going to go away.

What we need to introduce into the picture is more thinking in terms of how these different silos play together to make the business work. That has to happen on both the business side, looking at business processes, and the technology side, looking at systems.

To me, it's a real open question as to where that responsibility actually lies. In a theoretical sense, I would think that it belongs with the chief operating officer (COO) of the company, whose job is essentially to execute the day-to-day business of the enterprise. I don’t see many companies that have evolved that far yet.

More often, you see the responsibility falling under the chief information officer (CIO), which is not necessarily inappropriate. In my book I argue that the CIO has another huge responsibility, which is simply managing the vast array of resources in the IT part of the world -- not only the systems, but the people that are there for their daily care and feeding.

That’s a huge responsibility in itself. I wonder whether a single individual has the bandwidth to manage this overall marriage of business processes and systems at the enterprise architecture (EA) level, and to also manage all those resources.

So I leave it as an open question. There are two roles that need to be played. I think each organization has to find its own solution for where those roles live in the organization.

Biske: One of the things that I liked about the book was that it introduced a traditional organizational hierarchy, but then the bulk of the book talked about this project organization. You’re trying to execute a project ... [but] what’s the structure of the organization that supports getting that project done?

Do you think that we’re going to require more fluidity in our organizations, less formal structure and more focus on this particular effort? What does the organization need to be to ensure the success of that effort?

When we go on to the next effort, you again change the organization for that effort to ensure that if it’s successful. It’s not so much just about putting out the hierarchical organization chart, and then everyone tries to operate according to those rigid silos.

Brown: I think the assembly of an ad-hoc project team, typically referred to as a cross-functional team, is very important. It’s not a new concept, and even today many projects are organized that way.

What I think is necessary to augment that, though, is active participation of the enterprise architecture group with a broader view of the end-to-end business processes and the end-to-end systems that are involved.

The project team is going to be narrowly focused on accomplishing specific tasks and achieving specific goals, but if we want our service portfolio to build out, if we want that kind of flexibility, the style in which that is done has to be integrated smoothly with the bigger picture of both enterprise business process and enterprise systems architecture.

So, we need a proactive enterprise architecture group that’s willing to roll up its sleeves and get its hands dirty, touching the individual projects that are going on. They really need to coordinate their work, so that we don’t end up with the chaos that we’re in today, and so that we have pieces that elegantly fit together and can be rearranged to achieve new business goals.

Gardner: You make a strong point in the book that, in very few cases, someone has the full understanding of an entire business process. You also make the point that when we do get those people in place, they need to have authority over that entire process. And you have set out a methodology about how to approach this from a political and power perspective.

For example, Paul talks about the need for a process charter, which is sort of prime objective for a project. There needs to be a business-executive sponsor, someone who can take some ownership and have authority. There should be an IT executive sponsor, as well as the business executive.

Let’s look at just this personnel level. How do you think organizations can start to get people in a role of actually owning and having authority over an entire business process? There seems to be a vacuum, that there hasn’t been anyone in this role before.

Brown: Well, in a pragmatic sense, we sort of evolve into the business process from below. By "from below" I mean by attacking it from a system’s perspective. The reality is that anytime we touch systems, intentionally or unintentionally, we tend to alter the business processes they live in.

As a starting point, I would encourage the technical architect simply to be inquisitive about the business process and make sure he or she understands what the process is supposed to be, and acquaint the business with the implications of the changes that are being made.

You can do a lot of it through question and answer, just by asking questions: "Nobody told me what’s supposed to happen if this goes wrong at this point in the process." So by answering questions, you can raise the visibility of the business process.

That’s a far cry from actually taking control and managing that business process. We see some of the control and management arising out of workflow-oriented projects, where usually, at the departmental level, a business person is consciously trying to grab a chunk of a business process, and then structure it so that it can be controlled and managed. We see these different fragments coming into play here.

With really large projects, if you look at an enterprise resource-planning initiative, you tend to build this project team that reports in at a very high level in the organization and gains a lot of authority from where it reports in. The enterprise resource planning (ERP) projects that are most successful are the ones that consciously address both the business-process re-engineering part, as well as the systems engineering.

What we want is a model that’s very similar to that. You need the same kind of authority from that business-executive sponsor and the IT-executive sponsor to make everybody play ball. But now the projects are small. They’re not the mammoth business ERP efforts anymore.

It’s your everyday integration project or services project that spans silos. What we need is a project organization for them to report into.

In my book, I argue that organizations should be part and parcel of the same organization in which the enterprise architects live. That ideally ought to report in on the business side of the house, probably to the COO, because that’s where the authority comes from to really make changes in the business.

Architecture: An Enterprise Issue

Gardner: You say in the book that architecture is no longer just an IT issue, but an enterprise issue. By that do you mean that we should dwell on the fact that the IT people need to understand business issues and what the goals of the business are?

It seems to me that you’re also saying that the business side of the house needs to understand the technology issues as well as the process issues. Is that the case?

Brown: Yes. When you design a business process, you’re making assumptions about what systems are going to do and what people are going to do. You can’t validate the assumptions until you actually start doing the technical design.

You need to have a systems architect involved when you are planning business processes. Conversely, you can’t really make modifications in the system without inadvertently changing the business processes.

So it doesn’t make sense to separate these activities. You really have to treat them as if you're architecting the business, which consists of both business processes and systems.

Gardner: Todd Biske is a practitioner in the field. Do you see some movement within enterprises that they recognize this, that they will be looking at architecture from a "total" perspective that will encompass both the IT systems issues, as well as the process issues?

Biske: I think you're starting to see some organizations on the leading edge take this approach. Overall, I think enterprise architecture is still -- surprisingly -- an emerging discipline in many organizations.

I can look at some of the companies I’m familiar with here in the St. Louis area, where I’m located, and not all of them have enterprise architecture organizations. They’ve got to get over that first step of recognizing the need.

Groups that have been practicing enterprise architecture for some time are clearly growing out of the IT roots, and realizing [that the solution] is about the business and it has to incorporate business architecture.

If you go to a Gartner enterprise architecture conference, you’re going to hear the same thing. There's this momentum building around the notion of business architecture. It’s not just about what technologies I have and how do I consolidate the number of vendors that I’m dealing with, and the more IT related issues.

So I definitely agree that it is total architecture. Ultimately it has to involve the business, if the company is going to continue to be successful in the long run.

Gardner: Do you think that for business people on the management and organizational side of the house to read this book would perhaps illuminate for them some of the interdependencies [between business and IT]? Would it set them up to take part in a discussion like this?

Biske: It will. It was really interesting reading the book, thinking of some of my own experiences, and realizing that this book is really written from a business perspective. I've worked in organizations where, as described in the book, there is this hand-off that occurs at some point. Whether people realize it or not, all of a sudden you’ve now got business projects and you’ve got IT projects, and you run the risk of their becoming disconnected from each other.

I kept that in mind, as I was reading the book. ... I considered whether it could be a challenge or not. If you really have the business involvement, as it’s described in this book, this makes a lot of sense.

The extent to which a business manager could read this and see it come together is going to depend on how they currently work with their IT staff, and how open are they to changing the way IT behaves.

If they’re set in their ways, are used to the traditional customer-supplier relationship, have these hand-offs, and then things get disconnected -- it may not occur to them that there is another way of doing this. But if they have recognized that they need to get better interaction with their IT departments and start to operate more in terms of a partnership, then this book can help them out a lot.

Gardner: One of the things that was impactful for me was the notion that SOA can be a catalyst to thinking about things differently and bridging this business-IT chasm. And also that SOA can be a precursor to a whole new conversation about how to think about businesses from this perspective of "Total Architecture."

Those people who are thinking of SOA as a technology discussion will have their eyes opened by this book.

Let’s move on to the notion of "Total Architecture Synthesis." It’s a methodology that you discuss in the book, Paul.

Can you explain what you mean by "Total Architecture Synthesis," now that we’ve set the table in terms of the larger problem. Once we recognize that this is business and technology coming together, how do we then create a form of architecture that is inclusive for IT and business?

Brown: When you start getting into a project, there are a couple of traditional ways of structuring the work in the project. The oldest one is the Waterfall Model. That's where I gather all the requirements and do the business process design. Then, I would do all the systems design. Then, I would do implementation ... and so on.

The problem with that is that in a lot of our more ambitious projects, the feasibility -- and by that I mean our ability to execute the project within the cost and schedule guidelines that we’re given at the outset -- is still to be determined.

You’re well into the project, before you have enough information to recognize that you’re headed toward what Ed Yourdon would call the "Death March Project," one where the cost or schedule is off by more than 50 percent. So, the Waterfall Model is an expensive and risky approach.

Another model that’s being talked about a lot these days is the Agile Development model. Agile Development bites off a small chunk, tries to drive it to implementation, and then iterates rapidly. The issue I have with that approach is that you tend to pick off the low hanging fruit in the early iterations. You pick on the easy parts of the problem.

The situation you end up in is one in which you haven’t really thought about the architecture, because the simple parts of the problem didn’t challenge the architecture. By the time you get to the tough parts of the problem you have sunk cost into what may turn out to be an inappropriate architecture.

So I try to marry the two together. We want to take the agile approach, but just for building the architecture, not necessarily driving it to implementation. We want to take a look at the business processes that are involved in the project and figure out which ones are liable to be most challenging to the architecture. They're not hard to identify. They are the ones that have the high volume of execution, move a lot of data, require a lot of automation, and are business-critical.

If you simply inventory the business processes that you need to achieve your business goal and rank them in these four categories, the ones that are liable to challenge your architecture bubble to the top very quickly.

You then focus on gathering the requirements for just a couple of those business processes, the most challenging ones, and drive them down into a business process design sketch and an architecture sketch.

Immediately you begin evaluating your architecture: Is this something we can actually build within the cost and schedule guidelines? Does it achieve the business goals? If the answer is "no" -- you can stop right there, go back to the business, and have a business discussion: Should we be doing this project at all? Should we grow the scope? Should we cut the scope? Or, whatever needs to be done.

That’s the whole idea of Total Architecture Synthesis. We want to turn this development of the architecture into an iterative process that tackles the tough problems first -- the risk problems, the feasibility problems -- and gets them right up front.

Gardner: Does this make sense to you, Todd?

Biske: Absolutely. I read through that section and saw a lot of things that I had seen in the past of how organizations approach things. I think it’s right-on.

The way it was put in the book that I thought was good was, when you’re in the feasibility stage, there are things that you already know how to do, and those are the things that you shouldn’t be spending your time on at that point.

You already have a good degree of confidence as to how much it’s going to cost and how many resources it’s going to take. The problems that you don’t know how to do are the ones that you’re at risk of these cost overruns.

So focusing on those larger problems and getting them the appropriate priority early on handles these things much earlier in the lifecycle, and yet can still be done in an agile manner.

Gardner: There’s the adage of "follow the money," when it comes to getting things done. I think this makes a lot of sense. There’s a great deal of logic in this book, but when you go into organizations, a lot of people are driven by profit and loss centers, budgets, who's responsible for what, and what's the "cost center" versus the "profit center."

I wonder about the point you make in the book, Paul, around the fact that a service being created and driven into a SOA probably won’t save money upfront. That is to say, there is a need for investment. The return on that investment (ROI) will come later as that service is used across more business processes, and those processes become more agile themselves.

What needs to happen in addition to discussing Total Architecture from the business and the systems perspective? What of incentives and the financial structure? Does that need to be adjusted as well?

Brown: Well, in order to get to a portfolio of services, to get that long-range return on investment, you need to have organizational discipline upfront to structure your IT work as services. To accept the fact that, in the very short-term, you’re probably going to pay a little bit more for services than you would for developing the same functionality in a non-service or non-reusable manner.

The way you cover that in the short-term is you simply accept a lower return on investment -- a positive return on investment -- but a lower one for the early projects. That implies that you’re structuring your projects and choosing your projects because the individual projects have inherent business value. They’re justifiable on their own, and you’re just getting a slightly smaller ROI. With this approach you can sustain the investment in services indefinitely.

We’ve seen organizations that are succeeding. I’ll use two companies as examples, Harrah’s and Con-way. They're in very different markets. Con-way is in transportation; Harrah’s is in the entertainment and casino business. Both sustained their investment in services for a couple of years and achieved significant business benefits after that two-year investment.

Harrah’s, prior to their SOA investment, acquired five new casinos, and it took them two years to fully integrate. What does that mean in their world? In their world, their competitive advantage is that when you earn points in one casino, you can walk across the street to another Harrah’s property and use them to buy yourself a meal at a restaurant.

That fluidity of the customers throughout all the properties was essential to Harrah’s. It took them two years to get there before their SOA investment. Two years later, they acquired Caesars, which effectively doubled the size of the company. But they had also made this two-year investment in services.

They achieved the systems integration [with the Caesars merger] in six months, rather than two years. So they gained 18 months of competitive advantage, as a result of this investment in SOA.

Con-way has seen similar improvements in their projects as well, as they sustain this portfolio investment and now they’re reaping significant business benefits.

Concession to Reality

Gardner: You say in the book, Paul, that Total Architecture is a "concession to reality." What do you mean by that?

Brown: I mean that your business processes and your systems are intertwined, and the design of one affects the other. That’s reality.

You can choose to ignore your business processes and let them evolve by accident, as Todd was talking about earlier. That’s how we got in the bind we’re in right now. To a large extent, we’ve been focused on individual profit centers and driving down cost in individual functions. We’ve lost sight of the end-to-end business processes.

The resurgence of interest in customer loyalty, customer service, and all of that is really a reflection that, as a business, we need to stand back and look at what we’re doing to our customers and our partners.

The investments we’re making and the individual activities in the business processes are fine, but in order to remain competitive we need to stand back and take the holistic view. The holistic view simply means you need to understand what that business process looks like before you can improve it.

I’ve seen horror stories of people who have tried to improve a business process by examining a small portion and making some changes, without realizing that the changes that they’re making are having an adverse impact on the bigger business process. So while they thought they were improving things -- they actually made them worse. You can’t live with that.

Gardner: Any response to that, Todd?

Biske: I’m in complete agreement. One of the things that I wanted to call out was the ROI discussion. It was great that the book pointed out that projects are successful when they achieve the desired business benefit.

I can’t tell you how many IT projects I’ve seen whose mission statement basically said, "We have to get this thing done by this date for this much money." Their success criteria was based upon meeting the date and meeting budget -- and had nothing to do with the perceived business benefit.

Suppose suddenly we change our view on projects to say the project isn’t done until the business benefit has been achieved? Or we say if it's not going to be achieved, we made a mistake somewhere along the way and we need to cancel the effort? It puts a whole different perspective on things in terms of how these IT solutions fit into the business, and it gives that appropriate visibility that we need.

It’s not about the ROI of one service: It’s about the ROI that’s going to be achieved by a whole sequence of services coming together to support a business process, which is going to yield either those cost reductions, the ability to integrate faster, or new market opportunities.

You have to put it all into that appropriate perspective, which brings it right back up to Total Architecture.

You wind up in these debates over whether this one particular service is actually going to achieve significant cost savings or not. You can’t possibly answer that question without viewing things in a broader perspective.

So phrasing our projects in terms of the business benefits and taking that broader view is critical to understanding how these things are going to fit in, and actually achieving the goal of agility and better IT business alignment.

Gardner: Paul, as you’ve been talking about the book with folks who come from a number of different perspectives and situations, has anything surprised you in terms of the reaction you’re getting to it?

Brown: No -- but that’s the good news. I've gotten very positive feedback from the people that I’ve talked to who have read the book.

If there is anything that I get in the feedback, it’s that the organizational solutions that are required to make this happen are very diverse in nature. They depend as much on the current organizational structure and current culture of the enterprise as they do on any of the principles that I lay out in the book.

So I do see some creative discussion there around how to organize and execute this. But in general I’m getting very positive feedback.

Gardner: So the desired effect is happening.

Brown: Yes, although it remains to be seen. I would like to look at [the market] again a year or two from now and see if these ideas are really taking hold and starting to influence the way people think about organizing their business, their IT and their projects.

Gardner: Well, thanks. We’ve been discussing the recent book, "Succeeding with SOA: Realizing Business Value Through Total Architecture."

We’ve been discussing how the concept of Total Architecture can help elevate the issues and spawn some discussion about what will make SOA successful, and ultimately make businesses more productive, more agile, and perhaps even cut some IT costs in the process.

We’ve been discussing this with the author of the book, Dr. Paul Brown, principal software architect at TIBCO Software.

We’ve also been joined by Todd Biske, enterprise architect at MomentumSI.

Any parting thoughts from either of you, please?

Brown: Well, I'd just like to encourage people to take a look at the ideas in this book and keep your mind focused on delivering business value. The rest will take care of itself.

Gardner: Todd?

Biske: I would like to add that there is no shortage of books on SOA that deal with the technical details. But if you listen to a lot of the experts out there, all of them say that it’s not the technical details that are going to make SOA so difficult.

It’s all the cultural details that come into play. Dr. Brown’s book really does a good job in trying to discuss some of those non-technical things in the organizational culture, brings up a number of excellent points, and makes a number of excellent suggestions on how to improve that situation, and then really be successful with SOA.

Gardner: Well, great. This is Dana Gardner, principal analyst at Interarbor Solutions. You’ve been listening to a BriefingsDirect sponsored podcast. Thanks for listening.

I do hope that people take a look at this book. I think it does have a lot to offer.

Listen to the podcast here. Sponsor: TIBCO Software.

Transcript of Dana Gardner’s podcast with Dr. Paul Brown on his book "Succeeding with SOA: Realizing Business Value Through Total Architecture." Copyright Interarbor Solutions, LLC, 2005-2007. All rights reserved.