Showing posts with label rogers. Show all posts
Showing posts with label rogers. Show all posts

Thursday, August 06, 2009

BriefingsDirect Analysts Debate the 'Imminent Death' of Enterprise IT as Cloud Models Ascend

Edited transcript of BriefingsDirect Analyst Insights Edition podcast, Vol. 43 on the health of corporate IT and whether reports of its demise are premature.

Download the transcript. Read the summary blog post. Charter Sponsor: Active Endpoints. Also sponsored by TIBCO Software.

Special offer: Download a free, supported 30-day trial of Active Endpoint's ActiveVOS at www.activevos.com/insight.

Dana Gardner: Hello and welcome to the latest BriefingsDirect Analyst Insights Edition, Volume 43. I'm your host and moderator Dana Gardner, principal analyst at Interarbor Solutions.

This periodic discussion and dissection of IT infrastructure-related news and events with a panel of industry analysts and guests comes to you with the help of our charter sponsor, Active Endpoints, maker of the ActiveVOS, visual orchestration system, and through the support of TIBCO Software.

Our topic this week on BriefingsDirect Analyst Insights Edition, and it is the week of June 8, 2009, centers on the pending purported death of corporate IT, and perhaps the unplugging of the last on-premises Web server any day now.

You may recall that in the early 1990s, IT pundits, and my former boss Stewart Alsop, glibly predicted at InfoWorld that the plug would be pulled on the last mainframe in 1996. It didn't happen.

Stewart apologized, sort of, and the mainframe continues to support many significant portions of corporate IT functions. But Stewart's sentiments are newly rekindled and expanded these days through the mounting expectations that cloud computing and software-as-a-service (SaaS) will hasten the death of on-premises enterprise IT.

Some of the analyst reports these days indicate that hundreds of billions of dollars in IT spending will soon pass through the doors of corporate IT and into the arms of various cloud-service providers. We might conclude that IT is indeed about to expire. Not all of us, however, subscribe to this extent in the pace of the demise of on-premises systems, their ongoing upkeep, maintenance, and support.

To help us better understand the actual future role of IT on the actual floors inside of actual companies, we're joined by our guests and analysts this week. First, Jim Kobielus, senior analyst at Forrester Research. Hey Jim.

Jim Kobielus: Hey, Dana. Hey, everybody.

Gardner: Tony Baer, senior analyst at Ovum.

Tony Baer: Hey, Dana. How are you doing?

Gardner: Brad Shimmin, principal analyst at Current Analysis.

Brad Shimmin: Hi, Dana.

Gardner: Ron Schmelzer, senior analyst, ZapThink.

Ron Schmelzer: Hi, guys, just unplugging my mainframe, as we speak.

Gardner: And, for the first time on our show, Sandy Rogers, former program director at IDC, and now independent IT analyst and consultant. Welcome, Sandy.

Sandy Rogers: Thanks, Dana. Great to be here.

Gardner: And, as our guest this week, welcome Alex Neihaus, vice president of marketing at Active Endpoints. Hey, Alex.

Alex Neihaus: Hi, Dana. Hi, everyone.

Gardner: Well, let's start with you, Jim Kobielus, if you don't mind.

Kobielus: I don't mind.

Gardner: We've heard this before, the same story, new trend, new paradigm shift, money to be saved, pull out the plug, you'll get it off the wire, or you'll get it from much lower cost approaches to IT.

I do believe that cloud computing is going to have a pretty significant impact and we've discussed that quite a bit on our show so far. What's your take? Do we have a sense of the mix? Is there any way to predict what's going to happen in, say, five years?

Death notice premature

Kobielus: There are plenty of ways to predict what's going to happen in five years. I need to buy a dartboard. That's one of the ways. I can predict right now, based on my conversations with Forrester customers, and specifically my career in data warehousing and business intelligence (BI). This notion of the death of IT is way too premature, along the lines of the famous Mark Twain quote.

If you look at a vast majority of enterprise data warehousing in BI environment, there is a bit of a movement toward outsourcing of the date warehouse into the cloud. There is a bit of a movement toward moving more of the report and dashboard and analytic application development to the end user or to the power user or subject matter expert and away from the priesthood of mathematicians, statisticians, professional data modelers, and data-mining specialists that many large companies have.

There is a bit of a movement in both directions. But it's only movement. In other words, there aren't a substantial number of enterprises that have outsourced their data warehouse or their marts. Probably there aren't that many commercial options yet that are fit to do so. Only a handful of data warehousing vendors offer a hosted solution, a SaaS, or cloud solution. I've been telling people that 2009 is not the year of the cloud in data warehousing, nor is 2010. I think 2011 will see a substantial number of data warehouses deployed into the cloud.

Gardner: Well, Jim, will that be taking them off of the corporate network and putting them in the cloud or will they just be new ones on the cloud?

Kobielus: The component of your data-warehousing environment that will be outsourced to public cloud, initially, in many cases, will not be your whole data warehouse. Rather it will be a staging layer, where you're staging a lot of data that's structured and unstructured and that you're pulling from both internal systems, blogs, RSS feeds, and the whole social networking world -- clickstream data and the like.

They will be brought into cloud storage services that will operate as a staging layer

First is security. You need strong control and you need to also be able to monitor it 24/7, because it's the most fundamental thing that you run your business on.

where transforms, cleansing, match and merge, and all those functions will be performed on massive amounts of data. We're talking about petabytes where it makes more sense, from a dollars-and-cents standpoint to use a subscription service in a multi-tenant environment.

Gardner: We're still going to see data growing on-premises as well.

Kobielus: Yeah, we're definitely going to see data growing a lot on-premises. The core data-warehousing hub where your master data is stored -- for most companies of most sizes -- will remain on-premises for lots of reasons. First is security. You need strong control and you need to also be able to monitor it 24/7, because it's the most fundamental thing that you run your business on.

There are lots of reasons why the centerpiece of your data-warehousing environment, the master tables, were made on-premises. For the foreseeable future, I sense strong reluctance from corporate IT to outsource that. As to the whole front-end mash-up side of these all sort of developments, I'm doing a report that will be published in about a month on the uptake of that approach. But, that's several years down the road, before we see that come to fruition. So, I don't think IT is dying anytime soon.

Gardner: Tony Baer, what about applications?

Cloud is transformational

Baer: Well, I just completed actually a similar study in application lifecycle management (ALM), and I did find that that cloud certainly is transforming the market. It's still at the very early stages, but it's not going to be basically a one single, monolithic, silver-bullet approach. And, not all pieces in the app lifecycle are as well suited for the cloud as others.

I found that two areas really stuck out. One is anything collaborative in nature, where you need to communicate -- especially as development teams go more global and more distributed, and of course, as the pace of business changes the business climate and accelerates -- it's more important than ever to get everybody on the same page, almost literally. So, what I found was that planning, budgeting, asset management, project portfolio management, and all those collaborative functions did very well.

At the other end of the scale, another side that did very well was something that I think Jim was sort of hinting at, which is anything that had very dynamic resource needs,

When you're developing code, you don't want to have to deal with any type of network latencies that are going to come up when you deal with cloud.

where today you need a lot of resource, tomorrow you don't. A good example of that is testing -- if you are a corporate IT department, which has periodic releases, where you have peaks and valleys in terms of when you need to test and do regression test.

Gardner: Platform as a service (PaaS)?

Baer: Yeah. What I found though that did not map well to the cloud was anything that related to source code. There were a number of reasons for that. One is, basically, that developers like to have the stuff on their own local machines.

There is a degree of control that you like, but there are some tactical reasons. When you're developing code, you don't want to have to deal with any type of network latencies that are going to come up when you deal with cloud. No matter how good the bandwidth, there are always going to be times when there are going to be some speed bumps.

But, the other part was also related to IP, which is source code before it's compiled in the binaries. It's basically pretty naked and it's pretty ripe for stealing. This is your intellectual property. Today, if you're doing development, it's because there aren't packages that are available to supply a generic need. It's something that's a process that's unique to your organization.

So, I got a lot of reluctance out there to do anything regarding coding in the cloud. There is the Bespin project on Mozilla, but that's the exception to the rule. So, in terms of IT being dead, well, at least with regard to cloud and on-premise, that's hardly the case in ALM.

Gardner: Brad Shimmin, why do we see these reports, some of them coming out of Wall Street? They're supposed to be smart money saying $120 billion of IT is going to be in the cloud in the matter of two or three years. Is it that they don't understand what cloud is, or are they dead wrong?

Shimmin: I don't think they're dead wrong. As Tony was saying, it depends on what you're putting in the cloud. Because I follow the collaboration area, I see that happening much, much more quickly, and, frankly, much sooner than even the discussion we've been having recently about cloud computing.

Way back in the late 90s, and early "0-dots," Microsoft and IBM were making big money out of their managed hosting services for Exchange and Notes, and they are pushing that downstream a little bit more now to get to the channel and the long tail.

Gardner: So there is not a lot of intellectual property in a messaging transfer agent?

Bothersome IT functions

Shimmin: That's just it. Those are the functions that IT would love to get rid of. It's like a diseased appendix. I would just love to get rid of having to manage Exchange servers. Any of us who have touched any of those beasts can attest to that.

So, even though I'm a recovering cynic and I kind of bounce between "the cloud is just all hype" and "yes, the cloud is going to be our savior," for some things like collaboration, where it already had a lot of acceptance, it's going to drive a lot of costs. If that's what Wall Street is talking about, then, yeah, I think they're pretty much accurate.

Gardner: Ron Schmelzer, we certainly heard a lot about cost reduction. It's certainly top of mind in a recession. I also think that cloud computing can offer some significant cost savings, but to what degree are we talking about disrupting the status quo in most IT departments?

Schmelzer: It's really interesting. If you look at when most of the major IT shifts happen, it's almost always during period of economic recession. The last time was in 2000-2001, when we first started really talking about service-oriented architecture (SOA). In the mid- '90s was when we really started pushing out the Web. In the early part of the '80s, when recession was kind of bad, that's when personal computers started coming about.

You kind of go back into this package every time. Companies are like, "I hate the systems I have. I'm trying to deal with inefficiency. There must be something wrong we're doing. Let's find some other way to do it." Then, we go ahead and find some new way to do it. Of course, it doesn't really solve all of our problems. We spend the next couple of years trying to make it work, and then we find something new.

The cost-saving benefit of cloud is clearly there. That's part of the reason there is so much attention on it. People don't want to be investing their own money in their own infrastructure. They want to be leveraging economies of scale, and one of the great things that clouds do is provide that economy of scale.

From my perspective on the whole question of IT, the investments, and what's going to happen with corporate enterprise IT, I think we're going to see much bigger changes on the organizational side than the technological side. It’s hard for companies to get rid of stuff they have invested billions of dollars in.

Gardner: Wait a minute. So, this is like a neutron bomb. The people die, but the machines keep running?

Schmelzer: Actually vice versa. The machines might change and the machines might move, but IT organizations will become a lot smaller. I don't really believe in 4,000-person IT organization, whose primary job is to keep the machines running. That's very industrial revolution, isn't it?

Gardner: Sandy Rogers, the theory is good, the vision is good, but so was the theory in 1995 that you'd pull out the last mainframe in a year. What's your perspective, given that you've been tracking enterprise infrastructure software for quite some time?

The cost of change

Rogers: Well, it's interesting. Many organizations have avoided legacy modernization projects due to the cost of change. It's not just about the technology replacement. It's a loss of capabilities. It's the change in human workflow and knowledge base. All that is a critical consideration. I see enterprises all the time that are caught between a rock and a hard place, where they have specialized technologies that were built out in the client-server era. They haven't been able to find any replacements.

So the idea of software-as-a-service (SaaS), that one-to-many model, means the kinds of replacements that are available will be very generic in nature, for the most part. There will be some niche capabilities, moving way out in the time horizon. But, the ability to take a legacy system that may be very specialized, far reaching, have a lot of integrations and dependencies with two other systems is a very difficult change. A company has to get to a very specific point within their business to take on that level of risk from change.

Gardner: It's one thing to change from a legacy system to a more modern standard-based hardware and operating system platform environment and to frameworks

It's not to say it won't be done, but it certainly has a big learning curve that the whole industry will be engaging in.

for development. That's not quite the same, though, as making a transition to cloud. Do you think they go hand-in-hand?

Rogers: One thing to think about is there are so many different layers of the stack that we're talking about. When we're talking about cloud and SaaS, it's going to impact different layers. So, there may be some changes in the types of deployments that go on, the target locations.

It reminds me of the film, Pretty Woman. That's "just geography," and that's the way I envision the first wave moving out. We may want to think about leveraging other systems and infrastructure, more of the server, more of the data center layer, but there is going to be a huge number of implications as you move up the stack, especially in the middle-ware and integration space, and pick and choose different applications and their capabilities.

There are a lot of systems out there that are not designed to be run in this kind of capacity. We're still at the very beginning stages of leveraging services and SOA, when you look at the mass market. What I've been discovering in speaking with enterprises that are either doing SaaS as a business or as an enterprise is that the first thing they're thinking about is that the architecture has to able to support this kind of dynamic access and the ability to scale.

So, there's a lot of work that needs to be done to just think about turning something off, turning something on, and thinking that you are going to be able to rely on it the same way that you've relied on the systems that have been developed internally. It's not to say it won't be done, but it certainly has a big learning curve that the whole industry will be engaging in.

Gardner: Not about just pulling a plug at all.

Rogers: Yeah.

Gardner: Alex Neihaus, you're someone who's actually in the software business -- unlike the rest of us. And, by the way, thanks very much for sponsoring the show. We really appreciate it.

Neihaus: Our pleasure,

Gardner: Tell me a little bit about your perspective as someone who is delivering software, productivity, and value to enterprises. Why not go up on someone else's cloud and deliver this strictly as a service?

Borg-like question

Neihaus: We think that this is a Borg-like question -- who assimilates whom? Ron was exactly correct that cloud and the associated technologies that we describe today is today's shining new toy. What we find more interesting is not the question of whether the cloud will subsume IT or IT will subsume the cloud, but who should be creating applications?

And, there is a meta question, or an even larger question, today of whether or not end users can use these technologies to completely go around IT and create their own applications themselves? For us, that seems to be the ultimate disingenuousness, the ultimate inability for all the reasons that everyone discussed. I mean, no one wants to manage an Exchange server, and I was glad to hear Brad include Notes Server in that list, but, in fact, IT is still doing it.

So for us, the question really is whether the combination of these technologies can be made to foster a new level of collaboration in enterprises where, frankly, IT isn't going to go away. The most rapid adoption of these technologies, we think, is in improving the way IT responds in new ways, and in more clever ways, with lot more end-user input, into creating and deploying applications.

You hear a lot of people talk about the generational shift in business people. I agree that there is a lot more familiarity with IT among business end users, but we don't

For us, the cosmic question is whether we are really at the point where end users can take elements that exist in the cloud and their own data centers and create processes and applications that run their business themselves.

hear from our customers that business end users even want to be in the business of creating or manipulating applications in IT, in the cloud, or anywhere else.

Gardner: What I hear you saying is that you see the IT department as your customer, but also, at some level, the end user is your customer. You need to make them both happy, but can you make that end user happy without the IT department?

Neihaus: Our answer is no, simply because of some of the things that Sandy was talking about. There are legacy systems -- there are plenty of things lying about, would be the right way to put it -- that need to be integrated, using technologies that are modern and appropriate.

For us, the cosmic question is whether we are really at the point where end users can take elements that exist in the cloud and their own data centers and create processes and applications that run their business themselves. And our response is that that's probably not the case, and it's probably not going to be the case anytime soon. If, in fact, it were the case, it would still be the wrong thing to do in enterprises, because I am not sure many CEOs want their business end users being IT.

Gardner: Now, your product is something that's designed to make crafting and managing business processes easier and more visual. You're trying to elevate this from a code-based or tool-based process to more of a visual, something that an analyst level person could do, but not necessarily a line-of-business person. So, you've already tested the waters here and your conclusion is that IT can't go away.

Model-based environment

Neihaus: Correct. We're a model-based execution environment, and you're exactly right that we try and expose those processes to the business. But, there are what I call "pretty pictures" kinds of approaches to this, and they can exist in the cloud and they can exist in IT. But, for most people, those are customizations of existing applications.

You might go buy a call-center application and allow end users to modify the workflow. But, once you get beyond the pure human workflow, and you begin to integrate the kinds of systems that Sandy was talking about, and I think Ron was talking about, you're beyond the skill, desire, or capability of an end user.

Now, can these things be composed from elements that exist in the cloud? They could be and they probably should be. But, whether the cloud represents something that can enable business users to eliminate IT is a huge stretch for us, based on what we experience in the marketplace.

Gardner: We haven't really explored that dimension where the cloud fits. Does the cloud get between the end user and IT, or is the cloud behind IT and IT gets between the cloud and the corporate user and perhaps even their customers out in the public domain?

Brad Shimmin, recently we saw some inkling about Google Wave. What that's going to represent? I found the demo and the implications very interesting.

We've all been end users at some point and still are in many ways, for what we do day-in and day-out. I think all of us here will attest to the fact that we can be incredibly stupid.

Google seems to think that they can go directly to the end user, at least for some elements of collaboration for bringing different assets together in a common view -- maybe some check-in, check-out benefits, using a spectrum of different communication modalities and synchronicities.

What's your take? Is Alex right that we're not going to get too much out to the end user directly, that IT is going to be part of that? Or, are we perhaps being a little bit too cautious about what end users are capable of?

Shimmin: We've all been end users at some point and still are in many ways, for what we do day-in and day-out. I think all of us here will attest to the fact that we can be incredibly stupid. Yesterday, when I was sitting on Microsoft's Virtual Analyst Summit, I heard them say that what they'd like to accomplish is for users to be able to open up an Excel spreadsheet and create a BI report that would normally take IT two weeks to do.

I thought, "Hey, that's terrific, but, oh dear Lord, you don't want anyone to do that, because they're going to use the wrong datasets, they're going to perhaps have the wrong transitions and transformations for data."

It's not as simple as the picture is being painted. With Goggle Wave, as we've said before, when they are talking about certain types of collaborative applications, that sort of mashability -- as Jim put it earlier -- is something users are capable of and comfortable with. It's within the bounds of something they know how to manage, and they know that what they get out of the application is right.

When I hear about customers being able to mash-up their own BI reports, for example, I think, "How would they know? How on earth would they know that what they've gotten out of it is correct?"

Gardner: And, would the security and regulatory compliance issues be maintained?

Loss of control

Shimmin: Sure, that's the other horn on the bull. The more you move into the cloud, the less control you have over the data. The vendors that I talk to realize that fact, but they still haven't come to a point in which you can control which data resides where and what happens to that data. This is even in the collaboration space, mind you, which is I've said is really getting out there ahead of a lot other ventures,

A lot of companies that say they are pure SaaS are really still using shared data resources on the back-end, which is not a good thing, if you really need to lock down that data.

Gardner: It's not really cloud. Is it?

Shimmin: No.

Gardner: Jim Kobielus, I'm sorry I cut you off earlier, but I wanted to get across the spectrum of our analysts, before we dug down too deeply. But, now is the time to dig deeply to this point that end users, even sophisticated power users in a corporate environment, are probably not going to be in a position of doing SQL queries or even queries that have been visually abstracted for them. We need a sort of intermediary group or capability between the consumers of data and the actual production of data. Isn't that right?

Kobielus: The intermediary group is the governance group. Alex, Brad, Sandy, and the others are talking about how, as you allow the end users or encourage them or require them to mash up the hone applications in their own data, in their own presentation layer, that becomes chaos unless you have strong governance.

As Brad said, when users are given a sandbox of their own, they should know that the whole sandbox, in fact, was built and is being monitored by IT, so that you're taking the right data, doing the right transforms, and applying the right presentation components, the right data model, the right calculation, as defined by your company, its policies, and its rules. You need strong governance to keep this massive cloud sandbox from just becoming absolute chaos.

So, it's the IT group, of course, doing what they do best, or what they prefer to be doing, which is architecture, planning, best practices, templates, governance control, oversight support, and the whole nine yards to make sure that, as you deal in new platforms for process and data, such as the cloud, those platforms are wrapped with strong governance.

Gardner: Tony Baer, perhaps what we are seeing is not the demise of IT, but the transformation and elevation in the role and importance of IT.

The other part is technical. If you're going to provide them the capabilities to mash up things, which is certainly valuable, you want to do this in a protected sandbox

Instead of doing support, maintenance, patches, and keeping the red lights out and the green lights on, they're going to be involved with the governance, provisioning, security, and more innovation in terms of getting closer to the productivity benefit than simply keeping the cycles going and the hard-drive spinning.

Baer: There's no question about that. It reminds me of some of the notion that to make things simple underneath the plumbing is very complex, so make things simple on top. As Jim is saying, you can't provide users the ability to mash-up assets and start creating reports without putting some sort of boundary around it.

This is process-related, which is basically instituting strong governance and having policies that say, "Okay, you can use these types of assets or data under these scenarios, and these roles can access this and share this."

The other part is technical. If you're going to provide them the capabilities to mash up things, which is certainly valuable, you want to do this in a protected sandbox. That's where I see technical innovations that could go to cloud, which would be like enterprise mash-up hubs -- probably a good example -- or like a report center.

I could use those Excel spreadsheets to generate those reports, but they're coming from a protected set of data for which there are very stringent access controls and governance. So, it's a combination of both process and technology.

The same cloud?

Gardner: Ron Schmelzer, I'm a neat person. I like things that follow in nice little neat packages that line up, and are not crooked. What I am starting to see now in this cloud evolution is one part of a cloud being something that end users would use, inside of companies or consumers at home through their mobile devices.

I'm also seeing the cloud providing these back-end infrastructure services, automation and lower cost, and building blocks for IT. And, IT has a value-added role on top of that. But, is it the same cloud? Is it a different cloud, and how would we manage this border between, "I want to use the cloud as an end user" and "I want to use the service from the cloud through the IT department control."

Schmelzer: It sounds like you have a future in interior decoration to put things in neat boxes, but that's why we call it a cloud, right? The reason we call these things cloud is because they're kind of amorphous. They don't have well-defined boundaries.

The whole reason for the metaphor "cloud" is that in network diagrams you want to show something outside the boundaries of the IT organization, but you don't know exactly how it's configured. You just represent it visually as a cloud, right? So, that's the conceptual model we are computing here, where you don't necessarily have all the details of the implementation.

Now, the question is: is the cloud boundary at the firewall or is the cloud boundary necessarily outside of the organization? Not necessarily. There maybe internal processes in IT or the IT organization that are leveraging aspects and elements that you don't have complete control over, in which case they are very cloud-like. They have all the same features and benefits of the cloud.

What we have to be aware is that there are a lot of different things that are wrapped up in the cloud. There's SaaS and application service provider stuff that we've been doing since late '90s. There's utility computing, grid computing, elastic computing, compute on demand, and all this sort of stuff.

The question is what benefits do we want? That's what differentiates cloud.

There's an increasing need to compose and integrate silos within organizations. That has a huge implication on governance activities.

It really is a third-party provider that we're paying for on a transactional model and leveraging infrastructure we have no visibility over, rather than a model that we have ownership of. We have cost visibility, but we have elastic consumption capability. So, we're using more of the implementations of the cloud.

Gardner: Sandy Rogers, you've been tracking governance capabilities, and is it the role of IT to further govern this amorphous boundary between what a cloud, off-the-wire set of services might bring to an organization in addition to governing the IT that goes on inside of their SOA activity. Is IT going to rise up to this or you are going to say, hey, that's outside of our purview and we are not interested.

Rogers: It's certainly within the purview of both IT and business, as partners, to address governance, whether it's internal to an organization or it's leveraging facilities that are external or outside the firewall. IT is still responsible for ensuring that whatever systems are used, how and where the technologies and being used, they accomplish the business goals.

It's off-loaded for support overall. They're going to have to be responsible to ensure that it fits in line with their governance policies in their meeting to set goal. I think the availability and maturity of technologies will evolve, and it will evolve in different spaces to be one-for-one able to be replaced.

The sophistication of the solution interfaces and the management in the administrative capabilities to enable governance, are very nascent in the cloud offerings. That's an opportunity for vendors to approach this. There's an increasing need to compose and integrate silos within organizations. That has a huge implication on governance activities.

Gardner: And, that doesn't even include these outside silos.

Step back and do the basics

Rogers: Yes. It's just being exaggerated with these cloud-based environments. What I've seen in looking at SOA governance is that for those companies that don't have good governance policies, programs, and procedures to start with really are in a situation where they have to step back and do the basics. Every time you end up with some type of distributed, federated environment, you have to look at all of those issues that relate to governance, whether it's compliance, security, management, or anything like that.

SOA, or any distributed environment, exaggerates this. Cloud will exaggerate it even further. Managing contracts and legal arrangements will be a growing emphasis within IT. What's interesting in the cloud space is that we're seeing a lot of packaged services, where one company may be engaging with a service provider, and that service provider is dependent on another service provider for, say, providing some compute infrastructure services.

Gardner: An ecology approach to this.

Rogers: Yes, having the visibility, having access to the right information to perform governance is going to be an area that needs to be worked on. It will have to be worked on sooner, rather than later, to win over those C-level executives who are very nervous about relinquishing control.

Gardner: Another area that I'd like to get into, before we run out of time, is the ability for the vendors, the software providers, to make a decent living. If they're only going to deliver what they do through a cloud model and they have a subscription they are going to charge per user per month, or some similar model, can they, in fact, cover their cost and make a profit?

JP Morgenthal, who has been on our show, has been critical and says that even open source is a threat, because of the same issue. The innovative, quality software won't get developed in the future, if the models don't support it. I'll take that to Alex as a software developer and provider of value. Is there a case here that the subscription model undercuts the viability of your business?

Neihaus: I don't think so, and I'll tell you why. Like any other vendor of any product in any marketplace, we'll sell our services or our products the way customers want to buy them.

The software market is very big. The market we exist in, the business-process management system marketplace, is very big. Companies like ours and others will adapt to what customers ask for

As of yet, at least in our case, we've had no substantive demand for subscription, which is closely associated with the open-source model. It turned out to be fairly expensive over a longer period of time, or per user per month hosted Exchange or Notes mailbox pricing. -- at least for the category in which we exist.

The software market is very big. The market we exist in, the business-process management system marketplace, is very big. Companies like ours and others will adapt to what customers ask for. We can be more nimble than some of the bigger players in this marketplace to responding to that, and that's the key point.

The very large, leviathan players in the space have the most to lose from any kind of change in pricing or distribution business models. So, there's a huge lethargy in the marketplace towards changing buying behavior.

Even if we wanted to promulgate and distribute a new business model, customers are so used to buying the way they have been buying from companies for such a long time that their internal processes from decision-making to contracting are wrapped around those models. It's something we would adapt to, but I think the market is going to change relatively slowly.

Gardner: Brad Shimmin, to Alex's point that the big players, the leviathans, have the most to lose from the wholesale move to cloud, that's in semi-agreement with this concept that moving to a services provisioning subscription model has its risks compared to a license on-premises, per processor type of model. Where do you come down on that?

Vendors will adapt

Shimmin: Well, I stand firmly on the side of broader ecosystems and the power to the people. So, my feeling is that the vendors will adapt to this, just as Alex was saying, but they're doing it slowly. When I look at Microsoft, Cisco, and IBM, for example, I see three very different approaches to that.

With Microsoft, they were pretty quick to roll out their Microsoft online services and firmly undercut the pricing that their partners could give their customers on hosted Exchange, for example. But, they set it up so that those partners could then build value-add on top of it to increase their revenues. As we've been talking about here, when it comes down to just a numbers game, it's hard to make money on just a pure services contract -- unless you have a huge scale to work with.

When Microsoft rolled out Azure -- last October, I think it was -- the plan was to allow their ecosystem, their channel partners, to build applications for vertical markets. These are the things they are good at and the things that Microsoft is not good, and they can make money on those by building into the cloud.

It's these channel partners that are going to benefit the most from these standardized interfaces and the mashability component that's built into these cloud services. It's not the end users who are going to be putting things together. It's the channel partners who are going to be assembling value that they can then deliver to customers.

Gardner: Tony Baer, it seems to me that the open source rollouts of the past 10 years may be harbingers of things to come into cloud.

A lot of customers have said, "Look, just handle the infrastructure for an extra fee, and we'll to continue to pay our perpetual license."

If a large vendor wants things to go slowly, they could perhaps time things. At the same time, they might offer certain elements of their services as a service for free in order to undercut competitors and/or to entice the use of a larger solution, rather than an application or feature set. Do you expect they will see that?

Baer: To a certain extent, where you will see it is in the commodity areas. Microsoft is obviously the poster child there, because they have the most to gain and the most to lose. Actually, it's more that they have the most to lose, not so much to gain. They are really in a defensive position there.

But, when you look at enterprise software or more specialized software, I don't think that's really the case. One of the notes I was jotting down here was that I thought this may actually be very particular to my market, to the software tools market, and that it may march to a different drummer, compared to customer relationship management (CRM) or Exchange.

IBM is struggling with the pricing for how it's going to price its cloud. Hewlett-Packard's (HP's) experience so far, at least from the Mercury side which has offered testing services going back a long ways, is that in many cases, the pricing is not on the subscription model. A lot of customers have said, "Look, just handle the infrastructure for an extra fee, and we'll to continue to pay our perpetual license."

The move to the cloud and subscription pricing are two different things. One does not necessarily follow the other. That's a finding that actually surprised me.

Gardner: Ron Schmelzer, Tony Baer made a point that you could be a victim of cloud, before you could be a beneficiary of it, if you are a provider and a vendor. That's a tough transition to go through.

All transitions are similar

Schmelzer: Maybe, and I think all these transitions are like that. If you look at what happened to the Web. I was on the CRM side of things back in the mid '90s, and we thought that the Web was going to kill client-server CRM applications, and, to a certain extent, it kind of did. It just took a lot longer than we thought. I remember Siebel's dominance and they're saying, "We are not going to move to the Web."

Obviously, Salesforce put the impetus behind it, but even before Salesforce was out there in the late '90s, we were asking, "Why are we using this in-house enterprise application software system with all this great Web stuff happening over there? Why can't we put this stuff online?" The same thing is going here.

We talked about this a couple of podcasts ago, this IT divide between the IT experience at work and the IT experience at home. The home IT experience is just so much richer than what we've got at work. So, it's the same question. Why are we still using these systems in the enterprise and we have all this cloud-based mash-up stuff when we go home?

The writing is on the wall. The smart vendors will learn how to transition themselves in a way that doesn't cannibalize their existing business model. The stupid ones will be pushed to the model anyways, They can't resist it, and they will, of course, suffer.

Gardner: I think this has been a very good and interesting discussion. I'd like to go around the table before we close out, because I haven't heard too much about the death of IT in these permutations of the subject that we've gone through here.

Jim Kobielus, first to you. On a scale of 1 to 10, with 1 being IT dead and 10 being

Much of the actual guts of IT within an organization will migrate to hosted environments, and much of the development will be done by end users and power users.

IT alive, robust, and growing vibrantly, where do you think we're going to see the IT department's role in say three years?

Kobielus: Okay, in three years. I'll be really wishy-washy and give it a 5. It's almost like Schrodinger's cat. You know it's in the box, but you don't know if it's dead or alive yet. It depends on how the quark falls. But, I think that in three years time, IT will be alive, kicking, robust, and migrating toward more of a pure planning, architecture, and best practices function.

Much of the actual guts of IT within an organization will migrate to hosted environments, and much of the development will be done by end users and power users. I think that's writing on the wall.

Gardner: So, the role and impact of IT will be about the same in three years?

Kobielus: Yeah.

Gardner: Tony Baer, how do you come down -- 1 to 10?

Baer: I was really confused about Jim's answer, because I thought he said at one point that IT's role is going to change as we go to hosted services.

Gardner: We may change his mind on the show.

Doing the cool stuff

Kobielus: Actually, 20 years ago I worked as a contractor for a government agency that outsourced a vast majority of their IT to contractors. I remember that the folks who remained as the government's employees running the shop were all procurement, planning, architecture, and all the high-level, cool stuff. They didn't get their fingernails dirty.

Baer: I don't subscribe to the death of IT, because I remember 20 years ago hearing about the death of IT, when Yankee Group did the announcement of that Kodak did a big outsourcing contract, because they decided that, as a company, they were not really in the business of IT. They were in business of photography. A few years later, they realized that the business of photography really did involve IT, and they very quietly backtracked on those contracts.

Gardner: JP Morgan Chase did the same thing about five or six years ago, right?

Baer: Exactly. As Sandy was saying before, there is a lot of complexity, even if you outsource. Outsource means that you need more management. Even if you use the cloud, that requires more governance.

So, I don't see IT's role diminishing. There may be a lower headcount, but that can just as much be attributed to a new technology that provides certain capabilities to end users and also using some external services. But, that's independent of whether there's a role for IT, and I think it pretty much still has a role.

Gardner: If you have 1 to 10, give me a number.

Baer: And 10 being that it does have a role?

Gardner: Vibrant, alive, thriving, and growing like crazy.

Baer: I am going to give it an 8.

Gardner: Excellent. Brad Shimmin?

Shimmin: I'm giving it a 7 for similar reasons, I think that it's going to scale back in size little bit, but it's not going to diminish in value.

IT is not going to go away. I don't think IT is going to be suffering. IT is just a continuously changing thing.

Back to what Sandy was saying, I think it's going to be very much alive, but the value is going to be more of a managerial role working with partners. Also, the role is changing to be more of business analysts, if you will, working with their end users too. Those end users are both customers and developers, in some ways, rather than these guys just running around, rebooting Exchange servers to keep the green lights blinking.

Gardner: So, more architects, fewer admins.

Shimmin: Yup.

Gardner: Ron Schmelzer?

Schmelzer: I'm going to be your lemming here. I think it's 10. IT is not going to go away. I don't think IT is going to be suffering. IT is just a continuously changing thing. Look, IT is only 60 years old. The whole life of the entire IT-as-an-organization department within the enterprise is only 60 years.

So, IT is going to be thriving in three years. It's going to be completely different than anything we may know today or maybe it'll be mostly similar. But, I guarantee that whatever it looks like, it will be still as important as an IT organization.

Now, of course, my information tells me that the world is coming to an end at three years, my Mayan Calendar. That was a good choice on time horizon, because if you had said four years, that would mean the world is not going to exist in four years. So what kind of trick question is that?

Gardner: Well, that's why I bring it down. Sandy Rogers -- 1 to 10?

Some IT is in deep trouble

Rogers: Probably in the 7 to 8 range. I agree with everything that's been said here. I think it's up to the individual enterprises. In some enterprises, IT is in deep trouble if they do not embrace new technologies and new opportunities and become an adviser to the business. So it comes down to the transition of IT in understanding all the tools and capabilities that they have at their disposal to get accomplished what they need to.

Some enterprises will be in rough shape. The biggest changeover is the vendor community. They are in the midst of changing over from being technology purveyors to solution and service purveyors. That's where the big shift is going to happen in three years.

Gardner: Alex Neihaus, how about your choice here? 1 to 10?

Neihaus: Our self-interest is in a thriving a segment of IT, because that's who we serve. So, I rate it as a 10 for all of the reasons that the much-more-distinguished-than-I panel has articulated. I wish to say one thing, though. The role of IT is always changing and impacted by the technologies around it, but I don't think that that could be used as an argument that it's going to diminish its importance or its capabilities really inside organizations.

Gardner: Well, I'll go last and I'll of course cheat, because I'm going to break it into two questions. I think their importance will be as high or higher, so 8 to 10, but their budget, the percent of spend that they're able to derive from the total revenues of the organization, will be going down. The pressure will be on, and it will be going down.

So, from a price and monetary budgeting perspective, the role of IT will probably be down around 4. That's my take.

Thanks very much for all of your input. I also want to thank the sponsors for the BriefingsDirect Analyst Insights podcast series, Active Endpoints and TIBCO Software.

And I also want to thank our guests this week. Jim Kobielus, senior analyst at Forrester Research. Thanks Jim.

Kobielus: Always a pleasure.

Gardner: Tony Baer, senior analyst at Ovum.

Baer: Great discussion as usual.

Gardner: Brad Shimmin, principal analyst at Current Analysis.

Shimmin: Thank you, Dana. It was great today.

Gardner: Ron Schmelzer? What's your name again? Brawn? No, Ron Schmelzer, senior analyst at ZapThink.

Schmelzer: Glad to be here, and I think my mainframe is taking about three years to turn off. I'll let you know in three years.

Gardner: Thank you also Sandy Rogers, now an independent IT analyst and consultant.

Rogers: It was great to participate and be here.

Gardner: And also a special thanks to Alex Neihaus, vice president of marketing at Active Endpoints.

Neihaus: It was a thrill to join you guys today.

Gardner: Thanks for listening to BriefingsDirect. Come back next time.

Download the transcript. Read the summary blog post. Charter Sponsor: Active Endpoints. Also sponsored by TIBCO Software.

Special offer: Download a free, supported 30-day trial of Active Endpoint's ActiveVOS at www.activevos.com/insight.

Edited transcript of BriefingsDirect Analyst Insights Edition podcast, Vol. 42 on on the health of corporate IT and whether reports of its demise are premature. Copyright Interarbor Solutions, LLC, 2005-2009. All rights reserved.

Wednesday, June 11, 2008

Live TIBCO Panel Examines Role and Impact of Service Performance Management in Enterprise SOA Deployments

Transcript of BriefingsDirect podcast on service performance management recorded live at TUCON 2008 in San Francisco on April 30, 2008.

Listen to the podcast here. Sponsor: TIBCO Software.

Dana Gardner: Hi, this is Dana Gardner, principal analyst at Interarbor Solutions, and you’re listening to BriefingsDirect. Today, a sponsored podcast discussion about service performance management in support of service-oriented architecture (SOA).

We are here live at the TUCON 2008 conference, TIBCO Software’s user event in San Francisco, to look into the issues around SOA integrity, particularly in the context of widespread enterprise use, the myriad demands that are going to be put on services, and how infrastructure is going to need to adapt and perform in a way that probably has not been the case for infrastructure up until now.

Helping us to weed through service performance management and how it relates to SOA governance and other issues of total architecture, we are joined by a panel of industry analysts, experts and representatives from TIBCO.

Let's start by introducing our panel. We are joined by Joe McKendrick, an independent analyst and SOA blogger. Welcome to the show, Joe.

Joe McKendrick: Hi, Dana, happy to be here.

Gardner: We are also joined by Sandy Rogers, the program director for SOA, Web services and integration at IDC. Welcome, Sandy.

Sandy Rogers: Thanks, Dana.

Gardner: We are also joined by Anthony Abbattista, the vice president of enterprise technology strategy and planning for Allstate Insurance Co. Welcome to the show, Anthony.

Anthony Abbattista: It’s good to be here, Thanks.

Gardner: And also joining us, Rourke McNamara, director of product marketing for TIBCO Software. Welcome Rourke.

Rourke McNamara: Thank you, Dana.

Gardner: We saw and listened to some presentations this morning at the TIBCO conference. One of the things that struck me is this notion of pulling together what had been not only in disparate technology silos -- but really joining what had been in functional and organizational silos. Particularly, I mean the design and process creation phases that we have heard so much about with SOA.

And then how that relates to the secondary aspect of functional activities, which is the operations -- keeping the trains running on time, and making sure that service-level agreements (SLAs) are met. That means making sure that users get very fine-grained services coming through uninterrupted in aggregated applications -- without hiccups, without slowdowns.

These processes are moving to mission-critical, and so there needs to be more opportunity for these two aspects of SOA, design and operations, to work together. Performance management of services gives more insight into what takes place beneath those services, and is, therefore, becoming essential.

First, let's take a look at this landscape of what's going on in SOA, and why, as we move toward enterprise-wide deployment, service performance management becomes so important.

Sandy at IDC, what do you see in terms of enterprises that are early adopters of SOA? How concerned are they that, when they throw the switch, so to speak, with these composite business processes -- made up of services from a variety of different sources with a variety of different support infrastructure -- that they really feel confident that this is going to hold up in real world situations?

Rogers: What I find is interesting is that even if you have one service that you have deployed, you need to have as much information as possible around how it is being used and how the trending is happening regarding the up-tick in the consumption of the service across different applications, across different processes.

So, most organizations need to present an environment where individuals and stakeholders in the company feel more comfortable in relying on services and also allowing others to potentially handle the operational dynamics of those services, once they are in production.

They need a lot more visibility and an understanding of the strains that are happening on the system, and they need to really build up a level of trust. Once they can add on to the amount of individuals that have that visibility, that trust starts to develop, more reuse starts to happen, and it starts to take off.

Eventually they get to a stage, where they are concerned about the scalability and how far they can push the limits of these deployments. It could be the way that they’ve designed it architecturally, or it could be just that they are getting familiar with the new technologies to support SOA infrastructure.

Gardner: It seems that at the very time when SOA is putting more emphasis on a diversified portfolio of services -- repurposing those services, extending visibility -- that, at the same time, IT as an organization is being tasked with behaving more maturely as a business within a business.

The Information Technology Infrastructure Library (ITIL) and other compliance standards are being placed on IT departments, so they behave more like we would expect a human resources department to behave. Let's go to Joe McKendrick.

Joe, just now that we are looking at the need for IT to perform like a mature business, is there a risk here of finger-pointing -- that when something goes wrong and so many constituents are involved with the support of a service, that no one will really be able to take responsibility?

McKendrick: Yes, that’s been a problem all along. There is always a lot of finger-pointing, and IT tends to get blamed for everything. Sandy made an excellent point that the foundation of SOA is trust.

The business units are being asked to sign on to a SOA to provide support, and perhaps even some of them to provide funding, and they are looking to the services that they will consume to be scalable, looking to the services to have uptime to be available, perhaps 24x7. And if this trust is not there, the SOA, the whole foundation of the SOA, breaks down and IT will get the blame again.

It very much hinges on IT and performance management. We're actually talking about two levels here, governance and performance management. They are integrated, and they need each other. But governance deals more with how the business addresses SOA. Performance management is an IT challenge and is rightly put into the IT "sphere of influence."

Gardner: I was struck, when I heard Anthony’s presentation this morning, by your example of what things used to be like, where you would get 40 people on a conference call when something went wrong, and you would be yelling out URLs in order to find the right server to either shut down or replace.

What's the issue from your perspective now on solving this issue when things go wrong? Is this something that we can rely on people to solve, or do we need to move more toward a systems-based approach?

Abbattista: First, I'd like to wind through an earlier question you asked. When we went to SOA, when we put in our enterprise service bus (ESB), and when we chose TIBCO for our bus, a lot of people thought of SOA as, "Well, I am just going to construct some WSDL and call some SOAP or HTTP, and that’s SOA."

But the first thing we did is talk through the governance part of why we want to "get on the toll road" and "pay a toll for the bus," and really that became the consistency in measurement and governance, and lets us operate the things once we have created them.

So the first thing we had to do was get through the whole idea of that. It was worth it, and it wasn’t a matter of whether the bus would work or not. For the first year-and-a-half that we put our ESB in and we started to market services on it, we would hear the words, "TIBCO is down."

It didn’t matter whether the back-end service is down. It didn’t matter whether the mainframe was broken, they would say "TIBCO is down." We finally started to get the root cause, saying, "No, so and so service is down." The basis for us having good measurement of performance is helping to "pay the toll," of getting on the bus and actually having measurement points that are well understood.

I also don’t agree exactly that governance is a business-unit thing. Governance for us is also a lot about the SLAs around the services, of having good expectations up front about how they will behave and how they will be called. That way, we have a benchmark or baseline to compare ourselves to on this. All of sudden, if we get a 100,000 calls a day to something that is designed for, or is expected to have, 1,000 -- we at least understand what to be looking for.

Gardner: Let's provide a level-set for our listeners. You are representing Allstate, which is a very large organization, with 17 million customers, $156 billion in assets. Give us a sense of the scale that we are talking about in terms of your IT organization?

Abbattista: Our claims organization, for example, has an IT shop of about 400 people that are employees, and we are not counting offshore or other people to support that. Each of our business units is a substantial IT shop in of itself, each with 500 to 1,000 people.

Then, what we choose to federate becomes an issue, because they need to talk to each other. They need to talk to themselves. They need to talk to the outside world. So what we layer then in my area is an infrastructure of components on how to do those tasks.

The massiveness of it is how do you measure and monitor that to get end-to-end composite services that we really can monitor and supply a good customer experience from? The massiveness is amazing. We have about 5,000 servers -- UNIX, Windows, mainframes, AS400s -- we have them all at this point.

Gardner: How many services do you have that have to "pay their toll" on the service bus, so to speak?

Abbattista: About 750.

Gardner: Wow! That’s pretty good.

Abbattista: We actually front our document management services and collapse all that into Oracle, but we fronted that with TIBCO. We did that so that we would have the measurement from day one, and it’s worked amazingly well.

People argued it would be just as easy to shove the document to the database and make an HTTP-SOAP call, but this governed ESB approach has paid off a 1,000 times over, because we now predicatively know when something is going awry.

Gardner: All right, now let's go to Rourke. We understand that enterprises are hesitant about going toward SOA on a holistic basis, if they haven’t got performance backstops in place. We are a little bit weary of finger pointing, because there is such a complex stew of components and services that makes it very difficult after the fact to point out and say who is responsible.

And, we're dealing with organizations like Allstate, which have massive size and scale, with 750 services. What do people need to be considering, as we moving into to yet more complexity with virtualization, cloud computing, utility grids? Give us a little bit of level-set about what's important to consider when moving toward a solution before the fact?

McNamara: SOA, virtualization, and governance -- all of these technologies have pluses and minuses. And, on the whole, when you finish computing out the equation, you are definitely on the plus side, you are definitely on the positive side.

But, you need to make sure that, as you move from the older ways of doing things -- from the siloed applications, the siloed business unit way of doing things -- to the SOA, services-based way of doing things, you don’t ignore the new complexities you are introducing.

Don’t ignore the new problems that you are introducing. Have a strategy in place to mitigate those issues. Make sure you address that, so that you really do get the advantage, the benefits of SOA.

What I mean by that is with SOA you are reusing services. You are making services available, so that that functionality, that code, doesn’t need to be rewritten time and time again. In doing so you reduce the amount of work, you reduce the cost of building new applications, of building new functionality for your business organization.

You increase agility, because you have reduced the amount of time it takes to build new functionality for your business organization. But, in so doing, you have taken what was one large application, or three large applications, and you have broken them down into dozens or tens of separate smaller units that all need to intercommunicate, play nice with each other, and talk the same language.

Even once you have that in production, you now have a greater possibility for finger-pointing, because, if the business functionality goes down, you can’t say that that application that we just put on is down.

The big question now is what part of that application is down? Whose service is it? Your service, or someone else’s service? Is it the actual servers that support that? Is it the infrastructure that supports that? If you are using virtualization technology, is it the hardware that’s down, or is it the virtualization layer? Is it the software that runs on top of that?

You have this added complexity, and you need to make sure that doesn’t prevent you from seeing the real benefit of doing SOA.

Gardner: So after the fact of failures, in trying to do forensics and root cause analysis and putting more agents and agent-less systems in place, if it's all telling you what's wrong after the fact that it’s wrong, it’s probably too late.

McNamara: Absolutely.

Gardner: How do we get to this vision of proactive, anticipatory systems awareness via service performance management? Let me first take this to Sandy. How important is it for us to get to this sense that something isn't quite right, in advance of it failing?

Rogers: Obviously, there are different use cases and different companies that are really interested in that dynamic, autonomic type of environment, where you can adjust to the demands of the environment, but we are also becoming much more Web-based.

What we are seeing is that, as services are exposed externally to customers, partners, and other systems, it affects the ability to fail-over, to have redundant services deployed out, to be able to track the trends, and be able to plan, going forward, what needs to be supported in the infrastructure, and to even go back to issues of funding. How are you going to prove what's being used by whom to understand what's happening?

So, first, yes, it is visibility. But, from there, it has to be about receiving the information as it is happening, and to be able to adjust the behavior of the services and the behavior of the infrastructure that is supporting. It starts to become very important. There are levels of importance in criticality with different services in the infrastructure that’s supporting it right now.

But, the way that we want to move to being able to deploy anywhere and leverage virtualization technologies is to break away from the static configuration of the hardware, to the databases, to where all this is being stored now, and to have more of that dynamic resourcing. To leverage services that are deployed external to an organization you need to have more real-time communication.

Gardner: So, the proposition remains, how do you do that? It’s clear that you want to get out in front of these problems, but with so many interdependencies, the large scale in number of services, different environments, probably inside and outside the organization, it raises questions. How do we move up in abstraction toward understanding the context of an entire business process, in order to go back and look for the signals that will tell us when something is approaching a breakdown, or when we need to provision more hardware and software resources?

Let me take this to you, Anthony. Where do you think that abstraction needs to be in order to forecast appropriately issues of SOA integrity?

Abbattista: I'll go back to the point on having some expectations or benchmarks of how the service should run when it’s designed and deployed in the first place. Then, you can understand if your baseline is correct and then, over time, you can look for fragmented behavior. But, I do think you need some level of end-to-end view of the process and of who is the customer on the end.

Ultimately, where these things show up en masse is at the end-points, and typically that’s in the consumer space, as we are frustrating an employee or someone on a website with a bad client experience. Those are unforgivable.

So, starting with the customer at the end-point of that business process and looking at some of those interactions, is part and parcel of deploying the service in the first place. If you don’t do that, you will be chasing your tail for the rest of your life in operations, until you go back and do that mapping. So I think it pays to do it upfront.

Gardner: You mentioned in your presentation that the "Walls must come down" between IT operations and development-deployment-requirements-test functions. It sounds like you're also saying it needs to go from end-to-end, beyond just that wall, but also across the entire event-processing landscape.

Abbattista: In that respect, I view our function in running the applications and supplying the applications as a utility. It's our job to point back to the groups that deploy the stuff. If I let them deploy junk, I am as complicit in that junk being delivered as anybody else. That’s a responsibility we take seriously. If you're going to put it in the shop and expect us to run it, I won't take junk.

Gardner: Right. So there is the adage of, "Garbage in, garbage out." Now, if garbage appears anywhere in the context of a complex process, it's garbage out. That’s even more difficult.

Let's go to Joe McKendrick. Tell us about the concept of complex event processing (CEP). How do you get any handle on a process? Do you look for the description of the process from a modeling perspective, through what's been done on the ESB, all of the above?

McKendrick: Definitely all of the above. CEP is something that’s just coming into the SOA realm. It is said that that’s the next phase for SOA. As was pointed out this morning, real time is not enough for a business. Business needs to be able to react and predict.

Rourke and I were talking about that a little bit earlier. You need to be able to predict what's going to happen, not only in the business, but in the systems. TIBCO is making some progress in this area in terms of being able to predict when the system may go down or when there will be spikes in demand. Predictive analytics, which is a subset of business intelligence (BI), is now moving into the systems management space.

Gardner: We're actually moving above the systems management space by an abstraction level or two. Let's go back to Rourke. You had a couple of product enhancement announcements today here at the TIBCO conference. You are getting out in front of service performance management, and your interest is to accomplish some of the things we have been describing, provide what the market is demanding for SOA in order to be trusted.

Tell us about CEP and why that is an important part of this predictive solution.

McNamara: One of our customers said it best last night over dinner, when I introduced the concept of the product I am going to mention in just a second. They saw immediately what problem it solved for them.

They said that their biggest fear is that their SOA initiative will be a victim of its own success. A service will be reused so many times so rapidly that the hardware it's deployed on, the manner in which it was deployed, won't be able to handle the load. That service, which is now used in a dozen different business applications, or exposed in a dozen different business applications, will go down or will degrade in its performance level.

That could make SOA a victim of its own success. They will have successfully sold the service, had it reused over and over and over and over again. But, then, because of that reuse, because they were successful in achieving the SOA dream, they now are going to suffer. All that business users will see from that is that "SOA is bad," it makes my applications more fragile, it makes my applications slow down because so many people are using the same stuff.

Gardner: What is it about CEP that gives us more visibility at the right abstraction, so that we can predict among all of these different complex components and assets where a problem is developing?

McNamara: The key is that we just can’t simply wait for the problem to develop or the problem to happen, because it will happen very quickly. We won't have a week’s warning, a month’s warning, or even necessarily a few hours’ warning. And we won't understand, when we deploy that service, all the places or all the manners in which it will used. So, we need to be able to predict these problems before they occur and do something to prevent those problems from occurring.

TIBCO is taking our CEP technology, the business events technology that we have, and applying the problem to our internal software, our infrastructure, the same way our customers apply it to their business problems.

We are using business events to monitor what's going on with service load and performance -- what the load profiles look like in a given organization, allowing it to understand some of the programs and marketing efforts that are going on within that company. Then, when it sees that a service load is approaching a dangerous level; when it sees that based on the events that are occurring that the service will become overloaded and will violate its SLAs, it’s able to tell other parts of the infrastructure to take action to prevent that problem.

Gardner: Let me see if I understand this. This sounds like a schematic about a business process and, by reverse engineering from that process down to the constituent ingredients to support it, you can predict where the loads will be building or will become erratic. Therefore, you can also detect what's going on within that system, put the two together, and come up with a heads-up?

McNamara: That’s exactly right. You need to understand what the interdependencies are between your services and what the load characteristics of the different component parts in that dependency graph in that environment are. Then, based on that, you need to understand what sort of events in your business or in your IT infrastructure will cause performance problems or overload conditions.

Gardner: Let's go back to Sandy. You mentioned earlier about how to automate toward these goals. It sounds like it’s going to be a bit of journey to get to full automation. On the other hand, having 40 people on a conference call to try to manually bear-wrestle these problems down doesn’t work either. How do we find a balance between too much automation, automation that can’t be attained, and purely manual, after-the-fact approaches?

Rogers: Everyone has to walk before they run with any type of new technology implementation. But, we are finding that most organizations are keying in on those services that are most important, and making sure that they are instrumented appropriately regarding the technologies that support management as being able to define what those thresholds are.

Being able to correlate those thresholds to real business needs and business value -- that’s one of the interesting things about what we were doing in a service level. We can start to associate the services that are most relevant and what there are going to have the most impact for.

We can make sure that information that is contained either in the payload or form the service itself as provided. So, you have that insight. I think that organizations are starting to realize that, in order to prove the value of the services, in order to prove that the value of having this level of coordination around management, they need to be able to make that association.

From an inventing point of view, what’s interesting is that there are a lot of parallel types of processing going on in this environment. Rather than wait until something happens in some linear, straight-through process, we're seeing the ability to watch and correlate some of those events vis-à-vis the thresholds, understand which thresholds are the most important, and start automating how to define the behavior, how the system is going to react to those conditions, and do it from a cost-benefit perspective in moving forward.

Gardner: Okay, so companies can take this approach, use a moderate pace, learn as they go, and use complex event processing to offer insights into the context of what’s going on. But, if human nature is any indication, people usually react to whatever the rules are about their job, and for IT this is going to be the view from the SLAs.

It strikes me that what’s going to happen is a lot of these organizations are going to reverse-engineer from the SLA, and that the rules and the models in the SLA become extremely important. Am I going out on the limb here, Anthony, or do you think that it will pan out that the SLAs will be the rules that the service performance management then needs to line up around?

Abbattista: That’s right. Again, it's back to what do you expect, and are you living up to it? You talk about failure not being from the SOA, but we could have a case where a service got deployed, people learned about it, and, before you know it, we are taking 100,000 hits a day on a service that no body ever gave any design thought to.

I would have to reach in there and get some agent information once in a while. And all the sudden, the supplier of the service, who did us a favor, put this on the bus, and then did a point-to-point interface, calls up and says, "Help!"

Someone might publish this thing and it had no modeling, because they thought it was some low-volume thing and it wasn’t important. All of a sudden, it becomes important because everybody found it. So, as we get to composite services, SOA performance is about the service expectation.

Gardner: And the governance?

Abbattista: The governance, and do you let them do it? Do you have governors? Do you have a cost model that burdens the caller, rather than the supplier? These are real questions we'll get into, and they are why I was talking about breaking down the walls. If it's truly a valuable service, then it's my job to figure out and pay for upgrades -- or to help you redesign it.

We take very much an advocate approach to, "Okay, if you come on the bus, we will help you with being successful." And the SLA is the baseline for that. But it also sets up that, "Hey, did you do a good enough job? And what if you are wildly successful?"

Gardner: Right. Let's throw this back to TIBCO. There's clearly a need in the market for a full lifecycle approach, feedback loops, many moving parts. What is it that you can do from a product perspective that helps get to that level of automation? That, in a sense, fills in the cracks about whom and what performs some of these necessary communications between the operations side and those associated with the ongoing requirements?

McNamara: Taking a step back, TIBCO offers a single user interface from the business analyst all the way through to the operational administrators who are running our applications. The idea is that, when you sit down to build out your services, when you sit down to build out your business processes, you use one tool to define what the business processes look like, what the touch points are between folks. Then, that diagram gets handed off from the business analyst to the implementer, who sits down and actually builds the services or builds the business process management (BPM) process that meets those requirements.

There is a direct link between the two. There is a round-tripping built into that tool, largely because it's a single data model and a single user interface with different views for people with different roles in your enterprise. That’s one major thing we do to help facilitate that communication, and that’s part of what we call the TIBCO ONE initiative. The product in question is the TIBCO Business Studio product, which forms that single user interface.

Gardner: And you’ve got hooks in a lot of the other parts of the SOA infrastructure for service enablement and delivery. How do you pull these parts together in a concerted effort?

McNamara: The other side of things is, even once you’ve built things out and deployed things to production, you need to make sure you can keep track, because a number of the folks on this panel have said exactly what’s going on. Ideally, you want to identify early on, as Sandy and Anthony said, which services are important to your enterprise and which services will have heavy load.

Unfortunately, you can’t always do that. Sometimes a little service, as Anthony said, where you think it's just helpful turns out to be a service used in 60 percent of the applications you are deploying. All of a sudden, you’ve got an issue.

You need to understand what the usage characteristics are on your services, not just the designed usage characteristics on your services. We’ve embedded both policy and performance management capabilities in our underlying service infrastructure. All the TIBCO ActiveMatrix products, all of our SOA enablement products, will transparently monitor for performance and usage of the services deployed in that environment.

Anything that you build in TIBCO ActiveMatrix BusinessWorks, ActiveMatrix Service Bus, ActiveMatrix Service Grid, and so on, is automatically monitored. And, you can automatically do some things around policy and access and control and rules.

So, even if you build that little service and you don’t think it's important, and you don’t want to go to the extra trouble to build some governance into it, it's there. It's already been embedded in that infrastructure. When you need it, you can just turn it on and make use of it, and you will automatically have some information about how people are using it with a fairly nice visual dashboard?

The key here is not just the ability to see some numbers in a report, because people miss that. You can have a report on, as Anthony said, more than 750 services running. By going through the performance numbers on each of those services on a regular basis, things get lost if it's just numbers. You need to have very good visualization tool, so you can see in "living color" what's going on with those services and how that relates to the SLAs and the rules you’ve set -- the expectations you’ve set for those services.

Gardner: All right, let's go back to Allstate. Anthony you’ve heard the announcements today, you’ve understood this vision, and you understand the need very well. Do you think that we are getting very close to realizing more of an automated approach to service performance management in an SOA environment?

Abbattista: Yes, we are getting closer and making rapid strides. We need to be careful though. We are being careful to manage the service deployment, the service bus grid, and the parts about how to operate it. What makes me a little nervous or restless is the idea that we start taking all that back into the system parameters and the Java environments and Oracle databases, and that sort of thing. I would hate to see us not solve this first.

I really don’t think we’re at a stage where I want to automatically be adjusting heap sizes in Java virtual machines, or Oracle database parameters, which could be a next logical extension. I did see a little twinkle in people's eyes today, when they looked at products like the BMC Suite and Matrix. I don’t know that I want to have system programmer types around, trying to debug the debugging environment. I think it could become very complicated, very quickly.

Gardner: So we need to keep this at that higher abstraction in order to appreciate the whole and not get down into the weeds?

Abbattista: That’s my belief. I would say that if this service is not performing, then maybe we get the three people on the phone, the database administrator, the platform person, and the network person -- and we take a look at it. But I don’t think we should drill too far into that, until we solve the other layer.

Gardner: I suppose the good news and bad news about all of this is that the metrics for success or failure will be quite evident. You are not going to be able to cover this up across a service-support environment and the business processes that those contribute to, if it doesn’t work. Any failures are going to be readily apparent, not just to a systems administrator, but also to the entire organization that’s affected.

Joe, let's go to you on this whole notion of metrics of success. We have seen some caution, but we also see great promise around SOA. If we got into an economic environment where the pressure becomes higher for better productivity -- of doing more with less -- it's likely we are going to see more companies look to virtualization, outsourced services, software as a service. When do you think the switch on wider SOA use will get thrown, and to what degree does service performance management contribute to that?

McKendrick: Wow, that’s the $64-billion question. It's interesting, I was speaking with the enterprise architect for a major distribution company a little bit earlier. She pointed out to me that, when they started out their service enablement years ago, even before Web services came on the scene and evolved over the past 10 years, they built their infrastructure to be service-enabled from the get-go.

There was no effort to identify what can be service enabled and try to build a service around and try to get acceptance of it. And I asked her, "Well, what do you consider to be success in terms of adoption of the SOA, and in terms of reuse -- and do you even measure a reuse success?"

Basically, to that company, if a service gets reused, fine. If it doesn’t get reused at all, that’s fine too. It doesn’t matter. The reason I'm bringing that up is because reuse is often brought up as the ultimate metric for a SOA success, as the most tangible metric, I should say. But, I think the best approach is to design applications or pieces of applications from the initial start to be service-enabled and employing the latest standards.

Gardner: Okay, so the risks are high, the rewards are high. It sounds like we are getting closer to a less manual, more automated approach, something that has visibility and hooks up and down, deep and wide. Let's wrap up with some last thoughts on this subject.

Sandy, if you are a CIO, a decision maker in the enterprise, and you are listening to this, what do you think that you want to hear that’s going to make you confident, given that you’ve already made a lot of investments in services-enablement? You have to recognize that this is the way for the future, but what are you going to want to put in place in order to start protecting yourself when it comes to your performance management?

Rogers: What we are seeing with IT executives today is a real interest in leveraging what you have, of being able to have speed for deployment, not having to worry about all of the issues, and to have people on board that understand all of the technical dynamics of how everything needs to be implemented from an infrastructure point of view.

So, they need to be able to support fast time to market, and not worry about throwing something out there. When you are deploying it, you have to step back and make sure all of the resources that you need are lined up to make that happen. You want to have an automated way to handle deployment, to handle governance, to handle all of these different issues.

There is also the self-service nature that’s starting to happen -- the ability to create services and allow anyone in the enterprise to be able to get at the information they need as quickly as possible, not have to have a whole army of developers out there. That means you need to feel comfortable that you are creating an infrastructure that could be consumed by multiple parties.

Setting up that infrastructure is really going to save cost. It's going to save time to market, and you need that level of assurance, so that you don’t need to baby sit every single service. There is also an issue of being able to outsource to different parties. You want to be able to leverage that, cost-effectively.

You need to set up an infrastructure, all the processes and rules that everyone needs to follow. And by doing that, you can now leverage whatever resources you want to develop and create what's necessary, and not have to worry about everyone falling in line and having their own infrastructure and having all of that reference architecture put together at each different resource.

It’s really that whole concept of creating a centralized type of platform and a framework to consume all these services. It’s going to be very, very important going forward. Everyone is talking about the issues of the economy, and it’s really the trade-offs of what do you need to do in order to move forward and think about things in more of a total cost of ownership (TCO) manner versus that of direct return on investment (ROI) -- that immediate cost-per-service type of measurement.

Gardner: It sounds like we are describing what could be thought of as insurance. You’ve already gone on the journey of SOA. It’s like going on a plane ride. Are you going to spend the extra few dollars and get insurance? And wouldn't you want to do that before you get into the plane, rather than afterward? Is that how you look at this? Is service performance management insurance for SOA? I am throwing that out to Anthony at Allstate.

Abbattista: It’s interesting to think of it as insurance. I think it’s a necessary operational device, for lack of better words.

Gardner: Service performance management -- not an option?

Abbattista: I don’t think it’s an option, because what will hurt if you fall down has been proven over and over again. As the guy who has to run an SOA now that's on insurance -- it’s not an option not to do it.

Gardner: Last words from you, Rourke? Do you view this as an insurance policy? I guess you have the choice of different insurers, right?

McNamara: I do. I actually do look at service performance management as insurance -- but along the lines of medical insurance. Anthony said people fall down and people get hurt. You want to have medical insurance. It shouldn't be something that is optional. It shouldn't be something you consider optional.

It’s something that you need to have, and something that people should look at from the beginning when they go on this SOA journey. But it is insurance, Dana. That’s exactly what it does. It prevents you from running into problems. You could theoretically go down this SOA path, build out your services, deploy them, and just get lucky. Nothing will ever happen. But how many go through life without ever needing to see a doctor?

Gardner: Okay, now we are going to take some questions from the audience.

Tony Baer: This is Tony Baer with OnStrategies. I want to seize on something that Anthony Abbattista from Allstate had mentioned before, which is that you hope that service performance management doesn’t degrade into getting down to "Java heap sizes." I surely don’t blame you on that one, but what I am wondering is, at what point does this become an IT service management issue?

Abbattista: Because we have gathered that responsibility together, I guess it all falls under one roof in our particular organization. I would think it was external services. One thing we are doing is measuring some of our external providers outside the organization. I guess it’s sort of the same phone call. You are calling yourself or you are calling the person who is responsible and holding him accountable. So, I don’t know it changes much.

McNamara: I would like to add something to that. With something like a Tivoli or a BMC solution, something like a business service management technology, your operational administrators are monitoring your infrastructure.

They are monitoring the application at the application layer and they understand, based on those things, when some thing is wrong. The problem is that’s the wrong level of granularity to automatically fix problems. And it’s the wrong level of granularity to know where to point that finger, to know whom to call to resolve the problem.

It’s right, if what's wrong is a piece of your infrastructure or an entire application. But if it’s a service that’s causing the problem, you need to understand which service -- and those products and that sort of technology won’t do that for you. So, the level of granularity required is at the service level. That’s really where you need to look.

Rogers: What I find is that it’s inevitable that we are going to go down that path, but standards between the systems that do IT management traditionally and this level of detail really haven’t been fleshed out. Most organizations are looking for a single, unified type of dashboard on some of the key indicators. They might want to have that for the operations team that has traditionally run IT service management.

A lot of the initiatives around ITIL Version 3.0 are starting to get some of those teams thinking in terms of how to associate the business requirements for how services are being supported by the infrastructure, and how they are supported by the utility of the team itself. But, we're a long way away from having everything all lined up, and then having it automatically amend itself. People are very nervous about relinquishing control to an automated system.

So, it is going to be step-by-step, and the first step is getting that familiarity, getting those integrations starting to happen and then starting to let loose. What's interesting is in some of the areas of virtualization technologies, where you might have some level of management that’s abstracted from the physical infrastructure, and then you have this level of abstracted management of services how they come together. It hasn't really been defined in the industry, but down the road -- two, three, four, five years from now -- I think you will be seeing a lot more around that.

McKendrick: Let me add that we're still in the very early stages of SOA. In fact, a lot of companies out there think they have SOA, when they actually have just the bunch of Web services, JBoss architecture, and point-to-point types of interfaces and implementations. A lot of companies are just starting to get their arms around exactly what SOA is and what it isn't.

Gardner: Very good. We have been discussing the issues around service performance management for SOA environments. We are talking with a panel of industry analysts and practitioners. I want to thank our panelists, Joe McKendrick, Sandy Rogers, Anthony Abbattista, and Rourke McNamara. Thanks.

This is Dana Gardner, principal analyst at Interarbor Solutions, and you have been listening to a sponsored BriefingsDirect podcast. Thanks and come back next time.

Listen to the podcast here. Sponsor: TIBCO Software.

Transcript of BriefingsDirect podcast on service performance management recorded live at TUCON 2008 in San Francisco on April 30, 2008. Copyright Interarbor Solutions, LLC, 2005-2008. All rights reserved.