BriefingsDirect SOA Insights Analysts on Defining SOA and IBM's DataMirror Acquisition

Edited transcript of weekly BriefingsDirect[TM] SOA Insights Edition podcast, recorded July 20, 2007.

Listen to the podcast here. If you'd like to learn more about BriefingsDirect B2B informational podcasts, or to become a sponsor of this or other B2B podcasts, contact Interarbor Solutions at 603-528-2435.

Dana Gardner: Hello, and welcome to the latest BriefingsDirect SOA Insights Edition, Volume 22, a weekly discussion and dissection of Services Oriented Architecture (SOA) related news and events with a panel of industry analysts and guests. I’m your host and moderator, Dana Gardner, principal analyst at Interarbor Solutions. Our panel this week, and this is the week of July 16^th, 2007, consists of Joe McKendrick. Joe is a research consultant, columnist, and blogger. Welcome back, Joe.

Joe McKendrick: Hi, Dana, glad to be here.

Gardner: We’re also joined by Tony Baer. He's a principal at OnStrategies. Hey, Tony,

Tony Baer: Hey, Dana.

Gardner: We're joined once again by Neil Ward-Dutton, research director at Macehiter Ward-Dutton. Welcome back, Neil.

Neil Ward-Dutton: It's great to be here, Dana. Thanks for having me.

Gardner: Also, filling out our roster -- and we have a large group today -- Todd Biske. He is an Enterprise Architect at MomentumSI. Welcome back, Todd.

Todd Biske: Good morning, Dana.

Gardner: Also, Dave Linthicum, CEO of Linthicum Group. We’re glad you’re here Dave.

Dave Linthicum: Glad to be here, Dana.

Gardner: Last but not least, Brad Shimmin, principal analyst at Current Analysis. Good morning, Brad.

Brad Shimmin: Good morning, Dana.

Gardner: Our topics today are, on one hand, a large amorphous topic, which is to try to once again define and categorize SOA -- what it means and where it's going. Then second, a bit more refined, direct topic, and that is the recent acquisition by IBM of DataMirror, or perhaps the intention to acquire DataMirror, and what that means for services, the data-services layer, and the inclusion of more data for IBM in their Information Server Suite approach.

Let's start with the larger topic of the day. What's been coming up in some of the blogs and in some analyst reports is this notion of people overusing the term SOA categorically, and the suggestion that we should go back to the roots and get pure again about SOA. Some are saying that this other business around it being application types, application support, runtimes, and actual products and platforms needs to be muted a little bit.

At the same time, we’re also seeing discussions crop up around such things as platform as a service, particularly around the Salesforce.com ecology and approach, and also integration as a service. I recently did a sponsored podcast with Cape Clear Software on this topic. Then, of course, there's the the long-term discussion around software as a service (SaaS), and the ability to use Web services and a variety of application interfaces and APIs to avail yourself of hosted online applications and services.

So, let’s take this out to the group. Are we confusing topics here? There's this notion of things off the wire, outside of the domain of the enterprise. Should that remain separate and distinct, as a discussion point and as a direction for enterprises, from SOA, which is more of an architectural initiative and philosophy within the confines of their enterprise IT systems.

Let's go to Todd Biske first, because, Todd, you're an enterprise architect and you’re working with clients in the field. How do you come down on this? Is SOA inclusive of all of these "x as a service" or "blank as a service" or are they actually part and parcel of the same direction?

Biske: I really think it should be inclusive of all these external platforms. SOA, in my opinion, is really something that’s driven out of enterprise architecture. To go back to the simplest view, it's about the services. If you are able to get services from outside your firewall, those should be included as part of your SOA. You absolutely need to include the availability of external solutions as contributors to your overall SOA for your enterprise. I know Dave has talked about this a lot as well.

There's a lot of debate as to whether SOA is the whole kitchen sink. There's a Yahoo! mailing group, and they’ve had the same debate going on just recently on not just the technical aspects, but whether SOA should encompass all of the culture change that people are saying is so critical to success with SOA. Are we just muddying the term by trying to lump all of this in together?

If we look at it from what it takes to be successful, if you leave those pieces out and focus strictly on the technologies associated with it, you’re not going to get success. At that point we’re not achieving the goals we set out to do. So, it needs to include all of these concepts and have that factored into your overall enterprise approach.

Gardner: Dave Linthicum, what do you think? Do you think that we’re going to water down SOA by including all this stuff, or is there another category that we should put all of this stuff into, in which SOA is a component?

Linthicum: I agree with Todd. It's inclusive. In fact, look at my career in starting up Grand Central and Bridgewerx and all those companies, where it was all about creating SOA outside the enterprise and delivering it as a service. It's what Salesforce.com is doing with their Apex platform today, what they call Salesforce SOA.

So, it's an architecture, and ultimately lines are blurring between where the enterprise stops and the Internet starts. With mashups, outside-in services, all the stuff Google and Microsoft are working on, and all the services that are being consumed within the enterprise, it definitely is inclusive and it has to be considered in the context of the architecture, enterprise architecture, and architectural patterns like SOA.

Gardner: Tony Baer, do you think that this is begging too much of a typical enterprise IT department to both be managing transition and transformation to SOA internally, and, at the same time, start to work with these off-the-wire elements -- be it platform, integration, or services and applications?

Baer: I’m going to take this to another level, which is the business level. If you look at the way business has evolved, or supply chains have evolved, over the past 20 or 25 years, they're really evolving into an ecosystem that is very partner dependent. When you buy a product that is manufactured by a company or that’s branded by a specific company, chances are it is manufactured by somebody else.

So, SOA and enterprise architecture really need to echo or support the direction in which business is going. More and more enterprises are becoming virtual. As a result, if you're running your internal processes, the chances are you’re going to be interacting pretty intimately with one of your business partner’s external processes.

Gardner: So, this for you is really just the evolution of business in general, which is to outsource as best you can.

Baer: It's not just outsourcing. You’re working in coalitions. For example, look at how a car is put together today. There have always been suppliers, but suppliers are now taking higher-level roles. They're actually co-designing. When one of the automakers or OEMs designs a car, chances are they’re designing it with their partners. In the old days, they designed the car and then sent out parts orders to partners.

So, SOA in IT and enterprise architecture really needs to support the way the business is going. Business is becoming more virtual. It includes external processes and partners, and SOA must support that.

Gardner: Okay, but about five or eight years ago, a large number of companies really explored outsourcing, but then pulled back. There was something they didn’t seem to like about it. They seemed to do it for economic reasons, but there was this backlash against it. Do you think that was growing pains? What's up with that in terms of still wanting to retain a lot of control over IT internally?

Baer: Well, it's cyclical. I remember back around 1990, there was a grand announcement by Kodak, "Well, we’re outsourcing IT, because, guess what, IT is not part of our business." I’m not even going to fast-forward this 20 years. I’m going to fast forward it three years. Kodak realized, "You know something? IT is part of our business." They had to take large parts of those 10-year contracts with IBM and, at that time, Digital and, I believe, EDS, and after about three years those contracts essentially collapsed in. They dropped a number of provisions. Today, Kodak is saying, "We’re a digital business."

It's a cyclical thing. If you outsource without managing it, if you just throw something over the wall, you’re going to get what you paid for.

Gardner: Neil Ward-Dutton, this notion of the virtual enterprise, do you see that as being on the same wavelength with SOA activities. Do you agree so far with the discussion?

Ward-Dutton: I thought Tony made some really interesting points there. Certainly, one of the arguments that you could make for a long time about enterprise architecture was that it was locked into a way of thinking that was very last century. It was locked into this evolutionary level of thinking that started around the kind of information-engineering movement that happened in the 1970s and 1980s. It assumed that you were building everything yourself, in long-running waterfall projects, and things didn’t really touch other things. Everything was built in isolation.

That was a big problem for enterprise architecture, because, as Tony pointed out, businesses have been pursuing this kind of virtualization agenda for a long time. Think about EDI and value-added networks (VANs), all that kind of stuff. All of that was designed outside the purview of IT and was done by business teams. The enterprise designers or the architects had no real input into that whatsoever. They were focused on the stuff they build internally for the organization. It's long overdue that architecture practice starts to take a more holistic view of an ecosystem of technology provision.

This thinking is long overdue. SOA, shouldn’t be constrained, as Todd just said, by a particular hosting model, where service is provided from, or indeed a technical choice. It's all about the services and it's all about the architecture, but that shouldn’t be constrained by where that service happens to be delivered or who it happens to be delivered by. You can pursue SOA in an environment where the resources are widely distributed, where responsibility is federated. Indeed, more and more, that’s how people are going have to do it. This is all natural, and as Tony said, really, really needed.

Gardner: Brad Shimmin, we’re getting into this notion that IT, by being isolated, has been a handicap, as companies explore extended supply chains and VANs and reach out and capture clients through the Internet. Do you see it that way? Do you think that IT has been a handicap? And, on a related note, is SOA therefore a catalyst to greater virtualization of companies, greater efficiency, and firm productivity?

Shimmin: Well, yes to both. IT really has been isolated, and I think we can all look at the cataclysmic failure of the idea of the waterfall method of development, as a pointer to that. You really can’t work in a vacuum, just toss code over the wall, and hope that it runs somewhere. As we were saying with the failures in early investments and outsourcing, I think those were all built in that regard.

Nowadays, when you start to look at what SOA has done to the industry and how it has impacted IT, it has become an enabling notion that allows companies to employ methodologies like the agile development method, that are more kind in how they view the lifecycle of software and where software runs. It's a partnership with the business and perhaps with outsourced parties that might be hosting that software.

SOA, as a collection of standards, technologies, and ideas -- which is how I look at it -- is already playing a very active role in allowing that pendulum to swing back towards platform-as-a-service, integration-as-a-service, SaaS, infrastructure-as-a-service. It's actually enabling those.

If you look at the vendors that we cover all the time you will see that everybody from VAN vendors like GXS, Otics, and Sterling to the traditional platform vendors are all looking at how they can utilize SOA to better enable outsourced models. SAP, for one, has always offered managed services, but they offer hosted services for their software.

Microsoft is doing the same with their BizTalk Services and with their dynamic CRM, which is tied with BizTalk Services. I think we’re seeing everybody really jumping on this bandwagon.

The best example is one we've already alluded to, Salesforce.com. When I look at them, I see a hosted service that utilizes SOA ideas to facilitate a secured, reliable, manageable, business-savvy domestic environment between your backend systems and their backend systems, which just happen to be on somebody else’s servers.

Gardner: Joe McKendrick, you’ve written recently about Internet Service Bus (ISB). Tell us a little bit about what you mean by that and how that relates to this discussion.

McKendrick: ISB is something that Microsoft has out there. They calls it BizTalk Services, and it’s now in the community technology preview phase, where they’re offering the functionality of their BizTalk -- essentially their ESB -- on an online basis to companies that don’t necessarily want to invest the time, capital, money, or staff to build their own ESB or SOA.

Gardner: So, does this mean that you get the option of hanging your clothes on their clothesline, as it were, on their servers, and then you can coordinate applications or mix and match across this bus? Help me understand conceptually what Microsoft is getting at here.

McKendrick: That’s exactly what they’re hoping to accomplish. You see it happening across the breadth of their product line, with the whole Microsoft Live. You can see them moving to this notion of SaaS, platform-as-a-service. I think they see the handwriting on the wall.

One thing I always say is that Microsoft doesn’t make a move unless it sees a mass market available to pursue. The fact that they’re pursuing the ISB shows that they see it's reaching critical mass in the market. Their natural base is the small- to medium-sized business. It’s not the large corporations that IBM, Oracle, and Sun serve.

Gardner: Or even departments within large organizations, which can actually behave like small- and medium-sized businesses.

McKendrick: Exactly. One model that makes it easy for the business to understand SOA could be an internal SaaS that units of the company are making available to the rest of the enterprise.

What happens is you have this software and it’s something the business can understand. There is a division, maybe at the CIO’s office and maybe somewhere else in the enterprise, that’s deploying these service-oriented applications. The services are available for reuse across the enterprise, across the walls of the enterprise. The enterprise has the option of either picking up these services internally or picking up services externally from another partner. The enterprise becomes both a provider and a consumer of services. It’s two-way outsourcing.

Gardner: There is a tangential trend around ITIL and some of these other initiatives, whereby IT departments take on more of the service bureau or a shared services role, and move from being a cost center towards more of a services provider. Their budgets then become like a P&L. They’re motivated a little bit differently than they would have been as just "Tell us what you need and we’ll figure out how much money we can come up with at the end of year for you" type of approach.

Does anyone else have a reaction to this related area about IT departments shifting in their business models and how that relates to SOA? Todd Biske.

Biske: One of the things that I heard as we went through this conversation, and you hit on it with ITIL and becoming a service provider, is the notion of service management. When we’re looking at why there’s been an outsourcing trend and a backlash against it, a key factor is the degree of service management that’s provided.

It’s one thing to need this capability and go off and find it somewhere, but once you start using that, you realize that there’s this whole degree of transparency that’s needed to understand how that service is being used, where the results are coming from.

In the initial stages of outsourcing there was a lot of people saying, "We want to be a black box and we don’t want to expose any of our dirty laundry, because then our customers may go away." Information needs to be provided to give good service management, as well as to help the customers of those services understand how it’s being used and where they can make incremental improvements on it. It creates a feedback loop.

We can look at virtually anything, whether it’s blogging or podcasting. Before we all got on the call. we were talking about the metrics of how many people are listening to the podcast. By looking at that information we can make things better. We can target audiences or understand exactly who the audience is.

The same thing applies now in the service management domain. I need to have visibility into how the consumers are using the service, where they’re getting good experience, where they’re not, are there trends, and how it’s used. If I’m not getting that information, whether it’s from an internal service provider or from an external service provider, that’s going to start to have an impact over time.

That’s where it leads into culture change -- adopting things like ITIL and understanding service management and what it means to be a service provider. IT is used to building solutions. They put it out there and they go on to the next project. That’s not service management; that’s project management.

Gardner: We’re also seeing discussions -- and we’ve had them on this show, of course -- around governance, where you’re looking at internal governance of services. Then, you’ve got to multiply that in terms of how you would govern across platform as a service and integration as a service.

It raises the question of whether the complexity starts spinning out of control, but we could combine a couple of these observations today. We’ve talked about governance in terms of comparing it to a government, whether you do a democracy, federalized, local, or state. But, if we consider ITIL Service Bureau, we recognize that one of the best governance mechanisms for the most complex things -- business and the economy in general -- is capitalism, competition, or even more crassly, Darwinism.

Suppose we open up services and allow them to compete for the users and/or the lines of business managers to be able to choose among an external service, an internal service, a shared service, a supply chain service within an ecology. Does anyone think that a capitalistic or Darwinistic approach, if this follows its course, will allow for the best form of governance, which is, "Let the best service at the lowest cost win?"

Ward-Dutton: I say yes and no to that. Competition, or the threat of competition, is sometimes a very, very valuable thing to produce the mix. However, you've got to remember that if you’re talking about free market, one of the side effects of a free market is that an awful lot of companies go bust, and that’s fine. If you’re at the top, looking down, you’re just looking at the performance of a market or an economy overall. It’s fine if companies go bust. If it’s your company, though, that’s not so good.

If you’re looking at IT provisioning in this context, you've got to be careful. You've got to put some regulations in place. You don’t want your own internal IT provider spending a heap of money on stuff that isn’t going to be used, not going out of business, but essentially causing business problems, because they’re wasting a lot of money.

So, you’ve got to have layers of governance, i.e, rules and frameworks about how planning gets done, how priorities are set, what’s important, and what’s not. Those kind of things have to be decided, so that a company knows collectively where it wants to go, what it’s trying to do, and what that means for IT. When you’ve got that in place, then in constrained places you can start to drive competition by saying, "In these places, we want to look for competitive solutions, but it’s not always going to be the right thing to do."

Gardner: So, the jungle might work in markets at large, but internally that could be self-destructive. What do you think Brad Shimmin? Do you think that competition has a role or should we have a balance like China, where we have a market, but also central control, and a little bit of socialism thrown in too?

Shimmin: Well, as much as I love socialism I have to say that I’m on the capitalism side of this. I just want to throw out a caveat that if we’re talking about IT departments becoming providers of services, we need to make a distinction between internal provision and external provision, because not every company is a Google that’s going to throw out a Google Maps widget or WSDL, and suddenly everyone is going to be utilizing it, and everyone is going to be happy.

Companies have a better shot at that, however, than doing this internally, because internally you have a lot of federated environments, as we were just saying, with departments acting as their own companies. The whole notion of charging back for services and being accountable for how much you use those services is something that a lot of companies are certainly not willing or prepared to embrace right now. So, the internal adoption of this notion is a long ways off for a lot of companies.

Externally, there are providers and consumers, and the average enterprise is most likely not a provider of services like that. They’re consumers of services from companies, who, as Neil was just saying, hopefully are going to be around for a while. If I'm going to subscribe to one of those services, as a Web service, I want to know a couple of things.

One, I want to know that the company is going to be around a while, because I have an investment in that service, in setting up the interface for it -- for example -- and testing it out, because you don’t just grab a URL WSDL, throw into your designer, and poof, you have a service running. You have to test it and make sure the data models match, etc. So, I want to make sure that my investment is going to be protected.

The second thing is I want to know that the service level agreement (SLA) that I have with that provider is acceptable. If I'm getting the Yahoo! Search WSDL, and I’m limited to 10,000 hits a day, but don’t know that, and I’ve rolled out this enterprise application based on it, I’m going to get some angry phone calls from people around the company when we’ve gone over our limit.

Gardner: So, the notion of how to procure a service responsibly and appropriately becomes very important. That’s a little bit different than governance. IT departments have been historically good at procurement around systems, servers, hardware and software, and support maintenance, but the advancement, up an abstraction, of procuring a service or set of services, would fall more into the role of an SOA architect and not necessarily an IT architect.

Linthicum: Absolutely. The SOA architects ultimately need to figure how these services are consumed and how they have value with their internal systems. I would argue, however, that we are getting more experienced at picking outside-in service providers through the whole SaaS notion. However, they deliver it through a visual interface which is, in essence, a Web site, but it’s becoming much more than that.

Salesforce.com, Amazon, eBay, all those guys, are making tons of money now renting their Web services, and the ability to provide the behavior and the value through Web service is going to be a core component. Businesses are probably more advanced than we think they are in integrating those things into the core systems. It’s at a rudimentary level, but ultimately becomes outside-in services, integration with the SOA, and that’s where the architects come into play.

Gardner: There is a role here for free market influence. That is to say, as this matures and more services from different environments and providers become available, that SOA architect is going to be picking and choosing based on pure market types of forces. That creates the market dynamic, which has an overall governance impact itself. Furthermore, we've also talked on this show about the rationales for SOA. Why should you do it? Why should you do it sooner rather than later? What are the cost benefits? What are the analyses from a business and technology perspective?

For those that are listening, perhaps they get the sense that creating SOA internally, identifying and cultivating SOA architects, will allow them to, in the future, avail themselves of services across a wider panoply of sources and business models, and ultimately they can become the beneficiary of these market forces. This capitalistic approach towards lower costs, higher productivity, and less waste becomes another economic impetus to pursue SOA. What's your reaction to that, Tony?

Baer: I don’t think there's question that this really follows how companies are running their own businesses right now. It's just not possible to become agile and be competitive if you try and make every thing yourself. So, it will be inevitable that you will, at some point, consume external Web services.

I want to get back to the governance point from before. With all the regulatory compliance mandates that we have to deal with at this point, we have to worry about who is handling financial data. Who is handling customer data? Who is protecting identities?

There's no question that a free market is going to emerge for services, but, at the same time, it's going to have to be accompanied by some very heavy doses of transparent governance. I may need to know, for example, my partner’s mechanism for federated trust and authentication. The short of it is, I don’t think you can stop continental drift in this case. You are going to have a free market of services, but you're also going to need a very transparent governance to support that.

Gardner: That's governance at even yet a higher level. That's governance at a market level. In what sort of organization, what authority, under what auspices would that type of governance occur?

Baer: You have to conduct your own governance. It’s a federated model. It’s not going to be any sort of United Nations agency that's going to be top-down type. There probably will be industry best practices that will be drawn up by industry bodies, but ultimately the enforcement has to be at the enterprise level. You have to decide what protections you need and what you expect from your partners?

Gardner: Wouldn’t that federation at the level just above enterprise, or even above supply chains, have to happen under some sort of a brand. We’re already seeing Salesforce.com. We're already seeing Microsoft.

Baer: I thought you were going to go towards something more like the Underwriters' Laboratory model, where we have some sort of external certification agency. Actually, you made a good point where you take a company that's very established and say, "Well, we're going to trust Microsoft, because they're too big to try and risk having a bad reputation."

Gardner: You're more likely to see a business side approach to this than a regulatory side.

Baer: I would agree with you. I don’t see any new top-down agencies parachuting in. It’s going to be, "I'm going to deal with trusted partners. I expect Microsoft is going to be around for a while. I expect that, considering the type of business they’re in, that they’re going to conduct themselves with the highest standards of good governance."

Gardner: Did we go right back to where we had been, which is five or six major suppliers, and "I'm going to be a IBM shop -- or a Microsoft shop -- or an SAP shop," but it’s going to have to do with the ecology of services and not just the packaged applications or platforms? Is this just going to evolve into another instance of major vendors controlling this. Does anybody have a reaction?

Shimmin: Well, I think it will be the same ecosystem we see now, where you have different tiers in the marketplace. Depending upon how important that service is to your business and your application, and whether or not you can withstand a change in that service, of that service going away or not, having the service availability or not, having the governance or the ability to control the data as you would like, then you might be able willing to go with somebody that's smaller than Microsoft.

Gardner: You might be willing to go with perhaps an open-source community that might perk up around a set of services and applications, where the governance is based on community input, rather than just a single entity or large influence?

Shimmin: I don’t see why not.

Gardner: Anybody else have some thoughts about what impact an open-source approach to a services ecology might include?

Ward-Dutton: You’re getting towards a cooperative model in that kind of environment.

Gardner: I guess you would.

Ward-Dutton: You are, if you're talking about a community banding together to make its own of autonomous decisions about who's to be trusted and how things are going to happen, without any kind of patronage from a benevolent dictatorship like a Microsoft or whatever.

Gardner: Or perhaps if the benevolent dictatorships become less benevolent over time.

Ward-Dutton: There is space for all sorts of models. I know the B2B exchange thing in the late '90s was over hyped, but Cognizant is still doing some pretty healthy business in the automotive sector, for example. That's not an open-source model, but it is a kind of industry driven, bottom-up collaborative effort. There are all sort of models, a spectrum of commercial models which can work.

It wouldn’t be open source particularly, but a kind of a non-profit community-driven type of approach could be equally valid. I don’t think it’s likely to be one model winning over any other. There are a whole load of things that are likely to have a role to play.

Shimmin: I feel that the open-source vendors are well positioned to get into this space, because you're buying a service, whether it’s support, installation, custom development, etc. It's a service that you're buying from them. You're not buying software. Companies are becoming much more accustomed to that and comfortable with it. Let’s say you have an open-source Mule version that's hosted within the ESB, Mule ESB hosted, maybe New Source is the company that runs the data center somewhere, and they provide SaaS. There's no reason why they can’t, and it would fit well with their model.

Gardner: I suppose another important trend that is out there now is this whole notion of social networking.

If people have transparency on how services are used and consumed, they should also have transparency to share their experiences with those services. If openness and the ability to create community dialog around whether services are being well provisioned or governed, or if costs and benefits are out of alignment, that this might also provide some grease on the skids of competition and efficiency towards the best solution that the community then adopts or shifts.

Any thoughts about how openness in networking, from a sociological point of view across the network, might keep this in some sort of checks and balances?

Biske: It’s part of any kind of capitalistic approach to this. Customer feedback is just part of the market dynamic, and that certainly can make or break somebody. If you're not factoring that in and incorporating what your customers think in leveraging that information, it’s going to come back to haunt you.

I had another comment on the open-source piece as well. There is still going to be plenty of room for smaller companies, closed-source approaches, in these vertical domains. One of the bloggers that I follow has commented that you don’t see a lot of open-source efforts in niche vertical areas, and we're talking about governance and the impact of regulations.

You don’t see people who, in their spare time, are following the new SEC regulations that are coming out and are trying to offer open advice on how to best meet source regulations.

Those who develop software have always had formal jobs and careers based upon it. It’s also been a hobby for lots of people, and has helped the open-source movement. I don’t know too many people who like to follow regulations as a hobby. So, to have this kind of open-source model to handle governance, the broad horizontal domains will continue to see pushes in that.

In these niche areas, where it requires somebody being paid to understand all of those regulations from the various government agencies, that has to go to a proprietary model. One, it’s just not as interesting, and two, it’s not as big as a marketplace. So, you have to get a level of niche expertise and understand who your customers are going to be.

Gardner: That follows the pattern we’ve seen already with the boundaries between where open source works well and where commercial works even better. Okay, let’s wrap up this interesting discussion we've had about SOA and its role, services, ecologies, governance, competition, and the viral impact of social networks and openness.

Moving on to our next subject, it's a little less global or universal in scope. The IBM acquisition announcement about DataMirror. Brad Shimmin, tell us a little bit about how you view this announcement and how this will or will not strengthen IBM’s hand in information management?

Shimmin: If you look at this from purely a master-data-management, or just data-management, perspective, it makes sense. It’s something they should have done, and they already have a good deal of energy being extended toward that. So, yes, "yippee-skipee," but what gets me going about it is just thinking about how IBM, and every other vendor that has its toes wiggling in the SOA pool, is really looking to build complex event-stream processing and event-driven architectures on top of their solutions. This allows companies to stop thinking about their processes.

Just as you build the process, you throw it out there. If it maxes the server out, that's a bad thing and you go from there. You have your process running. It has interdependencies upon these other processes, these other servers, these other resources, etc. Over time, there are a number of events that take place relative to that process. Companies want to be able to look at that stream of events in context and over time and be able to make decisions. They also want to have automated actions taken upon those. Something like DataMirror gives IBM the opportunity to do that.

Gardner: Is there any question now that having a strong data story is essential to having a strong SOA story for any of these major vendors?

Shimmin: It’s a necessity. Everyone has come to understand that you need to understand the data in the transaction as well. Most of the vendors that we cover in the space either do that or have woken up and smelled the coffee, and are making a lot of efforts in that direction.

Gardner: What about HP and Sun Microsystems? They don’t seem to have quite the same data story. What do you expect to happen there? Let’s go to Joe McKendrick. You track data stuff. Are we getting to a point where HP and Sun are going to be at a disadvantage, or is there actually a benefit for their being a bit more agnostic or neutral vis-à-vis data and data services?

McKendrick: Well, that's a tough one to call, Dana, because they are not actively in the data-management space per se. They don’t have the robust data architecture products that IBM, Oracle, Microsoft have. Sybase as well.

I could see HP emphasizing its service aspect in this regard, really getting into this with its service’s space. Sun is a tough call. It’s sometimes tough to keep track of what Sun is trying and trying not to do.

Gardner: It seems that the software business is a pretty good place to be right now. Looking at the results just this week from SAP, Microsoft, and IBM, software is selling pretty well, and we're not just talking one or two elements, but across the board. Enterprise software is a strong growth business, somewhere between 10 and 12 percent, even after currency fluctuations are taken into account. What do you think, Tony Baer? Is software in SOA important, and is this large set of data capabilities essential? How do you see it shaking out?

Baer: Well, actually I was doing a study at, of all places, a Telco software company. What struck me was that as part of their SOA strategy they also embraced a very heavily federated data model. If you think about it, it’s the perfect compliment to a SOA strategy, because SOA is built on the idea of a loose coupling of all your processes. If you're really going to realize the promise of that, and you have a lot of different data sources, which is the case in most large organizations today, a federated data strategy, which essentially abstracts the data from its source, is the perfect compliment to realizing the promise of SOA.

I'm not surprised to see that the software business is doing very well, and especially companies that are very heavily involved with managing data in some ways are performing. It's not a big surprise.

Gardner: Doesn’t this DataMirror acquisition by IBM point to the fact they want to be more federated. It’s not all going to be a DB2 world. They want to expand on the ability for real-time data management?

Baer: No question. Think about the one key piece that DataMirror brings in there, which is the whole changed-data capture. I can imagine a lot of business scenarios, where that itself could be packaged as a very high in-demand service.

For example, if you're doing any type of market watch or real-time supply-chain tracking, you don't care about all the routine data, when there's no change in the stock or when things are moving on schedule in the supply chain. You want to know when exceptions happen or when things change. I can see this DataMirror capability being packaged into a service that’s not just changed-data capture, but event notification. It fits perfectly in that picture.

Gardner: Neil Ward-Dutton, how about the last word, the role of data within the SOA value? And, how about the competitive landscape question around whether certain vendors might be at a disadvantage for not being too aggressive on the data side?

Ward-Dutton: How long have I got? 30 seconds?

Gardner: 30 seconds.

Ward-Dutton: The role of data is very, very complicated and gives people a lot of headaches. Certainly, this seems to be like a wave-particle duality thing between service and data. If you pursue SOA too simplistically, you just say, "Listen, what's important in the way I design things and architect my environment are services and that's all that matters. That's the primary organizing concept." Then, what happens when you get to data is a whole heap of trouble.

Equally, if you take a very data centric perspective, it can cause problems, when you think about services. When you just think about a service that you're going to reuse in multiple contexts, you have to think quite carefully about implications for the way that data is stored. Clearly, consistently mapping data architecture and SOA architecture is a challenging thing.

What's clear from the adopters I've spoken to, is that you can’t ignore information architecture when you're pursuing an SOA initiative. It’s vital to understand the implications for data architecture and capabilities like federation, changed-data capture, and synchronization of data across boundaries. So, there is a very strong play here.

I wouldn’t go as far as to say that IBM, HP, and Sun are at a disadvantage. I think it was Brad just said, I am not sure, so many people on this call. It’s a cast of stellar participants. Someone was just saying that these guys have never been data-management players, and there are so many things to say about Sun's software strategy, I'm not going to get into it now. HP has managed to survive and is now thriving, but focusing in a completely different area of software delivery. They are much more about management process, quality, and so on, and they are very agnostic in terms of the middleware layer and data layer, and that works very well for them. So, I don’t see why there should be a bigger problem.

Gardner: Right, I suppose there is one thread that we can pull together on the two elements of our discussion today. Perhaps, as you shop around for services, as you look towards the natural-selection benefits of an open ecology in communication, even with services being cheaper, better, faster -- over the wire in some cases -- you still might want to keep your data services very close to your vest. So, perhaps there is a delineation in our discussion for another podcast about whether as we look towards a mixture of in-sourcing, out-sourcing, and multi-sourcing, the data issue is something to keep an intellectual property and ownership vise around.

I want to thank our participants. It's been another interesting discussion. Joining us today have been, Joe McKendrick, Tony Baer, Neil Ward-Dutton, Todd Biske, Dave Linthicum and Brad Shimmin. Thank you all for joining.

This is Dana Gardner, principal analyst at Interarbor Solutions. You've been listening to Volume 22 of BriefingsDirect, SOA Insights Edition. Thanks and come back again.

Listen to the podcast here. Produced as a courtesy of Interarbor Solutions: analysis, consulting and rich new-media content production. If any of our listeners are interested in learning more about BriefingsDirect B2B informational podcasts or to become a sponsor of this or other B2B podcasts, please fill free to contact Interarbor Solutions at 603-528-2435.

Transcript of Dana Gardner’s BriefingsDirect SOA Insights Edition, Vol. 22. Copyright Interarbor Solutions, LLC, 2005-2007. All rights reserved.

Apache Camel Addresses Need for Discrete Infrastructure for Services Mediation and Routing

Edited transcript of BriefingsDirect[tm/sm] podcast with Dana Gardner, recorded July 27, 2007.

Listen to the podcast here. Podcast sponsor: IONA Technologies.

Dana Gardner: Hi, this is Dana Gardner, principal analyst at Interarbor Solutions, and you’re listening to BriefingsDirect. Today, a sponsored podcast discussion on a top level Apache Software Foundation project, known as Apache Camel.

It’s a way to help developers do better integration and mediation for routing in software-intense environments, and moving lots of objects and component services around, but also relying on rules and patterns to do so.

To help us understand more about Apache Camel, we’re joined today by James Strachan, technical director of engineering at IONA Technologies. Welcome, James.

James Strachan: Hi.

Gardner: Tell us a little bit about Apache Camel. It seems to be a subset of many other integration and infrastructure projects in the open-source community. Tell us a little bit about the history of how Apache Camel came to be.

Strachan: Earlier this year, Apache Camel grew organically from code and ideas from a bunch of other Apache projects, particularly Apache ActiveMQ and Apache ServiceMix. We found that people wanted to create and use patterns from the "Enterprise Integration Patterns" book in many different scenarios.

Some people wanted to use these patterns inside an Enterprise Service Bus (ESB), some people wanted to use these patterns inside a message broker, and other people wanted to use these patterns inside an application itself or to talk between messaging providers. Still other people wanted to use them inside a Web services framework or some other communication platform. So, rather than tie this routing code to a particular message broker or ESB, we tried to extract this code to be a standalone framework that can be used in any project.

Gardner: I assume that this is to simplify it and make it easier for developers? Is that right?

Strachan: Definitely. What we tried to do with Camel was give it the smallest footprint possible, so that it can be reused anywhere, whether in a servlet, in the Web services stack, inside a full ESB, or a messaging application.

Gardner: And, this is a plain old Java object (POJO), I believe, also built in Java. Is that correct?

Strachan: Absolutely.

Gardner: Tell us a bit about the team behind this and perhaps a little background for yourself?

Strachan: I'm an Apache committer. I’ve been in Apache for over five years now, and I am an Apache member for a few years too. I can’t remember how many. The Camel team is comprised mostly of people from the ActiveMQ team and also from ServiceMix and Apache CXF communities. Plus, a bunch of other people have joined since then.

Gardner: Tell us a little bit about your relationship with Apache. You’re involved with a number of activities there. Is that right?

Strachan: I've been around Apache for many years now, and I am involved in all kinds of different projects -- ActiveMQ, ServiceMix are the big ones -- but also on a bunch of others, like Jakarta Commons, Maven, and many others.

Gardner: There are a number of terms we’ve been bandying around. Let’s assume that not everyone listening is that deeply into technology. When we talk about mediation and patterns, let’s flesh some of these out. Mediation and routing, how is that different from what people might understand generally under enterprise integration?

Strachan: I definitely recommend people read Gregor Hohpe's book "Enterprise Integration Patterns." He offers a really good patterns catalog of how people should do mediation and integration. Rather like the original Gang of Four "Design Patterns" book, which describes low-level programming things, Gregor’s book describes very well how enterprise integration patterns (EIPs) can work and gives us a language for describing them.

Aggregate, re-sequencer, message filter, and message translator generally describe patterns people commonly want to do within an integration solution. One of the simplest patterns is "content-based router," where you want to route messages around your network, based on the content. It might be, for example, that for gold-level customers you want to use the big, shiny, fast machines in your data center, but for bronze-level customers you use the crappy, old, worn Linux box in the corner of the basement.

Gardner: You mentioned earlier ESB, enterprise service bus. Explain to me how this works in terms of other ESBs. This is ESB-agnostic. Is it something that people will use to complement ESBs or perhaps replace them in some instances? What’s the relationship between Camel and an ESB?

Strachan: That’s a good question. In some ways the term "ESB" is a little bit like the terms "object" and "component," in that ESB is used for many different things. It has kind of lost a lot of its meaning. We typically don’t tend to think of Camel as being an ESB. We think of it as a rules-based routing and mediation framework that you may deploy inside an ESB.

We work very closely with the ServiceMix community at Apache which has created a complete Java Business Integration (JBI)-compliant container of JBI components. Camel can be deployed within the ServiceMix ESB among JBI components, but some people don’t use JBI and they may just use Java Message Service (JMS) or they may just use Web services or they may just be JAX-WS clients or whatever. So, we try to make Camel agnostic to technologies. You can use it within patterns like Spring, or JBI or OSGi, or you can use it within any application.

Gardner: I’ve also heard you refer to this as a domain-specific language project. What do you mean by that?

Strachan: Programming languages are often very general purpose. They can be used to solve any kind of problem. Domain-specific languages have become popular lately, and they’re basically tied to typically a business domain, such as investment banking, telco, or whatever. In Camel’s case, we’ve made a domain-specific language that focuses purely on the EIPs.

The language itself describes how to do the various things you typically do in the patterns, such as wire tap, aggregator, content-based router. These are first-class concepts within the language itself. We’ve implemented this domain-specific language in both Java and XML, so you can stick to a Java IDE if you prefer, or, if you’d rather, you can use XML to describe this language and deploy it inside your Spring XML.

Gardner: So, we’ve had vertical industry domain-specific languages. This seems to be more of a functionally domain-specific language. Is that a correct statement?

Strachan: Yes. The EIP is the functional domain, or rather the business domain.

Gardner: And, this is all about getting more granularity, more specialization for integration that nonetheless can play across a more general or horizontal capability. Is that fair?

Strachan: Definitely. Across all of IT we’re seeing increased specialization in many different areas, where the specialization helps us solve a problem at a higher level. If you spend your life solving problems, with, say, JavaBeans everywhere, you have a lot of code to comprehend and understand, and things can get quite complicated. The higher level of abstraction helps us solve problems easily.

Gardner: And, because we’re doing it an open-source environment where there’s a community involved, the more likelihood that this will be applied across many other different types of platforms and technology?

Strachan: Exactly. The other benefit of being with Apache is that we have a very liberal license, so this can be embedded in any commercial product very easily. We've seen lots of partners using this technology.

Gardner: Right. A few moments ago, we talked about ESBs. There has been, as you say, some divergence in the term ESB. There are also a number of products and approaches in the field. How about for Camel as a rules-based routing and mediation engine? Are there many others of those? Who else is trying to solve the same problem, or can we safely say that the Camel is unique?

Strachan: It depends on how wide you cast that net. Camel is unique in a number of ways. What we’re doing with Camel is defining a high-level language to describe EIPs, which I don’t think anybody else has done before. The other thing that’s unique is that this language very closely maps to components that work inside Spring 2.

We use a lot Spring 2 features, such as declarative transactions, inversion of control configuration, and various utility classes for working, with such things as JMS and JDBC and Java Persistence API (JPA). What we’re doing is raising the abstraction level to make things very simple, reducing the amount of XML we have to write, but still exposing the wire-level access if you need to do the really hard stuff and roll your sleeves up and get down and dirty.

Gardner: Let’s move this a bit closer to the business side. I wonder what sort of a problem set it is that we’re facing here. Is this a new approach to older problems or a new approach to newer problems? What's the problem solution fit here in the market?

Strachan: The problem we’re trying to address is the routing and mediation problem, which lots of people have. They're taking data from various components and sources -- whether it’s files, databases, message queues, Web services, instant messaging, or other data systems, integrating them together, formatting them, and connecting them to the systems. From a higher level perspective, this could be for legacy integration of systems, for smart routing, performance, monitoring, or testing or monitoring transaction flows.

In general, integration is a very old problem. As soon as we relate to couple of different computer programs, we already have an integration problem and that just grows with time. The thing that’s new is the approach that’s used. We're increasing the abstraction level by making it very easy to work at the EIP layer.

People don’t have to worry about the low-level details of how to use JMS, how to use JBI, or how to wire together the Spring components correctly, and so forth. We're giving people a nice, simple, high-level abstraction, but yet we are exposing all the power of frameworks like Spring and still exposing the low-level details if you need them.

Gardner: Over the last couple of years, because of the popularity and use of Web services, the business side wants to expand the interactions across not only applications, but organizations, and perhaps in a supply chain setting. It sounds like this helps solve the issue of inclusiveness across domains and businesses, and then adds the opportunity to start moving toward event-driven computing. Does that sound fair?

Strachan: Definitely. And, as soon as people start moving toward distributed systems, service oriented architecture (SOA), event-driven architectures, and all these other terms, the one thing that keeps happening is that people need a pervasive technology they can apply anywhere. Camel is lightweight, and can be used anywhere, in a smart end point, in an ESB, or a message broker.

ActiveMQ5 is coming out in [August, 2007]. That's going to have EIPs integrated in the client and the broker by Camel. ServiceMix has integration with Camel and CXF, as a project at Apache, which implements the JAX-WS standard that’s got EIP integration via Camel. We’re seeing a large number of projects that have EIP integration via Camel. It’s already proven in the open-source world that it’s very easy to reuse this routing and mediation technology anywhere. Hopefully, we'll see this just grow and grow.

Gardner: It’s curious to me. It seems like yesterday, but I suppose we're going on 10 years now, that a lot of these functions -- singular, distributed, perhaps even proprietary -- were combined into what became conceptually the application server. Now, it seems like we’re decoupling things. That one large application server approach might not be the best fit, as we get more types of routing and messaging in these integration patterns.

Can you expand on that? What's been the larger trend, in terms of architecture?

Strachan: There’s been a definite move away from J2EE apps, the big application servers, and people are looking for small, lightweight solutions that solve only the problem they have and not problems they don’t have. A lot of software vendors suffered from the kitchen-sink syndrome, where they tried to have one of everything in a big box. That just led to lots of complexity and redundancy.

People really want small and simple-to-use components that solve the problems they have. I've seen that throughout all of our customers. People ask for very specific solutions. They don’t say, "Give me a SOA." They say, "I need a message router," "I need a message bus," "I need an ESB," or "I need a services framework." Often, people have very specific requirements and are very much cherry-picking the best-of-breed components from the open-source tool set.

Gardner: So, it’s the Swiss Army Knife approach.

Strachan: Exactly.

Gardner: That works well for developers. I wonder over time how Camel might find itself being used in terms of operators, specifiers, and architects on the operational side of things. Do you see this moving in that direction? What’s the short-, medium-, and long-term implementation roadmap that you foresee for Camel?

Strachan: Today it's very much developer-focused. The current Camel is aimed at developers who are savvy with Java, XML, or Spring. What we want then is to gradually raise the abstraction level, so the architects and operational people can monitor services, monitor business processes, perform operational behavior, plus be able to design and deploy EIPs in a simple way.

For example, building tools and making it easier to design EIPs, as well as to choose a monitoring tool, are important.

Gardner: How would the specifying roadmap shake out? Do you think this is something that would get bundled in or integrated? How would this be something easily rationalized in production environments for those people that are going to be looking at large data center implementations?

Is this something that would be part of another grouping of products, other Apache kind of projects? How would this combine to make a whole larger than the sum of the parts?

Strachan: What we’ve seen over the last few years is a huge diversity in how people actually deploy products. Maybe five years ago, people assumed the world was J2EE app servers and everything would deploy in a J2EE deployment unit. These days, we see people make their own deployment units. We're seeing a lot of people move towards OSGi bundles. So we see a huge difference in the ways in which people deploy software now.

One of the reasons we’ve taken extreme steps to split Camel up and make it a very simple and easy to use bundle is that we know that there is a huge range of deployment platforms. Some people want to use Camel within their own smart end points. Some people want to host it inside the message broker, in the ESB and so forth. So, we see it going into production in many guises and many different forms.

Gardner: In IONA’s case, how would this be used in the context of its offering? I'm assuming that, as with other IONA offerings, it will be the commercial open-source support approach.

Strachan: Yes. Camel’s pretty much the core foundation for smart routing and mediation within an entire product suite. We have a product called FUSE, an open-source SOA backplane that provides a services framework, a message broker, an ESB, and a mediation and routing engine. The mediation and routing engine is based very heavily on Apache Camel, and we're using Apache Camel throughout other projects as well.

Gardner: Are there forums or community sites? How are some of the dialog and collaboration around this taking place?

Strachan: If you go to http://open.iona.com, there is a whole raft of documentation online forums, a wiki, and so forth.

Gardner: I know this is fairly early, but maybe this will be a good time to lay out some of the milestones. Are we expecting some significant developments and/or visions within the overall Camel project? Can you give us a bit of a timeline?

Strachan: It’s still fairly early for the timeline, but the next version of Camel 1.2 should be out fairly soon, and it's going to offer improved monitoring and management tools and particularly improved visualization and tooling.

Toward the end of the year, we’re hoping to improve various other capabilities we’ve added recently for extract, transform and load (ETL). It’s a way of loading data into systems and load testing systems with data. Plus, another area we’ve been developing what we call business activity monitoring (BAM), which helps monitor transaction flows across different systems.

For example, you might want to track that a purchase order in a system maps through to an invoice that comes out of another system. You might be processing a 100,000 of those a day, but there is one that goes along every few hours, and you want to find out which one that is within a small time window. That’s another area where we are extending the reach of Camel to do very complex and powerful monitoring.

Gardner: That's interesting, because this provides the actual routing mechanisms that provide a window into what's good, and also what's not good about how those processes are being built and used.

Strachan: Definitely. We’ve found that as soon as you have this very flexible and reusable integration patterns, it’s very easy to build on that, to build higher level abstractions, whether this is powerful testing frameworks, stimulation frameworks, ETL or BAM. We’re seeing the levels of abstraction increase to remove layers and layers of complexity.

Gardner: Is this something that could be applied to the ongoing -- and I imagine increasingly arduous -- task of dealing with semantic issues in terms of data across applications? That is to say, helping to define a mediation and a rules-based approach, such as this help in how companies can deal with that diversity and then come up with more common approaches to data and content that can then be injected into a business process.

Strachan: Definitely. This is a very large and complex topic. There are just a few bullet points on the top, and we’ve already got a range of transformation and validation capabilities in Camel, but we’re looking to improve this in various ways.

IONA recently acquired a company, called C24, which has a whole range of technologies for doing semantic mapping and transformation of different data formats. For example, they've got models for pretty much all the standard financial formats, like SWIFT, FpML, FIX, and all these other things. And, they've got other models for telcos. We’re looking to open source some of that and integrate that into Camel for doing lots of data service type integration.

Another area that’s kind of similar, but slightly different, is in the business activity monitoring ground, where you sometimes want to just check the effect of systems. So, it’s a little bit like I am doing real time reconciliation systems, and you’re doing all this transformation between systems, but you want to make sure of the end results, like that final purchase order or that final settlement instruction does contain the values that you expect. For example, we might want to check that the purchase order amount matches the invoice amount, or whatever.

Gardner: As you say, it’s early in the evaluation of Camel, but are there instances where we can see how it's being used or how developers have identified this as a productivity benefit, or are there metrics of productivity? How is it helping in terms of getting a better job done?

Strachan: We’ve got a bunch of different customers using Camel in different ways. We seem to have a few different kinds of customers finding Camel for the first time. Some come from an ActiveMQ background. They're doing lots of messaging, and they want something a bit more powerful to do more complex routing within the message broker itself.

Some come from the ServiceMix background. They are very ESB focused. They love JBI. They like the idea of standards-based ESB. They’ve got a bunch of standards-based integration components, but they’re working together, and they just want a slightly more powerful routing engine to work with.

Some of our customers have been using Mule. They found Camel a little bit easier, and they’ve migrated across. Finally, there are people who’ve just found Camel, that's all they know, and they just dive straight into it.

One of the things we’re seeing from customers is that using APIs, like JMS and JBI in particular, and to a lesser extent JAX-WS, could often become complicated, but people find that wiring things together with Camel is almost trivial. People get things going really quickly without having to read pages and pages of specifications for things like JMS or JBI, or dealing with the lower-level details of Spring.

We see quite a rapid productivity gain, and it means that people can then use lots of different technologies, whether it’s JMS or JBI without having to spend a long time figuring out things in the Spring manuals.

Gardner: Among these early users are there any vertical industries popping out, where it seems to be making sense first, perhaps financial services?

Strachan: Yes, finance and telcos definitely are big markets, but we’re also fairly strong in the e-tailers. Those areas will be our big three markets right now.

Gardner: Can you explain the business problems that they're using it for?

Strachan: Pretty much all of our customers have similar kinds of problems. They have lots of silos in applications. There are lots of messages being exchanged between systems, whether those are messages via flat files, email, JMS, legacy message brokers, or web services. They generally want to do one or more of the patterns, whether counter-based routing, or message transformations. We see more people starting to investigate a business activity monitoring feature of Camel, and that’s something I'm quite excited about.

Lots of companies, once they realize they can tie together messages to business processes, start running rules off the back of that. That has been extremely useful for people. One thing about SOA that’s quite difficult is knowing what’s really happening in your system.

Today, people have often done such things as monitor message queues or use a JMX-based management console to look at numbers and statistics. Being able to tie together all the different things flowing through your SOA to actual business processes is extremely useful and valuable for customers.

Gardner: There seems to be some murkiness in how feedback will occur, when you have events-driven architectures, and how to create common approaches to runtime and design time with this BAM emphasis that we are seeing associated with Camel. Do you expect that Camel could fill some of that role in terms of a balancing act between what takes place in operations and what needs to be brought into a service level agreement of some kind?

Strachan: Definitely. I have worked with quite a few customers on various BAM-related problems, and it does seem that BAM solves those problems. It’s quite hard to have an off-the-shelf tool to do everything people need. Almost as soon as people start doing any kind of BAM, they find that they want to put their own code in there. You might be in financing, and maybe there is a trade in U.S. dollars. You want to take that trade to the settlement instruction, but you want to do a foreign exchange conversion in there.

Almost as soon as you start doing the “hello world,” BAM-type scenarios, you tend to want to roll your sleeves up and actually do some coding of some kind, to do custom reconciliation, custom calculations, custom visualizations. BAM is very much a framework-driven thing, rather then a black-box tool. Certainly, we see people take on the Camel framework itself and then expand and extend it to do more powerful things.

Gardner: I suppose that’s part and parcel of open source, community-driven projects. They tend to be pushed out in a variety of directions. Do you have any sense of what other functional parts or functional capabilities we might see brought into Camel?

Strachan: A big area is visualization. It’s incredibly hard for people to follow what’s happening in general. Once you can create a wire tap into any system, it’s very easy to glean information from it and find out what’s happening, what its throughput is, what it is actually doing. You can then do things like trace messages around the system, correlate messages to business processes, reverse engineer whole business flows from legacy systems.

That whole area is a visualization and tooling problem. I see that as a very large area of research with the Camel project. We hope to be to be rolling out in Q4 the first batch of visualization tools. It's very much a work in progress in the next few years to get even better, more powerful operational controls and visualization engines.

Gardner: By getting more visual, it seems like we can bring more people into the process, and that maybe elevates us beyond BAM into business intelligence. Is that fair?

Strachan: Definitely. Absolutely.

Gardner: I think a lot of business people would like to get more of a real-time sense of what’s going on among these processes to provide more confidence, as they move processes from manual and older message into this new SOA approach.

Strachan: Definitely. Plus people want early warning notice of any kind of failure. One of the nice things is that you can be alerted that maybe something's going on that hasn't been found yet. Maybe it's running a bit slow today, or maybe it’s just a heavy trading day. But, it might not be, and maybe something is really wrong. So, rather than waiting for some extreme failure that results in a fine or extra interest being paid, you can respond more rapidly than that.

Gardner: Let’s just circle back quickly. Developers seem to be the core short- to medium-term audience for this. What sort of developers specifically should be more aware of what Camel’s capable of?

Strachan: I’d say anyone who’s doing any kind of SOA or distributed programming. They should at least take a good look. Definitely, anybody who ever read the EIP book. If you read that book, you’d probably just see Camel and you’ll immediately just figure out exactly what it’s doing. It’s very simple.

If you're doing Web services, you might need mediation. If you're doing any kind of messaging using multiple databases, if you have any kind of integration problems, and certainly if you’re using any kind of ESB, then definitely take Camel for a ride.

Gardner: Now, when we look toward getting more information on this, we have the resources at Apache Foundation, and also, as you mentioned, http://open.iona.com?

Strachan: Yes.

Gardner: Any other resources? You mentioned the book. Are there any other resources on this that people should be aware of?

Strachan: Those are the big three. There’s also my blog: http://macstrac.blogspot.com/.

Gardner: Very good. Well, we’ve been discussing a budding Apache Foundation project known as Apache Camel. It’s a rules-based routing and mediation engine for POJO-based implementation of EIPs.

Discussing this with us today, we had James Strachan, technical director of engineering at IONA Technologies.

I'm your host and moderator Dana Gardner, principal analyst at Interarbor Solutions. Thanks for listening and learning more about Apache Camel.

Listen to the podcast here. Podcast sponsor: IONA Technologies.

Transcript of Dana Gardner’s BriefingsDirect podcast on Apache Camel. Copyright Interarbor Solutions, LLC, 2005-2007. All rights reserved.

BriefingsDirect SOA Insights Analysts on IBM’s Telelogic Deal and Open Source ESBs

Edited transcript of weekly BriefingsDirect[TM] SOA Insights Edition podcast, recorded June 15, 2007.

Listen to the podcast here. If you'd like to learn more about BriefingsDirect B2B informational podcasts, or to become a sponsor of this or other B2B podcasts, contact Interarbor Solutions at 603-528-2435.

Dana Gardner: Hello, and welcome to the latest BriefingsDirect SOA Insights Edition, Vol. 20, a weekly discussion and dissection of Services Oriented Architecture (SOA) related news and events with a panel of industry analysts and guests. I’m your host and moderator, Dana Gardner, principal analyst at Interarbor Solutions.

Our panel this week consists of Jim Kobielus, principal analyst at Current Analysis. Welcome back, Jim.

Jim Kobielus: Thanks, Dana. Hi, everybody.

Gardner: We’re also joined by Todd Biske, he’s an enterprise architect with MomentumSI, an Austin, Texas consultancy. Thanks for joining, Todd.

Todd Biske: Thanks Dana.

Gardner: Joining our show for the first time is Brad Shimmin. Brad is also a principal analyst at Current Analysis. Welcome, Brad. Tell us a little bit about your area of coverage.

Brad Shimmin: Thanks for having me, Dana. I focus on application infrastructure and spend most of my time thinking about middleware and SOA.

Gardner: Terrific. That’s the right mix for our discussion. We’re here to discuss some news and events for the week of June 11^th, 2007. I attended the Rational Developer Conference in Orlando, Florida this week. It was a little bit warm down there, but a busy time nonetheless. There was lots going on within the Rational Brand Division, within the software group, and within IBM. I came away with some observations I'd like to share. Have we been joined by another guest?

Dave Linthicum: It’s Dave Linthicum.

Gardner: Thanks. Also joining us today, Dave Linthicum. He's the CEO of Linthicum Group. Good to have you, Dave.

As I was saying, we’re just getting into our topics, and we’re going to look at some of the IBM news this week that came out of the Rational Division. There was the intent to buy Telelogic, a Swedish firm that’s got a lot of product across requirements, tests, QA, architecture and modeling, as well as embedded and system development. So, we’ll talk a little bit about that.

We’ll look at some of the announcements out of the Rational Developer Conference, including the Jazz Community, an innovative commercial/open-source community approach to development. Also, one of the more interesting product announcements was Rational Asset Manager, essentially a design-time metadata repository that can be used in conjunction with an operational or run-time registry and repository for a lifecycle approach to services.

What’s more, we’ll go around the table and look at what research folks have been conducting this week. That might include a look at WS02, the IONA Artix announcement, and some announcements also from BEA.

So, let’s start with the Telelogic acquisition. Telelogic is a publicly held company in Sweden, and IBM had to jump through hoops in order to acquire this company. They also announced it at their developer conference, which to me was an indication that this timing was not entirely their choosing.

Why don’t we start with Jim Kobielus? IBM and embedded -- we haven't seen anything along those lines for a while. They jettisoned the Rational embedded drive before IBM had acquired Rational, but now they seem to have a new-found interest in embedded. That's increasingly focused on end-to-end development, recognizing that the entertainment and media sectors are going to be creating devices different from a personal computer. IBM doesn’t want to miss out on the opportunity to leverage its back-end systems vis-à-vis these new types of devices. Any thoughts, Jim?

Kobielus: Clearly, IBM over the last few years has made stronger moves into the appliance space and really helped to define it. Obviously, with the DataPower acquisition a few years ago and increasingly taking their data warehousing appliances -- I think they’ve already revamped that entire product family -- they can scale from very small, or relatively small, data warehouses to very large ones.

IBM is trying to fuse their chief software products and technologies with their hardware engineering expertise to develop products for various markets. Embedded operating systems, embedded application components, and so forth, are key to all that.

So, when I saw that they were acquiring Telelogic, it made perfect sense. IBM is very much bringing together the hardware and software worlds into appliances, both for the business and potentially for the consumer market.

Gardner: They mentioned synergies several times. Synergies would include the silicon. You know IBM is big with Power PC. They also have this new Cell family of customizable, modularized silicon designs. So, there’s an opportunity for them to go to the embedded market with an optimized silicon and platform approach. Of course, Rational is in the business of helping developers be productive across lifecycle and to manage development, not so much in the tools category itself.

I think they see a wide-open opportunity in embedded that is still a fairly wide-open, roll-your-own test, roll-your-own tools, even roll-your-own IDE or real-time operating system. So, I wonder if anyone else has some thoughts on the embedded angle on this?

Shimmin: Well, Dana, not so much about the embedded angle, sorry to say, but perhaps something that might transition us into talking about Rational. When I look at what IBM is doing here -- and this is pursuant to what Jim was just saying about their synergies between hardware and software -- I see IBM doing two things.

One is renewing the focus on software development on the design time and development time side of things. Then, I see them taking their expertise in hardware and putting the two together to build what a lot of the software companies or platform companies really wish they had in the space, and that is the ability to do two things: create well-performing software and actually have software that performs well in the production environment. They’re going to be pretty well positioned to take advantage of those two things.

Gardner: That’s interesting because that fires high-level observation I made between the Rational Conference and the Impact Conference around SOA innovation just several weeks earlier. That was that IBM is anticipating the change in the definition of applications and systems. It is really focused on trying to get very vertical in terms of expertise about businesses in verticals, align that with their systems, and then allow for some level of consulting that joins these, whether it’s their consulting or someone else’s.

They seem to be involved now more and more with getting very close to the application, almost to the point of being in a position to be dominant in terms of custom application development. Do you have any sense from your perspective, Brad, about the whole notion of the shift in the definition of applications and how to create them?

Shimmin: Absolutely, Dana. It’s funny. Both IBM and Oracle are the two companies leading the chart with what you just said there. They’re taking tentative baby steps, because this is a pretty daunting task they’ve undertaken. That is, as you said, dive deep into a given vertical and a given set of business processes within that vertical to provide literally out-of-the-box functionality. You’ve got your data models. You’ve got your actual BPEL processes. You’ve got everything that surrounds whatever it is you need to actually get some sort of process up and running within their environments.

Maybe five years ago or so, platform vendors really focused on the foundation infrastructure of what these applications run on. They’ve realized in maybe the last six months that the success that they’re going to enjoy is going to come from their ability to save their customers money from the exorbitant consulting and professional engagement fees that usually come hand-in-hand with rolling out software like this.

So, what they're doing is pretty cool, and I applaud them for it. As I was saying a second ago it’s a pretty long row to hoe and it’s going to take a lot of energy on Oracle’s and IBM’s part to actually fulfill this and to say, “Okay, we’re going to give you a set of business processes with all the data you need to set them up and get them running for your vertical.”

Gardner: I think they’ll try to come downstream to a certain level and then hope that there’s an ecology of other providers that are even more in the weeds of vertical by vertical that they can help support and become friendly with. Right?

Shimmin: I think IBM is actually leaning that way and they have a very strong ecosystem already in place.

Gardner: Hey, Todd Biske. Telelogic also has an enterprise architecture product. Have you evaluated that or been familiar with it at all?

Biske: I have not evaluated it myself. Momentum’s CEO, Jeff Schneider, said maybe we’ll see some more activity on that product. It’s interesting, because I’ve yet to run into an organization that’s really leveraging some of the EA-specific products that are out there. They still tend to do 90 percent of what they need to do in Visio, rather than some of the specialized tools in that area, but, as the discipline matures, we’re going to see a lot more of that.

I want to talk a little more about this notion of it bringing IBM closer to the application business that you first commented about in your blog. At least, that was the first time that that was brought to my attention.

I have a friend from college who has worked in the embedded space pretty much since we graduated, and he contacted me recently about applying SOA to some of the work that he was doing. It surprised me a little bit, because it’s not a space that I’ve had to deal with. I’ve typically been in big IT and big enterprises. It opened my eyes that their environment for developing is maturing as well. They are getting to higher levels of abstraction and taking advantage of the same types of programming models a typical enterprise developer is.

I thought they were going to be so focused on performance and real-time behavior of these systems, that they may not have a strong interest in some of the things that Web service standards and XML have to offer, because of the tradeoffs from a performance standpoint. He said, “No, we are looking at all of that.”

Now, with IBM making this acquisition of a company that’s dealt in the embedded space, it really shows that development is still development. IBM is now recognizing that it’s not all just about “build whatever you want.” We are getting more specialized, and maybe the right way to get into the applications market is to create specialized tools for particular vertical domains, rather than providing the applications themselves.

It’s definitely something to pay attention to, as we go along. They did it on the tool side; they did it on the software side with the Webify acquisition last year; and I would guess that we’ll continue to see more in this direction.

Gardner: I suppose if you get very specialized in managing the process of creating applications at that very fine level, then that’s as good as creating the applications themselves. Let’s go to Dave Linthicum. Any thoughts about the Telelogic acquisition?

Linthicum: What’s missing in the space is a holistic design tool around SOA. I looked at Telelogic at the EA Conference in New Orleans a couple of months ago. I was impressed with it, but it didn’t have a lot of the components it needed to drive an SOA. I think IBM saw the potential for providing not only a design tool, which they have with the Rational stuff that you mentioned -- basically development lifecycle -- but also a holistic architecture tool to deal with artifacts and requirements and all those sorts of things that Telelogic does, and they’re looking to connect the dots. If they do come out with a holistic SOA-oriented design tool built around their technology, they’re going to have a huge hammer to beat into the market.

Gardner: Interesting. Did you get a chance to take a look at this new Rational Asset Manager, and what did you think of that?

Linthicum: I did look at it from the online piece, and I think that it’s going to have value in this space as well. The folks at IBM are not dumb. They’re out in the back, trying to figure out how all this stuff is going to fit together. They want to have not only the mega-stack in terms of deploying technology and development technology, but the mega-stack in terms of the design time stuff, including holistic enterprise architecture, asset management, service management, and SOA governance.

So, it’s going to be very difficult not to see IBM in almost all the larger SOA implementations out there, once they have a critical mass of tools. They’re investing right now. They see this as a long term strategy and a way to gain revenue 10-15 years down the line. I think they’re making some smart moves. I would have acquired Telelogic as well, if I were IBM.

Gardner: It seems that $745 million isn’t heck of a lot, given what companies go for these days. It seems like they got a pretty good deal.

Linthicum: Oh, it’s a bargain. They did very well buying it, and they’re going to reap a lot of benefits from it. This is the right move from IBM and the investors are going to love that three or four years down the line.

Gardner: One of my takeaways was that Rational, as an entity and a functional component within IBM, is rising. It’s really bringing together some of the other brands, including Tivoli and the SOA and WebSphere activities. There’s also a role for Lotus with collaboration among and between developers. So, it’s interesting that Rational, which seemed like an odd man out a few years ago, is really becoming somewhat of a mortar between the IBM brands, and also gives them more interception points in these accounts. That is to say, they can work with OEMs, now that they’re in the embedded space. They can work with ISVs through Rational. It just gives them more traction points to pull other aspects of their value into play.

Kobielus: That’s exactly what I’ve seen too, Dana. Rational is becoming the crown jewel within IBM. In my area of focus, master data management (MDM), the Rational tool has become the primary master data modeling and domain modeling tool for all of IBM’s MDM products. I agree. It was probably their most important acquisition in the last 10 years.

Gardner: Just for an element of balance, there is significant product overlap between Telelogic and IBM Rational, particularly on requirements, gathering, and management. However, they took pains during their press conference and analyst discussions to say that there is product overlap. They’re bound by the laws in Sweden not to get too specific about what they plan to do, but they say that there is very little market overlap. Where IBM plays in requirements and where Telelogic plays in requirements are quite different. So, I’ll be curious to see how they do, let’s say, RequisitePro, a popular Rational tool, vis-à-vis the other aspects within Telelogic.

One last item in the IBM news. They announced this Jazz community, jazz.org, and it’s essentially an open environment, which people can join and help contribute to the development of Rational products. This is somewhat of a trial balloon with IBM saying, “Wow. Look, how successful Eclipse was as a governance environment and a community development force in the market. How can we take what was good about Eclipse, but apply it to commercial product development, not just open-source development?”

It strikes me that if the companies who partner with IBM that have a vested interest in how their products relate to the Rational products contribute and help define the Rational products, then the same model could be applied to other commercial aspects within IBM, and they could then perhaps even take the model to other products. Has anyone had a chance to look at Jazz? Do you think that this is a wacky idea, or do you think it will get traction?

Shimmin: I looked at it a little bit. This is not the first time this has been done, and it certainly won’t be the last. As you said, they saw the success of Eclipse and they saw that it was an environment that fostered innovation. As we all know, it’s very hard for large closed-source vendors to innovate quickly, while maintaining a customer base accustomed to once-a-year big upgrades, punctuated with little patch here and there.

I look at what IBM has done, as well as TIBCO, Sun, BEA, and Red Hat -- even though they’re not closed source -- and I see them using the tools that the open-source community fostered in order to collaborate over a large-scale network of developers, and they’re applying it, just as you said, to closed source. There are many benefits to that.

First and foremost is a quicker turnaround on bug fixes and getting to a GA. When you’re dealing with the traditional closed-source development cycle, you build your software, you send it out to maybe 10 trusted customers. They hammer on it a little, and you have your own internal people hammering on it, and that’s maybe a three month venture.

Using 10 customers, who have their own jobs to do and don’t give this a lot of shrift, sets you up for failure. That’s why we see so many post-release patches going out. What this is going to do, if it succeeds and can be applied to closed source, is let these large vendors get their code out quicker in a much more tip-top, enterprise-ready fashion.

Gardner: I suppose it also allows end users to have more of a say in what they’d like to see in the products that they buy or use -- how they evolve.

Shimmin: Absolutely. IBM is taking a tiered approach to it, and some of the others have too. As we were saying with these trusted customers and partners, partners in particular are going to play a big role in this, but they would get access to source code at deeper levels. Folks that maybe are smaller customers or just interested parties, who want to make this product go forward, will have more limited access, unlike a traditional open source. They’ll have more limited access to the details of the source code.

Gardner: As I thought about this, it seemed to me that there has been an enabler in the market that’s allowed this to take off more than it would have in the past. That’s because of companies like Black Duck and Palamida that are able to evaluate code very quickly and determine its origins. IBM seems to be a little bit more flexible about opening the code up for people to look at and use. They’re assured that if it were to go into some commercial production, they’d able to find out and put an end to it, or at least get it into their royalties and licensing business. Do you think we’re at the point now where we can manage code to such a level that this permeability around the use of code is much more open and free?

Shimmin: Should we be thanking Microsoft, Novell and SUSE for that a little bit?

Gardner: Well, bringing it to people’s attention that it’s not black and white. It’s grey, right?

Shimmin: Absolutely.

Gardner: Anybody else have thoughts on Jazz community development in a commercial production effort?

Biske: It’s an interesting idea. I wonder how different it is from some of the efforts already going on. Clearly, we have commercial efforts built on open-source products. The key question here involves open-source products that are not available for free. If developers are working on it and they can build it and use it, how is that model going to come together if they say, “No, you only have a license to run it in a development mode and nothing more than that?” Is that going to be followed or not? Are they going to bother to enforce it, or do they really know in the long run it’s going to take the same direction that Eclipse did.

IBM has a commercial version of Eclipse, but largely people just go with the free product, because they can still include all the plug-ins that they need to. If they need to buy add-ins to it, they’re okay with that. So, they can focus their attention on Eclipse and create a framework, and can plug-in commercial components as they need to.

The other risk that they take is that the community is just going to look at this and think they’re just looking for free work. trying to take advantage of developers who just love to code and could care less whether they’re getting paid for it or not.

Gardner: I guess it all comes down to incentives and motivations. It might propel people to “donate” to an open-source project, similar to the incentives and motivations that drive them to contribute to a proprietary closed commercial environment.

Shimmin: There are no selfless acts. Right, guys? When I scroll some message boards for these development efforts, I see people in enterprises saying, “I need access to this API because I need to extend the product to work with something I’ve built in-house.” I think that’s the kind of work that’s going to drive us forward.

[[[Speaker:]]] It’s the same phenomenon as the developers who are employed by a vendor like IBM. They quite often work excessive unpaid overtime, just because they’re committed to their jobs, their products, etc. In a sense, now you’re roping in the partner ecosystem as well. They’re putting in essentially unpaid overtime to help out the mother-ship vendor get its products debugged and developed.

Gardner: If I were an ISV in the Rational universe or ecology, and I had an opportunity to have a say in what their products did or didn’t do, or if I could contribute some minor hooks, that might greatly benefit my business, I might be very motivated to do that.

Okay, let’s move on to some other topics. WS02 announced ESB 1, which is largely based on the Apache Synapse ESB. I want to make a disclosure that WS02 is a client of mine, and we should consider that as I present comments. I wonder if anyone else took a look at this, and had some thoughts on, “Wow. Yet another ESB and yet another open-source support maintenance business model entrant in the SOA ecology?”

Shimmin: Dana, I talked to them briefly about this before they released it. Like you, I saw this as, “Oh, yes, here’s another one, and maybe Red Hat should worry.” But, I don’t think anyone else is going to worry, except maybe the pure play ESB vendors like Cape Clear.

They’re focusing on what everybody in this space is trying to focus on, performance. Everyone has realized that ESB is at its level of maturity. You need to really be focusing on availability, reliance, reliability -- the “ilities” -- of deployment. This is the third vendor in the last month -- this would include IONA, Cape Clear, and WS02 -- who has talked primarily about the performance of their ESB and their ability to parse through messaging in a very highly scalable manner.

Gardner: They also seem to feel that they’re coming from a pure Web services heritage, without trying to drag a legacy business model into the mix. Therefore, their ESB is more ecumenical. Does that make sense?

Shimmin: I got the same vibe from them too, but I feel as if every vendor with an ESB these days feels the same way. They realize that, as with databases, you’re going into a heterogeneous environment regardless, and most likely inter-departmental, inter-company you’re going to have multiple ESBs and different messaging platforms that need to interoperate.

Gardner: Let’s take that to Todd Biske. He’s an enterprise architect. On one hand, choice is good and on the other hand, choice is bad. Are you are getting too much choice when it comes to ESBs?

Biske: It doesn’t bother me. I’d rather see a lot more in the open-source space. They’ve got the freedom to keep it more focused on some of the target areas. In the case of WS02, they really are focused more on what I call the middle capabilities, rather than on service development and execution capabilities. You see a lot of the commercial ESBs going in the direction of giving you an orchestration platform and a composition platform. All of it is about building new services, and not about connecting existing consumers and existing service providers.

Some of these open-source ones are keeping it a little bit more constrained and targeted. Now, with the open-source model, if an enterprise needs to augment that for their particular needs it gives them the ability to do so. I’ve run into a few clients who are looking at some of these products and have a potential need to do that. The openness is a plus for them.

I don’t necessarily see it as too many ESBs out there. The market naturally will shake them out on its own. This is just the way these product spaces work. I don’t view it as an SOA bad thing.

Gardner: As a system integrator, you must feel pretty good about high quality open-source code coming around. Doesn’t that suit your business model pretty well?

Biske: Absolutely. At Momentum, we’re vendor neutral, and I know Dave talked in a recent podcast about the importance, when dealing with system integrators, of having one that is not closely tied to a particular partner, unless your company has already decided that you’re married to this particular vendor. Then, it makes sense.

If you’re looking for your pure systems integrator, you have got to select solutions that are in the best interests of the client, not the best interests of the consulting company or the partner ecosystem around this. Having open-source products gives us a lot more flexibility in meeting the needs of the clients.

Gardner: Dave Linthicum, do you want to weigh in on this?

Linthicum: I don’t put as much value as everybody else does into the open-source equation. In fact, my client base -- and it’s Global 2000 and government folks -- are indeed buying ESBs and other things based on the notion of having access to the source code. I just can’t imagine in my lifetime that they would ever want to become a product development vendor and would have the skill set to actually maintain a middleware product, having built a bunch of those in my lifetime. I’m a little skeptical about the ultimate value there. To me, open-source needs to have marketing value. I’m not sure it’s going to have a lot of technical value going forward.

My larger concern with the number of ESBs out there is that, in many instances, these have a tendency to be queuing systems with service interfaces on top of them. Therefore, they’re more information- or data- oriented than transaction-oriented. That has a tendency to limit some of the emerging patterns I’m seeing within SOA. People are looking basically to automate these high-end business transactional systems well beyond just data consumption and production.

Most of the ESBs don’t really address that. They basically become queuing systems with a nice interface on them. To Todd’s point, they do have orchestration layers and other development technology. Some of the higher ESBs out there definitely have that capability. That seems to be nice, but it still seems to be limited by the underlying infrastructure that they’re selling.

I’m concerned about the number of the ESBs in the market, and I’m concerned about the ability of those ESBs to deliver the ultimate value as the SOA we’re building becomes much more sophisticated.

Gardner: Any other thoughts on ESBs before we move to our next subject? Let’s go to Brad. He is the new guy on the block. Let’s pick on him a little bit. What did you write about this week, Brad, and share a little bit about your insights if you wouldn’t mind?

Shimmin: This week I looked at two things primarily. One was the release AmberPoint made regarding their SOA validation system and their SOA management system. They have two products and they just versioned them to 6. I found what they were talking about very interesting, because, as I was just saying a second ago, they too are focusing on performance.

What they’ve gotten at is their desire to be in a run-time environment, because, as you guys know, they are like the Switzerland of SOA run-time governance in our industry. They are focusing on being able to scale their platform, and they have a number of partnerships and potential partnerships with hardware manufacturers, going back to our earlier discussion about IBM and their DataPower acquisition.

They seem to have this idea of, “Well, we’re run-time governance only, and we have the capability to be design-time governance as well, but we’re not going to get into that space.” I found that interesting, because they’ve just announced this as a part of this pre-flight check that they do. This is similar to the type of the service that IONA launched with their Registry Repository, in which when you check your WSDL into the Registry Repository, it actually goes out, looks for dependencies, etc.

In the AmberPoint product, but a little bit more predominantly positioned, they actually go look at that WSDL as it relates to other processes across the entire application and looks for any interdependencies, broken relationships, and anything within the production environment that may cause a problem.

I think about companies like CA, HP, and IBM, who all really are trying to come at the same problem, but from the design and development side, and I think of AmberPoint coming at it from the run-time side. I feel like, “Well, why don’t you guys just get together? Let’s put these two notions together and make it so that we have a more coherent lifecycle management of our codes for SOA implementations that starts in design and ends in run-time.”

Gardner: What’s going to fill that middle role or the gap between these run-time and design-time capabilities, so that we get that feedback loop, perhaps even an automated approach to optimizing and assuring quality and reliability. Is that going to have to be wetware? Will people fix that problem, or can it be productized?

Shimmin: It’s a combination. When I talked to AmberPoint about that very question, Dana, we were talking about it with regards to this throttling technology they have, where they can look at PKIs and SOAs that have been defined inside the Registry Repository and tell the process to slow down on Thursday afternoons, for example, because I need these other process to take priority on that date and time.

I said to them, “Well, gosh, I know there are a lot of tools out there at designed/development time – a lot of BPM products for example -- let me establish these and stick them in a registry. You just pick those up automatically and start executing them.” They said, no, because the schema and the amount of data that comes from this is not really enough to do it.

Gardner: These are just different orbits, right?

Shimmin: Right.

Kobielus: One of the glues, of course, is a common registry and repository infrastructure between the design-time and the run-time environments, but just as important is the wetware, as you indicated, a common governance environment with roles and workflows. People who are doing the design and optimization of the Web services and the people who are administering the services in a run-time can be collaborating on a ongoing basis.

The common rules and policies of this common infrastructure, be it AmberPoint on the run time or tons of other vendors on the design time, they can all share. So, it’s a bit of the registry and it’s a bit of the governance human administrative workflow.

Biske: It’s interesting that you bring that up, Jim, because one of the things that I wanted to come back to was the Rational Asset Manager. It seems to be typical of IBM that in their SOA offerings they’ve got three of everything. If the registry repository is the key, and I agree with you that this is really the unifying component on some of these things that are dealing with policy on the metadata, they’ve got Rational Asset Manager, which is effectively a metadata repository, and then they’ve got WebSphere Registry Repository, which is another metadata repository.

I’ve had conversations with them going back a couple of years on this subject, asking “How are you going bridge those, and what’s Tivoli using on top of this as the common metadata storage for all this?”

Brad hit on the other point, in his conversation with AmberPoint, that it’s not the fact that you have a metadata repository out there, but it’s the information that’s going into it. Until there’s some standard level of policy domain languages that these tools can leverage, you’re going to still see people just building their own fiefdoms and saying, “Well, you’ve got to have either AmberPoint everywhere or HP everywhere -- or whatever your management system of choice is -- to be able to do some of these things, like throttling across systems, and some of the run-time policy enforcement. It does need to bridge all the way back into the development tools.” So, again, IBM’s in a great position, providing products in all of those spaces, but it’s going to be difficult to pull off.

Gardner: Thank you. Now, you brought up an interesting point, which is that they would love to be able to have that chokehold. So, obviously, vendors have some vested interest in their own business models of making this less heterogeneous than other aspects of the environment -- this ability to create the feedback loop, manage exceptions and change, and optimize for performance and design. What do you think about that, Dave Linthicum? Is that possible or is it too late for them to do that, given the general heterogeneity and nature of SOA?

Linthicum: It’s possible for the newer offering, but they will have some pushback, given the fact that every domain is extremely different. One challenge that people have, when they try to get out in the market with this kind of stuff, is that ultimately what they think they’re implementing is different than what’s actually being implemented. I see a huge chasm between the perceptions.

For example, I was at the Gartner event this week to do a talk on ROI, and I got a chance to wander around and talk to a number of the people who are pushing in the market, both customers and users. There’s a very different perception as to what vendors think the problems are and what the problems are that users are actually experiencing. There’s going to be a bit of a sobering [[[-- come to Jesus --]]] that’s going to happen over the next year or so, when these guys push out there.

Gardner: Where are the two camps in terms of the difference between what they think they’re changing?

Linthicum: In the user community, all the problem domains I’m seeing, definitely in my client base, are unique. There doesn’t seem to be any one set of solution patterns you can apply across the board. You see bits and pieces of a stack and how simplistic those problem domains are. The vendors don’t see that when they design SOA in general. They typically give you the same stack and the same problem description. I’m not seeing a consistent problem description out there to work with the client.

Gardner: Well, of course, you can understand the perspective of the vendors. In order for them to have a volume business and repeatability and automate, they’d like to be able to take one hammer for all nails.

Linthicum: You can’t do that, and that’s the problem people are running into right now. I saw the same thing back in the integration days, back in the AI days. We tried to take one problem domain, all the spaghetti code, and put it through a single hub. While that was applicable to a small percentage of the problem domains, it wasn’t widely applicable. It wasn’t applicable to all problem domains. SOA is even more complex than that. Vendors are going to find that they’re missing the boat in terms of understanding the needs of the people they’re trying to serve.

Gardner: Once again, we come up against these issue of technology and people, where people are much better at matching nails to specific hammers or types of hammers, so that technology can run with it once they’ve established the proper relationship. SOA again becomes very people intensive. You need to be in the weeds to understand the business, and specifically you need to understand the particular company.

Linthicum: Dana, it’s all about people. As I’m getting further along in this stuff and learning more about it, I find that the people issues are really the core of all this all. I can solve any problem with technology, and probably everybody in this space can do the same thing. However, getting people aligned with how that’s going to happen and setting them up for success is the ultimate challenge right now, and the vendors need to understand that.

Shimmin: Back to our example of AmberPoint. If I have a nice BPM tool that lets me find my PKIs and SOAs, and if it’s not in my best interest to include that additional data the additional artifacts that AmberPoint is going to need to make that throttling automatic, why am I going to do it? I won’t.

Gardner: This is interesting, because I hear two different things in the market as well. On one hand, there’s a recognition that you can’t get the labor you want. You can’t get it on the continent that you want it on. And, people are trying to find ways in which to reduce the number of people, because they’re costly and they’re hard to hold on to. Yet, what we’re hearing here today, and I agree with it, is that we’re actually entering a phase where more people, with more specific knowledge and talent, are going to be required, and they’re going to be required onsite, not necessarily doing this through instant messaging. Any thoughts on that?

Kobielus: That explains why I’m seeing more emphasis in the SOA space on pre-built domain models, essentially solutions that package up the rules, the best practices templates, the workflows, the policies, and so forth, for a particular problem domain, be it in the MDM space or be it in the ESB space. The customers are demanding these accelerators so they don’t need to hire people who are smart enough to build all that stuff from scratch. If the vendor and their partner ecosystem have already frozen all that expertise into the solution, the customer can be productive from day one. So, the customer could have a little bit longer to go find the appropriate smart people, wherever they happened to be, whether in Indonesia or Chicago.

Shimmin: That’s why you look at the Rational and Jazz announcements this week and you see the reality that IBM is seeing, which is that, although we want to have this deep knowledge that’s in-house, that’s not likely to happen. You need to go where the talent is. So, the software is hopefully bridging those gaps.

Back to what you were saying earlier, Dana, about even Lotus being able to play in this, I think what’s going to become a much more predominant paradigm is that sort of telepresence for the entire life cycle of software for all involved in that.

Gardner: Well, let’s just hope the human resources people don’t get too involved.

Biske: I have a little bit different take on this. I still feel that it’s not that we need better developers, but that the technologists need to become more business aware and more business savvy.

There are a lot of businesses that may be looking offshore for a lot of these efforts, strictly from a cost reduction effort, but that doesn’t necessarily mean they’re going to get any better technical solutions than they would have with resources in-house. It just may mean that they’re going to get it less expensively, and even that is debatable.

So, they haven’t really improved things from that standpoint. In terms of business agility, they’ve reduced their cost, but is the IT solutions actually helping the business any more than it did before when all the work was being done internally? The only thing that’s really going to help push it along is to get people who are both knowledgeable about the business, as well as knowledgeable about the technology, and being able to bring those two worlds together.

Shimmin: I don’t think BPEL is the only way to do that, but that seems like that’s all vendors are focusing on right now.

Gardner: I’m a little bit concerned about that. I had this conversation with Sandy Carter at IBM about how they’re looking for “T-ish people” who have horizontal business acumen and understanding and then a vertical stem around deep technology. I’m thinking to myself, these are fundamentally different kinds of thinking. You’ve got right-brain people; you’ve got left-brain people. Most people favor one or the other. There are not necessarily going to be very many who are very good at both.

Maybe we should be thinking about this as small pods or teams, where you’ve got a business person who is savvy, you’ve got a technologist who is savvy. And, then you’ve got a facilitator, a person who is very good at motivating and communicating, and create three-person pods to approach this, rather than think you’re going to get it in just one individual.

Kobielus:[[[???]]] Right, because these are all separate domains of complexity. You can be really astute on business issues, if you focus on that and you continue to refresh your understanding and the nuances of all of that day in and day out. Likewise, all the technology areas are themselves entirely stand-alone spheres of complexity. Imagine one person trying to juggle those different spheres of complexity all day, every day, and do a good job of it. That’s really hard from a wetware perspective.

Shimmin: Do you guys know the notion of extreme programming? The idea is to have these micro teams with two people who are always working on a given aspect of a project. Why not have one of them be the IT technologist and the other one be the business analyst?

Gardner: There’s an opportunity here for someone like a McKinsey to come in and start analyzing the organizational dynamics of approaching SOA, what sort of teams should be put together, and what sort of people should have certain skills. This whole notion of one person doing it seems to me as farfetched. These teams could be much more capable and much more distributed. You could push them into different activities within this problem set and they won’t necessarily have to be physically there.

Biske: Companies that have started to practice user-centered design and some formal usability practices are probably in a much better position. One thing you find in doing that is that you immediately have to get out of this customer-supplier relationship and into a team environment, as you describe, for developing solutions for business users. They’re part of the team. They’re not a customer.

Pointing to other potential groups, someone like Patricia Seybold Group, and their focus on customer innovation, some of those concepts really need to be brought in here to stop viewing IT as a supplier to the business and instead as a partner and working from a team standpoint.

You’re absolutely right that the T can be built by creating a team, rather than looking for one superstar individual that understands it all, because there aren’t too many of them.

Gardner: Well, I’m afraid we’re out of time, but I think we’ve stumbled into a topic that I’d like to pick up again in another show. That’s the organizational approach and the relationships between customers, supplier, integrator, and how they come together, or not, and what we need to do to come up with some new approaches around that. So, thank you everyone for joining. Once again, we’ve had Jim Kobielus, principal analyst at Current Analysis. Thank you, Jim.

Kobielus: My pleasure.

Gardner: Todd Biske, enterprise architect with MomentumSI. Please come back, Todd.

Biske: Thank you.

Gardner: Dave Linthicum, CEO of Linthicum group. Always good to have you, Dave.

Gardner: Also, a newcomer, who we hope he will become a regular, Brad Shimmin, also principal analyst at Current Analysis.

Shimmin: It’s been a pleasure everyone, thank you.

Gardner: This is Dana Gardner, principal analyst at InterArbor Solutions. You’ve been listening to BriefingsDirect SOA Insights Edition, Volume 20. Come back again next week. Thanks, everyone.

Listen to the podcast here.

Produced as a courtesy of Interarbor Solutions: analysis, consulting and rich new-media content production. If any of our listeners are interested in learning more about BriefingsDirect B2B informational podcasts or to become a sponsor of this or other B2B podcasts, please fill free to contact Interarbor Solutions at 603-528-2435.

Transcript of Dana Gardner’s BriefingsDirect SOA Insights Edition, Vol. 20. Copyright Interarbor Solutions, LLC, 2005-2007. All rights reserved.