BriefingsDirect SOA Insights Analysts on Defining SOA and IBM's DataMirror Acquisition

Edited transcript of weekly BriefingsDirect[TM] SOA Insights Edition podcast, recorded July 20, 2007.

Listen to the podcast here. If you'd like to learn more about BriefingsDirect B2B informational podcasts, or to become a sponsor of this or other B2B podcasts, contact Interarbor Solutions at 603-528-2435.

Dana Gardner: Hello, and welcome to the latest BriefingsDirect SOA Insights Edition, Volume 22, a weekly discussion and dissection of Services Oriented Architecture (SOA) related news and events with a panel of industry analysts and guests. I’m your host and moderator, Dana Gardner, principal analyst at Interarbor Solutions. Our panel this week, and this is the week of July 16^th, 2007, consists of Joe McKendrick. Joe is a research consultant, columnist, and blogger. Welcome back, Joe.

Joe McKendrick: Hi, Dana, glad to be here.

Gardner: We’re also joined by Tony Baer. He's a principal at OnStrategies. Hey, Tony,

Tony Baer: Hey, Dana.

Gardner: We're joined once again by Neil Ward-Dutton, research director at Macehiter Ward-Dutton. Welcome back, Neil.

Neil Ward-Dutton: It's great to be here, Dana. Thanks for having me.

Gardner: Also, filling out our roster -- and we have a large group today -- Todd Biske. He is an Enterprise Architect at MomentumSI. Welcome back, Todd.

Todd Biske: Good morning, Dana.

Gardner: Also, Dave Linthicum, CEO of Linthicum Group. We’re glad you’re here Dave.

Dave Linthicum: Glad to be here, Dana.

Gardner: Last but not least, Brad Shimmin, principal analyst at Current Analysis. Good morning, Brad.

Brad Shimmin: Good morning, Dana.

Gardner: Our topics today are, on one hand, a large amorphous topic, which is to try to once again define and categorize SOA -- what it means and where it's going. Then second, a bit more refined, direct topic, and that is the recent acquisition by IBM of DataMirror, or perhaps the intention to acquire DataMirror, and what that means for services, the data-services layer, and the inclusion of more data for IBM in their Information Server Suite approach.

Let's start with the larger topic of the day. What's been coming up in some of the blogs and in some analyst reports is this notion of people overusing the term SOA categorically, and the suggestion that we should go back to the roots and get pure again about SOA. Some are saying that this other business around it being application types, application support, runtimes, and actual products and platforms needs to be muted a little bit.

At the same time, we’re also seeing discussions crop up around such things as platform as a service, particularly around the Salesforce.com ecology and approach, and also integration as a service. I recently did a sponsored podcast with Cape Clear Software on this topic. Then, of course, there's the the long-term discussion around software as a service (SaaS), and the ability to use Web services and a variety of application interfaces and APIs to avail yourself of hosted online applications and services.

So, let’s take this out to the group. Are we confusing topics here? There's this notion of things off the wire, outside of the domain of the enterprise. Should that remain separate and distinct, as a discussion point and as a direction for enterprises, from SOA, which is more of an architectural initiative and philosophy within the confines of their enterprise IT systems.

Let's go to Todd Biske first, because, Todd, you're an enterprise architect and you’re working with clients in the field. How do you come down on this? Is SOA inclusive of all of these "x as a service" or "blank as a service" or are they actually part and parcel of the same direction?

Biske: I really think it should be inclusive of all these external platforms. SOA, in my opinion, is really something that’s driven out of enterprise architecture. To go back to the simplest view, it's about the services. If you are able to get services from outside your firewall, those should be included as part of your SOA. You absolutely need to include the availability of external solutions as contributors to your overall SOA for your enterprise. I know Dave has talked about this a lot as well.

There's a lot of debate as to whether SOA is the whole kitchen sink. There's a Yahoo! mailing group, and they’ve had the same debate going on just recently on not just the technical aspects, but whether SOA should encompass all of the culture change that people are saying is so critical to success with SOA. Are we just muddying the term by trying to lump all of this in together?

If we look at it from what it takes to be successful, if you leave those pieces out and focus strictly on the technologies associated with it, you’re not going to get success. At that point we’re not achieving the goals we set out to do. So, it needs to include all of these concepts and have that factored into your overall enterprise approach.

Gardner: Dave Linthicum, what do you think? Do you think that we’re going to water down SOA by including all this stuff, or is there another category that we should put all of this stuff into, in which SOA is a component?

Linthicum: I agree with Todd. It's inclusive. In fact, look at my career in starting up Grand Central and Bridgewerx and all those companies, where it was all about creating SOA outside the enterprise and delivering it as a service. It's what Salesforce.com is doing with their Apex platform today, what they call Salesforce SOA.

So, it's an architecture, and ultimately lines are blurring between where the enterprise stops and the Internet starts. With mashups, outside-in services, all the stuff Google and Microsoft are working on, and all the services that are being consumed within the enterprise, it definitely is inclusive and it has to be considered in the context of the architecture, enterprise architecture, and architectural patterns like SOA.

Gardner: Tony Baer, do you think that this is begging too much of a typical enterprise IT department to both be managing transition and transformation to SOA internally, and, at the same time, start to work with these off-the-wire elements -- be it platform, integration, or services and applications?

Baer: I’m going to take this to another level, which is the business level. If you look at the way business has evolved, or supply chains have evolved, over the past 20 or 25 years, they're really evolving into an ecosystem that is very partner dependent. When you buy a product that is manufactured by a company or that’s branded by a specific company, chances are it is manufactured by somebody else.

So, SOA and enterprise architecture really need to echo or support the direction in which business is going. More and more enterprises are becoming virtual. As a result, if you're running your internal processes, the chances are you’re going to be interacting pretty intimately with one of your business partner’s external processes.

Gardner: So, this for you is really just the evolution of business in general, which is to outsource as best you can.

Baer: It's not just outsourcing. You’re working in coalitions. For example, look at how a car is put together today. There have always been suppliers, but suppliers are now taking higher-level roles. They're actually co-designing. When one of the automakers or OEMs designs a car, chances are they’re designing it with their partners. In the old days, they designed the car and then sent out parts orders to partners.

So, SOA in IT and enterprise architecture really needs to support the way the business is going. Business is becoming more virtual. It includes external processes and partners, and SOA must support that.

Gardner: Okay, but about five or eight years ago, a large number of companies really explored outsourcing, but then pulled back. There was something they didn’t seem to like about it. They seemed to do it for economic reasons, but there was this backlash against it. Do you think that was growing pains? What's up with that in terms of still wanting to retain a lot of control over IT internally?

Baer: Well, it's cyclical. I remember back around 1990, there was a grand announcement by Kodak, "Well, we’re outsourcing IT, because, guess what, IT is not part of our business." I’m not even going to fast-forward this 20 years. I’m going to fast forward it three years. Kodak realized, "You know something? IT is part of our business." They had to take large parts of those 10-year contracts with IBM and, at that time, Digital and, I believe, EDS, and after about three years those contracts essentially collapsed in. They dropped a number of provisions. Today, Kodak is saying, "We’re a digital business."

It's a cyclical thing. If you outsource without managing it, if you just throw something over the wall, you’re going to get what you paid for.

Gardner: Neil Ward-Dutton, this notion of the virtual enterprise, do you see that as being on the same wavelength with SOA activities. Do you agree so far with the discussion?

Ward-Dutton: I thought Tony made some really interesting points there. Certainly, one of the arguments that you could make for a long time about enterprise architecture was that it was locked into a way of thinking that was very last century. It was locked into this evolutionary level of thinking that started around the kind of information-engineering movement that happened in the 1970s and 1980s. It assumed that you were building everything yourself, in long-running waterfall projects, and things didn’t really touch other things. Everything was built in isolation.

That was a big problem for enterprise architecture, because, as Tony pointed out, businesses have been pursuing this kind of virtualization agenda for a long time. Think about EDI and value-added networks (VANs), all that kind of stuff. All of that was designed outside the purview of IT and was done by business teams. The enterprise designers or the architects had no real input into that whatsoever. They were focused on the stuff they build internally for the organization. It's long overdue that architecture practice starts to take a more holistic view of an ecosystem of technology provision.

This thinking is long overdue. SOA, shouldn’t be constrained, as Todd just said, by a particular hosting model, where service is provided from, or indeed a technical choice. It's all about the services and it's all about the architecture, but that shouldn’t be constrained by where that service happens to be delivered or who it happens to be delivered by. You can pursue SOA in an environment where the resources are widely distributed, where responsibility is federated. Indeed, more and more, that’s how people are going have to do it. This is all natural, and as Tony said, really, really needed.

Gardner: Brad Shimmin, we’re getting into this notion that IT, by being isolated, has been a handicap, as companies explore extended supply chains and VANs and reach out and capture clients through the Internet. Do you see it that way? Do you think that IT has been a handicap? And, on a related note, is SOA therefore a catalyst to greater virtualization of companies, greater efficiency, and firm productivity?

Shimmin: Well, yes to both. IT really has been isolated, and I think we can all look at the cataclysmic failure of the idea of the waterfall method of development, as a pointer to that. You really can’t work in a vacuum, just toss code over the wall, and hope that it runs somewhere. As we were saying with the failures in early investments and outsourcing, I think those were all built in that regard.

Nowadays, when you start to look at what SOA has done to the industry and how it has impacted IT, it has become an enabling notion that allows companies to employ methodologies like the agile development method, that are more kind in how they view the lifecycle of software and where software runs. It's a partnership with the business and perhaps with outsourced parties that might be hosting that software.

SOA, as a collection of standards, technologies, and ideas -- which is how I look at it -- is already playing a very active role in allowing that pendulum to swing back towards platform-as-a-service, integration-as-a-service, SaaS, infrastructure-as-a-service. It's actually enabling those.

If you look at the vendors that we cover all the time you will see that everybody from VAN vendors like GXS, Otics, and Sterling to the traditional platform vendors are all looking at how they can utilize SOA to better enable outsourced models. SAP, for one, has always offered managed services, but they offer hosted services for their software.

Microsoft is doing the same with their BizTalk Services and with their dynamic CRM, which is tied with BizTalk Services. I think we’re seeing everybody really jumping on this bandwagon.

The best example is one we've already alluded to, Salesforce.com. When I look at them, I see a hosted service that utilizes SOA ideas to facilitate a secured, reliable, manageable, business-savvy domestic environment between your backend systems and their backend systems, which just happen to be on somebody else’s servers.

Gardner: Joe McKendrick, you’ve written recently about Internet Service Bus (ISB). Tell us a little bit about what you mean by that and how that relates to this discussion.

McKendrick: ISB is something that Microsoft has out there. They calls it BizTalk Services, and it’s now in the community technology preview phase, where they’re offering the functionality of their BizTalk -- essentially their ESB -- on an online basis to companies that don’t necessarily want to invest the time, capital, money, or staff to build their own ESB or SOA.

Gardner: So, does this mean that you get the option of hanging your clothes on their clothesline, as it were, on their servers, and then you can coordinate applications or mix and match across this bus? Help me understand conceptually what Microsoft is getting at here.

McKendrick: That’s exactly what they’re hoping to accomplish. You see it happening across the breadth of their product line, with the whole Microsoft Live. You can see them moving to this notion of SaaS, platform-as-a-service. I think they see the handwriting on the wall.

One thing I always say is that Microsoft doesn’t make a move unless it sees a mass market available to pursue. The fact that they’re pursuing the ISB shows that they see it's reaching critical mass in the market. Their natural base is the small- to medium-sized business. It’s not the large corporations that IBM, Oracle, and Sun serve.

Gardner: Or even departments within large organizations, which can actually behave like small- and medium-sized businesses.

McKendrick: Exactly. One model that makes it easy for the business to understand SOA could be an internal SaaS that units of the company are making available to the rest of the enterprise.

What happens is you have this software and it’s something the business can understand. There is a division, maybe at the CIO’s office and maybe somewhere else in the enterprise, that’s deploying these service-oriented applications. The services are available for reuse across the enterprise, across the walls of the enterprise. The enterprise has the option of either picking up these services internally or picking up services externally from another partner. The enterprise becomes both a provider and a consumer of services. It’s two-way outsourcing.

Gardner: There is a tangential trend around ITIL and some of these other initiatives, whereby IT departments take on more of the service bureau or a shared services role, and move from being a cost center towards more of a services provider. Their budgets then become like a P&L. They’re motivated a little bit differently than they would have been as just "Tell us what you need and we’ll figure out how much money we can come up with at the end of year for you" type of approach.

Does anyone else have a reaction to this related area about IT departments shifting in their business models and how that relates to SOA? Todd Biske.

Biske: One of the things that I heard as we went through this conversation, and you hit on it with ITIL and becoming a service provider, is the notion of service management. When we’re looking at why there’s been an outsourcing trend and a backlash against it, a key factor is the degree of service management that’s provided.

It’s one thing to need this capability and go off and find it somewhere, but once you start using that, you realize that there’s this whole degree of transparency that’s needed to understand how that service is being used, where the results are coming from.

In the initial stages of outsourcing there was a lot of people saying, "We want to be a black box and we don’t want to expose any of our dirty laundry, because then our customers may go away." Information needs to be provided to give good service management, as well as to help the customers of those services understand how it’s being used and where they can make incremental improvements on it. It creates a feedback loop.

We can look at virtually anything, whether it’s blogging or podcasting. Before we all got on the call. we were talking about the metrics of how many people are listening to the podcast. By looking at that information we can make things better. We can target audiences or understand exactly who the audience is.

The same thing applies now in the service management domain. I need to have visibility into how the consumers are using the service, where they’re getting good experience, where they’re not, are there trends, and how it’s used. If I’m not getting that information, whether it’s from an internal service provider or from an external service provider, that’s going to start to have an impact over time.

That’s where it leads into culture change -- adopting things like ITIL and understanding service management and what it means to be a service provider. IT is used to building solutions. They put it out there and they go on to the next project. That’s not service management; that’s project management.

Gardner: We’re also seeing discussions -- and we’ve had them on this show, of course -- around governance, where you’re looking at internal governance of services. Then, you’ve got to multiply that in terms of how you would govern across platform as a service and integration as a service.

It raises the question of whether the complexity starts spinning out of control, but we could combine a couple of these observations today. We’ve talked about governance in terms of comparing it to a government, whether you do a democracy, federalized, local, or state. But, if we consider ITIL Service Bureau, we recognize that one of the best governance mechanisms for the most complex things -- business and the economy in general -- is capitalism, competition, or even more crassly, Darwinism.

Suppose we open up services and allow them to compete for the users and/or the lines of business managers to be able to choose among an external service, an internal service, a shared service, a supply chain service within an ecology. Does anyone think that a capitalistic or Darwinistic approach, if this follows its course, will allow for the best form of governance, which is, "Let the best service at the lowest cost win?"

Ward-Dutton: I say yes and no to that. Competition, or the threat of competition, is sometimes a very, very valuable thing to produce the mix. However, you've got to remember that if you’re talking about free market, one of the side effects of a free market is that an awful lot of companies go bust, and that’s fine. If you’re at the top, looking down, you’re just looking at the performance of a market or an economy overall. It’s fine if companies go bust. If it’s your company, though, that’s not so good.

If you’re looking at IT provisioning in this context, you've got to be careful. You've got to put some regulations in place. You don’t want your own internal IT provider spending a heap of money on stuff that isn’t going to be used, not going out of business, but essentially causing business problems, because they’re wasting a lot of money.

So, you’ve got to have layers of governance, i.e, rules and frameworks about how planning gets done, how priorities are set, what’s important, and what’s not. Those kind of things have to be decided, so that a company knows collectively where it wants to go, what it’s trying to do, and what that means for IT. When you’ve got that in place, then in constrained places you can start to drive competition by saying, "In these places, we want to look for competitive solutions, but it’s not always going to be the right thing to do."

Gardner: So, the jungle might work in markets at large, but internally that could be self-destructive. What do you think Brad Shimmin? Do you think that competition has a role or should we have a balance like China, where we have a market, but also central control, and a little bit of socialism thrown in too?

Shimmin: Well, as much as I love socialism I have to say that I’m on the capitalism side of this. I just want to throw out a caveat that if we’re talking about IT departments becoming providers of services, we need to make a distinction between internal provision and external provision, because not every company is a Google that’s going to throw out a Google Maps widget or WSDL, and suddenly everyone is going to be utilizing it, and everyone is going to be happy.

Companies have a better shot at that, however, than doing this internally, because internally you have a lot of federated environments, as we were just saying, with departments acting as their own companies. The whole notion of charging back for services and being accountable for how much you use those services is something that a lot of companies are certainly not willing or prepared to embrace right now. So, the internal adoption of this notion is a long ways off for a lot of companies.

Externally, there are providers and consumers, and the average enterprise is most likely not a provider of services like that. They’re consumers of services from companies, who, as Neil was just saying, hopefully are going to be around for a while. If I'm going to subscribe to one of those services, as a Web service, I want to know a couple of things.

One, I want to know that the company is going to be around a while, because I have an investment in that service, in setting up the interface for it -- for example -- and testing it out, because you don’t just grab a URL WSDL, throw into your designer, and poof, you have a service running. You have to test it and make sure the data models match, etc. So, I want to make sure that my investment is going to be protected.

The second thing is I want to know that the service level agreement (SLA) that I have with that provider is acceptable. If I'm getting the Yahoo! Search WSDL, and I’m limited to 10,000 hits a day, but don’t know that, and I’ve rolled out this enterprise application based on it, I’m going to get some angry phone calls from people around the company when we’ve gone over our limit.

Gardner: So, the notion of how to procure a service responsibly and appropriately becomes very important. That’s a little bit different than governance. IT departments have been historically good at procurement around systems, servers, hardware and software, and support maintenance, but the advancement, up an abstraction, of procuring a service or set of services, would fall more into the role of an SOA architect and not necessarily an IT architect.

Linthicum: Absolutely. The SOA architects ultimately need to figure how these services are consumed and how they have value with their internal systems. I would argue, however, that we are getting more experienced at picking outside-in service providers through the whole SaaS notion. However, they deliver it through a visual interface which is, in essence, a Web site, but it’s becoming much more than that.

Salesforce.com, Amazon, eBay, all those guys, are making tons of money now renting their Web services, and the ability to provide the behavior and the value through Web service is going to be a core component. Businesses are probably more advanced than we think they are in integrating those things into the core systems. It’s at a rudimentary level, but ultimately becomes outside-in services, integration with the SOA, and that’s where the architects come into play.

Gardner: There is a role here for free market influence. That is to say, as this matures and more services from different environments and providers become available, that SOA architect is going to be picking and choosing based on pure market types of forces. That creates the market dynamic, which has an overall governance impact itself. Furthermore, we've also talked on this show about the rationales for SOA. Why should you do it? Why should you do it sooner rather than later? What are the cost benefits? What are the analyses from a business and technology perspective?

For those that are listening, perhaps they get the sense that creating SOA internally, identifying and cultivating SOA architects, will allow them to, in the future, avail themselves of services across a wider panoply of sources and business models, and ultimately they can become the beneficiary of these market forces. This capitalistic approach towards lower costs, higher productivity, and less waste becomes another economic impetus to pursue SOA. What's your reaction to that, Tony?

Baer: I don’t think there's question that this really follows how companies are running their own businesses right now. It's just not possible to become agile and be competitive if you try and make every thing yourself. So, it will be inevitable that you will, at some point, consume external Web services.

I want to get back to the governance point from before. With all the regulatory compliance mandates that we have to deal with at this point, we have to worry about who is handling financial data. Who is handling customer data? Who is protecting identities?

There's no question that a free market is going to emerge for services, but, at the same time, it's going to have to be accompanied by some very heavy doses of transparent governance. I may need to know, for example, my partner’s mechanism for federated trust and authentication. The short of it is, I don’t think you can stop continental drift in this case. You are going to have a free market of services, but you're also going to need a very transparent governance to support that.

Gardner: That's governance at even yet a higher level. That's governance at a market level. In what sort of organization, what authority, under what auspices would that type of governance occur?

Baer: You have to conduct your own governance. It’s a federated model. It’s not going to be any sort of United Nations agency that's going to be top-down type. There probably will be industry best practices that will be drawn up by industry bodies, but ultimately the enforcement has to be at the enterprise level. You have to decide what protections you need and what you expect from your partners?

Gardner: Wouldn’t that federation at the level just above enterprise, or even above supply chains, have to happen under some sort of a brand. We’re already seeing Salesforce.com. We're already seeing Microsoft.

Baer: I thought you were going to go towards something more like the Underwriters' Laboratory model, where we have some sort of external certification agency. Actually, you made a good point where you take a company that's very established and say, "Well, we're going to trust Microsoft, because they're too big to try and risk having a bad reputation."

Gardner: You're more likely to see a business side approach to this than a regulatory side.

Baer: I would agree with you. I don’t see any new top-down agencies parachuting in. It’s going to be, "I'm going to deal with trusted partners. I expect Microsoft is going to be around for a while. I expect that, considering the type of business they’re in, that they’re going to conduct themselves with the highest standards of good governance."

Gardner: Did we go right back to where we had been, which is five or six major suppliers, and "I'm going to be a IBM shop -- or a Microsoft shop -- or an SAP shop," but it’s going to have to do with the ecology of services and not just the packaged applications or platforms? Is this just going to evolve into another instance of major vendors controlling this. Does anybody have a reaction?

Shimmin: Well, I think it will be the same ecosystem we see now, where you have different tiers in the marketplace. Depending upon how important that service is to your business and your application, and whether or not you can withstand a change in that service, of that service going away or not, having the service availability or not, having the governance or the ability to control the data as you would like, then you might be able willing to go with somebody that's smaller than Microsoft.

Gardner: You might be willing to go with perhaps an open-source community that might perk up around a set of services and applications, where the governance is based on community input, rather than just a single entity or large influence?

Shimmin: I don’t see why not.

Gardner: Anybody else have some thoughts about what impact an open-source approach to a services ecology might include?

Ward-Dutton: You’re getting towards a cooperative model in that kind of environment.

Gardner: I guess you would.

Ward-Dutton: You are, if you're talking about a community banding together to make its own of autonomous decisions about who's to be trusted and how things are going to happen, without any kind of patronage from a benevolent dictatorship like a Microsoft or whatever.

Gardner: Or perhaps if the benevolent dictatorships become less benevolent over time.

Ward-Dutton: There is space for all sorts of models. I know the B2B exchange thing in the late '90s was over hyped, but Cognizant is still doing some pretty healthy business in the automotive sector, for example. That's not an open-source model, but it is a kind of industry driven, bottom-up collaborative effort. There are all sort of models, a spectrum of commercial models which can work.

It wouldn’t be open source particularly, but a kind of a non-profit community-driven type of approach could be equally valid. I don’t think it’s likely to be one model winning over any other. There are a whole load of things that are likely to have a role to play.

Shimmin: I feel that the open-source vendors are well positioned to get into this space, because you're buying a service, whether it’s support, installation, custom development, etc. It's a service that you're buying from them. You're not buying software. Companies are becoming much more accustomed to that and comfortable with it. Let’s say you have an open-source Mule version that's hosted within the ESB, Mule ESB hosted, maybe New Source is the company that runs the data center somewhere, and they provide SaaS. There's no reason why they can’t, and it would fit well with their model.

Gardner: I suppose another important trend that is out there now is this whole notion of social networking.

If people have transparency on how services are used and consumed, they should also have transparency to share their experiences with those services. If openness and the ability to create community dialog around whether services are being well provisioned or governed, or if costs and benefits are out of alignment, that this might also provide some grease on the skids of competition and efficiency towards the best solution that the community then adopts or shifts.

Any thoughts about how openness in networking, from a sociological point of view across the network, might keep this in some sort of checks and balances?

Biske: It’s part of any kind of capitalistic approach to this. Customer feedback is just part of the market dynamic, and that certainly can make or break somebody. If you're not factoring that in and incorporating what your customers think in leveraging that information, it’s going to come back to haunt you.

I had another comment on the open-source piece as well. There is still going to be plenty of room for smaller companies, closed-source approaches, in these vertical domains. One of the bloggers that I follow has commented that you don’t see a lot of open-source efforts in niche vertical areas, and we're talking about governance and the impact of regulations.

You don’t see people who, in their spare time, are following the new SEC regulations that are coming out and are trying to offer open advice on how to best meet source regulations.

Those who develop software have always had formal jobs and careers based upon it. It’s also been a hobby for lots of people, and has helped the open-source movement. I don’t know too many people who like to follow regulations as a hobby. So, to have this kind of open-source model to handle governance, the broad horizontal domains will continue to see pushes in that.

In these niche areas, where it requires somebody being paid to understand all of those regulations from the various government agencies, that has to go to a proprietary model. One, it’s just not as interesting, and two, it’s not as big as a marketplace. So, you have to get a level of niche expertise and understand who your customers are going to be.

Gardner: That follows the pattern we’ve seen already with the boundaries between where open source works well and where commercial works even better. Okay, let’s wrap up this interesting discussion we've had about SOA and its role, services, ecologies, governance, competition, and the viral impact of social networks and openness.

Moving on to our next subject, it's a little less global or universal in scope. The IBM acquisition announcement about DataMirror. Brad Shimmin, tell us a little bit about how you view this announcement and how this will or will not strengthen IBM’s hand in information management?

Shimmin: If you look at this from purely a master-data-management, or just data-management, perspective, it makes sense. It’s something they should have done, and they already have a good deal of energy being extended toward that. So, yes, "yippee-skipee," but what gets me going about it is just thinking about how IBM, and every other vendor that has its toes wiggling in the SOA pool, is really looking to build complex event-stream processing and event-driven architectures on top of their solutions. This allows companies to stop thinking about their processes.

Just as you build the process, you throw it out there. If it maxes the server out, that's a bad thing and you go from there. You have your process running. It has interdependencies upon these other processes, these other servers, these other resources, etc. Over time, there are a number of events that take place relative to that process. Companies want to be able to look at that stream of events in context and over time and be able to make decisions. They also want to have automated actions taken upon those. Something like DataMirror gives IBM the opportunity to do that.

Gardner: Is there any question now that having a strong data story is essential to having a strong SOA story for any of these major vendors?

Shimmin: It’s a necessity. Everyone has come to understand that you need to understand the data in the transaction as well. Most of the vendors that we cover in the space either do that or have woken up and smelled the coffee, and are making a lot of efforts in that direction.

Gardner: What about HP and Sun Microsystems? They don’t seem to have quite the same data story. What do you expect to happen there? Let’s go to Joe McKendrick. You track data stuff. Are we getting to a point where HP and Sun are going to be at a disadvantage, or is there actually a benefit for their being a bit more agnostic or neutral vis-à-vis data and data services?

McKendrick: Well, that's a tough one to call, Dana, because they are not actively in the data-management space per se. They don’t have the robust data architecture products that IBM, Oracle, Microsoft have. Sybase as well.

I could see HP emphasizing its service aspect in this regard, really getting into this with its service’s space. Sun is a tough call. It’s sometimes tough to keep track of what Sun is trying and trying not to do.

Gardner: It seems that the software business is a pretty good place to be right now. Looking at the results just this week from SAP, Microsoft, and IBM, software is selling pretty well, and we're not just talking one or two elements, but across the board. Enterprise software is a strong growth business, somewhere between 10 and 12 percent, even after currency fluctuations are taken into account. What do you think, Tony Baer? Is software in SOA important, and is this large set of data capabilities essential? How do you see it shaking out?

Baer: Well, actually I was doing a study at, of all places, a Telco software company. What struck me was that as part of their SOA strategy they also embraced a very heavily federated data model. If you think about it, it’s the perfect compliment to a SOA strategy, because SOA is built on the idea of a loose coupling of all your processes. If you're really going to realize the promise of that, and you have a lot of different data sources, which is the case in most large organizations today, a federated data strategy, which essentially abstracts the data from its source, is the perfect compliment to realizing the promise of SOA.

I'm not surprised to see that the software business is doing very well, and especially companies that are very heavily involved with managing data in some ways are performing. It's not a big surprise.

Gardner: Doesn’t this DataMirror acquisition by IBM point to the fact they want to be more federated. It’s not all going to be a DB2 world. They want to expand on the ability for real-time data management?

Baer: No question. Think about the one key piece that DataMirror brings in there, which is the whole changed-data capture. I can imagine a lot of business scenarios, where that itself could be packaged as a very high in-demand service.

For example, if you're doing any type of market watch or real-time supply-chain tracking, you don't care about all the routine data, when there's no change in the stock or when things are moving on schedule in the supply chain. You want to know when exceptions happen or when things change. I can see this DataMirror capability being packaged into a service that’s not just changed-data capture, but event notification. It fits perfectly in that picture.

Gardner: Neil Ward-Dutton, how about the last word, the role of data within the SOA value? And, how about the competitive landscape question around whether certain vendors might be at a disadvantage for not being too aggressive on the data side?

Ward-Dutton: How long have I got? 30 seconds?

Gardner: 30 seconds.

Ward-Dutton: The role of data is very, very complicated and gives people a lot of headaches. Certainly, this seems to be like a wave-particle duality thing between service and data. If you pursue SOA too simplistically, you just say, "Listen, what's important in the way I design things and architect my environment are services and that's all that matters. That's the primary organizing concept." Then, what happens when you get to data is a whole heap of trouble.

Equally, if you take a very data centric perspective, it can cause problems, when you think about services. When you just think about a service that you're going to reuse in multiple contexts, you have to think quite carefully about implications for the way that data is stored. Clearly, consistently mapping data architecture and SOA architecture is a challenging thing.

What's clear from the adopters I've spoken to, is that you can’t ignore information architecture when you're pursuing an SOA initiative. It’s vital to understand the implications for data architecture and capabilities like federation, changed-data capture, and synchronization of data across boundaries. So, there is a very strong play here.

I wouldn’t go as far as to say that IBM, HP, and Sun are at a disadvantage. I think it was Brad just said, I am not sure, so many people on this call. It’s a cast of stellar participants. Someone was just saying that these guys have never been data-management players, and there are so many things to say about Sun's software strategy, I'm not going to get into it now. HP has managed to survive and is now thriving, but focusing in a completely different area of software delivery. They are much more about management process, quality, and so on, and they are very agnostic in terms of the middleware layer and data layer, and that works very well for them. So, I don’t see why there should be a bigger problem.

Gardner: Right, I suppose there is one thread that we can pull together on the two elements of our discussion today. Perhaps, as you shop around for services, as you look towards the natural-selection benefits of an open ecology in communication, even with services being cheaper, better, faster -- over the wire in some cases -- you still might want to keep your data services very close to your vest. So, perhaps there is a delineation in our discussion for another podcast about whether as we look towards a mixture of in-sourcing, out-sourcing, and multi-sourcing, the data issue is something to keep an intellectual property and ownership vise around.

I want to thank our participants. It's been another interesting discussion. Joining us today have been, Joe McKendrick, Tony Baer, Neil Ward-Dutton, Todd Biske, Dave Linthicum and Brad Shimmin. Thank you all for joining.

This is Dana Gardner, principal analyst at Interarbor Solutions. You've been listening to Volume 22 of BriefingsDirect, SOA Insights Edition. Thanks and come back again.

Listen to the podcast here. Produced as a courtesy of Interarbor Solutions: analysis, consulting and rich new-media content production. If any of our listeners are interested in learning more about BriefingsDirect B2B informational podcasts or to become a sponsor of this or other B2B podcasts, please fill free to contact Interarbor Solutions at 603-528-2435.

Transcript of Dana Gardner’s BriefingsDirect SOA Insights Edition, Vol. 22. Copyright Interarbor Solutions, LLC, 2005-2007. All rights reserved.

BriefingsDirect SOA Insights Analysts on Defining the New Role of 'SOA Architect'

Edited transcript of weekly BriefingsDirect[TM] SOA Insights Edition podcast, recorded March 23, 2007.

Listen to the podcast here. If you'd like to learn more about BriefingsDirect B2B informational podcasts, or to become a sponsor of this or other B2B podcasts, contact Interarbor Solutions at 603-528-2435.

Dana Gardner: Hello and welcome to the latest BriefingsDirect SOA Insights Edition, Vol. 15. This is a weekly discussion and dissection of service-oriented architecture (SOA) related news and events with a panel of industry analysts and guests. I'm your host and moderator, Dana Gardner, principal analyst at Interarbor Solutions, ZDNet software strategies blogger and Redmond Developer News Magazine columnist.

Our panel this week consists of Jim Kobielus. He’s a principal analyst at Current Analysis. Welcome to the show, Jim.

Jim Kobielus: Hi, Dana. Hi, everybody.

Gardner: Also joining us from the U.K., Neil Macehiter. He's a research director at Macehiter Ward-Dutton. Welcome, Neil.

Neil Macehiter: Hi, Dana. Hi, everyone.

Gardner: Also joining us today we have Steve Nunn. He's the vice president and COO of The Open Group. Welcome to the show, Steve.

Steve Nunn: Thanks very much, Dana, and good morning, everyone.

Gardner: Also joining us is John Bell, an enterprise architect at Marriott International. Hello, John.

John Bell: Hello.

Gardner: We're going to be discussing the role and concept of what’s becoming defined as the "SOA Architect." This is a different role, as we’re finding out, than the enterprise architect, but certainly seems to be part of an evolution of the role of architect within the enterprise and within IT in general.

We’ve invited a representative from The Open Group, in this case Steve Nunn, to join us, because The Open Group has taken some steps to try to define the role of the SOA architect, has created some certification around that role, and is trying to get in front of this role in terms of what will be required in the marketplace. That is, to try to encourage more people to step up and understand this role and to certify themselves, so that the progress and maturity of SOA practices can continue and not face a human resources crunch.

So, with that, why don’t we hand it off to you, Steve? Why don’t you tell us a little bit more about what The Open Group is doing and why?

Nunn: Thanks, Dana. The Open Group and its members have been working in the architecture space for over a decade now, primarily developing something called The Open Group Architecture Framework (TOGAF), but, as you’ve mentioned, we're running certification programs in two specific areas. One is in relation to TOGAF, but perhaps more relevant to this discussion is our IT Architect Certification (ITAC) program.

I guess that sets a caveat at the outset. The terminology around what type of architect it might be -- IT architect, enterprise architect, SOA architect -- is still very much settling down as a topic of debate in its own right, but our program that I can talk about is the IT Architect Certification Program, which is a broad skills- and experience-based program. It's aimed at creating a vendor-neutral program by which individuals can be certified. It provides them with a transferable qualification in the industry, and it enables employers to know that if they prefer recruiting certified individuals, they would be getting somebody who has been through an accreditation process.

Briefly, the process would be that a resume is compiled, which can be quite extensive, up to 52 pages in some cases.

Gardner: Wow!

Nunn: Yeah and then there’s a peer review by a panel of three certified architects themselves who would probe a little on the resume, ask questions of the candidate, and conclude whether or not that individual meets the conformance requirements.

Gardner: Is this process already up and running, or is it something you're still pulling together in terms of how you want to approach it?

Nunn: No, this is up and running. We launched this in July 2005, and as of today, we have just a shade under 2,000 individuals from all sorts of companies and all over the world who are certified under this program.

Gardner: This is the SOA Architect Certification?

Nunn: This is actually what we call ITAC, the IT Architect Certification.

Gardner: I see.

Nunn: It has several levels and covers various disciplines. The SOA-specific part of it is one that we are still working on. We have various horizontal levels under this program. The conformance requirements for meeting those levels have been agreed upon. There’s an entry level and a higher level. We are working on the highest level right now, but what’s also going on is work on the individual aspects of that certification, of which SOA is one. What we’re quite proud of in this program is the conformance requirements for the overall program, and what we're now focusing on are the conformance requirements for the individual disciplines.

Gardner: Perhaps this is a good time to go around the room, so to speak, and see if we’re in some agreement that an SOA architect is fundamentally different from an enterprise architect and why? Why don’t we start with you Jim Kobielus. Do you see these as significantly different roles?

Kobielus: Not really. You have to be an IT architect to be an SOA architect. It seems to me that an SOA architect, or that discipline, is a subset of the overall enterprise architect. I would like to know precisely what other disciplines or practices that one needs to be certified in to be a SOA architect, versus just an overall enterprise architect, I’m still unclear on that.

Gardner: When we hand it back to you, Steve, one of the helpful concepts for me in understanding this was the notion of the "city planner" or "town planner" role. The analogy is that an SOA architect needs to like a city planner, looking at all the resources and infrastructure and how the entire community comes together, managing constituencies and political relationships, whereas an IT architect might have a smaller role. Can you expand on that a little bit?

Bell: Actually, an enterprise architect from my perspective would have the view of the town planner. When they’re looking at the entire city, they're looking at how the various neighborhoods, how the various business zones, etc., fit into that city. The SOA architect, from my point of view, is really more interested in, "Hey, how does that underlying infrastructure allow different neighborhoods to communicate with each other and exchange messages? How are health services delivered across neighborhoods?"

So, it’s more interested in, "Okay, I’ve got a firehouse. Can the fire truck get to the house before it burns down?" The vision of the SOA architect is more associated with the communications pieces within the community.

Gardner: So, the hierarchy might be that the enterprise architect is in the town planner role, the more holistic, oversight uber-architect role, and then a subset of that is making sure that the communication channels between and among these different facets of resources and functionality are behaving well and conforming to what needs to happen.

Bell: Conforming to the standards so that they’ve got a consistent set of standards for exchanging information, those kinds of things.

Gardner: Okay. Neil Macehiter, what you make of this?

Macehiter: In that that classification of the difference between enterprise architecture and the SOA architect, it sounds to me that the premise is that the SOA architect is focused primarily on the plumbing. From a bottom-up perspective, the challenge that many SOA architects face is more around understanding what the services are that need to be delivered in a business-meaningful way, not just about communication and plumbing. It’s also about understanding the high-level, business-meaningful services.

There is a business strategy, there are business processes and priorities, and there are the services we need at a business level. Then, there's a handoff to what’s currently defined as the SOA architect, who will actually define how those services are deployed in technology terms. So, the distinction is quite blurred. A service-oriented approach is one of the methodologies and the approaches that you can utilize to deliver or to support an enterprise architecture initiative. So, I still find a distinction difficult.

Kobielus: I second what Neil’s saying. I’m uncomfortable with just reducing SOA to the plumbing. In the three-layer stack that I carry in my head, the plumbing level is the enterprise service bus, and then SOA refers to development and reuse practices within the development organization to enable maximum sharing and reuse. Then, there’s the layer above that, which is the applications, services and data -- the business processes.

I’ll put enterprise architecture at that very top layer, concerned with the end-to-end set of resources: app services, data, etc. The SOA architect would be the middle layer of the development and reusing. The layer below that, the enterprise service bus (ESB) or whatever, I call that "IT architect" in the sense that it's the infrastructure architect.

Bell: And to be fair, I didn’t mean to imply that the SOA was limited to the plumbing. My intent was saying that the enterprise architect has a much broader spectrum and scope that they have to deal with than your SOA architect has to deal with. Putting it into that city paradigm, you kind of limit it as to how to describe some of the roles. I try to clarify that by saying it’s not just how they communicate, it’s things like, "Hey, where’s the fire station? Do you have a fire station? Where’s your police station? Where are your schools? Where are all those pieces that are providing services to that community and are they adequate for providing the services to their community?" That’s a subset of what a city planner has to do but it’s still an important city-planning kind of function.

Gardner: John, you’re in the trenches, you’re an enterprise architect in a large global concern. How do you see this hierarchy and is this really the right discussion that we’re having?

Bell: My view is that the enterprise architect is at the top of the hierarchy, and at some place, working with the enterprise architect is an SOA architect, and their focus is on, "What are the services that are being delivered, how am I delivering them? What’s the infrastructure I am using to deliver it? Do I need – using that town model -- a police station? Do I need a fire station? Do I need a school? Do I need a museum? And, if I do, how do I get that service out to the community or to the entire city, not just an individual neighborhood?"

So, from my perspective, using a city planner paradigm, the role of the SOA architect, is identifying what are the services that need to be available to the city and how to deliver those services out to the city.

Gardner: Back to you, Steve Nunn. It seems to me that there needs to be a fair amount of flexibility, enterprise by enterprise, and circumstance by circumstance, as to how this SOA architect role pans out. How much standardization and methodological consistency can we bring to something that, in fact, will probably be dealing with huge variability from organization to organization?

Nunn: Something that we’ve had to address in putting the program together is that there are huge differences. Even taking the frameworks that might be used in implementing an enterprise architecture, there are huge differences among organizations. Some organizations are required to use a certain framework. Our approach is not to specify exactly how enterprise architecture or SOA architecture is done in the certification program, but more about the experience of the individual who's implementing it.

It doesn’t seek to define a particular way of implementing the architecture, but is more about the skills and experience of the individuals who are playing that role inside an organization. They could be part of a team in larger organizations, or could be one person in a much smaller organization who is playing this role. The whole idea of raising the value of the architects of various titles in the organization is what we are seeking to achieve with our efforts in the certification program. It’s about raising the standards of that role, and getting people to understand that it’s a valuable role and, apart from anything else, it should be compensated as such.

Macehiter: Dana, could I chip in with a quick question there?

Gardner: Certainly.

Macehiter: It’s about the approach and the experience, rather than framework, and I agree completely with that. Given that the SOA discipline is currently within the IT architect certification, to what extent do you look at the approach and experience in terms of the interface to the business, business understanding, or collaboration with the business? I think that key elements of the SOA architect role are skill and capability, as well as the more IT-oriented skills and capabilities.

Nunn: Well, that's not so different between SOA and enterprise architecture. I’d say exactly the same about the enterprise architect -- that ability to translate the business need into the systems underlying or delivering the needs of the business. It’s something the enterprise architect absolutely needs to have, and that’s why we think there’s a special set of individuals who play this role. So, there isn’t really a difference in that respect between the SOA architect and the enterprise architect.

Kobielus: I like what you do. Thinking about the whole notion of certification, there are two ways to go about it. One approach that you can easily take -- and this is the way it’s usually perceived -- is when you are certifying a CPA, you’re certifying somebody as a skilled in an established and knowledgeful body of practice, be it law, medicine, whatever. But when there’s no consensus body of practice that everybody agrees upon -- for example SOA, which is still evolving -- a certification in that regard is more like when somebody is applying to college. You’ve got to send in your transcripts, write essays, and you also might have to go and do an interview in the admissions office. They look you over and say, "Oh, this is a smart person. Yeah."

So, they consider the sum total of everything you’ve done and who you are in certifying that. They say, "Yes, you’re good enough to be admitted into this college," and then proceed from there. That’s sounds like what you’re doing. You’re certifying the competency of a particular individual in this general field called a enterprise architect (EA) or a SOA architecture.

Nunn: That’s right, Jim, and it’s not a bad analogy at all. It's about assessing the individual. It’s a relatively young discipline in its own right. One of the things that we look at in a conformance requirement is the role that those individuals have played in the projects that they have been involved in, and whether they had been in a lead role or support role, or a combination of the two, but it certainly is about the individual, rather than the specific approach that they take or any particular body of knowledge.

Macehiter: Just one other quick comment on this, if I may. The other dimension for this, I think, is rather than thinking about the role, thinking about SOA as an approach. Then, thinking about how that approach applies to different types of architect. What I mean by this is that a lot of the emphasis and focus on SOA today has been around application development and integration, when, in fact, there’s a broader perspective that really extends across more traditional IT architecture and other disciplines, for example, a service-oriented approach to infrastructure architecture and the service-oriented approach to the operations and operational management of IT.

So, there are two dimensions that a body such as The Open Group might want to think about. One is the role of a service-oriented architect, and the second is how service orientation impacts other architecture disciplines and other IT functions or operation capabilities. If we don’t do that, we risk driving SOA into a particular stovepipe focused on application development and integration, and aren't thinking about it more broadly as an approach that it is an enabler of whatever the enterprise architects are driving out.

Gardner: Thanks, Neil. I suppose, too, that the role of the SOA architect will shift, as the maturity of SOA principles and methods evolves inside of an organization. They might have to start out a bit more focused on application development and deployment issues, move up toward being mindful of the business issues, and then move up more toward being the communications conduit between the fire house and the police station, for example. Does that make sense?

Macehiter: Absolutely. It’s about gradually extending through the life cycle.

Gardner: One of the reasons we are discussing this is that we’ve seen some warnings from analysts and others saying that we are moving toward SOA, but we really might find ourselves without the people with the background and abilities to move this. So, we’re worried a little bit about a dearth of qualified people, which might, in fact, stifle the progress here for SOA.

Do you see that is the case, Steve Nunn? Do you have any sense of numbers and what the demand is going to be? The second part of the question is, if there aren’t enough people, aren’t these roles going to fall upon the enterprise architect anyway?

Nunn: Dana, what we’re hearing is there aren’t enough enterprise architects to start with. So, I think it’s a given, therefore, that the SOA specialists are in short supply too. We’re hearing from our members that if they spot a good enterprise architect or somebody they think has potential for that, then they try and grab them. They’re pretty few and far between right now in terms of folks with experience.

Obviously, there are other folks that we just talked about with the college analogy who might well be groomed into that role in the future. So, certainly there is a shortage right now. That's what we are hearing from our membership. I don’t have specifics on numbers, but the message we hear is that demand is out-stripping supply right now.

Macehiter: This is not new. We’ve been through this with every major technology advance or discipline advance. You used the word "potential" there. I think there are individuals within organizations that have the potential to fulfill that role. Part of the benefit of this certification approach, providing conformance and definitions of what constitutes the role, is that it will help organizations identify the individuals within that company that have that potential, even if they don’t have the 50-page resume that demonstrates that they have been there and done it, because the key element in a service-oriented approach, and an enterprise architect more broadly, is an understanding of the business.

If you’ve got individuals within the organization that understand how the business works, have been around, and know the right individuals to talk to, that that can be of much benefit in terms of enabling effective EA and SOA, as can be going outside, finding someone from a different vertical market or different industry, and bringing them in because they’ve done six SOA projects elsewhere.

Gardner: Thanks Neil. Let’s take that point back to John Bell. Now, as an enterprise architect with Marriott International, I assume that you’re going to be in the position of having to hire or find SOA architects in this climate of scarcity. Where do you think these people will come from and what kind of backgrounds would you look for?

Bell: I think what we are going to have to do -- fortunately or unfortunately, however you look at it -- is end up training our own people. A lot of it goes back to what was just said about having to have an understanding of the business. You have to know the people in the business. You have to understand what the business of the business is. You have to have a lot of domain knowledge in order to create an effective SOA environment.

Because of that, when you pull somebody from outside, they may understand the technology, but they have to come up and learn the business, which is harder to train than somebody who has a general technology background, but knows the business pretty well, because he has been working in the business. Marriott happens to be a company that retains employees for years and years on end. So, in our IT department we have people who may already have 5, 10, 15 years of experience working directly in the business. And we can’t afford to lose that.

Gardner: Do you find that a developer is a fast-track path to SOA architect or a business analyst, even though it makes great sense to have someone with longevity in your organization? Is there particular type of role that they would have played that seems to conform to this need for SOA process management and evangelism?

Bell: In our experience, we are finding developers, as they move through their technology career path, since they have been developing within the context of the business, if they take that broader view, they understand the basis for the SOA architecture that’s installed in this particular company. They tend to make a good SOA architects with the proper training, and sometimes that training isn’t the technology training; it’s the people training -- teaching them how to conduct interviews, how to talk to people, how to get information from people, particularly in a company like Marriott where the business is not technically oriented.

Kobielus: This is all very good, but it doesn’t address the need that many companies have which is, "Hey, we need to hire people straight out of college who have some background in architecture, and where are we going to find these people and how are they going to get certified?"

Everything I'm hearing says that, an EA or a SOA architect is somebody who has experience and, by definition, somebody right out of college doesn’t have experience. So, is this the kind of thing that we can actually train in school or does somebody have to be in their career for 5, 10, 15 years before they’ve been steeped enough in all of this architectural infrastructural development and integration stuff to the point where they can be certified?

Bell: I’m also an adjunct faculty member at Towson State University, and this is an issue that we’re dealing with at the university level so that the university can provide the skills that the local businesses in the Baltimore area need. So, at the graduate school level, we are looking at what we offer in the way of architecture courses that take architecture from an enterprise or SOA perspective, so that we can enable our students who are finishing graduate school to be more and better prepared as they enter their new job market.

Gardner: These are excellent points. Steve, you and I discussed, when we spoke about some of these issues a month or so ago, that you were also trying to encourage universities to create the curriculum and the definition of these jobs. Can you fill in our listeners a little bit on what you’ve seen?

Nunn: That’s right, Dana. Something The Open Group launched at the end of January this year was the Association of Open Group Enterprise Architects (AOGEA), which really is -- the analogy here is the one somebody used earlier with attorneys or CPAs -- to do for enterprise architects what the Bar Association, for example, would do for attorneys, all joking aside. I think one of the things that we’re trying to do is partner with various types of organization in creating this community and this professional association.

One of those groups is the academic community, so we are putting out feelers to various universities to explore the possibility of getting enterprise architecture on the curriculum. There is one university that we are aware of where there is actually a TOGAF module in some of their courses. Obviously, changing a curriculum is a multi-year project, or multi-year plan. It’s not going to happen overnight, but in the interim, one of the categories of membership for the association is students.

So, those who are on a course of some description inside the university or even working in a job and doing part-time study, can join the association, be part of the community, get the information that’s available there, be on the news groups, maybe take part in the local chapter, or whatever they want to do to start building up some experience.

I had somebody come up to me a couple of weeks ago, after a talk I gave, and said, "This is exactly what I’ve been looking for, because in my organization I’m quite junior and the people above me really aren’t that interested in enterprise architecture, and certainly not SOA, but they won’t listen to me because I’m too junior. So, I need to get some experience or immerse myself in this field. Some kind of virtual community that allows people to do that is going to be a great help to me."

So, that’s one of the thing we’re trying to do. There are various membership categories, and student is just one of those.

Gardner: Now, it does seem that we have a climate of opportunity here. There’s the track for developer to move above that role and embrace more business understanding in domain expertise, and that would be a track. We’re looking at more universities preparing people for these types of roles. We’re probably going to find, again, variety within organizations in terms of business analysts or non-tech people coming into this role, because it requires influencing and consensus building, and so forth.

Usually, in markets, when there’s opportunity and there’s scarcity -- and these are probably well-paying jobs -- we would expect for the supply and demand to even out at some point. For those in the field like yourself, John Bell, am I overly optimistic that this supply and demand is going to mesh, or are we looking at something a bit more serious in terms of the next three to five years where there’s going to be a serious deficit of talent?

Bell: I think that the supply and demand will eventually mesh, but there may be a gap in the next year or two. I don’t know if it will carry out for three to five years.

Gardner: Well, thank you very much. Let’s move on to our next subject of the day.

In March an announcement came from the consortium of large IT vendors including SAP, IBM, Oracle, BEA, and Cisco. They have formed a series of proposed standards, the Service Component Architecture (SCA) specification and the Service Data Objects (SDO) spec. We are not quite at the standards level, but that seems to be the goal, to take this approach through OASIS, which is the organization that’s overseeing the Web services specifications and standards, WSDL, UDDI and SOAP and so forth and all the WS-* specifications.

It seems that the vendors have stepped up and said, "Listen, we need a level of standardization. We are going to do some heavy lifting, create some specifications, and then we are going to hand them off to the standard organization." This strikes me as an important juncture in the maturity and real-world applicability of SOA, and I wanted to test that hypothesis on Jim Kobielus.

Kobielus: Yeah, it’s very important. It’s clear that the SOA paradigm requires ever more high-level abstractions to enable easy development of very complex, orchestrated, end-to-end services. The SCA and SDO specifications – the initiative has been being going on for a couple of years – have come a long way and they’ve got pretty significant support throughout the industry. Microsoft is one of the few important holdouts. It's not only the high-level abstraction for developing competence services, but also, especially, in my area, the SDO, the high-level abstraction for working with heterogeneous data. I see the SDO, in itself, becoming potentially the standard industry framework for what’s called the semantic layer for any data integration.

So, I’m very keen on the potential for SDO, for example, within the business intelligence space, the data warehousing, and the enterprise, information, integration space. The fact that now OASIS will be taking over ongoing development of SDO, puts it on a very important fast track. Hopefully, we’ll get some of the business intelligence (BI) vendors like Business Objects and Cognos behind it. That’s one of my fond hopes.

Gardner: Jim, you’re tracking the data management side on this quite deeply. Do you think that SDO has the potential to become the ODBC/JDBC of SOA in terms of what those things enabled and empowered for distributed architecture?

Kobielus: It’s quite likely, because it’s leveraging the whole WS-* stack and the whole notion of semantic web that’s been kicking around for long time. Tim Berners-Lee keeps this going. It’s really a utopia of interoperability, where the semantic layer is the resource description layer for describing metadata. If you look at the so called semantic web’s specifications like OWL, RDF and a few others, they have not achieved takeoff velocity in the data management world. I can count on one hand the number of the BI or data warehousing vendors that are implementing OWL, for example. The semantic web has not really gotten any traction with standards or specifications where it counts.

With SDO, I still don’t see significant traction yet in the whole BI space, but the fact is that every BI vendor is SOA-focused and enabled and getting ever more so. That’s one of the clear gaps I’ve been seeing in the whole enterprise information integration (EII) side of it all in terms of distributed master data management (MDM). Every vendor, including Business Objects and the others, have their own semantic layer. That’s what Business Objects calls it.

As yet, there is no federated semantic layer specifications, but customers are asking for federation of say, Business Objects, Teradata, Microsoft, Oracle, and I believe that at some point BI and EII will converge around a common set of standards. I’m getting further and deeper into SDO, and it really looks like this is a strong potential framework for them all to work together going forward.

Gardner: A framework for a common and federated metadata approach, is it not?

Kobielus: Yeah, exactly.

Gardner: Now, the politics here struck me as a little bit interesting. Ed Cobb of BEA, who was on the call describing the movement of these specifications to OASIS, said that he hopes that this does for SOA what J2EE did for n-tier in distributed computing, which is to create a climate of growth with application server vendors coming together and ISVs building applications that take advantage of these. That sort of exploded during the mid- and late-1990s into what is now a predominant architecture for enterprise applications, as well as large Web commerce and online commerce types of applications.

Neil Macehiter, what do you make of the politics here? If J2EE did for distributed computing what they hope this does for SOA, why aren't SDO and SCA going into the Java process?

Macehiter: You've hit the nail on the head there. I think there are a couple of issues here. First is, if it went into the Java Community Process (JCP), you’re talking about SOA based on purely Java. As my colleague put it at the time, it’s like a three-legged dog running in a race. If you’ve only got Java, then you’re not really addressing one of the core propositions of SOA -- that it is about heterogeneous interoperability, with services based on multiple languages.

Gardner: What happened to "write once, run anywhere?" Wasn’t that heterogeneity and interoperability?

Macehiter: One programming language, and that’s the distinction. The SCA and the SDO are multi-program, multi-language.

Gardner: So, an abstraction above Java their pointers will make sense?

Macehiter: Yeah. Actually, the second point is that, in part, the creation of SCA and SDO was motivated by the frustration with the J2EE process. Enterprise JavaBeans (EJBs) and things like that never really took off. Some of the lightweight programming frameworks, Spring and Hibernate, were just taking great chunks out of J2EE in terms of deployment.

Then there was a significant amount of discontent among the Java community around the support for Web services, which is clearly one of the key enablers of SOA. Those three things, plus what Microsoft was doing with the Windows Communication Foundation (WCF), and the work they’ve been doing around it, caused the big J2EE players to think, "Well, actually we need to do something different." That was the motivation.

Gardner: I suppose it’s water over the dam at this point, but perhaps if the J2EE and a variety of Java framework specifications had moved into a standards organization like OASIS five years ago or so, the SOA specifications and the Java specifications could find themselves in the same organization. That seems now not to be the case.

On the other hand, if Microsoft has been the holdout in terms of embracing SDO and SCA, and is focusing more on .NET Framework -- but OASIS was a place where Microsoft felt comfortable going when it came to working with folks like IBM on the Web services standards -- do you think that this might move the hearts and minds of Redmond, Washington toward a bit more SOA compatibility and the programmatic approach to SOA, rather than just the interoperability approach?

Macehiter: My gut feeling is "no." And the reason is that Microsoft has collaborated with the likes of IBM, BEA and others, its historical competitors, up to a certain level up the stack. But the level at which SCA and SDO are operating is at the level where Microsoft has a massive investment, and a significant proportion of its business has been driven out of this at the programming model level.

So, I think it would take a lot for Microsoft to move to support SCA and SDO within the composition framework that they have, which is fundamentally around Visual Studio. Whether we are talking about BizTalk or Sharepoint or Office, it’s all around that programming model, which is tied into WCF and Windows Workflow Foundation. So, I just think the battle line is drawn at that level.

Gardner: What we are facing is perhaps an important decision within enterprises and service providers, software-as-a-service (SaaS) providers, and ISVs as to which role you perceive for .NET playing in Microsoft’s tools and process runtimes. Are they a subset of SOA, or are they in fact the master -- and the rest of the SOA componentry is the slave?

That would be one way of looking at it. The other would be that .NET should be just another spoke in the hub of all SOAs. Jim Kobielus, do you think we are going to be facing this sort of a face-off between the role of Microsoft as the hub or the spoke?

Kobielus: I think Microsoft has gotten much more open to being just one spoke. But I wouldn't use the hub vs. spoke analogy here. They’ve become more comfortable with the notion that they’re just one node in a vast mesh on the Internet in terms of Web services and SOA. So I don’t see Microsoft in face-off mode in the SOA world, or where SCA or SDO are concerned. A couple of years ago it might have been different, but it has changed.

Gardner: How about in terms of the role of their communications, their ESB, WCF (the former Indigo) and the role of BizTalk? It seems as if they are happy to be a spoke on one level, but they’d also like to be where business process is coded and logic is instantiated, and therefore become "the place" where SOA is driven, the dashboard from which SOA is driven.

Kobielus: Well, pretty much every BPM vendor wants to be that hub, that business-process hub, and Microsoft is not alone. Companies like TIBCO with ActiveMatrix, are interesting, because basically what it is doing is it is virtualizing the app server or the integration server, so you can run your .NET logic and your J2EE logic, and so forth in different containers on the same platform. That kind of architecture is more where the industry is going. In that case, TIBCO necessarily isn’t trying to be the one and only integration and logic hub out there. It is simply trying to be the hub of all hubs, but one of the hub of many hubs federated to each other.

Gardner: Is it that you don’t agree with Neil then, that if Microsoft is a bit more ecumenical on this, would we expect them to embrace SCA and SDO? Is that what you expect?

Kobielus: It’s a relative thing -- getting ecumenical. They get ecumenical when they are good and ready, like they’re doing right now in the identity space, with this whole notion of user-centric identities. It’s taken them a couple of years of sitting back, watching things like OpenID and Higgins develop. And then finally they make a token offering that says, “Okay, we’ll implement OpenID in the next generation of the Vista Card Space."

And, once again, my sense is they’re going to wait quite a period of time, at least a year, before they make any public pronouncements on the extent to which they are going to work with OASIS on SCA and SDO. It’s just their nature, and they are going to pursue their proprietary approach, as long as it holds out, and as long anybody will implement it.

Macehiter: I’m not suggesting that this a face-off. What I am suggesting really is that it comes down to the point of control that an organization or vendor has. Microsoft does not want to be denigrated to the plumbing, even if that’s interoperable plumbing. The point you raised about TIBCO with ActiveMatrix is that it’s actually using the SCA programming model in order to provide this abstraction.

So, ultimately in that environment who’s controlling it? TIBCO is becoming the master, and it will be the same in an IBM environment. SCA may be under the hood, but ultimately there will be a point of control that IBM wants to wrest for its process server for its development tools, and that’s what Microsoft wants to do. The challenge is that SCA and SDO are trying to do the same thing as Windows Communication Foundation and Microsoft tooling around .NET and SOA.

Gardner: To wrap up, it seems that it's not a face-off, but perhaps there are, as Jim points out, a lot of vendors who would like to be that over uber-dashboard, point of control. Some are BPEL-focused and others are taking different tacks. SCA seems to be playing a role for many of them, and Microsoft would like to play that role as well -- but perhaps thinks that it has an advantage through the way it’s architected up to this point. The market will, I suppose, determine who the ultimate winner is.

Macehiter: I think a lot will depend how quickly the tooling comes out around SCA and SDO, as an alternative to Visual Studio.

Bell: The other piece too is, as Microsoft tends to build those kinds of capabilities in as part of the operating system, other vendors tend to create them as standalone products and infrastructure pieces. If you are a small company, having it built into the operating system is a value to you, but in large, heterogeneous environments that can be costly to you. So, that’s always been used by Microsoft. If you look at CORBA versus COM and DCOM, it’s the same story.

Macehiter: Absolutely.

Gardner: You’ve been listening to yet another BriefingsDirect SOA Insights Edition, Vol. 15, for the week of March 19, 2007. We’ve been joined by Jim Kobielus, principal analyst at Current Analysis. Neil Macehiter, a research director at Macehiter Ward-Dutton. Steve Nunn, the vice president and COO of The Open Group, and John Bell, enterprise architect at Marriott International.

I'm your host and moderator, Dana Gardner, principal analyst at Interarbor Solutions. Thanks everyone, and thanks for listening.

Listen to the podcast here. If any of our listeners are interested in learning more about BriefingsDirect B2B informational podcasts or to become a sponsor of this or other B2B podcasts, please fill free to contact Interarbor Solutions at 603-528-2435.

Transcript of Dana Gardner’s BriefingsDirect SOA Insights Edition, Vol. 15. Copyright Interarbor Solutions, LLC, 2005-2007. All rights reserved.