Webinar: IDC Research Shows SOA Adoption Deepens in Enterprises Based on Key Implementation Practices

Transcript of Oct. 14, 2008 webinar on SOA research and how companies are implementing SOA more strategically based on essential adoption best practices.

Listen to the podcast. Download the podcast. Access the Webinar. Learn more. Sponsor: Hewlett-Packard.

Download the IDC report "A Study in Critical Success Factors for SOA."

Introduction: Hello, and welcome to a special BriefingsDirect presentation, a podcast created from a recent Hewlett-Packard (HP) webinar on Service Oriented Architecture (SOA) adoption trends. The webinar examines recent findings and analysis from original IDC surveys and research into actual enterprise SOA use and their reported business outcomes.

We'll hear from executives at both IDC and HP on how SOA is rapidly increasing in its importance and value for developers, architects and IT strategists. The presentation is followed by a question and answer session from the live webinar audience.

Please now welcome the webinar moderator, BriefingsDirect's Dana Gardner.

Dana Gardner: Hello, and welcome to our live webcast, “Expanding SOA Adoption to Mainstream IT,” brought to you by HP and InfoWorld. I’m your moderator Dana Gardner, principal analyst at Interarbor Solutions.

Today we’re going to examine fresh research from IDC on SOA adoption patterns and what users of SOA identify as success factors. In other words, for those doing SOA right, what it is that led them there, and what is it that they’re doing that others can learn from, in terms of best practices and insight?

We’ll hear from Sandy Rogers, program director for SOA, Web services, and integration research at IDC. We’ll also hear from Kelly Emo, SOA product marketing manager for HP Software.

Now let's dig into SOA use today, and the adoption patterns that show how and why SOA is moving into mainstream IT. We’re seeing a move to a strategic SOA value that support business goals and not just SOA that supports IT goals, such as the benefits around code reuse or development agility.

We’re starting to see a great deal of movement to the strategic value of SOA, and that means moving toward the aspects that create the need for governance, develops SOA benefits across larger business processes, and starts to show the paybacks in terms of actual business outcomes.

What are the factors that determine the success of SOA, generating that strategic and business level payback? Let's now go to Sandy Rogers at IDC, and learn more about her research into success in a SOA world.

Sandra Rogers: Thank you, Dana. Hello, everyone. First of all, what we want to do is take a look at what has brought us to this period of time -- where we now need to create enterprise-level systems.

Right now, we’re seeing a lot of systems where their foundations are basically breaking, and we’re dealing with a mixture of different generations, different types of systems, different ways that they were developed, different technologies, and different ways that they are and continue to be procured. Enterprises are challenged to address new and changing business requirements, and that volatility of business change is increasing at very rapid rate.

Organizations are looking for much more consistency across enterprise activities and views, and are really finding a lot of competitive differentiation in being able to manage their processes more effectively. That requires the ability to stand across different types of systems and to respond, whether in a reactive mode or a proactive mode, to opportunities.

The types of individuals who are being served by these systems are different, and that’s because of the extended value change, new types of workers entering the workforce, and many different business models that require either some type of self-service capability, or even more of a high-touch personalized type of engagement and experience with systems.

What we’re finding is that, as we go to this generation, SOA, in and of itself, is spawning the ability to address new types of models, such as event-based processing, model-based processing, cloud computing, and appliances. We’re really, as a foundation, looking to make a strategic move. With that kind of structure, it's also balancing freedom.

So, moving on, what we see -- and this is a poll that was recently run by IDC this summer, primarily with mid- and large-sized organizations -- is that if they haven’t already adopted SOA, they are planning on it, and at greater levels of engagement. So, if it is not going to be "the" standard for most or all systems, it's important, and will be used for all new projects, or it's a preferred approach.

The issue is not necessarily deciding if they should go toward SOA. What we're finding is that for most organizations this is the way that they are going to move, and the question is just navigating how to best do that for the best value and for better success.

According to the same poll, what we show is the top IT objectives and challenges for SOA. We also asked for business objectives and IT objectives. What's different from past surveys that we've run is that the flexibility and agility to respond to changing business needs is actually number one now. In previous responses, that had been in the top tier of three, but not necessarily the first one.

What are most interesting are the top challenges in implementing SOA. All of our past studies reinforced that skills, availability of skills, and training in SOA continue to be a number one challenge. What’s really noticeable now is that setting up an SOA governance structure has reached the second most-indicated challenge. This is the top 3 of 18 options.

In the past you may have seen security or other technical elements, interoperability, or maturity of standards. What this is telling us is that we have reached another stage of maturity, and that in order to move forward organization will need to think about SOA as an overall program, and how it impacts both technology and people dimensions within the organization.

We find that when we ask this from a business objectives and challenges view, the business is looking at more efficient processes at greater levels of service and customer service throughout their entire environment. Some of their top challenges involve gaining agreement on what processes and services should exist, how to define those, and how to agree upon those, and also rallying individuals around support of budgeting and funding for SOA. This all points to an overall need to step up the ability to address this as a managed business discipline, versus a technology discipline.

Defining SOA Success

We wanted to look at how SOA's success is actually defined, even though SOA can have varying definitions amongst individuals, and what factors and practices in these organizations that are successful have the most impact. We wanted to see what tactics and technologies are being leveraged, and how they are being leveraged, and how they are being introduced and expanded within the enterprise.

Then, we wanted to see what other words of advice experienced leaders want to impart to others, as we are seeing a next wave of adopters that may be a little bit new to SOA, versus those that had come before.

So, with this study, we did primary research, mostly U.S.- and European-based companies who had successfully implemented enterprise SOA programs. Most of them had from two, to two-and-a-half, to over eight years of experience. Some of these companies actually had started their SOA endeavors at the turn of the century.

They’re senior level individuals with enterprise perspective and they’re primarily from the IT ranks. They also have had certain levels of experience that might range from CIO to enterprise architect, to quality manager. So, we got a broad-brush view, but most of these individuals were actually charged with, or were a core part of who was driving their SOA initiative in their organization.

This was based on a semi-structured interview format, so that while we wanted to capture some basic information about the overall IT environment structure, the SOA initiatives in particular, the technology, topology, the business goals, and drivers of the organization, we really won't have that broad brush view to present a context.

We also did this in a semi-structured way, in order not to skew the results and to unearth varying elements that may have influenced their success, despite what these individuals brought to the table. And, there was representation across various sized companies and industries sectors.

A few of the overarching trends reinforced what we have been seeing in some of our studies. We are indeed moving from project- and application-level SOA to more of a system and enterprise scale. And, the short- and long-term impact of SOA needs to be better understood and addressed. Enterprises need to manage the expectations of the individuals in the organizations as to how their roles will be impacted, and what kind of benefits they may get on a short term basis, versus that long term view and accumulation, and they need to try to balance strategy with tactics.

While technologies are key enablers, most of the study participants focused on organizational and program dynamics as being key contributors to success. Through technology, they are able to influence the impact of the activities that they are introducing into the overall SOA program.

That success can be defined in multiple dimensions, but rising to the top, we found that, in part because of their roles, the pervasiveness of SOA adoption in the enterprise was a key determinant of how they were looking at it, whether their programs were gaining traction in the right ways, and were being successful. They were achieving clear business results, and those that can be measured.

When we gathered all of this information, we had many different tactics and activities, and some of them started to become repetitive in our research. We created a framework of varying components, and elements that impacted success. Then, we aggregated these into seven key domains, as we call them.

Domains of SOA interest

While different elements can impact each different domain, and vice versa, it was interesting trying to think of a way to present his. All the different domains impact one another. Therefore, if you’re able to handle trust, you’re able to influence organizational change management effectiveness. If you’re able to address business alignment, then you’ll have much more success in understanding the impact on architecture and vice versa.

With this, it's much more of a network of different activities and components. The technologies interact on the network's foundation. When you really think about it, services is basically a network topology. SOA puts a wrapper around this environment, and tries to give it a foundation and a framework, for which it will function effectively.

The seven domains are: Business Alignment, Organizational Change Management, Communication, Trust, Scale and Sustainability, Architecture and Governance.

Now, we’re going to briefly go over these seven domains, and give some key trends that we discovered about how different activities and tactics can make each of these areas much more effective?

With business alignment we discovered that with these organizations and these individuals, SOA is truly understood as a business discipline. Now, many of them did admit that within their organization. They still have a way to go to educate, and incorporate SAO as more of a business view. It's often seen as more of an IT agenda, but that is starting to change. But, they themselves, in the way they have approached the issues, and the way that they are thinking about their program, see it as a business discipline.

That alignment of business to what is being incorporated in the program, services, and processes that are being created means the involvement of key individuals who have a keen understanding of the business. Sometimes, it might mean involving someone at a high enough level within a business division who can see how activities within that business division may be impacted or may impact other divisions, yet have enough understanding of some of the details around what transpires, what incorporates the business, what defines the business -- the elements and the processes -- and get them involved in the overall initiative.

These individuals also can influence others. They not only influence outcomes of what is actually being created, but they can actually influence the acceptance rate, the understanding of services of SOA within their network of individuals, and within the activities that they are involved in.

Many of these individuals can help with reflecting on what the current state of the environment is. That can actually help define what future state might need to be created. Think about the overall framework. I believe that you can get access to this and that HP is making it available to you for more detail. I know it's difficult to see here on the screen.

One of the factors highlighted here is indeed getting those individuals really involved, and helping them with determining a key business taxonomy, and vocabulary. It's very important that everyone get on the same page of how they are going to communicate with one another, and define that within an aligned business appropriate to what is being presented.

If we look at the next domain, organizational change management, one of the key factors we found is that many of these individuals did what they could do to ease the transition to ensure that individuals may not have overly complex requirements posed to them at the onset. They tried to figure out what the existing modus operandi was, incorporate what they could into that, and move the organization along. The key is to disseminate that knowledge, and give tools and different technologies that can help with that change, without imparting too much complexity, which disrupts, and/or impacts the goals that they are trying to achieve on a day-to-day basis.

This is very important, versus task initiatives that we have seen, where organizations put together a road map, expected the individuals to follow certain protocols, but really didn't think about how that would impact everything else that these individuals were involved in.

It also gets them to think outside the box across the enterprise. A way to do that is to show how different services are being created and what it looks like, giving them an overall understanding of the program, and driving cooperation through examples, and helping with their understanding by presenting them with concrete examples in their context. Then, they can start to envision what other types of services and processes it could be impacting, and who to get also involved if they want to build out this network impact.

One thing we found is that it's important to have that overall view of the program, of the enterprise, and to have key individuals as part of the central architecture team, the center of excellence, or whatever, work side-by-side with individuals on the different divisional development teams, the different business analysts involved. They can start to understand and impact the outcomes of initial projects.

Then through a train-the-trainer approach, they can get more individuals involved in utilizing what could be at their disposal. As time goes on and these organizations start automating more of the capabilities in the foundation for SOA, all of these different parties understand how to engage with the systems, take advantage of it, and then can start envisioning how to utilize them to their own success.

Successful communication proves essential

The next major domain is around communication. It was very important to these leaders that they were seen as a business leaders, as well as an IT leader. They were also evangelists, and politicians. They actually went out and did one-on-one discussions with key stakeholders. They would have discussions about what policies, protocols, and standards they were thinking about incorporating.

By the time those decisions needed to be agreed upon to gain that buy-in, a lot of that lobbying had happened behind the scenes, and there were a lot of lunch-and-learn type of sessions. That was really making those connections, and showing individuals how they could not only impact the success of the overall initiative, but also how they could actually gain some things from cooperating and drive that network effect.

In order to do that, a lot more visibility needed to be presented in a variety of different forms that these institutions used, and accessibility to this information is key. Trying not to have too many middlemen and trying to automate it, so individuals find what they needed, was very important.

Many of them had designed their own dashboards and wikis, different ways to present information on the overall program. They were thinking about the details, introducing things like registries and repositories from a more technical dimension. Engaging in more of a collaborative atmosphere really drove a lot and also allowed individuals to communicate with one another, and not always through the central team. This was very important.

Some smaller initiative may be able to run manually through a central group, but as soon as it started to extend, that's where they found that allowing different conversations and negotiations to happen, being able to have a potential policy consumer, finding various services, knowing how to engage with the service provider, and having that as automated as possible will not disrupt their daily routine. It will make it easier and will also capture all the necessary information to go forward, without having a lot of rework and lot of hand-holding. This was again very important that that kind of visibility and accessibility is so key.

That dovetails with the next domain, which is really about building trust, enabling cooperation, and building in a sense of security that these services are going to run and be available for someone to consume. Developers can now think about that loose coupling and drive more of a SOA architecture in what they’re developing. That sense of security is beyond just traditional security. It's performance. It's ensuring that the availability of that service is going to be there under load. There are lots of different concerns.

We found that many companies standardize security in the form of services embedded within the framework, foundation, and infrastructure that they had created for the SOA program. That, in and of itself, started alleviating the pressure on individuals to know exactly what security protocols to cover. They ensured identity rights and they could be protected, and mitigated their risks on inappropriate use of services.

They understood that this secure environment is important, and that the reputation of a service matters. That means validating that standards and policies were upheld and integrating what they could regarding testing and validation in the course of the lifecycle of developing their services. By the time a service was posted and available, that service could be depended on and there were no concerns about it. If this was being done manually or ad hoc, people found that they had different experiences.

One organization we interviewed found that over 50 percent of the services that were in production didn’t adhere to policies. As soon as they automated that and brought it up to 100 percent adherence, the network effect started to take over, regarding the consumption, use, and even the development of services.

The transparency and visibility into past behaviors and the history of the service is very important to individuals who are going to take that risk and take that step that they are not going to develop something on their own, but are going to reuse and capitalize on what is built out as this network of resources and services.

Architecture over technology

Of course, we can't talk about SOA without mentioning architecture, and what we kept hearing over and over again was that architecture should come first over technology. Many of these companies had to come up with reference architecture and had to prototype the technology and the reference architecture, so that it would be able to address reality.

In the past, I’ve heard, some of these enterprise architecture teams actually did, beyond a proof of concept, prototype the varying scenarios that were likely to serve in rolling out the architecture to more entities, more systems, more and system type.

So, setting the standards, setting the reference architecture, defining messages, schemas, and protocols -- more so than mandating the specific use of certain technologies -- was a real learning experience for these organizations. Large organizations, especially, may not be able to influence every technology that will be used or can't envision every single technology that will be introduced to the environment in the future.

What they needed to do was focus on all the architectural dimensions, best practices, and standards, and define those, so that when they needed to test whether technology could interact appropriately, they had much of this defined. That allowed that level of flexibility to flow through to the different divisions and teams. Thinking about architecture beyond technology, about the process of how you are dealing with setting up architecture, how you are engaging with architecture, and seeing it through it's different stages and lifecycle are very important.

Updating process and SOA technologies, vis-à-vis the overall IT environment, started to surface as a key concern in what needed to transpire. You can't treat SOA as a silo. You need to think about this as an overall environment that incorporates many different options. One of the major reasons people are moving to SOA is to take advantage of a heterogeneous mix of resources, whether internal to an organization, or external through an organization.

Thinking through those dynamics leads to the next domain, which is scale and sustainability, to prepare for that viral network effect and to automate and test for higher volumes of demands early on. We found some organizations that had either built some of their own technologies, or had sort of incorporated things that were available very early on in the decade. When it came time to scale out, and there was more pressure imposed on these systems, they really couldn’t handle the demand.

What we learned from them is to really make sure that you test that early. Even though you may not have the volume now, you may some day, and even one service could get hit by an unknown amount of consumption. Being able to prepare for that, and prepare your architecture, as well as your policies and your governance processes, to be more distributed and federated to prepare for more of a federated type of environment, means that these policies and technologies can scale out.

We found that it's a learning experience to get the right definition around a service, the right fit, and the right granularity. That's something that comes with experience, and you might need to go through a couple of iteration of services before people understand how to best keep volatility down on services, put the right level of abstracted tiers for processes and rules, and plan and test that you have the right levels. Then, you can scale out.

That may mean atomic services at some levels and more broad-grained services at others, and seeing how that impacts the infrastructure, and testing that out in all the different scenarios and dependencies that could exist.

Governance helps assure ongoing success

Last, but not least, the final domain, where we start impacting everyone, was the issue of bringing in proper governance. Thinking about it in balancing control with empowerment and driving consistency throughout the environment is very important. People needed to plan on the processes and ramp up the speed of development and deployment into production, until having that level of consistency was giving them huge amounts of business value, savings efficiencies, and opportunities in the market to differentiate and compete.

Making moves early to automate was a learning experience for a lot of these organizations that didn't do this. They found that they could have not only expanded their programs more effectively, but they could have mitigated their risks. They could have avoided a lot of rework, as they had automated governance processes earlier on, integrating it into the overall flow, as we stated before, and thinking about it not as a just “these are the things that need to be met and this is the information that needs to be captured.” It's really thinking about the processes.

What happens is that there are these exceptions, dealing with review committees that should be involved, determining the right roles and responsibilities, and sometimes that may mean amending what you have.

In other instances, it was creating anew. We found in other studies that a lot of organizations did not have strong governance. SOA almost forces these companies to do what they should have been doing all along around incorporating the right procedures around governance, and making that a non-intrusive approach.

If you make it too complex, no one is going to follow it. If you make it a mandated activity, without a lot of tools to help facilitate, it becomes a chore to do that non-intrusively. Also speaking to non-intrusive runtime governance, many of these organizations found that you really should have a centralized foundation of runtime governance incorporated into the fabric of the SOA architecture, and technology infrastructure.

From all the right monitoring and management around services, in particular portions of this network that's integrated together, you can gain that overall perspective and drive what’s necessary to move forward, both from a business goal perspective and from an IT topology perspective.

To quickly wrap up, here are some additional words of advice from the field. We found that enforcing policies, not putting off governance till later on, was very important, putting more efforts into business modeling, which many of these organizations are doing now. They said that they wished they had done a little bit more when thinking about the services that were created, focusing on preparing the architecture for much more process and innovation.

So, with that, I’d like to hand this off to Kelly Emo to speak on HP's offerings in the SOA space.

Kelly Emo: Thank you, Sandy. And good morning and afternoon, everyone. I’m just going to wrap up the webinar with about 10 minutes or so. I’m going to hit four main points that I think will dovetail nicely with what you heard today from Sandy Rogers, some great insights coming from customers who have “been there, done that,” and have been working with SOA for while. It’s real advice that we all can learn from.

I’m going to dive now into a little bit more context around why, if you’re doing SOA, now is absolutely the right time to be thinking about and working on a governance program. I’m going to share a few key governance-specific best practices that we've also gleaned from our customers who have been down the road of their SOA journey.

We’ll talk a little bit about the value of using an automated SOA governance platform, to help automate those manual activities and get you there faster. And, I’m going to wrap up with one customer success story, a customer who is almost complete with their SOA journey -- about 70 percent there -- and who sees significant business benefits through their investments in not only SOA, but SOA governance, and SOA management.

You heard from IDC the seven critical SOA success factors that came from this in-depth analysis of customers. The point that I want to reiterate here that was so powerful in this discussion is the idea that the seven domains are linked. By putting energy and effort in any one of them, you are setting yourself up for more success across the board.

What we are going to do now is drill down into that domain of governance. You’ll see as we talk through some of the key capabilities for SOA moving to the enterprise from a governance perspective, how it will help establish other success factors, like building our trust, or facilitating communication across IT silos, for example.

I’m just going to touch on this briefly. It's interesting here. HP has used this graphics for quite a while in talking about the need for governance, having a governance program that helps bring together the different IT stakeholders that play a part in the successful delivery and realization of business services that return the results that the business expects and that behave, so that they can be easily consumed, and reused for even more responsiveness and agility.

This graphic rings true even more now with the kind of pressures that our businesses and IT are under to realize the results of their investment in SOA faster with existing resources. What we’re seeing again and again with customers who have been implementing SOA and going down to the path of scaling it out, is that they have to invest in processes and best practices to not only deliver services, but to ensure that the services are of the highest quality, that they can be managed over time, and that policies are consistently applied, so that we can handle events like change and new consumption in a way that delivers the result that we expect.

We see many of our customers now crossing the enterprise scalability divide with their SOA, looking to incorporate SOA into their mainstream IT organizations, and they’re seeing the benefits of that initial investment in governance help them make that leap.

So why invest in SOA governance now? It's an interesting question I’ve been getting a lot lately. “Hey, you know, we’re under a lot of economic pressure, budgets are tight, there's fewer resources to do the same work.” This sounds counter-intuitive, absolutely, but this is the right time to make that investment in SOA governance, because the benefits are going to pay off significantly.

SOA governance is all about helping IT get to the expected business benefits of their SOA. You can think of SOA governance, in essence, as IT's navigation system to get to the end goal of SOA. What it's going to help IT do, as they look to scale SOA out, is to more broadly foster trust across those distributed domains. It's going to help become a catalyst for communication and collaboration, and it's going to help jump-start that non-expert staff.

You may recall that Sandy mentioned one of the biggest challenges with SOA is building out the education and expertise among the staff. If governance can assist in shrinking that learning curve, enable IT organizations to understand the unique attributes of SOA and what process is need to be applied to successfully realize their SOA goal, that will help accelerate the transformation that needs to occur from the SOA perspective.

The thing that's key about governance is that it helps integrate those silos of IT. It helps integrate the folks who are responsible for designing services with those who actually have to develop the back end implementations and with those who are doing the testing of performance and functionality. Alternately, it integrated them with the organizations that are responsible for both deploying the services and the policies and integration logic that will support accessing those services.

So, governance becomes the catalyst for integrating these silos and facilitating communication. One of the keys, one of the best practices we are seeing across these customers, is that they approach governance from a lifecycle perspective. They are not just thinking of one aspect, but they are actually thinking of all the different collaboration points, all the different key decisions that need to be made, as a service goes from initial concept, into its design, into the development organizations that are responsible for delivering the implementations, out into the QA organizations that are responsible for defining the requirements for testing aspects of those services.

This includes the functional test, the performance test, the security test, and then out into the operational teams. These teams will be responsible for deploying services into the network, and understanding the implications on the stacks, and data sources, services access, and those that are responsible for deploying policies, such as authentication policies, authorization policies, the protocol mediations, and all the way back into the change process.

On ramps to governance

I'm not saying that an organization has to automate and create a complete governance infrastructure for all aspects of the lifecycle on day one. Certainly, there’s going to be the starting point that's going to make the most sense, based on the organization's maturity. Maybe the first thing to bite off is automating how organizations can get visibility of services and putting some automated policy checks in place on the design side for testing supportive standards and interoperability.

By keeping a perspective on lifecycle governance, your organization can be primed and ready to handle SOA, as it scales, as more and more services go into production, and more and more services are deemed to be ready for consumption and reuse into new composite applications.

The key is to keep a service lifecycle governance perspective in mind, as you go about your governance program, and automation is key. Sandy touched on this, and my intent here is not to talk through all aspects of this slide, but just to show you that there are a number of different aspects of governance that can be automated. If they are, that will have significant efficiency pay off downstream.

Automating policy compliance can bring a huge pay off. Sandy mentioned an example where a customer went into their governance program assuming that people were doing the right thing and found initially that fewer than 50 percent of the services being built were actually in compliance with the design policies that they have established to meet their corporate and IT objective.

Automation that will ensure that those issues can be caught quickly, and the collaboration and processes can be put in place to affect that behavior, and alternately work into that network effect that Sandy talked about can not only be in compliance, but also be something to brag about.

This next slide makes a point that I think is important here. I talk to a lot of folks about SOA. When you talk about a service lifecycle, it's very different than a traditional application lifecycle. It builds upon the concept of an app lifecycle, where you have a planning, building, testing, deploying, and changed cycle, but you also have the impact of consumption. Those are the consuming services and will have a lifecycle of their own.

It involves planning what services they are going to consume in the building out of the composite application, locating those services, engaging in a contractual relationship with the service provider, and alternately testing and delivering that composite application. Then, if there is a change on either side of the equation, if there is a change to the underlying service or a change to the composite application that, in turn, impacts, both the service and the composite application from a expectations perspective and, potentially, in performance and quality perspective.

The SOA lifecycle can actually be thought of as concentric circles. It's an iterative, living, breathing thing, and that's why service lifecycle governance is so key, to keep all these dependencies and the synthetic relationship, moving smoothly forward in terms of meeting its business objective over time, not just the initial deployment, but as both services and consumption patterns change. And governance is a powerful tool to manage this power, and complexity downstream.

So what I am going to show you next are a couple of key things you can do with the governance program to help you manage and scale. The first is really thinking about this idea of the service manager. I talked about the lifecycles of providing a service and consuming a service. A service manager is either a real or virtual person who is responsible for ensuring that the service is delivering against the goals of the business, not just at its initial deployment, but as different people are consuming it.

When I say a “virtual person,” I realize how funny that sounds. I don't literally mean a virtual person. What I mean is that it could be the service provider, or could be a committee of service providers, key service consumers, and maybe a line-of-business owner.

Manage the services like a service

What we are finding more and more now is that organizations are actually investing in a role known as service manager, someone who oversees the implication of not only delivering a service over time, but those that are consuming it. I see this as a best practice that can be supported by SOA governance, and which helps empower them by giving them a foundation to set up policies and have visibility in terms of how this service is meeting its objective and who is consuming the service.

Sandy talked a little bit about this as well. This is another best practice we are seeing mature customers that are being successful, and that's the way SOA governance deploys out there. That's bringing together the robust and well-developed processes around SOA governance and quality assurance, and creating a collaborative environment between those that are responsible for managing the entire testing process of new applications and services and those that are involved with the initial planning, with the design of services, and the governance of services.

You can actually get a dialog going between your enterprise architecture and planning teams, your development teams, and your testing teams, in terms of the expectations, and requirements right upfront, as the concept of the service is being ferreted out.

Get that communication occurring, so that everybody knows what are the key aspects we are going to test, how we are going to deliver it, what the expectations are, and what the policies are from a quality perspective that’s going to drive governance decisions downstream.

A really simple example comes from a customer who automated a very simple policy, which said that if a service has any critical or serious defects, we won’t allow it to be pushed into the staging environment, but rather we’ll flag it, bring it back into the testing process, and have it a discussion around how we’re going to mitigate those requirements. That doesn't sound like a lot of work, but it was important as the number of services scaled up to really automate that collaboration, and ensure that things didn't fall in the cracks.

Ultimately, it's about driving collaboration between the enterprise architecture and development team and the quality assurance team through automated governance, connected to quality. The same thing can be said about operations, by aligning and integrating the processes and knowledge that’s gleaned through the planning, and design processes, and the governance processes about how services are expected to behave once they go into deployment.

With the information that flows to the operations team, what we avoid is throwing a service over the wall, and then testing to figure it out. As a result, the SOA and the operational aspects of a service that ultimately get realized in production align with the original expectations that we are established.

You get the service behavior that was originally intended, and as your SOA scales and you get more and more services out there, this becomes essential. It keeps that line of communication seamless and flowing between the planning side and the delivery side, so that you get the behavior that meets the needs of the service consumers, and ultimately the business. And again, automated governance can help with this.

HP just recently announced the third generation of our automated governance platform, HP SOA Systinet. We’re positioning this as IT's navigation system for SOA. It’s designed at the core to guide our customers through their SOA journey by automating the governance aspects of the service lifecycle. This will ensure that the policies that are defined and automated, at the design, planning, and build side, as well as at the testing and run side, will map to the goals of the business in IT.

There are a number of functions inside Systinet designed to empower that concept of the service manager, supporting both parts of the lifecycle by providing a service, and the consumption of services, and supporting that collaboration between those responsible for providing services, and those who want to engage in a contractual relationship with them. Ultimately, the idea behind Systinet 3.0 is IT's objective in scaling out their SOA across their enterprise to realize the business value of their SOA investments faster, which is so important in today's environment.

Success stories from the field

Let's talk about a customer who has actually done this. This is the success story of a major European telecom provider. They’ve implemented approximately 70 percent of their SOA objective and right from the get-go, they made an investment in SOA governance. What they have seen over the last three years is an ROI of 327 percent, and it's really benefited them in four main dimensions.

First, they’ve been able to reduce the amount of downtime in the provisioning and delivery of new mobile subscribers services. A lot of this has to do with the fact that the services that are being delivered have been designed compliant to policies and have been tested and confirmed that they will deliver the behavior that was expected in terms of how they execute an operation.

The customer has also been able to increase customer retention, which has really resulted from two things, faster delivery of new services, and reduced downtime.

They've been able to reduce the time to market behind the delivery of new services, because of the timing of the communication flows, and ensuring interoperability and compliance.

And, they have seen an overall IT cost reduction of a significant amount of money, almost a million dollars, with their investment in SOA governance from the get-go.

Investing in SOA governance, as I mentioned at the very beginning of my presentation, while it may require a little investment upfront, can have a significant and powerful payoff downstream, when you go to move SOA out into the mainstream.

With that, I’m just about out of time, and I want to make sure that we have some time for Q and A, for both Sandy and myself. So, here is a pointer to where you can learn more, as both Dana and Sandy mentioned upfront. The IDC research is available on www.hp.com/go/soa, so you can download the reports and dig in to the specific detail behind what was shared with you today.

So, I’m going to turn it back to Dana and let him lead Q and A.

Gardner: Thank you, Kelly. Now that we've had a few questions, I’m going to direct the first one -- from me, actually -- to Sandy. I wanted to find out if there were any real surprises, any unexpected results, when you went out to the field to uncover SOA practices. Was there anything that caught you by surprise?

Rogers: What was interesting was that when it comes to providing metrics around the SOA experience, we have got a long way to go. A lot of organizations knew about different approaches to SOA management in monitoring, and understanding dimensions around the environment well before they made those investments.

Then, they still followed the same pattern that we have seen in past generations of technology where, when we were first being introduced in the marketplace, people said they weren't going to make the same sort of mistakes. It seems that SOA is very much like other initiative.

What they are doing now is a really fast catch-up in finding a lot of immediate value from doing that. Part of that had to do with gaining the buy-in, justifying the investments. What most of these organizations are discovering is that the last mile is dealing with that funding hurdle, showing that kind of value. What they’re realizing is that if they have these kinds of capabilities, they would have been able to measure that value much earlier.

It's more of a word of advice that we were getting than it was a real surprise, but it's something that, as an industry watcher, you just sort of see. Also, a lot of these organizations are indeed starting to tackle Web 2.0, and mashups for other kinds of dimensions within their organization. They really see it as an overarching type of trend. They don't see these as separate technology initiatives, and that's actually a pleasant type of view and a surprise to me as an analyst to see that. That sort of holistic view is starting to take hold.

Gardner: Here's another question directed at Sandy. You mentioned that the automating governance is an important element. What best practices have you seen for convincing the management in these enterprises to start an automated governance program?

Rogers: I was mentioning that just a moment ago. The kind of visibility that you are able to give to management presents the information on what services are being incorporated. But, if those services are designed well, you can actually be able to track that to key performance indicators (KPIs) in business measures and understand how this can be justified and funded? That kind of visibility is very important.

The other thing with automating governance, and I think Kelly referred to this, is that you do not need to do it all at once. That really targets protecting the environment, being able to automate as much as possible, have standards services, and schema automatically populated in the tools, and have that shared metadata concept start to expand. So, whether you’re creating services from one tool or another, that metadata is being captured. As time proceeds, you act on that metadata in the form of what kinds of policies you need, and so the threshold you measure that you can achieve is an overall process.

Gardner: This is a question for Kelly. The European company mentioned that had the 320 percent-plus return on investment. Did they start their SOA with SOA management or governance? What best practices have you learned about when to start managing SOA?

Emo: It’s a very good question. In that particular scenario, they actually started first with management, because they were having downtime issues. What they found right away, as they started to instrument their SOA environment and understand where the issues were taking place, was the part of the problem was inconsistency, in terms of what the operational team understood they were supposed to do, when they provisioned out of service from a load balancing, a security, and an overall performance perspective.

They found very rapidly that if they put governance in place to start to capture those expectations back on the design side, and then communicate those expectations to operations, they were able to alleviate that gap. They are now at a point where they're automating a production deployment of services using templates that come out of the governance side. So, they’re seeing additional timesavings in terms of how quickly they can provision new mobile subscriber services to their customer base.

Gardner: Okay, I have another. I think it’s directed to Sandy. SOA is not happening in a vacuum. There are other major undertakings in IT departments and across enterprises of virtualization, cloud computing, data center consolidation have all have been quite prominent lately. The question is, how does SOA help or align with these other types of goals that IT is tackling?

Rogers: When it comes being able to support an effort like consolidation, the whole idea behind what a lot of people are doing with SOA is to try to consolidate on core functional and information elements, and then share those to the rest of the enterprise, and through different applications and systems.

So, it dovetails very nicely with consolidation. What a lot of organizations might do is try to consolidate first and then think about enabling with services. However, being able to expose varying parts of different systems and have that visibility into the core services and how they are being used can actually facilitate on both consolidation and modernization efforts.

When it comes to initiatives like cloud computing and virtualization, it's really thinking about the overall architecture, what kind of interfaces that you have, what kind of service needs to be supported in a virtualized environment being able to componentize, and modularize it, and allow for that necessary interoperability.

When you think about virtualizing systems, and when you think about that overarching idea of on-demand cloud computing, the first step is interoperability. We found a lot of even on-demand providers who didn't go down the Web services and services-oriented route, having to go back, and re-architect their solution. It's important to have that kind of interoperability facilitated on a standardized basis to enable those kinds of activities to proceed.

Gardner: I believe we’ve run out of time. I certainly want to thank our guests. We’ve been speaking with Sandra Rogers of IDC, and Kelly Emo of HP Software. This webcast is sponsored by HP Software and includes commissioned research from IDC.

You can get more information at www.hp.com/go/soa.

This is your moderator, Dana Gardner, principal analyst at Interarbor Solutions. Thank you all for joining the webcast.

You’ve been listening to a sponsored BriefingsDirect Podcast on SOA adoption patterns based on original research from IDC and HP. Thanks for listening and come back next time.

Download the IDC report "A Study in Critical Success Factors for SOA."

Listen to the podcast. Download the podcast. Access the Webinar. Learn more. Sponsor: Hewlett-Packard.

Transcript of Oct. 14, 2008 webinar on SOA research and how companies are implementing SOA more strategically based on essential adoption best practices.

BriefingsDirect Analysts Discuss IT Winners and Losers in Era of Global Economic Recession

Edited transcript of BriefingsDirect Analyst Insights Edition podcast, Vol. 31, on the outlook for IT in the face of the economic downturn, recorded October 10, 2008.

Listen to the podcast. Download the podcast. Find it on iTunes/iPod. Learn more. Sponsors: Active Endpoints, Hewlett-Packard.

Special offer: Download a free, supported 30-day trial of Active Endpoint's ActiveVOS at www.activevos.com/insight.

Dana Gardner: Hello, and welcome to the latest BriefingsDirect Analyst Insights Edition podcast, Volume 31. This periodic discussion and dissection of IT infrastructure related news events with a panel of industry analysts and guests comes to you with the help of our charter sponsor, Active Endpoints, makers of the ActiveVOS visual orchestration system.

I’m your host and moderator Dana Gardner, principal analyst at Interarbor Solutions, and our panel this week consists of Jim Kobielus, senior analyst of Forrester Research. Welcome, Jim.

Jim Kobielus: Hi, Dana. Hi, everybody.

Gardner: Tony Baer, senior analyst at Ovum. Welcome back, Tony.

Tony Baer: Hey, Dana, good to be here again.

Gardner: And, Dave Linthicum, independent consultant with the Linthicum Group. Is that the correct designation these days, Dave?

Dave Linthicum: That's right. Thanks guys, good to be back.

Gardner: Very good. We’re going to talk primarily today about the burning issue of the moment, and hopefully not for the next 10 years, and that's the financial situation of fairly well-defined panic. We‘re not sure why, but there’s certainly a panic at this point in the global markets, and bailouts and other attempts by governments around the world not necessarily helping, so far. We’re coming to you on October 10, 2008.

Hopefully, when you hear this in the next few weeks, things won't seem quite as dire, but we are going to take a pulse of whether this is panic or whether this is a prelude. We’re certainly not going to look at this through the full lens of the economy. We’re not economists, and people will probably think we don't know what we are talking about, but we wouldn't be alone in that category right now.

So, we will focus it on what we do know a little about, and that is the IT sector, the software business, how this will affect IT vendors, users and enterprises.

First, we've heard a couple of different takes on this whole situation. IBM just came out with some fairly encouraging results, 2 percent real top-line growth and 20 percent bottom-line growth. So IBM says, “Not so bad,” HP had similar results and Oracle as well. They’re saying that we’re seeing some bumps in the road, but certainly not a meltdown. On the other hand, companies like SAP and Dell are saying that they’re really feeling it.

For my first question, I want to take this out to Tony Baer. Is this going to be something that drops the tide on all boats in IT? If not, who are the winners and losers likely to be?

Baer: Well, I think the winners are those who are likely to be more diversified into services, services that can help companies harvest more of what they already have. I was actually doing a mental comparison before we got on the call between, for example, IBM and SAP. In other words, why has IBM reported positive results and SAP hasn't. On first blush, they are both global companies, they both have incredible penetration into the Global 2000.

So, part of it is fairly hard to explain, you have to drill down a little bit deeper into the SAP’S acquisition of Business Objects, a two-product company With maybe some exceptions on the Business Objects side, it’s not so much new sales, but essentially maintenance and upgrades to new versions In a tightening economy, putting in a new version of SAP or NetWeaver is probably a discretionary expense.

Just look at IBM, which, besides the fact that it's much more diversified, has services. The fact is that in an economic situation like this, especially where there are a lot of known unknowns, having a services business is a good way of helping clients to discover new economies. And it's also potentially a much more flexible arrangement than having to put in an upgrade of a new version of SAP software.

Gardner: What I think I hear you saying is that companies that are in the services business, and that have primarily revenue through subscription, might fare better than those who are in a product cycle, where licensing and actual product upgrade, in addition to their maintenance, might be in a situation where people will postpone those upgrades.

Baer: Absolutely!

Gardner: Where does that put Microsoft?

Baer: Good question, because they are in a transition. I just had a fairly detailed briefing with them yesterday on their Software Plus Services strategy and that's clearly where they want to go, and they do have some impressive early wins. But it's obviously still not the majority of their business. In the short-term, I think it's going to hurt their business, because clearly take-up of Vista has pretty well-flagged, especially on the corporate side.

Obviously they are trying to cultivate the Software Plus Services side, but that business is still very much in its early in its cycle. In the long run, it will be a good strategy for Microsoft, but they are so early along that it accounts for a pretty slow proportion of their revenue. In the short-term, Microsoft is more vulnerable.

Gardner: I was at a Red Hat conference earlier this week. Their model built very much on subscription and support, not on licensing the software. They give it away essentially. They felt pretty confident too that this wasn't going to be a cliff for them. So, I guess that further substantiates our trend.

Jim Kobielus, how do you see the shaping up for IT vendors? Is there going to be a dichotomy between those who have a recurring revenue model around subscriptions, versus those who have little bit more reliance on software licensing?

Kobielus: By the way, full disclosure, I have a degree in economics from way long ago and I am not going to even try to be dangerous. . . .

Gardner: Well, you might as well, because they don't see that what's going on either, right.

Linthicum: There’s an instant CNN gig out there for you, dude.

Gardner: By the way Jim, you are pretty dangerous, so go right ahead.

Kobielus: Okay, I do see in any economic downturn in the things that get cut from corporate budgets, for example, large capital expenditure (CAPEX) projects. That's going to hurt a number of IT vendors in particular niches, for example the hardware vendors, and where it's a discretionary software upgrade purchase. Those are going to feel the crunch.

Ongoing maintenance of existing systems, existing solutions that will relatively weather the storm. In other words, just to keep on keeping on.

So, the business model that open-source companies like Red Hat have established, and likewise, very mature software vendors like SAP and also Business Objects in the business intelligence (BI) space, they will do relatively okay because a large percentage of their revenue is from maintenance and support.

Those who will get hurt are those vendors who rely on new-product sales, especially new product sales that are very much hardware-centric. And where that comes in now ties in with my core focus areas, BI and data warehousing. We see in the data warehousing arena more of a focus on appliances, the hardware-software bundles that are pre-configured and so forth.

So, all the vendors in the data-warehousing space, pretty much all of them have re-geared their entire go-to-market strategy around hardware optimization of their own with turnkey solutions.

How will this economic crunch shake out the data warehousing appliance industry, really the data warehousing market? In any downturn, users, large corporate IT, look to rationalize and streamline their vendor commitments. In other words, they consolidate to a few very large, very strategic vendors. So, the big guys will get bigger and the small, pure-play data-warehousing appliance vendors will be acquired or will vanish.

Gardner: Is that the flight-to-quality kind of effect, do you think?

Kobielus: “Flight-to-quality,” explain that Dana?

Gardner: Well, you are not sure about where vendors might be and you might want to have one throat to choke, a bit more opportunity to deal with them, and that they can bargain with you because they want you long-term business. They are in a more powerful position and so quality, not unnecessarily the buy side but on the sell side, makes some sense.

Kobielus: Okay, yes, it's very much the phenomenon. They are the dynamic in play here. I think that the larger data-warehousing vendors will do relatively okay, especially those who are well-established and have a substantial amount of maintenance and support revenue themselves. I’m talking about the likes of Teradata and Oracle and IBM and a few others.

But, right now, with the data warehousing and BI vendors, every time I talk to them, I ask them, “Okay, a substantial proportion of your business is in the financial services vertical. How are you feeling? Are you seeing any softness in demand for your solutions?” And pretty much uniformly, they say, “Well, so far so good. We’re not really seeing a huge cut back in orders, or even any substantial delays in placement of orders that were expected,” but everybody is sort of bracing for the worst.

Gardner: Alright, so what I heard from you is that there is certainly a benefit of subscription, but there are also certain niches within IT that are specialized and that are hot right now, like BI and warehousing, that adds such a competitive advantage that they are probably going to continue to invest there.

Let’s look at this not necessarily just through the selling but on the buy side, those people who are in IT shops. Let's go to Dave Linthicum. You have been in the situation of specifying and buying. About 70-80 percent of these budgets are already locked into maintenance, not a lot of discretionary spending. What kind of pressure do you think they are going to feel?

Linthicum: They are going to feel a lot of pressure with anything that can be cut in the short-term. It's really going to be more that there is so much stress in there, instead of just definite cutting, just tactical pulling of expenses. They are looking to morph the way in which they consume IT. I just did a survey yesterday. I basically talked about the economic downturn and their plans to implement strategic technology into their enterprise. And everybody came back with, it's going to increase in interest but decrease in cost.

In other words, people are going to move into more efficient technologies. They are going to look at a little bit more at cloud computing and other ways to save money and start moving aggressively in those directions.

I think IT and some of the IT leadership were just waiting for an excuse to drive in this unfamiliar, risky area. If their budgets are sliced, they still have the responsibility for doing very intense IT business processing, and they are looking for new innovative ways to do that. That's inclusive of cloud computing and services-oriented architecture (SOA).

I don’t know if you looked at the SOA market just in terms of services, but it seems to be exploding right now. I’m not sure about the adoption of technology and the selling of technology. That may be an after effect, after all of these SOAs start taking more strategic positions within these enterprises. It's definitely a game changer right now. I’m not sure if it's positive or negative, but it's changing the game.

Gardner: When we look at how these organizations, these enterprises will move to, as you say risky, unproven, or just innovative new ways. What aspects of IT do you think they are going to be more willing to offload to a cloud first? Clearly, there is going to be too much risk in some areas and acceptable risk in others. Where do you think we are first going to start to see business activities and IT functional sets and applications offloaded -- just because it's so much cheaper to do it that way?

Linthicum: I think it's initially going to be the office-automation technologies, moving to more of the lighter-weight processes, and then moving to more of the heavy-weight processes.

Gardner: Can you be more specific on an application-by-application basis?

Linthicum: Yeah. Instead of having a huge Microsoft infrastructure just for e-mail and calendar-sharing in groupware, and those sorts of things, moving to things that are in the cloud. This is obviously Google, but there is also a ton of other guys that are offering some pretty good technology -- information-sharing using similar infrastructure. They’ll start outsourcing that, versus maintaining all these data centers that are just dealing with e-mail and communication between people within the company.

Gardner: Sure, there are plenty of hosted exchange too. Even if you don’t want to move from Microsoft, you can go off-premises.

Linthicum: You can go off-premises with lots of stuff and the cost is always cheaper, and also it allows you to upgrade and innovate into new technological areas you haven’t driven before.

Next, would be tactical, software-as-a-service (SaaS) applications. Take some of the HR processing, which is driven by some kind of in-house system in the data center, and outsource that to the dozen or so SaaS vendors who are offering HR processing. That's kind of a light-weight business process.

Then, the next generation is even more risky, and I don’t see a ton of guys doing that initially. It involves some of the core business processes, and getting into an SOA kind of an initiative. Re-automating those, but also outsourcing a tremendous number that haven’t been done before for the primary reason of cost saving.

Gardner: I think I’m hearing from Dave here that not only we are now going to make baby steps towards significant innovation, but the economic pressure that's going to come down on CIOs and IT departments forces them more towards that transformational level of change. So, that could include a lot more SOA, a lot more virtualization, internal on-premises cloud infrastructures, and so on.

Jim or Tony, how do you feel about the possibility that more economic pressure is actually a catalyst towards transformation rather than iterative change?

Kobielus: You mentioned my name first so I’ll respond first and I’ll be brief, so Mr. Tony can go right after me. I see definitely the economic downturn is going to expand the footprint, as it were, for the cloud in data warehousing, where data warehouses are becoming ever larger in the hundreds of terabytes and now into the petabyte.

I’m seeing an upsurge in the number of start-ups and data warehousing vendors that now have cloud based offerings. For example Vertica and Oracle now support databases that can run in the Amazon EC2. There are other vendors, like 1010data, that are very much pure plays in the fact that they only operate in the cloud and they are very highly scalable, share nothing, and parallel process.

There are, of course, SaaS-based offerings on a subscription basis. In other words, where there is a capital expenditure crunch or a budget crunch, and users can’t afford to pay the millions of dollars to bring one of these petabyte-scale data warehouses in house, they are going to go outside to the likes of a 1010data or using Amazon EC2 to aggregate, persist these huge datasets.

They can do very complex analyses and also run a greater degree of their data mining and predictive analytics algorithms in that very cloud. It just saves them money, and it's not a huge capital expenditure. It's a pay-as-you-go kind of thing. I think that's going to be the trend and those vendors who are already out there could be the major beneficiaries of this current economic crunch.

Gardner: So, that might mean if you are going to go to market, you want to have a cloud avenue for your go-to-market activities in addition to on-premises, or even say an open-source support model, right?.

Kobielus: Yes, for sure.

Gardner: Tony, what's your take on the possibility of harsh economic times as actually a catalyst towards the increased transformation?

Baer: Well, I am going to pair a couple of words that would otherwise seem like an oxymoron, which is tactical transformation. In times like these, obviously you have changing economic conditions, changing in a very unpredictable manner. On the other hand, the financial crunch and the credit crunch is going to restrict the amount of resources you have at your disposal. So, you’re basically going to look very opportunistically. You are going to look at, let's say, the low-hanging fruit that will give you the greatest gain in savings or a way to respond to the market in a more agile manner.

That will be very much in the way that Dave and Jim mentioned, which is that you will be taking advantage of specific services in the cloud. You won’t necessarily do a global top down or enterprise-architectural SOA transformation, if you haven't done SOA already. But, opportunistically, if you are trying to take advantage of some of these cloud-based services to start doing mining on a more massive scale, at the same time trying to lower your risk, it will require certain applications or data source that you may have. You may need to conduct a transformation, where you will implement, more flexible architectures, data SOA architecture.

But you will do it opportunistically in these tactical areas, where you can take advantage of services in the cloud that give you the advantages of the transformation to solve the problem you need to deal with, and at the same time, minimizing your risk.

Gardner: So, they are going to be looking for innovation without a big CAPEX, and if they can do that at the same time they are shutting down their own high-cost, high-labor applications in data centers that will be particularly attractive.

Baer: Or put it another way, “Capital, what capital?”

Gardner: Remember, not all companies are like banks. They have cash on hand, or they have ability to raise capital in a variety of different ways, rather than just going to a bank. So, we don't need to lump all these different types of enterprises into just the financial crisis problem.

Baer: Agreed. It's not to say that capital is totally shut-off, but the fact is that it's going to be rationed and a lot more carefully. I was just reading the advice that all these VCs are reading, and what they are saying is that if you have capital, find ways of stretching it.

Gardner: Save more cash, hold your cash basically. Speaking of verticals, let's look at this now through the lens of verticals, which verticals will do well and which will not.

My first take on this is that the government vertical is actually going to explode and might even start going down this road towards transformation in a much more significant way. Now, we can't read the tea leaves entirely on the economy, but politically we do start to see quite a momentum around the Democratic ticket and potentially a substantial majority for Democrats in Congress. They have put down platforms that include significant investments in such things as energy, healthcare, and of course they are going to need to transform how the government and the financial sector work together to calm the markets down.

On the other hand some, verticals that don't look good include retail and manufacturing. The auto industry is getting whacked. So, as IT spending is sliced and diced according to vertical, do we get a net-net up, down, or flat, when we look across verticals. I want to take a look at that. Dave Linthicum.

Linthicum: Yeah, it’s great living in Washington DC, let me tell you, because I think no matter where this thing goes, there is going to be full employment. The housing prices have actually crept up.

I think that you’re absolutely right. People are going to look to government to solve some of these issues and bureaucratic changes are going to be built here in different divisions, and people are going to have oversight of the financial industry.

If the Democratic administration comes in, there is going to be more civilian spending, and there is going to be probably a little shift from the spending in the Department of Defense on the military side.

So, this area is going to be explosive yet again, based on some things that are occurring and based on the government taking power in particular industries that they think they can be helpful in taking power. You can argue whether that's a good thing or a bad thing, but you are definitely going to see a lot of job shifts as things shift to that vertical.

The retail space is going to suffer tremendously. They already have very narrow razor-thin margins. I think we are going to see a lot of the larger retailers suffer and perhaps go away. I think healthcare is going to remain fairly static, and I think some of their costs maybe reduced. As they start moving into more of a socialized medicine, if the Democrats take it there, there is going to be some big shifts there.

Believe it or not, even though you are moving into a healthcare-for-everyone kind of an environment, you are going to see that actually cost probably will go up, as a bureaucracy is put in place to maintain and administer that.

Finance is obviously going to be killed for a long time, especially the banking industry. That's going to be an area that isn't going to recover very quickly from what's going on right now, but I think that manufacturing ultimately will recover and we are going to see some good growth in the year 2009-2010.

Gardner: Why do you see manufacturing as doing okay?

Linthicum: Because, the need for products worldwide is down right now, because people don't have the capital or access to the credit to make that happen. However, they are going to continue to have to replace airplanes, factory equipment, those sorts of things. It's just going to be a pent-up demand, and I think that's going to basically get unleashed in 2009-2010.

You’re going to see the large durable goods, large manufacturing kind of systems. People are going to just spend money on that area and that's going to be a worldwide driven thing. It's not going to be just driven from the United States.

Gardner: Great. Jim Kobielus, you mentioned earlier that you saw financial organizations buying data warehousing services and solutions as sort of still growing, if not at the same rate. I'd like to have your take on the financial sector alone Sure, there’s lots of turmoil, lots of contraction, but that doesn't necessarily mean you can shut off your IT systems. Mergers and acquisitions, consolidation sometimes can have a short-to-medium increase in IT requirement.

Kobielus: Right, and one of the things, Dana, that occurred to me is that the financial vertical and the government vertical are becoming overlapped. There is a degree of nationalization already that's taking place. The government is taking back Fannie Mae and Freddie Mac. I think they have taken over AIG, but all around the world, you hear governments, especially in Europe saying, “Hey, we need to re-nationalize or, to some degree, exert tighter control over the financial vertica., I think this is everywhere in the world.

What we’re already seeing is that the government vertical, as they have indicated, will continue to grow, because it's going to exercise much greater oversight and equity positions within the financial vertical. I think the early part of this decade is a prelude to what we’re going to see in even greater abundance in the next 10 years.

After the whole Enron fiasco, with Sarbanes-Oxley and so forth, we saw the growth of this market and this technology called governance, risk management, and compliance (GRC) to exert tighter control over the financials of private enterprise, and bring greater transparency.

I think we are going to see now, the government exert ever tighter GRC reigns over the financial sector, to a degree unprecedented, because we now have government actually owning or controlling a number of the key firms in that space. So, the whole GRC sector is in an embryonic stage. There are a number of vendors like SAP and Oracle who have taken sort of a leading-edge position in that area. That will expand greatly, and we are going to see more of these risk dashboards and controls being implemented in the context of BI and the data warehousing investments that enterprises have already made.

In terms of the horizontals, the GRC sector will come into its own, and it will be primarily the driver. There will be the financials, and then it will be around the world. All governments will enforce the use of this kind of technology.

Gardner: Right, and at a higher abstraction, that really means governance, and as much as internal governance it's perhaps governance from the extended enterprise sense, where there is going to be governance that crosses organizational boundaries. That's not going to be done with folks holding clipboards. That's going to be largely automated.

It’s going to have to be enforced through policies and rules and governance engines, it sounds an awful lot like SOA, but we are not going to apply the infrastructure we have developed for SOA. Just like services, we can apply it across a multitude of different business processes and activities in order to satisfy what you are talking about.

Baer: This reminds of something I heard from Microsoft this week. I was in Seattle at their BI conference, and they were talking about how Microsoft internally is using their own BI tools and stack. They described a number of roles -- like marketing, sales, and finance -- and how they use BI. Then, I asked the person, “Okay, your CEO, Steve Ballmer, obviously uses BI, but does he have a risk dashboard or a compliance dashboard or tools?”

Clearly, Microsoft is under a number of legal and regulatory mandates, compliance and so forth, and the people from Microsoft couldn’t answer that question immediately. They weren't really quite sure what's on Steve's dashboard.

In three years time, every CEO in the world will have a GRC dashboard that tells them on any given day the hoops they need to jump through to satisfy the regulators, I think that's coming fairly soon.

Gardner: Not just regulators, but the market doesn’t want to be caught unaware, as we apparently have been with this meltdown. In the future, they are going to want to know not just what they have to do to comply, but what the unknown risks are in terms of how the markets themselves are behaving.

Let's go to Tony Baer. Tony, what's your take on the opportunity for governance infrastructure to move beyond SOA, and is the new environment for business a growth area for SOA governance infrastructure?

Baer: Yeah, big time. I was talking before about these opportunistic areas. In the case of governance, I don't know if I would call it “opportunistic,” but it is an area in which you do not have an option as to whether you comply or not. Therefore, the only economic way to provide all the information and to do all the audits without having to rip apart all of your existing back-end infrastructure is through a service's layer on top of all that.

Maybe I can come up with a cheap buzzword here, a buzz-line or a tag-line, such as “Son of SOX,” for what's going to become a changing regulatory environment. You’ll need a governance layer that can contend with changes in this moving target.

Obviously, the only feasible way, from an architectural standpoint, to deal with that is do a flexible architecture, and that's essentially what a SOA is.

I very much agree with Dave and with Jim in terms of what are likely to be the growth sectors, but there are a couple of extra points I want to plug in there. This ties in with this question. The financial industry itself will not be a growth sector over the next few years, it will be very much a consolidating sector, but guess what, as you consolidate, you need to invest in consolidation.

Imagine all these huge mergers going on. Wells Fargo just finally got the agreement to acquire Wachovia, but of course there will be a some litigation from Citibank. Also, Bank of America acquired Merrill and there’s the whole reorganization of Wall Street, from investment banks into banking institutions.

The fact is, there is going to be a lot of transformation going on, and it's not transformation to support a growing business. It's transformation to support a changing business. There will be a lot of investment there, in addition to whatever investment will be necessary to deal with the new governance risks in compliance requirements.

Another area -- and I wanted to slip this in because it's nothing intuitive -- but if you look back at past history during economic downturns, and I hate to use the 'D' word but back in the depression, and I hope we are not heading into one, what area boomed during that era? Hollywood, the film industry. People were going out to the movies for cheap thrills.

In today's environment, the equivalent of that is, if you already have an Xbox 360 out there, you are going to be buying more games. Those are cheap thrills. It's going to be cheaper than going out and buying a new HDTV or going out to Six Flags.

Gardner: That's interesting. We haven't talked about one sector, and that is the Entertainment/Web 2.0/Internet. We’ve seen some downturn in advertising, including Internet advertising, but is there an opportunity for buying $3 movie and downloading it, a $2 song, a $3 game. How might our Internet /Media/Entertainment economy fare and will it be sliced and diced between those who depend on advertising and those who are not?

Baer: Very much so. The only downward pressure on this would be downward pressure on households to cut expenses and, if they consider that broadband is a discretionary expense, that would be the ceiling there. My sense though is that today to participate in the modern economy, broadband is becoming a necessity.

Gardner: Yeah, it's a utility. It’s like water, electricity. It's one of the last things that will go, right?

Gardner: My mother is 93, and I finally got her to get broadband. So we won’t give it up.

Kobielus: I have to jump in here and be dangerous one more time. I have another degree in Journalism and I was primarily a student of the mass media. If you look at the depression of the 1930s, historians and people who lived through the period talk about, what kept them company, in the dust bowl or wherever when they didn't have a job. It was the radio, which had been introduced in the previous decade.

Now, if gosh forbid, we have something similar coming up in the teens of this decade, what is the new radio? It's the Web. And so, who are the new entertainers? Well, actually in many ways it will be each other. I mean, through the whole Web 2.0 user-generated content paradigm. If you think about it, that's cheap entertainment, because it's generated for free and there is an unlimited supply of it available over some pipe that you've got coming into your home.

Gardner: I'd like to point out that this podcast is coming to you completely free. Continue.

Kobielus: And we are free to say what we want on this podcast.

Gardner: Does anybody else have some thoughts out there on the impact on Internet and startups? What's the impact with startups? We have seen this slide deck from Sequoia Capital saying “batten down the hatches, no discretionary spending, hoard your cash. Is this the VCs overreacting, because it's their pool of money that's its stake, or aare there actually opportunities beyond what they are saying in these dire predictions?

Linthicum: There are huge opportunities out there. If you saw my column I did in SOA World Magazine, I think this is a great time to do a startup.

Number one, VCs be damned at this point. You don't need their money at all, just some angel investors to invest in some very minute infrastructure. With cloud computing out there and the number of things you can do from a marketing, application developer's, and outsourcing perspective, you can basically get a technology company up and running -- and profitable -- probably for the least real cost we've seen in years. It's a great time for people who are innovative, able, and resourceful to get out there and start technology companies.

There are two types of companies out there right now. There seemed to be the big behemoths that are very slow and cumbersome and strategically challenged, even though they are making a lot of money and grabbing a large share of the market. Then, there are the old maids and basically a lot of small startups that just haven't been able to get acquired to do their exits.

Now is a great time for small innovative new startups to get out there and help create new spaces, such as Web 2.0, and I think there are a number of SOA problems that needs solving as well. I'd love to see some startups get out there and take those problems on.

Gardner: So, unintended consequence of the VCs contracting might be laying off a bunch of engineers and entrepreneurs. They'll go out there and say, “Okay, what am I going to do, sit in my garage and cry or am I going to look for platform-as-a-service (PaaS) providers and cloud providers that will allow me to develop a whole new set of applications on the cheap that I could put on my credit card. Then, I only pay for infrastructure as I need it and as I can create a business model?

Linthicum: Yeah, one of the things I would love to see come out of this whole mess that we are in right now is some of the Sarbanes-Oxley stuff contracting a bit. Quite frankly, a lot of the startups out there are unable to do any kind of exit other than acquisitions. You have no chance to take anything public. It's economically not viable for you to do so, because of the cost of maintaining the regulations around the whole publicly traded company opportunity.

I would love to see the government reopen that market a bit and make it much easier for startups that are profitable, that have a good track record and good technology to get access to the public marketplaces. Right now, they have to keep going back to the venture capital community. In many instances, those guys are strategically challenged. They are not focused on a particular industry, they are basically just focused on investment. That's going to be difficult to going forward.

Gardner: So in the ‘30s, we had the Works Progress Administration (WPA), which got people out there with shovels -- and my grandfather was one of them -- moving stuff around in the city in order to create works. Perhaps with an Internet Public Assistance Program, we can let the government be the seed and even steer them towards solutions of the government’s needs.

Now, the government wants to hire investment bankers to solve the problem that investment bankers created, but perhaps there is an opportunity for technologists to be brought in to solve some of these problems too.

Linthicum: Absolutely. What if a couple of the billions of dollars we are pumping back into the banks just went off to assist organizations and start-up companies around the technology space. I think there would be a huge boom in the area, and it would create jobs and be profitable fairly quickly.

I think some of them would probably go away, but overall, I think that it would have a positive effect on the economy. If you think about 1999, we were doing so well, because of the innovations around the Internet technology and other things that were booming. I think we are able to do that again, but we are just putting so many regulations, so much bureaucracy out there, that it makes it very difficult for the upstarts to get going.

Gardner: One little subset on this media discussion would be the press. Jim Kobielus, press has been under a tremendous amount of pressure lately. How are folks like Sam Zell going to fare on their traditional media, as advertising dries up, going to the Internet, seeing appreciable advertising business uptake there. It seems to me they are in the dead-end situation.

Kobielus: There is an ongoing crunch in the whole media sector that continues to ripple and ripple. It forces people out of being full-time journalists. So, it's not a happy thing. There was a Doonesbury cartoon recently in which Rick Redfern had been forcibly retired from the Washington Post. He was told, “Go and be a blogger!” He said, “Yeah, I will be one of a trillion bloggers out there.” “Well, you have a special differentiation. You are ex-Washington Post.”

Everybody is going to be from the journalism space, and even publishers are going to be “ex-journalists.” They have to find some next stage in their career, and I think a lot of smart people are going to become, as Dave indicates, entrepreneurs, but who will be self-funded from whatever remaining savings they have. It's not going to be a happy thing until the credit crunch eases.

Gardner: We only have a few more minutes. Let's look at some other potentially unintended consequences of all this.

If technology company stocks plummet some more, we might see some interesting things there. Somebody floated the idea that Sun Microsystems might just take its cash and buy itself out when its stock is trading at $5 -- and that was a stock that had a reverse four-way split. So it's down like a buck and change from what it was a few years ago.

Also RIM, still a strong company, a potentially for a takeover, is looking back to the buy side. What sort of interesting unintended consequences might we see among the vendors. Any thoughts?

Kobielus: I have no thoughts. Guys?

Linthicum: Just from your first point, I think you’re going to see some guys who are going to buy themselves off from the market for now, and I can't blame them.

If I were CEO of a publicly traded company, and my stock price was below my market capital, with cash in the bank -- where some of them are -- I’d get off the market quick, because it's a good deal.

Gardner: Absolutely.

Kobielus: I think we've talked in a previous podcast about the upsurge of private investing, of companies going private. I think the difference this time will be that if companies are going to go private, they are going to have to basically bootstrap it. They are not to be able to get a Silver Lake or anybody like that in the short-term.

Gardner: So a takeaway might be, if you can ride this out for two or three years, there is a buying opportunity, even buying yourself.

Kobielus: Right, if you get cheap enough. The other dividend of all that is once you go private, of course, you don't have to worry about all the GRC.

Gardner: One other subject that we haven't talked about is the analysis business. Is this an opportunity for people that need to know more about what's going on, and are folks like us going to be okay? Any thoughts?

Baer: Folks like us. Yeah. I think everybody is becoming an analyst. There is a whole blogosphere. Everybody in the blogosphere, to some degree, is an analyst. So, we’re going to be okay in the sense that we can still do analysis to our hearts delight for free, if we so choose.

In terms making a living on it, I think more-and-more analysts need to be half analyst, half consultant, doing projects for those who will pay us to actually show up and attend to only their needs and help them out with projects and also to make sense of what's going on in the space.

In any good time or bad time, analysts are essentially like reporters or journalists. We not only are in the industry, but we are in a sense above the industry, surveying what's going on and reporting to everybody else what we can see in terms of broad patterns and trends.

So I think there is a greater requirement on analyst to come in and offer reassurances or to tell people, “Okay, this strategy that you have been taking is not going to pan out. You better jump ship and try something different.”

Gardner: So changes are growth engines.

Kobielus: Yeah, and from that standpoint, it basically supplements the fact that there is going to be a decline in the journalist population, essentially a migration towards the extremes, which is on one hand journalism and this is not a development.

I’m very happy to see is that, as the financial base and the business model for journalism businesses is evaporating at this point, you are seeing more-and-more citizen journalists taking up more of the load. People are reading more blogs. They are not buying newspapers.

On the other end, it will create an appetite, and it will create a demand for people who are above the level of citizen blogger to say, “I have some professional credentials, and I can provide you some value-added analysis on your positions, so that you can essentially improve the competitiveness of your business.”

Gardner: Traditional and trade media will contract, which opens up a vacuum that can be filled by the expert-blogger function.

Kobielus: Right, expert blogger, but also the fact is that you get what you pay for. If you are a business and you are trying to improve your chance of surviving the market, you are going to work with key experts, key thought leaders out there, and you will pay for that.

It's not to say this is an infinite market for analysts. The business model for analyst firms is going through some stresses. Especially when you take a look at blogging. A lot of analyst firms have really not adapted to the blogosphere very well, or the more rapid flow of information.

So, even though I think they will continue to be a need for analysis and for paid analysis. The analyst industry or the analyst-firm industry needs to adapt to the new world of faster more instantaneous communications.

Gardner: Well, great. We've had a well-rounded discussion about the situation. We found some bright spots and some counter-cyclical possible growth areas within this sad situation we find ourselves in. But, as we exit I want to go around the panel and on a 1-to-10 scale, with 10 being flush, financial nirvana, and 1 being a dead-pool bankruptcy, where on a scale of 1 to 10 at a median level will the software business be in a year from now, let's start with you, Jim Kobielus.

Kobielus: I give it 5, straight down in the middle. I am trying not to lean towards either the manic or the depressive ends of the spectrum here. I think that some will do quite well and some will not. It's just a matter of taking a deep breath and recognizing that the economy goes through cycles, and the economy occasionally goes through panics -- the banking panics of the early 1900s and the late 1800s. We are sort of in the middle of one right now, which is an interesting phenomenon. I say interesting in the old Chinese sense of may you live through interesting times.

This has been a harsh decade. We started off with a tech-crunch and we are going to end with a tech-crunch, and a financial crunch, and it's going to take some time to sort it through, so just breathe easy.

Gardner: Tony Baer, 1 to 10, software industry.

Baer: Well, I'll give it 4, only because there are different headwinds on this go around. On the positive side, as Dave was mentioning before, the fact is that the barriers to entry are so much lower. So, if you can take advantage of the cloud, you can start in your own garage, and essentially marshal resources for very little cost. Basically, if you can sustain yourself and live close to the ground for the next two or three years, you and many others who are taking advantage of platform-as-a-service will have a whole new generation of solutions that will be ready for the next uptake.

Gardner: Dave Linthicum.

Linthicum: I am going to say 7.5. There are huge opportunities for the innovative and resourceful few out there in the market space. I think that technology shift, moving to higher regulations, you’ve got this “mother of all Sarbanes-Oxley” coming. Everybody is going to need folks in there to re-architect and re-automate and re-cast these businesses.

Then I think if there is going to be some upside in the future. Every cloud has a silver lining and those who are smart out there can certainly find the silver linings in this cloud. I think IT is going to stumble a bit, but a lot of more innovation is going to come into play, and people are going to use the cost-reduction capabilities, become a little bit more modernized and innovative in moving to cloud computing and SOA. All that stuff is going to accelerate tremendously in the next couple of years.

Gardner: I am going to go with 6.5. I agree that this is a transformation period, not just a contraction. I think this is going to necessitate a lot of the things that people have been working towards, but accelerate that, and force them to cut bait on the old stuff that doesn't work and adopt the new stuff that does. So, I’ m fairly bullish on IT, but with a lot of spottiness. There are going to be some pockets of certain failure and the ability in people to move among and between those is what's going to become essential.

I want to thank our panel for a very interesting discussion about the IT sector in this economic maelstrom.

We have been talking with Jim Kobielus, senior analyst of Forrester Research. Thanks, Jim.

Kobielus: Thank you. It was great!

Gardner: Tony Baer, senior analyst at Ovum. Thank you, sir.

Baer: Hey, thanks, Dana!

Gardner: Dave Linthicum, independent consultant with Linthicum Group. Thank you.

Linthicum: Thank you!

Gardner: I also want to thank our sponsor, the charter sponsor for the BriefingsDirect Analyst Insights Edition is Active Endpoints, makers of the ActiveVOS visual orchestration system. I am Dana Gardner, principal analyst at Interarbor Solutions. Thanks for listening, and come back next time.

Listen to the podcast. Download the podcast. Find it on iTunes/iPod. Learn more. Sponsors: Active Endpoints, Hewlett-Packard.

Special offer: Download a free, supported 30-day trial of Active Endpoint's ActiveVOS at www.activevos.com/insight.

Transcript of BriefingsDirect podcast on the outlook for IT in the face of the economic downturn. Copyright Interarbor Solutions, LLC, 2005-2008. All rights reserved.