Showing posts with label Kobielus. Show all posts
Showing posts with label Kobielus. Show all posts

Sunday, February 07, 2010

BriefingsDirect Analyst Panelists Peer into Crystal Balls for Latest IT Growth and Impact Trends

Edited transcript of BriefingsDirect Analyst Insights Edition podcast, Vol. 49, with panel of analysts discussing the future of cloud computing, SOA, social networks and the economy.

Listen to the podcast. Find it on iTunes/iPod and Download the transcript. Charter Sponsor: Active Endpoints.

Special offer: Download a free, supported 30-day trial of Active Endpoint's ActiveVOS at

Dana Gardner: Hello, and welcome to the latest BriefingsDirect Analyst Insights Edition, Vol. 49. I'm your host and moderator Dana Gardner, principal analyst at Interarbor Solutions.

This periodic discussion and dissection of IT infrastructure related news and events, with a panel of industry analysts and guests comes to you with the help of our charter sponsor, Active Endpoints, maker of the ActiveVOS business process management system.

Our topic this week hones in on the predictions for IT industry growth and impact, now that the recession appears to have bottomed out. We're going to ask our distinguished panel of analysts and experts for their top five predictions for IT growth through 2010 and beyond.

To help us gaze into the new IT trends crystal ball we are joined by our panel. Please join me in welcoming Jim Kobielus, senior analyst at Forrester Research. Hey, Jim.

Jim Kobielus: Hey, Dana. Hi, everybody.

Gardner: Joe McKendrick, independent analyst and prolific blogger. Howdy, Joe.

Joe McKendrick: Hi, Dana. Very nice to be here.

Gardner: Tony Baer, senior analyst at Ovum. And, Brad Shimmin, principal analyst at Current Analysis. Hi, Brad.

Brad Shimmin: Hey, Dana.

Gardner: Dave Linthicum, CEO of Blue Mountain Labs. Good to have you with us, Dave.

Dave Linthicum: Hey, guys.

Gardner: Dave Lounsbury, vice-president of collaboration services at The Open Group. How do you do, Dave? [Disclosure: The Open Group is a sponsor of BriefingsDirect podcasts. See more on the consortium's recent conference in Seattle.]

Dave Lounsbury: Hello, Dana. Happy to be here.

Gardner: Jason Bloomberg, managing partner at ZapThink.

Jason Bloomberg: Good morning, everybody.

Gardner: And, JP Morgenthal, independent analyst and IT consultant. Good to have you with us, JP.

JP Morgenthal: Good to be here.

Gardner: I've decided to do this in a random order this time. So, based on the pick of the short straw, Brad Shimmin, you're up, what are your top five predictions for IT in 2010?

Brad Shimmin

Shimmin: Thanks, Dana. And, I have got a set of five. Obviously, mine are geared toward collaboration and conferencing, so I'll just put that out there as a caveat, but I think it will help if we're going to try to strive for consensus later on.

Let me just begin with the first and most obvious, which is that clouds are going to become less cloudy. Vendors, particularly those in the collaboration space, are going to start to deliver solutions that are actually a blend of both cloud and on-premise.

We've seen Cisco take this approach already with front-ending some web conferencing to off-load bandwidth requirements at the edge and to speed internal communications. IBM, at least technically, is poised do the same with Foundations, their appliances line, and LotusLive their cloud-based solution.

With vendors like these that are going to be pulling hybrid, premise/cloud, and appliance/service offerings, it's going to really let companies, particularly those in the small and medium business (SMB) space, work around IT constraints without sacrificing the control and ownership of key processes and data, which in my mind is the key, and has been one of the limiting factors of cloud this year.

Next up, I have "software licensing looks like you." As with the housing market, it's really a buyer's market right now for software. It's being reflected in how vendors are approaching selling their software. Customers have the power to demand software pricing that better reflects their needs, whether it's servers or users.

I think the weapons will be user facing enterprise apps that work in concert with line-of-business solutions on the back-end.

So, taking cues from both the cloud and the open-source licensing vendors out there, we will see some traditional software manufacturers really set up a "pick your poison" buffet. You can have purchase options that are like monthly or yearly subscriptions or flat perpetual licenses that are based on per seat, per server, per CPU, per request, per processor, or per value unit, with a shout out at IBM there -- or any of the above.

You put those together in a way that is most beneficial to you as a customer to meet your use case. We saw last year with web conferencing software that you could pick between unlimited usage with a few seats or unlimited seats with limited usage. You can tailor what you pay to what you need.

Third for me is the mobile OS wars are going to heat up. I'm all done with the desktop. I'm really thinking that it's all about the Google Chrome/Android. I know there's a little bit of contention there, but Google Chrome/Android, Symbian, RIM, Apple iPhone, Windows Mobile, all those devices will be the new battle ground for enterprise users.

I think the weapons will be user facing enterprise apps that work in concert with line-of-business solutions on the back-end. We'll see the emergence of native applications, particularly within the collaboration space, that are capable of fully maximizing the underlying hardware of these devices, and that's really key. Capabilities like geo-positioning, simultaneous web invoice and, eventually, video are really going to take off across all these platforms this year.

Win or lose

But, the true battle for this isn't going to be in these cool nifty apps. It's really going to be in how these vendors can hopefully turn these devices into desktops, in terms of provisioning, security, visibility, governance, etc. That, to me, is going to be where they're going to either win or lose this year.

Four is "The Grand Unification Theory" -- the grand unification of collaboration. That's going to start this year. We're no longer going to talk about video conferencing, web conferencing, telepresence, and general collaboration software solutions as separate concerns. You're still going to have PBXs, video codecs, monitors, cameras, desk phones, and all that stuff being sold as point solutions to fill specific requirements, like desktop voice or room-based video conferencing and the like.

Special offer: Download a free, supported 30-day trial of Active Endpoint's ActiveVOS at

But, these solutions are really not going to operate in complete ignorance of one another as they have in the past. Vendors with capabilities or partnerships spanning these areas, in particular -- I'm pointing out Cisco and Microsoft here -- can bring and will be bringing facets of these together technically to enable users to really participate in collaboration efforts, using their available equipment.

It will be whatever they have at hand. They're not forced to go to a particular room to participate in a conference, for example. They can just pick up their mobile phone or their preferred method of communication, whether they just want to do voice, voice/video, or chat.

For enterprise-focused vendors, we're going to see them playing in the waves in a number of ways.

And last but not least ... I'm sorry. I'm probably going to get kicked for this, but, because I'm a technical optimist . . . the Google Wave is really going to kick in in 2010. I may be stating the obvious, or I maybe stating something that's going to be completely wrong, but I really feel that this is going to be the year that traditional enterprise collaboration players jump head long into this Google Wave pool in an effort to really cash in on what's already a super-strong mind share within the consumer ranks.

Even though they have a limited access to the beta right now, there are over a million users of it, that are chunking away at this writing code and using Wave.

Of course, Google hosted rendition will excel in supporting consumer tasks like collaborative apps and role playing games. That's going to be big. For enterprise-focused vendors, we're going to see them playing in the waves in a number of ways. They're going to embed them within existing collaborative applications. They're going to enable existing apps to interact with Google Waves.

This is the case with Novell’s recently announced Pulse. You guys saw that. They're going to extend existing apps to make use of wave-like capabilities. They're going to create some competitive functionality that looks like a Google Wave but isn't a Google Wave, and doesn't really care what Google is doing with Wave. And that's it, Dana.

Gardner: Well, Brad, that was an excellent list. If I can plumb through this a little bit, it sounds like we are going to be using Google Wave to do unified collaboration on a mobile operating system, coming from the cloud and we are going to get to negotiate for the price we will pay for it.

Shimmin: Perfect. You strung them together like jewels on a thread. Thanks.

Gardner: Dave Linthicum, you're up next. What are your top five?

Dave Linthicum

Linthicum: My top five are going to be, number one, cloud computing goes mainstream. That's a top prediction, I'm just seeing the inflection point on that.

I know I'm going out on the edge on this one. Go to and do a search on the cloud-computing jobs postings. As I posted on my InfoWorld blog few weeks ago, it's going up at an angle that I have never seen at any time in the history of IT. The amount of growth around cloud computing is just amazing. Of course, it's different aspects of cloud computing, not just architecture with people who are cloud computing developers and things like that.

The Global 2000 and the government, the Global 1, really haven't yet accepted cloud computing, even though it's been politically correct for some time to do so. The reason is the lack of control, security concerns, and privacy issues, and, of course, all the times the cloud providers went down. The Google outages and the loss of stuff with T-Mobile, hasn't really helped, but ultimately people are gearing up, hiring up, and training up for cloud computing.

We are going to see a huge inflection point in cloud computing. This can be more mainstream in Global 2000 than it has been in the past. It's largely been the domain of SMBs, pilot projects, things like that. It's going to be a huge deal in 2010 and people are going to move into cloud computing in some way, shape, or form, if they are in an organization.

People are pushing back on that now. They’ve had it. They really don’t want all of their information out there on the Internet ...

That's going to continue going forward. I don’t think we are going to outsource everything as a cloud, but, in the next five years, there is going to be a good 10-20 percent existing on the cloud, which is huge.

The next is privacy. I’ll shift gears a bit. Privacy becomes important. Facebook late last year pulled a little trick, where they changed the privacy settings, and you had to go back and reset your privacy settings. So, in essence, if you weren’t diligent about looking at the privacy settings within your Facebook account and your friends list, your information was out on the Internet and people could see it.

The reason is that they're trying to monetize people who are using Facebook. They're trying to get at the information and put the information out there so it's searchable by the search engines. They get the ad revenue and all the things that are associated with having a big mega social media site.

People are pushing back on that now. They’ve had it. They really don’t want all of their information out there on the Internet, who their friends are, who they are dating, all these sorts of things. They want it secured. I think the rank and file are going to demand that regulations be set.

People are going to move away from these social media sites that post their private information, and the social media sites are going to react to that. They're going to change their policies by the end of 2010, and there's going to be a big uproar at first.

Cloud crashes

Next, the cloud crashes make major new stories. We've got two things occurring right now. We've got a massive move into the cloud. That was my first prediction. We have the cloud providers trying to scale up, and perhaps they’ve never scaled up to the levels that they are going to be expected to scale to in 2010. That's ripe for disaster.

A lot of these cloud providers are going to over extend and over sell, and they're going to crash. Performance is going to go down -- very analogous to AOL’s outage issues, when the Internet first took off.

We're going to see people moving to the cloud, and cloud providers not able to provide them with the service levels that they need. We're going to get a lot of stories in the press about cloud providers going away for hours at a time, data getting lost, all these sorts of things. It's just a matter of growth in a particular space. They're growing very quickly, they are not putting as much R&D into what these cloud systems should do, and ultimately that's going to result in some disasters.

Special offer: Download a free, supported 30-day trial of Active Endpoint's ActiveVOS at

Next, Microsoft becomes cloud relevant. Microsoft, up to now, has been the punch line of all cloud computing. It had the Azure platform out there. They've had a lot of web applications and things like that. They really have a bigger impact in the cloud than most people think, even though when we think of cloud, we think of Amazon, Google, and larger players out there.

Suddenly, you're going to see Microsoft with a larger share of the cloud, and they're going to be relevant very quickly.

With Azure coming into its own in the first quarter of next year in the rise of their office automation applications for the cloud, you are going to see a massive amount of people moving to the Microsoft platform for development, deployment, infrastructure, and the office automation application. The Global 2000 that are already Microsoft players and the government that has a big investment in Microsoft are going to move in that direction.

Suddenly, you're going to see Microsoft with a larger share of the cloud, and they're going to be relevant very quickly. In the small- and medium-sized business, it's still going to be the domain of Google, and state and local governments are still be going to be the domain of Google, but Microsoft is going to end up ruling the roost by the end of 2010.

Finally, the technology feeding frenzy, which is occurring right now. People see the market recovering. There is money being put back into the business. That was on the sidelines for a while. People are going to use that money to buy companies. I think there is going to be a big feeding frenzy in the service-oriented architecture (SOA) world, in the business intelligence (BI) world, and definitely in the cloud-computing world.

Lots of these little companies that you may not have heard about, which may have some initial venture funding, are suddenly going to disappear. Google has been taking these guys out left and right. You just don’t hear about it. You could do a podcast just on the Google acquisitions that have occurred this week. That's going to continue and accelerate in 2010 to a point where it's almost going to be ridiculous. Well, with that, Dana, those are my predictions.

Gardner: Excellent, Dave. We appreciate that. Let's go to other Dave today. This is Dave Lounsbury. Tell us please from your perspective at The Open Group, what your top five predictions are?

Dave Lounsbury

Lounsbury: I'm going to jump on the cloud bandwagon initially. We’ve seen huge amounts of interest across the board in cloud and, particularly, increasing discussions about how people make sense of cloud at the line-of-business level.

Another bold prediction here is that the cloud market is going to continue to grow, and we'll see that inflection point that Dave Linthicum mentioned. But, I believe that we're going to see the segmentation of that into two overarching markets, an infrastructure-as-a-service (IaaS) or platform-as-a-service market (PaaS) and software-as-a-service (SaaS) market. So that's my number one prediction.

We'll see the continued growth in the acceptance by SMBs of the IaaS and PaaS for the cost and speed reasons. But, the public IaaS and PaaS are going to start to become the gateway drug for medium- to large-size enterprises. You're going to see them piloting in public or shared environments, but they are going to continue to move back towards that locus of controlling their own resources in order to manage risk and security, so that they can deliver their service levels that their customers expect.

My third prediction, again in cloud, is that SaaS will continue to gain mainstream acceptance at all levels in the enterprise, from small to large. What you’ll see there is a lot of work on interfaces and APIs and how people are going to mash up cloud services and bring them into their enterprise architectures.

Of course all of this is set against the context that all distributed computing activities are set against, which is security and privacy issues.

This is actually going to be another trend that Dave Linthicum has mentioned as a blurring of a line between SaaS and SOA at the enterprise level. You’ll see these well on the way to emerging as disciplines in 2010.

The fourth general area is that all of this interest in cloud and concern about uptake at the enterprise level is going to drive the development of cloud deployment and development skills as a recognized job function in the IT world, whether it's internal to the IT department or as a consultancy. Obviously, as a consultancy, we look to the cloud to provide elasticity of deployment and demand and that's going to demand an elastic workforce.

So the question will be how do you know you are getting a skilled person in that area. I think you'll see the rise of a lot of enterprise-level artifacts such as business use cases, enterprise architecture tools, and analytic tools. Potentially, what we'll see in 2010 is the beginning of the development of a body of knowledge: practitioners in cloud. We'll start to recognize that as a specialty the way we currently recognize SOA as a specialty.

Of course all of this is set against the context that all distributed computing activities are set against, which is security and privacy issues. I don’t know if this is a prediction or not, but I wonder whether we're going to see our cloud harbor in 2010 its first big crash and the first big breach.

We've already mentioned privacy here. That's going to become increasingly a public topic, both in terms of the attention in the mainstream press and increasing levels of government attention.

There have been some fits and starts at the White House level about the cyber czar and things like that, but every time you turn around in Washington now, you see people discussing cyber security. How we're going to grow our capability in cyber security and increasing recognition of cyber security risk in mainstream business are going to be emerging hot topics of 2010.

Gardner: Thanks so much. Next up, Jim Kobielus. Tell us where you see things going in 2010. Your top five, please?

Jim Kobielus

Kobielus: Yes, my top five in 2010. In fact, I blogged that yesterday. I blogged six yesterday, but I'll boil it down to five and I'll make them even punchier. It's only going to be focused on analytics my core area.

Number one: IT more or less gives up BI. Let me constrain that statement. IT is increasingly going to in-source much of BI development of reports, queries, dashboards, and the like to the user through mash up self-service approaches, SaaS, flexible visualization, and so forth, simply because they have to.

IT is short staffed. We're still in a recession essentially. IT budgets are severely constrained. Manpower is severely constrained. Users are demanding mashups and self-service capabilities. It's coming along big time, not only in terms of enterprise deployment, but all the BI vendors are increasingly focused on self-service solution portfolios.

Number two: The users who do more of the analytics development are going to become developers in their own right. That may sound crazy based on the fact that traditionally data mining is done by a cadre of PhD statisticians and others who are highly specialized.

Basically, we're taking data mining out of the hands of the rocket scientists and giving it to the masses through very user-friendly tools.

Question analysis, classification and segmentation, and predictive analytics is coming into the core BI stack in a major way. IBM’s acquisition of SPSS clearly shows that not only is IBM focusing there, but other vendors in this space, especially a lot of smaller players, already have some basic predictive analytics capabilities in their portfolios or plan to release them in 2010.

Basically, we're taking data mining out of the hands of the rocket scientists and giving it to the masses through very user-friendly tools. That's coming in 2010.

Number three: There will be an increasing convergence of analytics and transactional computing, and the data warehouse is the hub of all that. More-and-more transactional application logic will be pushed down to be executed inside of the data warehouse.

The data warehouse is a greater cloud, because that's where the data lives and that's where the CPU power is, the horse power. We see Exadata, Version 2 from Oracle. We see Aster Data, nCluster Version 4.0. And, other vendors are doing similar things, pointing ahead to the coming decade, when the data warehouse becomes a complete analytic application server in its own right -- analytics plus transaction.

Predictive analysis

Number four: We're seeing, as I said, that predictive analytics is becoming ever more important and central to where enterprises are going with BI and the big pool of juicy data that will be brought into predictive model. Much of it is coming from the whole Web 2.0 sphere and from social networks -- Twitters, Facebooks and the like, and blogs. That's all highly monetizable content, as Dave Linthicum indicated.

We're seeing that social network analysis has a core set of algorithms and approaches for advanced analytics that are coming in a big way to data mining tools, text analytics tools, and to BI. Companies are doing serious marketing campaign planning, optimization, and so forth, based on a lot of that information streaming in real-time. It's customer sentiment in many ways. You know pretty much immediately whether your new marketing campaign is a hit or a flop, because customers are tweeting all about it.

That's going to be a big theme in 2010 and beyond. Social network analysis really is a core business intelligence for marketing and maintaining and sustaining business in this new wave.

Right now, we're in the middle of a price war for the enterprise data warehousing stack hardware and software.

And, finally, number five: Analytics gets dirt cheap. Right now, we're in the middle of a price war for the enterprise data warehousing stack hardware and software. Servers and storage, plus the database licenses, query tools, loading tools, and BI are being packaged pretty much everywhere into appliances that are one-stop shopping, one throat to choke, quick-deploy solutions that are pre-built.

Increasingly, they'll be for specific vertical and horizontal applications and will be available to enterprises for a fraction of what it would traditionally cost them to acquire all those components separately and figure it out all themselves. The vendors in the analytics market are all going appliance. They're fighting with each other to provide the cheapest complete application on the market.

Special offer: Download a free, supported 30-day trial of Active Endpoint's ActiveVOS at

You can see what Oracle has already done with Exadata Version 2, 20K per usable terabyte. We see other vendors packaging even more functionality into these appliances and delivering them to mid-market and large enterprises. Small companies can deploy a complete analytics environment with BI, ETL, and everything for much less than they could just a few years ago.

And, one last thing. There is a cloud twist in everything I am describing or discussing here. Analytics gets dirt cheap, and even more so, as more of this functionality is available in the cloud. We're seeing a boom of SaaS-based BI and data warehousing vendors. In the coming years, pay-as-you-go, subscription-based, low risk, fund it out of OpEx rather than CapEx, is coming to analytics everywhere. So, that will be a huge trend in the coming year.

Gardner: Thanks Jim. Next, we're going to Joe McKendrick. Joe, what's your top five for 2010?

Joe McKendrick

McKendrick: Thanks, Dana. You also gave us the option to talk about the decade ahead, and I was thinking whether I should talk about the year ahead or the decade ahead. It occurred to me that just as we had a 2000 problem a decade ago, we now have a year 2012 problem. I just saw the movie 2012 a couple of weeks ago. The world is going to end and it's going to get flooded.

Gardner: So, the cloud is going to be big, dark, and made of soot. That's it. It's all over. We are all going to – cloud.

McKendrick: Exactly. I might have some arks floating around, and you worry about the IT systems on those arks.

Gardner: Well, you are a pessimist. Back down to earth.

McKendrick: Back down to earth. Okay, 2010. My world, of course, is SOA, and the big question for 2010 is what will Anne Thomas Manes have to say about SOA to start off the year?

Gardner: What's dead this year?

McKendrick: Right. In the first week of this year, Anne came out and said that SOA is dead. That caused a lot of angst, anxiety, discussion, and brooding for pretty much the entire year. It really had an impact.

Gardner: It kept you in page views.

McKendrick: Yeah, thanks, Anne. So, I am hoping Anne will come out with something good at the beginning of 2010. She'll probably say that SOA is still dead. That's my prediction.

Gardner: What is the state of SOA in 2010, Joe?

McKendrick: Part of it will be tied into the economy. By all indications, 2010 is going to be a growth year in the economy. We're probably in this V shape. See, I'm actually an optimist, not a pessimist. The world may end in 2012, but for 2010, we're going to have a great economy. It's going to move forward.

For this decade, we're looking forward to the rise of something called "social commerce," where the markets are user-driven and are conversations.

What happened with SOA? SOA really proved itself through 2009. I know a lot of instances where companies had a service-oriented culture, had flexibility, had visibility into their applications, their services, and their data. This played a great role in helping them pull through in terms of visibility into the supply chains and logistics. I know of a home builder -- and that's a tough industry -- where a SOA implementation really increased its sales turnaround time and enabled it to tighten up, be more efficient, and pull through this economic dark hole we went through.

I think 2010 will be a year of growth. As I said in previous podcast, we had these economic downturns: 2000-2001, 1990-1991, 1981-82. These downturn periods were always followed by periods of spectacular growth, especially in terms of technology -- and usually a huge paradigm shift in technology.

It's hard to say what. Nobody at the time of those downturns could have predicted what was ahead. Nobody predicted the dot-com boom back in 1992. But, what we're seeing is the service-oriented thinking. It's not just IT. It's service-oriented across the board -- the idea of the loosely coupled business, businesses that could start on a shoe string budget in IT, thanks to the availability of cloud, and move forward in the market.

Ten years ago, we saw the rise of e-commerce. For this decade, we're looking forward to the rise of something called "social commerce," where the markets are user-driven and are conversations. To use the quote from the book "The Cluetrain Manifesto," markets will be driven by users who interact with each other. Companies that will succeed and get ahead will encourage this social commerce, the interaction with customers over social networking sites.

Gardner: Alright Joe, I'm confused. Are we still on number one prediction or are you on number two?

McKendrick: That was my number one prediction, the impact of the economy. We're going to start seeing some new paradigms rising. Folks here talk about cloud computing.

The new normal

Number two: Cloud computing. We’ve all been talking about that. That's the big development, the big paradigm shift. Clouds will be the new "normal." From the SOA perspective, we're going to be seeing a convergence. When we talk about cloud, we're going to talk about SOA, and the two are going to be mapped very closely together.

Dave Linthicum talks a lot about this in his new book and in his blog work. Services are services. They need to be transparent. They need to be reusable and sharable. They need to cross enterprise boundaries. We're going to see a convergence of SOA and cloud. It’s a service-oriented culture.

Number three: Google is becoming what I call the Microsoft of the clouds. Google offers a browser and email. It has a backend app engine. It offers storage. They're talking about bringing out an OS. Google is essentially providing an entire stack from which you can build your IT infrastructure. You can actually build a company’s IT infrastructure on the back of this. So, Google is definitely the Microsoft of the cloud for the current time.

Microsoft is also getting into the act as well with cloud computing, and they are doing a great job there. It’s going to be interesting to see what happens. By the way, Google also offers search as a capability.

Gardner: Is there anything that Google won’t do? That’s the easier list. What won’t Google get into this year?

McKendrick: They probably won’t get into building and selling hardware.

Gardner: I heard about a phone they’re into selling. Are they in partnership with a phone?

Everybody will be providing and publishing services, and everybody will be consuming services.

McKendrick: Right, with Verizon, but it's the only thing they won’t really touch.

Gardner: My prediction is that they won’t get into snow plowing. Google will not get into snow plowing in 2010. That’s my only safe bet.

McKendrick: That’s probably about it.

Number four: We're going to see less of a distinction between service providers and service consumers over clouds, SOA, what have you. That's going to be blurring. Everybody will be providing and publishing services, and everybody will be consuming services.

You're going to see less of a distinction between providers and consumers. For example, I was talking to a reinsurance company a few months back. They offer a portal to their customers, the customers being insurance companies. They say that they offer a lot of analytics capabilities that their customers don’t have, and the customers are using their portal to do their own analytic work.

They don’t call it cloud. Cloud never entered the conversation, but this is a cloud. This is a company that’s offering cloud services to its consumers. We're going to see a lot of that, and it’s not necessarily going to be called cloud. You're not going to see companies saying, "We're offering clouds to our partners." It’s just going to be as the way it is.

Number five: In the enterprise application area, we've seen it already, but we're going to see more-and-more pushback against where money is being spent. As I said, the economy is growing, but there is going to be a lot of attention paid to where IT dollars are going.

I base this on a Harvard Medical School study that just came out last month. They studied 4,000 hospitals over a three-year period and found that, despite hundreds of millions of dollars being invested at IT, IT had no impact on hospital operations, patient care quality, or anything else.

Gardner: And, that’s why I don’t go to hospitals.

McKendrick: There are ramifications for other industries as well. What’s the impact of all this IT expenditure? Ultimately, this may help the cloud model in the long run. Okay, that's my five.

Gardner: Excellent. Let’s go to JP Morgenthal. What are your top five predictions, JP?

JP Morgenthal

Morgenthal: First, I'm going to predict that Microsoft, Oracle, Google, IBM -- none of them are going to be supporting Tiger Woods as a sponsor next year.

Gardner: Another risk-taker.

Morgenthal: Sorry, man. I had to throw it out there. It was just sitting there, and no one else picked it up, like a $100 bill on the street. Okay, number one: Cyber security. As someone stated earlier, it's interesting what’s going on out there. I am beginning to understand how little people actually understand about the differences between what security is and information assurance is, and how little people realize that their systems are compromised and how long it takes to eliminate threat within an organization.

Because of all of this connectedness, social networking, and cloud, a lot of stuff is going to start to bubble up. People who thought things were taken care of are going to learn that it wasn’t taken care of, and there will be a sense of urgency about responding to that. We're going to see that happen a lot in the first half of 2010.

Number two: Mobile. The mobile platforms are now the PC of yesterday, right? The real battle is for how we use these platforms effectively to integrate into people’s lives and allow them to leverage the platform for communications, for collaboration, and to stay in touch.

It seems everywhere I go, people are willing to spend a lot of money on their data plan. So, that’s a good sign for telecoms.

My personal belief is that it overkills information overlook, but that’s me. I know that everywhere I go, I see people using their iPhones and flicking through their apps. So, they hit upon a market segment, a very large market segment, that actually enjoys that. Whether small people like me end up in a cave somewhere, the majority of people are definitely going to be focused on the mobile platform. That also relates to the carriers. I think there still a carrier war here. We've yet to see AT&T and iPhone in the US break apart and open up its doors to other carriers.

Gardner: Let that happen in 2010.

Morgenthal: We all say that, but this is a fertile ground for priming what’s been a notoriously dead pump. Two years ago, I wrote a blog entry about what happens to technology in an era where the economy is down? It seems everywhere I go, people are willing to spend a lot of money on their data plan. So, that’s a good sign for telecoms.

Gardner: Yeah, the human species has spoken. They like mobile and they like ubiquitous broadband, and that’s not going to change, right?

Morgenthal: I agree with you. But the question is, should people pay for it or should the government give to you free? In the US, I hear a lot of social groups saying, "Hey, everybody should have broadband like it’s electricity."

Gardner: So, maybe Tiger Woods pays for everybody’s broadband for six months. He's got the money to do it, and then everybody will forget about this marriage thing.

BI and analytics

Morgenthal: I think you’ve got a new business model. Number three: Business intelligence and analytics, especially around complex event processing (CEP). CEP is still in an immature state. It does some really interesting things. It can aggregate and correlate. It really needs to go to that next step and help people understand how to build models for correlation. That’s going to be a difficult step.

As somebody was saying earlier, you had these little Poindexters sitting in the back room doing the stuff. There's a reason why the Poindexters were back there doing that. They understand math and the formulas that are under building these analytical models. Teaching your average USA Today reader how to build an analytical model is akin to teaching everybody how to write programs by drawing pictures. It still hasn’t happened. There's a reason why.

Gardner: So, you are saying that this is a year of CEP, that’s your stake in the ground?

Morgenthal: CEP and analytics -- and the two tied together. You’ll see that the BI, and data aspects of the BI, side will integrate with the CEP modeling to not only report after the fact on a bunch of raw data, but almost be proactive, and try to, as I said in my blog entry, know when the spit hits the fan.

Gardner: Right. So, at this time next year, I won’t be having analysts on to predict that what’s going to happen in 2011. We’ll just plug it into a CEP engine and we’ll get all the right answers.

Morgenthal: That’s assuming you could find the right people to program it, which is a whole other issue. I had done as my number five, so I’ll save that, but number four is collaboration. We’ve crossed the threshold here. People want it. They're leveraging it.

The labor market has not caught up to take advantage of these tools, design them, architect the solutions properly, and deploy and manage them.

I've been seeing some uptake on Google Wave. I think people are still a little confused by the environment, and the interaction model is not quite there yet to really turn it on its ear, but it clearly is an indication that people like large-scale interactions with large groups of people and to be able to control that information and make it usable. Google is somewhat there, and we'll see some more interesting models emerge out of that as well.

Gardner: So, is there another way to say that, JP, which is the people stop living in their email and start living in something more like Google Wave?

Special offer: Download a free, supported 30-day trial of Active Endpoint's ActiveVOS at

Morgenthal: I don't see them doing that and wouldn't predict that, but they are clamoring for collaboration, and I think the market will respond.

Gardner: Alright.

Morgenthal: New and innovative ways to collaborate.

Gardner: Alright, number five for you.

Morgenthal: Labor. We're at a point where the market is based on all these other things based on the cloud. We had a lot of disruptive technologies hit in the past five years -- enterprise mashups, SOA, and cloud computing. The labor market has not caught up to take advantage of these tools, design them, architect the solutions properly, and deploy and manage them.

I think that 2010 has to be a year for training, rebuilding, and getting some of those skills up. Today, you hear a lot of stories, but there is a large gap for any company to be able to jump into this. Skills are not there. The resources are not there and they are not trained. That's going to be a huge issue for us in 2010.

Gardner: Thanks. We're on to our next analyst prediction, and that would be with Jason Bloomberg. Jason, what are your top five?

Jason Bloomberg

Bloomberg: Thanks for getting to me, Dana. I'm going to be a bit of the naysayer of the bunch. We work primarily with enterprise architects now, so we are on the demand side more than the supply side for IT capabilities. So, our perspective is colored through the glasses of the architect.

Dana, you asked us for not just the one- or 10-year predictions, but also positive and negative. So, my first four are things that I predict won't happen, and we can fill in the blanks in terms of what will happen.

First of all, sorry, Dave, I just don't see cloud computing striking it big in 2010. When we talk to enterprise architects, we see a lot of curiosity and some dabbling. But, at the enterprise scale, we see too much resistance in terms of security and other issues to put a lot of investment into it. It's going to be gradually growing, but I don't see such a point coming as soon as you might like.

Small organizations are a different story. We see small organizations basing their whole business models on the cloud, but at the enterprise level, it's sort of a toe in the water, and we see that happening in the 2010.

Another thing we don't see really taking off in any big way is Enterprise 2.0. That is Web 2.0 collaborative technologies for the enterprise. You know, "Twitter On Steroids," and that kind of thing. Again, it's going to be more of a toe in the water thing. Collaborative technologies are maturing, but we don't see a huge paradigm shift in how collaboration is done in the enterprise. It's going to be more of a gradual process.

Another thing that we are not seeing happening in 2010 is CIOs and other executives really getting the connection between business process management (BPM) and SOA. We see those as two sides of the same coin. Architects are increasingly seeing that in order to do effective BPM you have to have the proper architecture in place. But, we don't see the executives getting that and putting money where it belongs in order to effect more flexible business process. So, this is another work in progress, and it's going to be a struggle for architects to make progress over the course of the year.

Gardner: Alright, Jason, would today's announcement that IBM is acquiring Lombardi be a buttress to your point there?

Bloomberg: Well, that's a software story at this point. It's not a best practice story. IBM, being on the supply side, is attempting to push products like this into the market and they have this strategy for integrating the Lombardi technology with their existing technology. That doesn't necessarily mean that, from the buyer perspective, they see the full connection of how BPM and how SOA fit together and how leveraging architecture will support the business process optimization efforts in the enterprise.

Software vendors were hoping for a huge year, but they're going to be disappointed. It's going to be a growth year, but it's going to be moderate growth for the vendors.

So, tools are there and the tools are maturing, but as far as the demand, I see it growing slowly in fits and starts, as people figure out the role architecture plays.

Gardner: Okay, next one please.

Bloomberg: As far as the end of the recession, yeah, we're all hoping that the economy picks up, and I do see that there is going to be a lot of additional activity as a result of an improving economy, but I don't see a huge uptake in spending on software per se.

Spending in IT is going to go up, but in terms of what the executives going to invest in, they're going to be very careful about purchasing software. That's going to drive some money to cloud-based solutions, but that's still just a toe in the water as well.

Software vendors were hoping for a huge year, but they're going to be disappointed. It's going to be a growth year, but it's going to be moderate growth for the vendors.

Gardner: So that must be why Oracle bought Sun, right?

Bloomberg: Well, we'll have to see. There's been a lot of press on their core strategy in terms of what they are trying to do. Clearly, consolidation is in the cart. I'd agree with that. Part of that is because there are only so many software dollars to go around, and that's going to continue to be the case for a while.

Gardner: Okay, thank you. What’s your next point?

Bloomberg: Those are my first four. Those are the negatives. Not to be too negative, in terms of the positive, what we see happening in 2010 is increased focus on "MSW." You know what MSW is, right? Politely speaking it's "Make Stuff Work." Of course, you could put a different word in there for the S, but Make Stuff Work, that's what we see the architects really focusing on.

They have a good idea now of what SOA is all about. They have a good idea about how the technology fits in the story and the various technologies that have been mentioned on this call, whether it's analytics, data management, SaaS, and the cloud-based approaches. Now, it's time to get the stuff to work together, and that's the real challenge that we see.


The SOA story is no longer an isolated story. We're going to do SOA, let's go do SOA. But, it's SOA plus other things. So, we're going to do SOA, BPM, and the architecture driving that, despite the fact that the CIO may not quite connect the dots there.

SOA plus master data management (MDM) -- it's not one or the other now. It's how we get those things to work together. SOA plus virtualization. That's another challenge. Previously, those conversations were separate parts of the organization. We see more and more conversations bringing those together.

SOA and SaaS -- somebody already mentioned that SaaS is one segment of the cloud category. It's little more mature than the rest. We see more organizations understanding the connection between those two and trying to put them together.

Gardner: Are you that we're seeing services orientation of the enterprise?

Bloomberg: You can put it that way, and we like putting it that way, because we're the SOA guys. It depends on who you talk to whether the people in the organization see it that way or, rather, see that that there's a role for architecture as part of how you do things right. When we talk about architecture broadly, we're just talking about general best practices.

No one piece of the story is the whole story anymore. It's going to be a heterogeneity story in the enterprise and how we actually get this stuff to work together.

If you think about governance, for example, as a core set of best practices for running an organization, the key best practice is for it to be architecture driven, and that simply means best-practice driven. So, you can think of architecture as a way of codifying and communicating IT best practices as well as organizational best practices for leveraging IT.

We see that becoming more prevalent over time, as organizations understand the importance of connecting architectural best practices with the other things they're doing.

Before, we had this disconnect. We'll do middleware and we'll do SOA, but we don't really see the connection where we confuse one for the other, and that was a big issue. A large part of why Anne Manes said SOA was dead was because we were confusing SOA with the software enablers that vendors were trying to sell them. With the SOA label on the box, they opened the box and said, "Where's my SOA? I don't get it."

Well, organizations are getting that. Now, they're seeing that there is a connection, and they're trying to get this stuff to work together. In the enterprise context where it's heterogeneous, it needs to scale. It’s broad based, and there are a lot of moving parts. No one piece of the story is the whole story anymore. It's going to be a heterogeneity story in the enterprise and how we actually get this stuff to work together.

Gardner: A services-oriented whole greater than the sum of the IT parts?

Bloomberg: Yeah. We're happy to call this services-oriented, even though the organization, as a whole, may call it variety of different things, depending on the perspective of the individual.

Gardner: Great. Thanks so much. Okay, last but not least, Tony Baer, are you still out there? Thanks for your patience.

Tony Baer: I am here, present, and I am alive.

Gardner: You have to be quick, because we're almost out of time. What are your top five, Tony?

Tony Baer

Baer: Not a problem. I’ll make it very, very quick. Actually, I am just going to add various comments. On cloud and virtualization, basically I agree with Jason, and I don't agree with David or with Joe. It’s not going to be the "new normal." We're going to see this year an uptake of all the management overhead of dealing with cloud and virtualization, the same way we saw with outsourcing years back, where we thought we'd just throw labor costs over the wall.

Secondly, JP, I very much believe that there is going to be convergence between BI and CEP this year. I agree with him that there's not going to be a surge of Albert Einsteins out there. On the other hand, I see this as a golden opportunity for vendors to package these analytics as applications or as services. That's where I really see the inflection curve happening.

Number three: Microsoft and Google. Microsoft will be struggling to stay relevant. Yes, people will buy Windows 7, because it's not Vista. That’s kind of a backhanded compliment to say, "We're buying this, because you didn't screw up as badly as last time." It doesn't speak well for the future.

Google meets a struggle for focus. I agree with Joe that they are aspiring to be the Microsoft of the cloud, but it may or may not be such a good thing for Google to follow that Microsoft model.

Finally, I agree with Jim that you are going to see a lot more business-oriented, whether it's BI, BPM, or IBM buying Lombardi. I hope they don't mess up Lombardi and especially I hope they don't mess up Blueprint. I've already blogged about that.

I very much believe that there is going to be convergence between BI and CEP this year.

One other point -- and I don't know if this fits into a top five or not -- but I found what Joe was talking about very interesting in terms of the let-down on health-care investment in IT. There's going to be lot a of pushing in electronic medical records (EMR) this year. I very much believe in EMRs, but, on the other hand, they are no panacea. We're going to see a trough of disillusionment happen on that as well.

I don't know if that's fast, but that's my story and I am sticking to it.

Gardner: Well, that was great, very zippy, I appreciate that and I'm afraid we're out of time. I want to thank our guests and our panel for these very insightful predictions. It's going to be a fun year. Everything from Google and snow plowing to cheap, but not private and not secure, cloud -- a lot to look forward to.

Let me again thank our panel, Jim Kobielus, senior analyst of Forrester Research, thank you so much.

Kobielus: Have a good, happy new year everybody.

Gardner: Joe McKendrick, independent analyst and prolific blogger. Thank you, sir.

McKendrick: Thank you and looking forward to a great 2010.

Gardner: Tony Baer, senior analyst at Ovum, thank you.

Baer: Yes, thanks.

Gardner: Great insights from Brad Shimmin, principal analyst at Current Analysis. Thanks.

Shimmin: Thanks much, Dana.

Gardner: Dave Linthicum, CEO of Linthicum Group, again appreciating your insights.

Linthicum: Thanks, everybody.

Gardner: Dave Lounsbury, vice president, collaboration services at The Open Group, thanks so much for joining us.

Lounsbury: Thank you, Dana.

Gardner: Jason Bloomberg, managing partner at ZapThink. Very good. I appreciate your input.

Bloomberg: Thanks, Dana.

Gardner: And JP Morgenthal, independent analyst and IT consultant. Thank you, sir.

Morgenthal: Thank you, Dana. Thank you for inviting me. It's always a pleasure to be with this group.

Gardner: And, I would like to thank our sponsors for the BriefingsDirect Analyst Insights Edition, Active Endpoints.

This is Dana Gardner, principal analyst at Interarbor Solutions. Thanks for listening, and come back next time. Have a great and happy new year.

Listen to the podcast. Find it on iTunes/iPod and Download the transcript. Charter Sponsor: Active Endpoints.

Special offer: Download a free, supported 30-day trial of Active Endpoint's ActiveVOS at

Edited transcript of BriefingsDirect Analyst Insights Edition podcast, Vol. 49, with panel of analysts discussing the future of cloud computing, SOA, social networks and the economy. Copyright Interarbor Solutions, LLC, 2005-2010. All rights reserved.

You may also be interested in:

Monday, October 05, 2009

Part 2 of 4: Web Data Services Provide Ease of Data Access and Distribution from Variety of Sources, Destinations

Transcript of a sponsored BriefingsDirect podcast, one of a series on web data services, with Kapow Technologies, with a focus on information management for business intelligence.

Listen to the podcast. Find it on iTunes/iPod and Download the transcript. Sponsor: Kapow Technologies.

Dana Gardner: Hi, this is Dana Gardner, principal analyst at Interarbor Solutions, and you’re listening to BriefingsDirect.

Today, we present a sponsored podcast discussion on how to make the most of web data services for business intelligence (BI). As enterprises seek to gain better insights into their markets, processes, and business development opportunities, they face a daunting challenge -- how to identify, gather, cleanse, and manage all of the relevant data and content being generated across the Web.

In Part 1 of our series we discussed how external data has grown in both volume and importance across internal Internet, social networks, portals, and applications in recent years. As the recession forces the need to identify and evaluate new revenue sources, businesses need to capture such web data services for their BI to work better and fuller.

Enterprises need to know what's going on and what's being said about their markets across those markets. They need to share those web data service inferences quickly and easily across their internal users. The more relevant and useful content that enters into BI tools, the more powerful the BI outcomes -- especially as we look outside the enterprise for fast shifting trends and business opportunities.

In this podcast, Part 2 of the series with Kapow Technologies, we identify how BI and web data services come together, and explore such additional subjects as text analytics and cloud computing.

So, how to get started and how to affordably bring web data services to BI and business consumers as intelligence and insights? Here to help us explain the benefits of web data services and BI, is Jim Kobielus, senior analyst at Forrester Research.

Jim Kobielus: Hi, Dana. Hello, everybody,

Gardner: We're also joined by Stefan Andreasen, co-founder and chief technology officer at Kapow Technologies. Welcome, Stefan.

Stefan Andreasen: Thank you, Dana. I'm glad to be here.

Gardner: Jim, let's start with you. Let's take a look at what's going on in the wider BI field. Is it true that the more content you bring into BI the better, or are there trade-offs, and how do we manage those tradeoffs?

The more the better

Kobielus: It's true that the more relevant content you bring into your analytic environment the better, in terms of having a single view or access in a unified fashion to all the information that might be relevant to any possible decision you might make within any business area. But, clearly, there are lots of caveats, "gotchas," and trade-offs there.

One of these is that it becomes very expensive to discover, to capture, and to do all the relevant transformation, cleansing, storage, and delivery of all of that content. Obviously, from the point of view of laying in bandwidth, buying servers, and implementing storage, it becomes very expensive, especially as you bring more unstructured information from your content management system (CMS) or various applications from desktops and from social networks.

So, the more information of various sorts that you bring into your BI or analytic environment, it becomes more expensive from a dollars-and-cents standpoint. It also becomes a real burden from the point of view of the end user, a consumer of this information. They are swamped. There's all manner of information.

If you don't implement your BI environment, your advanced analytic environment, or applications in a way that helps them to be more productive, they're just going to be swamped. They're not going to know what to do with it -- what's relevant or not relevant, what's the master reference, what's the golden record versus what's just pure noise.

So, there is that whole cost on productivity, if you don't bring together all these disparate sources in a unified way, and then package them up and deliver them in a way that feeds directly into decision processes throughout your organization, whether HR, finance, or the like.

Gardner: So, as we look outside the organization to gain insights into what market challenges organizations face and how they need to shift and track customer preferences, we need to be mindful that the fire hose can't just be turned on. We need to bring in some tools and technologies to help us get the right information and put it in a format that's consumable.

Kobielus: Yes, filter the fire hose. Filtering the fire hose is where this topic of web data services for BI comes in. Web data services describes that end-to-end analytic information pipe-lining process. It's really a fire hose that you filter at various points, so that the end users turn on their tap and they're not blown away by a massive stream. Rather, it's a stream of liquid intelligence that is palatable and consumable.

Gardner: Stefan, from your perspective in working with customers, how wide and deep do they want to go when they look to web data services? What are we actually talking about in terms of the type of content?

Andreasen: Referring back to your original question, where you talk about whether we need more content, and whether that improves the analysis and results that analysts are getting, it's all about, as Jim also mentioned, the relevance and timeliness of the data.

There is a fire hose of data out there, but some of that data is flowing easily, but some of it might only be dripping and some might be inaccessible at all. Maybe I should explain the concept.

Think about it this way. The relevant data for your BI applications is located in various places. One is in your internal business applications. Another is your software-as-a-service (SaaS) business application, like Salesforce, etc. Others are at your business partners, your retailers, or your suppliers. Another one is at government. The last one is on the World Wide Web in those tens of millions of applications and data sources. There is very often some relevant information there.

Accessible via browser

Today, all of this data that I just described is more or less accessible in a web browser. Web data services allow you to access all these data sources, using the interface that the web browser is already using. It delivers that result in a real-time, relative, and relevant way into SQL databases, directly into BI tools, or to even service enabled and encapsulated data. It delivers the benefits that IT can now better serve the analysts need for new data, which is almost always the case.

BI projects happen in two ways. One is that you make a completely new BI. You get a completely new BI system, and then make brand-new reports, and new data sources. That's the typical BI project.

What's even more important is that incremental daily improvement of existing reports. Analysts sit there, they find some new data source, they have their report, and they say, "It would be really good, if I could add this column of data to my report, maybe replace this data, or if I could get this amount of data in real-time rather than just once a week." So it's those kinds of improvements that web data services also really can help with.

Gardner: Jim Kobielus, it sounds like we've got two nice opportunities here. One is the investments that have already been made in BI internally, largely for structured data. Now, we have this need to look externally and to look at the newer formats internally around web content and browser-based content. We need to pull these together.

Kobielus: There are a lot of trends. One of them is, of course, self-service mashups by end users of their own reports, their own dashboards, and their own views of data from various sources, as well as their data warehouses, data marts, OLAP cubes and the like.

But, another one gets to what you're asking about, Dana, in terms of trends in BI. At Forrester, we see traditional BI as a basic analytics environment, with ad-hoc query, OLAP, and the like. That's traditional BI -- it's the core of pretty much every enterprise's environment.

Advanced analytics, building on that initial investment and getting to this notion of an incremental add-on environment is really where a lot of established BI users are going. Advanced analytics means building on those core reporting, querying, and those other features with such tools as data mining and text analytics, but also complex event processing (CEP) with a front-end interactive visualization layer that often enables mashups of their own views by the end users.

When we talk about advanced analytics, that gets to this notion of converging structured and unstructured information in a more unified way. Then, that all builds on your core BI investment -- smashing the silos between data mining and text mining that many organizations have implemented for good reasons. These are separate projects, probably separate users, separate sources, separate tools, and separate vendors.

We see a strong push in the industry towards smashing those silos and bringing them all together. A big driver of that trend is that users, the enterprises, are demanding unified access to market intelligence and customer intelligence that's bubbling up from this massive Web 2.0 infrastructure, social networks, blogs, Twitter and the like.

Relevant to ongoing activities

That's very monetizable and very useful content to them in determining customer sentiment, in determining a lot of things that are relevant to their ongoing sales, marketing, and customer service activities.

Gardner: So, we're not only trying to bring the best of traditional BI with this large pool of valuable information from web data services. We're also trying to extend the benefits of BI beyond just the people who can write a good SQL query, the proverbial folks in the white lab coats behind the glass windows. We're trying to bring those BI analytics out to a much larger class of people in the organization.

Kobielus: Exactly. SQL queries are the core of traditional BI and data warehousing in terms of the core access language. Increasingly, in the whole advanced analytics space, SQL is becoming just one of many access techniques.

One might, in some ways, describe the overall trend as toward more service-oriented architecture (SOA), oriented access of disparate sources through the same standard interfaces that are used everywhere else for SOA applications. In other words, WS/XML, WSDL, SOAP, and much more.

So, SOA is coming to advanced analytics, or is already there. SOA, in the analytics environment, is enabled through a capability that many data federation vendors provide. It's called a "semantic virtualization layer." Basically, it's an on-demand, unified roll up of disparate sources.

Increasingly, in the whole advanced analytics space, SQL is becoming just one of many access techniques.

It transforms them all to a common set of schemas and objects, which are then wrapped in SOA interfaces and presented to the developer as a unified API or service contract for accessing all this disparate data. SOA really is the new SQL for this new environment.

Gardner: Stefan, what is holding back organizations from being able to bring more of this real-time, highly actionable information vis-à-vis web services? What's preventing them from bringing this into use with their BI and analytics activity?

Andreasen: First, let me comment on what Jim said, and then try to answer your question. Jim's comment about SOA as common to BI is really spot on.

The world is more diverse

Traditionally, for BI, we've been trying to gather all the data into one unified, centralized repository, and accessing the data from there. But, the world is getting more diverse and the data is spread in more and different silos. What companies realize today is that we need to get service-level access to the data, where they reside, rather than trying to assemble them all.

So, tomorrow's data stores of BI, and today's as well -- and I'll give you an example -- is really a combination of accessing data in your central data repositories and then accessing them where they reside. Let me just explain that by an example.

One Fortune 500 financial services company spent three years trying to build a BI application that would access data from their business partners. The business partners are big banks spread all over the U.S. The effort failed, but they had to solve this problem, because it was a legal and regulatory necessity for them.

So, they had to do it with brute force. Basically, they had analysts logging into their business partners' web sites and business applications, and copying and pasting those data into Excel to deliver those reports.

Finally, we got in contact with them, and we solved that problem. Web data services can encapsulate or wrap the data silos that were residing with their business partners into services -- SOAP services, REST services, etc. -- and thereby get automated access to the data directly into the BI tool. So, the problem they tried to solve for three years could now be solved with data services, and is running really successfully in production today.

This is also where web data services technology comes into play. Who knows best what data they want? It's the analysts, right? But who delivers the data? It's the IT department.

Kobielus: Dana, before we go to the next question, I want to extend what Stefan said, because that's very important to understand this whole space. This new paradigm, where SOA is already here in advanced analytics, is enabled by mashup. I published a report recently called Mighty Mashups that talks about this trend.

You need two core things in your infrastructure to make this happen. One is data mashups. In the back end, in the infrastructure, you need to have orchestrated integration, transformations, consolidation, and joining among disparate data sets. Then, you expose those composite data objects as services through SOA.

Then, in the front end, you need to enable end users to have access to these composite data objects through a registry, or whatever you call it, that's integrated into the environments where the user actually does work, whether it's their browsers/portal, Excel, or Microsoft Office environment. So, it's the presentation mashup on the user front end, and data mashup -- a.k.a. composite data objects -- on the back end to make this vision a reality.

Gardner: So, what's been holding back this ability to use a variety of different data types, content types, and data services in relation to BI has been proprietary formats, high cost and complexity, laborious manual processes, perhaps even spreadsheets, and a little older way of presenting information. Is that fair, Stefan?

Andreasen: I think so, yes. This is also where web data services technology comes into play. Who knows best what data they want? It's the analysts, right? But who delivers the data? It's the IT department.

Tools are lacking

Today, the IT department often lacks tools to deliver those custom feeds that the line of business is asking for. But, with web data services, you can actually deliver these feeds. The data that IT is asking for is almost always data they already know, see, and work with in the business applications, with the business partners, etc. They work with the data. They see them in the browsers, but they cannot get the custom feeds. With the web data services product, IT can deliver those custom feeds in a very short time.

Let me use an example here again. This is a real story. Suppose I am the CEO of one of the largest network equipment manufacturers in the world. I am running a really complex business, where I need to understand the sales figures and the distribution model. I possibly have hundreds of different systems and variables I need to look at to run my business.

Another fact is I am busy. I travel a lot. I'm often in the airport or where I don't have access to my systems. When I finally get access, I have to open my laptop, get on the 'Net’, and pull out my report.

What we did here was we took our product, service enabled the relevant reports, built a Blackberry front end to that, and delivered that in three hours, from start to end. So, suddenly, in a very agile fashion, the CEO could reach his target and look at his data anywhere he had wireless access.

Gardner: It must be very frustrating for these analysts, business managers, and business development people to be able to see content and data out on the web through their browser, but not be able to get it into context with their internal BI systems, and get those dashboards and views that allow a much fuller appreciation of what's really going on.

So, breaking down this barrier and giving them the key to the house, or actually giving IT a way to deliver the key to the house, is critical for the agility of BI going forward.

Andreasen: It's almost absurd. Think about it. I'm an analyst and I work with the data. I feel I own the data. I type the data in. Then, when I need it in my report, I cannot get it there. It's like owning the house, but not having the key to the house. So, breaking down this barrier and giving them the key to the house, or actually giving IT a way to deliver the key to the house, is critical for the agility of BI going forward.

Kobielus: I agree. Here's an important point I want to make as well. The key to making this all happen, making this mashup vision of reality in the final analysis, is expanding the flexibility of your data or source discovery capabilities within the infrastructure.

Most organizations that have a BI environment have one or more data warehouses aggregating and storing the data and they've got pre-configured connections and loading of data from specific sources into those data warehouses. Most users who are looking at reports in their BI environment are looking only at data that's pre-connected, pre-integrated, pre-processed by their IT department.

The user feels frustration, because they go on the Web and into Google and can see the whole universe of information that's out there. So, for a mashup vision to be reality, organizations have got to go the next step.

Much broader range

It's good to have these pre-configured connections through extract, transform and load (ETL) and the like into their data warehouse from various sources. But, there should also be ideally feeds in from various data aggregators. There are many commercial data aggregators out there who can provide discovery of a much broader range of data types -- financial, regulatory, and what not.

Also, within this ideal environment there should be user-driven source discovery through search, through pub-sub, and a variety of means. If all these source-discovery capabilities are provided in a unified environment with common tooling and interfaces, and are all feeding information and allowing users to dynamically update the information sets available to them in real-time, then that's the nirvana.

That means your analytic environment is continuously refreshed with information that's most relevant to end users and the decisions they are making now.

Gardner: So, we've identified the problem, and that's bringing the best of web services and web data into the best of what BI does and then expanding the purview of that beyond the white lab coats crowd, into the people who can take action on it. That's great. But, with the fire hose, we can't just start allowing this access to these data services without what the IT department considers critical. That is to keep the cost down, because we're still in recession and the budgets are tight.

We also need to have governance. We need to have manageability. We need to make the IT people feel like they can be responsible in opening up this filtered fire hose. So how do we do that, Stefan? How do we move from pure web static to an enterprise-caliber web data services?

The way our product works is that it allows you to instruct our system how to interact with a web application, just the same way as the line of business user.

Andreasen: Thank you for mentioning that. Jim, to get back to you on mashups, that's really relevant. Let's just look at the realities in IT departments today. They're probably understaffed. They've probably got budget cuts, but they have more demand from lines of business, and they probably also have more systems they have to maintain. So, they're being pushed from all sides.

What's really necessary here is a new way of solving this problem. This is where Kapow and web data services come in, as a disruptive new way of solving a problem of delivering the data -- the real-time relevant data that the analyst needs.

The way it works is that, when you work with the data in a browser, you see it visually, you click on it, and you navigate tables and so on. The way our product works is that it allows you to instruct our system how to interact with a web application, just the same way as the line of business user.

This means that you access and work with the data in the world in which the end users see the data. It's all with no coding. It's all visual, all point and click. Any IT person can, with our product, turn data that you see in a browser into a real feed, a custom feed, virtually in minutes or in a few hours for something that would typically take days, weeks, or months -- or may even be impossible.

Hand in hand

So a mashup is really an agile business application, a situational application. How can you make situational BI without agile data, without situational data? They basically go hand in hand. For mashups to deliver on the promise, you really need a way to deliver the data feeds in a very agile fashion.

Gardner: But what about governance and security?

Andreasen: Web data services access the data in the way you do from a web browser. All data resides in a database somewhere -- inside your firewall, at a customer, at a partner, or somewhere. That database is very secure. There's no way to access the database, without going through tedious processes and procedures to open a hole in that firewall.

The beauty with web data services is that it's really accessing the data through the application front end, using credentials and encryptions that are already in place and approved. You're using the existing security mechanism to access the data, rather than opening up new security holes, with all the risk that that includes.

Gardner: Jim, from some of the reports that you've done recently, what are customers, the enterprise customers, telling you about what they need in terms of better access to web data services, but also mindful about the requirements of IT around security and governability and so forth?

Kobielus: Right, right. The core theme I'm hearing is that mashups, user self-service development, and maintenance of user disparate data are very, very important, for lots of reasons. One, of course, is speeding delivery of analytics and allowing users to personalize it, and so forth. But, mashups without IT control is essentially chaos. And, mashups without governance is an invitation to chaos.

. . . users should be able to mashup and create their own reports and dashboards, but, from the perspective of the companies that employ them, they should only be able to mashup from company-sanctioned sources . . .

What does governance mean in this environment? Well, it means that users should be able to mashup and create their own reports and dashboards, but, from the perspective of the companies that employ them, they should only be able to mashup from company-sanctioned sources, such as data warehouses data marts, and external sources.

They should be able to only mashup that data, tables, records, or fields that they have authorized access to. They should only be able to mashup within the bounds of particular templates, reports, and dashboards that are sanctioned by the company and maintained by IT. There should be ongoing monitoring of access, utilization, and refreshes.

Then, users should be able to share their mashups with other users to create ever more composite mashups, but they should only be able to share data analytics that the recipient has authorized access to.

Now, this sounds like fascism, but it really isn't, because in practice what goes on is that users are usually given a long leash in a mashup environment to be able to pull in external data, when need be, with IT being able to monitor the utilization or the access of that data.

Fundamentally, governance comes down to the fact that all the applications are stored within a metadata environment -- repositories, and so forth -- that are under management by IT. So, that's the final piece in the mashup governance equation.

Gardner: I think I'm hearing you say that you really should have an intermediary between all of that web data and your BI analytics and the people making the decisions, not only for those technical reasons, but also to vet the quality of the data.

It’s in IT’s interest

Kobielus: Exactly. This is in IT's interest, and they know that. IT wants to insource as much of the development and maintenance of reports and dashboards and the like as they can get away with, which means it's pushed down to the end user to do the maintenance themselves on their own views.

IT is more than happy to go toward mashup, if there is the ability for them to keep their eyes and ears open, to set the boundaries of the sandbox, and insource to end users.

Gardner: Stefan, I want to go back to you, if I could. We talked about how to bring this into IT, but we also need to bring in to this the role of the developer, because we're just not talking about integration, we're also talking about presentation.

Does what Kapow brings to the table also allow those developers to get a task about trying to expose web data services within the context of applications, views, different audit presentation, dashboards, and what not? What's the role of the developer in this?

Andreasen: That's very important. We talked about this fire hose before. When I see a fire hose in front of me, I imagine the analyst can now open this fire hose and all the data in the world just splashing in their face, and that's really not the case. web data services allows the developer to incite the IT department to much more quickly develop and deliver those custom feeds or those custom web services that the analysts need in the BI tools.

Also, on governance, the reality is that the data that has value is data that comes from business partners, from government, or from sources where you have a business relationship, and therefore can govern it.

Also, on governance, the reality is that the data that has value is data that comes from business partners, from government, or from sources where you have a business relationship, and therefore can govern it. But, for various reasons, you cannot rewrite those applications, you cannot access those SQL databases in a traditional way. web data services is a way to access data from trusted sources, but access them in a much more agile way.

Gardner: Those services are coming across in a standardized format that developers can work with using existing tools.

Andreasen: Yes, that's very important. Web data services deliver the data into your standard data warehouse, into your standard SQL databases. Or, as I said earlier, it can wrap those applications into SOAP services, REST services, RSS feeds, and even .NET and Java API, so you get the API or you get the data access exactly the way you need it in your BI tool, in your data mining environment, etc.

Gardner: We've established the need. We've looked at the value of increasing BI's purview. We've looked at the larger trends around SOA and bringing lots of different data types into an architecture that can then be leveraged for BI and analytics. We've looked at the need for extending this to business processes outside the organization, as well as data types inside. We've looked at the role of the developer.

Are there examples, Stefan, of people who are actually doing this, who have been early adopters, who have taken the step of recognizing an intermediary and the tool and platform set to manage web data services in the context of BI? And, if they've done that, what are the paybacks, what are the metrics of success?

Andreasen: One of our early adopters is Audi. They've been using our product for five years. What was important for them was that, traditionally, it could take three to six months for them to get access to some data. But, with the Kapow Web Data Server, they were able to access data and create these custom feeds in a much shorter fashion, days rather than months.

What the business needs

They have been using it successfully for five years. They are growing with it, they're getting a lot of benefit around it, and couldn't imagine running the IT department without web data services today, because it gives them the way to deliver this agile custom data feeds that the business needs.

Gardner: Jim Kobielus, looking to the future, it seems to me that there is going to be more types of data coming from external sources. Perhaps, more of the internal data that companies have used in traditional applications -- BI and integration -- might find itself being housed in server farms, otherwise known as clouds, either on-premises, on some third-party grid or utility fabric, or some hybrid of the two.

When we factor in the movement and expected direction of cloud computing, how does that then bear down on the requirements for managed, governed, and IT-caliber, mission-critical caliber web data service tools?

Kobielus: It simplifies it and complicates it. It simplifies to some degree or enables this vision of self-service BI mashup, with automated source discovery, to come to fruition. You need a lot of compute power, you need a lot of data storage to do things like high volume, real-time text analytics.

A lot of that is going to have to be outsourced to public clouds that are scalable. They can scale out petabytes worth of data or can scale out some massive server farms to do semantic analysis and transformations and the like. So, the storage and the processing for most visions have to be outsourced to cloud providers. To some degree it makes it possible to realize this vision on the back end, at the web data services and data mashup side.

Public clouds are essentially silos from each other . . . They don't necessarily interoperate out of the box with your existing premises data environment, if you're an enterprise.

It also complicates it, because now you're introducing more silos. Public clouds are essentially silos from each other. There is Amazon, and there is the Windows SQL data or Azure, Then, of course, there is Google and a variety of others that are providing clouds that don't interoperate well, or at all, with each other. They don't necessarily interoperate out of the box with your existing premises data environment, if you're an enterprise.

So, the governance of all these disparate functions, the coordination of security, and the encryption and so forth across all these environments, as well as the coordination of the data archiving and auditing need to be worked out by each organization that goes this route with a disparate and motley assortment of internal and external platforms that are managing various functions within this analytic cloud.

In other words, it could complicate this whole equation considerably, unless you have one predominant public cloud partner that can do all the data integration, all the cleansing, all the transforms, all the warehousing in their cloud, and can provide you also with this SOA abstraction layer, the semantic virtualization layer, and can also ideally host your advanced analytics applications, like your data mining, in that environment.

It can do it all for you in a very streamlined way, with a common governance, security administration, and data modeling toolset. Remember, end users are a big part of this equation here. The end users can then pick up these cloud-based tools to mash up data within this unified cloud and mash it up in a way that makes sense to end users, not the professional black belt data modelers.

That vision cannot be realized right now with the commercial cloud offerings in the analytic market. I think it will take about two to three to five years for the cloud providers to go this route. It's not there yet.

Gardner: We're about out of time. I want to take the same question to Stefan about the cloud computing angle and the mixed sourcing for applications, datasets, and business processes. It seems to me this would be an opportunity for Kapow.

No master hub

Andreasen: Absolutely. What I don't see is one big vendor that solves all your data needs and becomes like the master hub for all information and data on the Web. History has shown that the way that companies compete with each other is to differentiate themselves.

If everybody was using the same provider and the same kind of data, they couldn't differentiate. This is really, I think, what companies realize today -- unless we do something different and better, than our competitors, we are not going to win this game.

What's important with web data services is hosting the tools and the facilities to access the data, but allowing the customers to create in a self-service fashion the custom data feeds they need. Our product fits perfectly into that world as well. We already have many of our customers using out product in the cloud. We become a tool where they can create ad hoc, on demand, or as necessary data feeds, and to share them with anybody else that needs them.

Kobielus: I've got one more point. In this ecosystem that's emerging, there's a strong role for providers of tooling specifically focused on self-service mashup and also for what's often called on-demand analytical sandboxing, which could be used by end users to create their own analytic workspace, and pull information.

What's important with web data services is hosting the tools and the facilities to access the data, but allowing the customers to create in a self-service fashion the custom data feeds they need.

Those that can provide the tooling that works in front of whatever the organization's preferred data management or data federation or data warehousing or BI vendor might be. So there's a plenty of opportunity for the likes of Kapow, and many others in this space too, for complementary solutions that are integrated with any of the leading data federation and cloud analytic solutions that are out there.

Gardner: Very good. I'm afraid we'll have to leave it there. We've been discussing the requirements around bringing web data services into BI, but doing so in a mission-critical fashion that's amenable to the IT department.

I want to thank our guests. We've been joined by Jim Kobielus, senior analyst at Forrester Research. Thanks, Jim.

Kobielus: Sure, no problem.

Gardner: We've also been joined by Stefan Andreasen. He's the co-founder and chief technology officer at Kapow Technologies. Thank you so much, Stefan.

Andreasen: Thank you everyone for a great discussion.

Gardner: This is Dana Gardner, principal analyst at Interarbor Solutions, and you've been listening to a sponsored BriefingsDirect podcast. This is just part of a series of four podcasts on the subjects around web data services and BI.

We look forward to future discussions on text analytics, cloud computing, and the role of BI in the future. Thanks for listening, and come back next time.

Listen to the podcast. Find it on iTunes/iPod and Download the transcript. Sponsor: Kapow Technologies.

Transcript of a sponsored BriefingsDirect podcast, one of a series on web data services, with Kapow Technologies, with a focus on information management for business intelligence. Copyright Interarbor Solutions, LLC, 2005-2009. All rights reserved.