Edited transcript of weekly BriefingsDirect[TM] SOA Insights Edition, recorded Jan. 5, 2007.
Listen to the podcast here. If you'd like to learn more about BriefingsDirect B2B informational podcasts, or to become a sponsor of this or other B2B podcasts, contact Dana Gardner at 603-528-2435.
Dana Gardner: Hello, and welcome to the latest BriefingsDirect SOA Insights Edition, Volume 7, a weekly discussion and dissection of Services Oriented Architecture (SOA)-related news and events, with a panel of industry analysts and guests. I’m your host and moderator, Dana Gardner, principal analyst at Interarbor Solutions, ZDNet blogger and Redmond Developer magazine columnist.
Our panel this week -- and that’s the week of Jan. 1, 2007 -- consists of Steve Garone, a former IDC group VP, founder of the AlignIT Group, and an independent industry analyst. Welcome, Steve.
Steve Garone: Hi, Dana, and happy New Year.
Gardner: Thanks. Also joining us again, Joe McKendrick, research consultant, columnist at Database Trends, and a blogger at ZDNet and ebizQ. Welcome back, Joe.
Joe McKendrick: Thanks, Dana, I’m glad to be back, and happy New Year as well.
Gardner: Also, Tony Baer, making a second appearance on this show, principal at onStrategies. Thanks for joining, Tony.
Tony Baer: Happy New Year, Dana.
Dana Gardner: Thank you, and making his SOA Insights debut, Jim Kobielus, he’s a principal analyst at Current Analysis. Hi, Jim.
Jim Kobielus: Hi, Dana.
Gardner: Also, welcome to our listeners. Our topics for today span the SOA spectrum of business and technology. We’re going to be discussing metrics of success, ROI, TCO, how do I know I’m doing? I guess we could call this the Ed Koch approach to SOA: "How am I doing?" Yet we can’t really judge how well SOA is doing, if we don’t have metrics.
We’re also going to work on the impact of SOA on outsourcing: outsourcing of IT, outsourcing of business process. Is this an accelerant? Is this a detractor? What’s the impact? We’re going to explore that. Also, the impact of SOA on services -- software as a service (SaaS) in particular; business applications coming across the wire.
So first off, let’s go back to the metrics topic. Someone out there, and I believe it was you, Tony, mentioned that at one point recently Verizon had come out and said that with a stable of 500 services that they were expecting to yield $20 million in savings over two years. Can you tell us where that information came from and how credible you think it is?
Baer: Good question. I actually spoke with the person who went by the title Executive Director of Strategic Systems at Verizon. Essentially, he reported to the CIO and he was in charge of their SOA program.
He gave a presentation on "SOA In Action." "In action" being two words there. I want to be very careful about that. He gave some metrics. I spoke with him about six or eight months ago, so if there is current data, I’m sure it’s different. But this has been going at Verizon for roughly a couple of years. He spoke of creating in excess of over 500, what he called "unique," services.
Now, I’m always a little skeptical when I hear the term "services," because what does that really mean? The definition can be very loose. It could be we have some functionality here and we’re somehow exposing it, or exposing it using standards, using official Web services standards.
Anyway, his point was that these 500 services, which were developed or reused by a community of over 7,000 developers, now account for -- when I say "now," this is as of six or eight months ago -- over 8.5 million transactions per day. He claims that within a couple of years it has saved the company $20 million. I would presume this is in development time, as opposed to having to create each of these things and reinvent the wheel, which is the traditional way of going about it.
My eyes tend to glaze over at such large numbers without having any greater detail, but the fact is, it’s always been a central tenet of reuse that if you can utilize something and not have to create it from scratch, you theoretically should save money and time. So, my sense is that I think the numbers are probably a bit high and a little loose, but I think they point to the potential that SOA can provide in delivering real savings.
Gardner: I wonder if the figures are high, especially on the number of services. I’ve talked to several companies that seem to think that even a smaller number of services could handle just about all of their back-office and front-office needs, perhaps not in vertical usage or industry usage. Then, for a company like Verizon, with the kind of IT budget they have, $20 million might actually not be that much.
Let’s go to Steve Garone. You’ve talked to a number of companies, and you were looking at ROI and business metrics, do these numbers make sense to you?
Garone: The numbers really aren’t out of the ordinary, but I think one of the challenges that organizations face in doing SOA and then trying to determine their ROI -- for want of a better way to put it -- isn't unique to SOA. It’s really a function of various other types of IT-related activities as well. How do you actually quantify what your ROI is, given the advantages of using an SOA approach? I’ve listed the main reasons why people would want to do SOA, in terms of the advantages, and they basically break down to four major areas.
First is what we just mentioned, which is the reuse of IT assets. That's fairly easy to quantify. As I’ve talked to customers and people who have implemented SOA, I find that this is one that they’re comfortable with and are fairly aggressive about putting numbers to, in terms of what they’ve actually saved. It’s not an easy task, because what actually gets saved when you reuse an asset? You certainly save them some development time and development cost. Do you also save maintenance cost, and how do you quantify that? Is it easier to upgrade assets, and to what extent is that dependent on the infrastructure you have in place?
A lot of factors go into it, so I see various levels of rigor. You see people claiming that they were able to reduce the expense associated with doing the application integration test they normally would have to do. That one is a real tough one, because just understanding what you would have had to do and breaking that down into, not only some specific cost, but also the relationship of the result to being able to do business better, faster with more customers and so on, is a real tough task to accomplish.
The other ones have to do with meeting compliance requirements, which we all know are growing by leaps and bounds every day.
Gardner: Regulatory compliances?
Garone: Yeah, and there’s an area where, again, I think the quantification of benefit might be a little bit easier because the requirements are fairly well identified. In fact, they are very well identified.
The fourth one, which I think is the hardest one, is the issue of how agile do you make your business. It’s a buzzword we hear all the time around SOA. That’s a real tough one because you have to ask such questions as: "If I didn’t do this, how many customers that I'm doing business with now, will I not be doing business with, and what affects that? How do I know what my reach was before versus now? How do I quantify how easy it was for a customer to get to me, and if they do, how comfortable they were given what I presented to them, doing business versus leaving because things were taking too long?"
Gardner: So, most of these are soft measurements, and they would take place over a long period of time, because it’s a journey, and you can’t just say, "Well, we went one quarter. What's our payback?"
Garone: That’s right.
Gardner: Now, let’s go over to Joe McKendrick. These are soft paybacks, and it also seems as if we’re comparing apples to oranges. If we look at mainframe, client-server, Web, and then SOA, which is such a different beast, you can’t really compare. It’s almost a leap of faith. Where do you see the right business metrics coming and getting people onboard in terms of dollars and cents here, Joe?
McKendrick: Well one thing I’ve said a lot on my blog is that the companies that are most likely implementing, or are implementing, SOA are probably the companies that don't really need SOA as much. The companies that really could use SOA in their operations are likely not to be the ones implementing, it’s kind of a paradox.
The reason that is the companies that have the management vision, the support to roll out and move an SOA project forward -- providing top management support, for example -- are likely to have lot of other initiatives on the table. They’re likely to be very forward-looking with their management. Their compensation structure, for example, is likely to be a bit more advanced. They’re likely to have other implementations -- a data warehouse, for example -- a very advanced data warehouse. Essentially what you have in these companies is an orchestra making a lot of beautiful music and SOA may be the horn section, but how do you measure the impact of SOA versus another type of implementation or another type of project?
Gardner: Okay. So the song, the piece of art that you’re creating with this orchestra rises up the charts, and if they're a hit, is it because of the woodwind. Is it because of percussion? Is it because of the conductor? We really don't know. We just know that it’s a hit. Right?
McKendrick: Exactly. There are a lot of case studies out there now. A couple of prominent ones are IBM and HP, who had cases of eating their own dog food. They’ve been able show some terrific significant impact from SOA or service deployment. IBM claims it’s reduced its application portfolio by 75%.
They have one quarter of the applications running their business versus back in 1998 when they really first started looking at this. HP claims it's seeing a payback of about $70 million as result of its SOA efforts. These are companies that tend to be forward thinking. They know that they’re being closely watched, and that their own operations are being closely watched.
In both cases, the benefit is coming from consolidation. They’re using SOA as a lever to help consolidate data centers, various far-flung parts of their enterprises. SOA is just part of that consolidation picture. There’s also virtualization going on, server consolidation. IBM, of course, is big with the mainframe, and they’re consolidating a lot of their operations on Linux running on the mainframe. It’s very much a mixed picture, and as I said, Dana, it’s a matter of being able to separate what’s making this beautiful music.
Gardner: I’ve spoken to HP and IBM as well, and they’re really undergoing IT transformation, probably business transformation, and there are many constituent parts to that, of which SOA is one. But SOA is one that has, I suppose, a lot of interdependencies and effects across many of these other activities, whether it’s server consolidation, application modernization, IT shared services, virtualization and what have you.
Let’s go over to Jim. Jim, if SOA is important, almost like a root system that cuts across a number of different trees that are growing, is it even worthwhile measuring it, or should people just be smart enough to recognize that this is the right thing to do?
Kobielus: That’s an interesting metaphor there -- SOA as a root system. My visual image of SOA is a very complex hyper-mesh. In other words, like a root system, where you have tendrils going hither and yon, the tendrils being simply interactions among services and client.
It’s very worthwhile to measure the ROI of SOA as a paradigm or an approach for enabling and for maximizing the sharing and reuse and interoperability of distributed resources across your network. You make an investment as an organization, as an enterprise, and in this approach you want to know whether you’re investing your funds and your resources wisely. When I think of SOA’s ROI, I think of two numbers, and those numbers are 100 and zero.
As we know, SOA focuses on how you maximize the sharing and reuse of services, of application functionality and resources. In other words, how do you enable a 100 percent reuse as a nirvana? We’ll never get there, but in any given organization, 100 percent reuse, service reuse, first and foremost is a consolidation topic. What that means is, if you do SOA right, you’re doing much more with much less.
You’re able to consolidate redundant silos of application functionality and data throughout the organization. You’re able to consolidate fewer software licenses and servers, with the associated translation and cost savings and capital on operating budgets, fewer redundant software components and so forth. The need for fewer programming groups, as we can consolidate that as well.
So 100 percent reuse is the nirvana. The zero comes in the sense that, if you’re doing SOA right, the marginal cost of billing the next application drops pretty close to zero. You’re able to reuse everything that’s already been built. You do not have to reinvent the wheel. So, basically, a 100 percent reuse means zero marginal cost of building the next application. Of course, as I said, you enable that vision through consolidation, both in software and hardware, and in programming teams, and so forth.
So, once again, getting back to your root-system-and-tree metaphor, SOA becomes this ubiquitous root system from which new sprigs can pop up, without needing to lay down their own root system. Rather they are simply branches on a huge underground system. In Northern Michigan, where I’m from, scientists have discovered the world’s largest organism as a mushroom or a fungus of some sort that spans 30, 40, or 50 square miles. They determined though DNA analysis that it's the exact same individual and has got the largest biomass in the world. In essence -- and it’s all underground pretty much. That’s what SOA is all about, essentially all the services in an SOA sort of share a common DNA.
Gardner: Well, there’s the message we need to take to the CEOs and the CFOs. Let’s make our IT like a fungus.
Kobielus: I think they probably already believe that!
Gardner: Let’s take this now to the outsourcing side. You mentioned between zero and a 100 percent in terms of reuse. That would be reuse for your internal resources, but what about reuse of external services? Things that are made available in a marketplace, because of the effects of economics and supply and demand in competition, could push the prices down while even enhancing the quality of services.
For those commoditized services that everyone uses, that really don’t differentiate anyway, why not go in that direction? What’s the relationship here, Steve Garone, between capturing ROI and lowering total cost of ownership, and starting to look toward outsourcing an SOA as being somehow a tag team?
Garone: Well, actually I think there is a fairly close relationship. Again, it’s not unusual that the drivers are really the same as they are in the case of other types of IT investments. Outsourcing can potentially drive costs down, increase quality, allow you to lighten up your IT staff, and so on. So, the same rules apply here. If you look at what’s going on, there are actually several companies now that are doing this. And, it’s not just commodity services. Look at companies like Salesforce.com, for example, which is essentially creating a services-based hosting environment, not only for their applications, but for applications and services that you actually produce in house.
Gardner: Incidentally, we should expect from Amazon.com similar activity around a retail and e-commerce sector.
Garone: I wouldn’t be surprised. So, the drivers are the same, and I think it’s going to happen. I think that if you’re talking about outsourcing services from the hosting standpoint, that’s certainly real and it’s certainly happening now. If you’re talking about outsourcing services from the development and IT support standpoint, meaning how do I go about building my SOA environment, there’s going to be a stage initially where there’s going to be a lot of that, for the same reasons people outsourced that kind of work before.
People look toward SOA and Web services to simplify their application integration environment. That may, in fact, be the case, but in the short term, people still really need to know how to do it and need, in some cases, to outsource for the expertise to get that done.
Gardner: It seems to me that in the not-too-distant future being able to access commodity-level services that you can incorporate into your SOA activities, off the wire, in a competitive environment, would be a really big economic benefit, to reduce your total costs. Maybe not grow your top line right away, but certainly cut your overall cost, which has been an important element of IT for certainly the last five years.
Let’s take this another step sideways. What about IT outsourcing? We had this phenomena a few years ago. Some of the largest entities on Wall Street -- and I’m thinking of JPMorgan Chase -- went to large companies like IBM and said, "Here, you take it. We don’t want to do IT anymore."
Then maybe a few years, even some months, later decided, "Well, maybe that’s not the right approach." If we can get into a marketplace of services, if we created an SOA, doesn't that really obviate the need for IT outsourcing? Don't you want to retain the competency of business process and composite application authoring, regardless of where those sources come from? Isn’t SOA and the outsourcing of services a better model than wholesale IT outsourcing? I’ll throw that to you, Tony.
Baer: I was just thinking back to what Steve was saying, and it kind of applies to this answer as well. If there is one benefit that SOA delivers, it’s that the value becomes the service rather than the plumbing. If you think about the way we've traditionally developed functionality or integrated systems, we’ve had to spend inordinate amounts of time in the plumbing and maintaining it. SOA theoretically, if it’s done right, standardizes the plumbing, makes everything declarative, so you take out the guess work. The result is that if you look at outsourcing, SOA separates the plumbing from the service. Therefore, what is probably ideal for outsourcing would be the plumbing, because that’s where the value is and that’s not where IT organizations should be spinning their wheels.
Gardner: So where is the value?
Baer: The value is in the -- and this will sound a little cliché-ish -- intellectual property, which is represented by the actual service. That’s the service that delivers the actual business logic. It’s the way a company does business. The way a company does business is not the way it puts together its SOA plumbing. That should be standard. That’s obviously an ideal scenario, but look at how this could impact the outsourcing market. If you follow SOA principles and comply with SOA standards, it allows for that kind of separation, and therefore it makes for an ideal opportunity for outsourcing services to say, "Hey, we’ll take care of your plumbing, and you take care of your services."
Gardner: So, you’re saying perhaps that I should look to commoditize services and find the cheapest best way to acquire them, either on premises, co-located, outsourced entirely, software as a service (SaaS). Then, for those differentiating services that I’m applying intellectual property and perhaps even patents to, then I could even host those somewhere off premises, and it’s bringing that together within my organization that makes me a winner in my market?
Baer: That’s one way of putting it, but another way is, "I create the service, and by the way I rely on you to create and maintain the plumbing." There are many different ways of slicing and dicing this.
Garone: To some extent it depends on what kind of vendor you’re dealing with. I’ll go back to the example I gave earlier of Salesforce.com, which will do some of both. They create a services-based infrastructure that you can use to not only host your IP-unique applications in services, but also to host the ones that they’re providing to you, and if you do it right, you can easily integrate with.
Gardner: Right.
Kobielus: On the issue of outsourcing, the only thing that the company should never consider outsourcing in terms of core competencies is the core competency of making money and delivering value. Everything else is fair game, and I think what everybody is saying here is that they agree with that. SOA -- or much of what we think of as SOA -- revolves round the plumbing, the protocols, the application servers, the middleware fabric, etc., that all is fair game for outsourcing.
Gardner: I like the terms that were used a moment ago. It can be "sliced and diced in a number of different ways." That’s the good news, but it’s also the bad news, because no one really has a model they can fall back on and say, "Ah, here’s the proper way to do this, when we apply outsourcing and software as a service to the SOA mix." Or have I got this wrong? Jim, can you think of any past instances in the history of IT we can look at to get some idea of the best way to slice and dice these many variables?
Kobielus: Going back to my sense of knowledge of IT history, I’m going to have to sleep on that one, Dana.
Gardner: Can anyone else out there think of any precedents that we can look to in terms of what’s the right balance of on-premises, commoditized, software as a service, SOA? It gets kind of messy.
McKendrick: I don’t know if there really is a precedent. This is all open for innovation, Dana. There are a couple of interesting examples, actually, if you look at the telecom industry. I like to use the term "loosely coupled business." SOA is based on loosely coupled components or services. A lot of telecoms actually don’t use their own internal resources to provide services to their external customers. They act as a broker.
Gardner: It shows sometimes.
McKendrick: Unfortunately, yes. I’ve heard a statistic that at Cisco, the networking giant, and 80 percent of its products and services come through outside services that it essentially brokers. I think that’s kind of the model SOA is helping us move toward -- this idea of a loosely coupled business that can pull together services from perhaps throughout the globe, package them, and offer them to an end user or customer that they identify as important to their market.
Gardner: So, maybe the metaphor has moved from a 30-square-mile fungus to a 30-square-mile amoeba.
Koblieus: Amoeba is a good metaphor here, because business models must continue or will always continue to evolve; therefore IT must continue to evolve to the next business model, the next IT sourcing model, whatever helps them to survive and make money.
Gardner: We came up against this when we talked to Jeff Pendleton. We thought about architecture and we looked at some of the architects, and we came to Frank Gehry, and we said, "Wow, the outside looks different and almost illogical, but inside, the stuff holding it together is very logically designed."
Then we moved from that to this notion of a sound stage. You would build a sound stage, or even a movie set, depending on whatever scene you were shooting. And, we wanted to have IT act like a flexible sound stage or a movie set. We would just build the set based on whatever the activity was, do what we needed to do, and then perhaps tear it down or reuse parts of it. So, we seem to be coming back to that, although we’re talking about not just our own services as an enterprise on premises, but increasingly a mixture of software as a service and outsourcing.
Anybody want to pick at that or expand on it?
Gardner: Well, we're in complete agreement, that’s great.
Garone: I’m not quite sure I understood it, to be honest with you.
Gardner: Do you remember that call we had about the sound stage show; I think you were on that call?
McKendrick: It’s an interesting analogy.
Gardner: MGM right?
Baer: Right. That actually is about the old movie studio metaphor, one studio and many different productions going on at the same time. Basically, the piece parts were all interchangeable, and piece parts we can call plumbing.
Gardner: So, we’re making five different movies on the lot, and we’re going to bring cameras and lighting equipment and backdrops and makeup artists, and just apply them in the same way, but with a different end outcome. Right?
Baer: I guess you can say the metaphor coming out of this discussion has evolved from fungus to amoeba to let’s bring back the studio system into IT.
Kobielus: Right, if you look at what SOA governance involves, if you examine a given service throughout its life cycle, it begins in the planning stage, it’s a gleam in some business person’s eye, and then it goes to the system engineers, the designers the business people. There’s a governance life cycle on those projects as well. That governance role structure in the movie industry has changed since the very start and it continues to evolve.
The notion of the classic studio system for Hollywood was there between the '20s and the '50s, and then it gradually gave way to more independent producers. Now, you’ve got all these indies everywhere. What I’m getting at is that, the governance of any given movie production project, how it’s done, the best practice for that particular industry, continues to evolve from generation to generation.
Likewise in the IT world. If you look at the life cycle of governance of the creation of any application or functionality, of the whole software is on the life cycle, that paradigm continues to evolve from generation to generation, with new platforms, new tools, and so forth. So, its one of those things where now you’ve got to look at the structure of the SOA governance environment.
You've got roles that are specific to the plan, specific to the development stage, roles that are specific to the deployment and optimization stages and so on. What I’m getting at, then, is that I think the movie studio analogy is very interesting, because it focuses on the core of governance as a workflow over a life cycle involving various roles, and those roles continue to munge into each other and get broken off from each other.
Garone: Dana, just jumping in here, now that I had a chance to think a little bit about your analogy, and just to follow up on that last thought. In essence, the movie studios as we know them now, don’t have the role that we used to know them as having. They used to actually make movies. My understanding is that the customer who is the independent producer, or maybe a larger producer who may bring on another independent producer under them, basically is outsourcing the physical plant requirements to make a movie to the studio.
Gardner: So it becomes an ecology rather than all-in-one.
Garone: Exactly. It’s an ecosystem.
Gardner: Is it the right direction, Steve? David Lean made "Dr. Zhivago" for probably $20 or $30 million. Now, it costs $200 million to make a pirate movie. Can we follow movies as the logical business example? It seems like they’re making less of a movie for more money?
Garone: I know I’m going off on a tangent here in terms of analogies, but the movie industry has almost gone the same way as Major League Baseball. You’ve got stars that get $10, $20, $30 millions a picture. Does it really matter all that much that there’s an independent producer outsourcing the physical plant requirements to a major studio? I don’t know the industry that well, but my gut feeling is that both parties feel that that’s the most efficient way to do it.
Baer: Actually, can I jump in there, because I’ve heard -- beyond the specifics -- of an entertainment industry where values have gotten hopelessly inflated. But look at the mechanism behind this. Before, you had monolithic studios, which had their own life cycles, their own software development life cycles.
Gardner: Exclusive stars that they’d sign up for years at a time.
Garone: Right, it was the Major League Baseball model.
Baer: Exactly, and it’s sort of analogous to the monolithic software marketing, except for the fact that all the piece parts underneath were more interchangeable.
I guess you can argue that if you are an SAP, your piece parts inside the infrastructure were designed to be a self-contained interchangeable ecosystem. But look at the way it has evolved now. Part of it is that the technology for producing films and videos has become much more accessible, and therefore you have now an independent film industry. So, that's analogous to the infrastructure that has become much cheaper, much more open, which is what SOA can do.
In turn, you look at the business model. It's become more interchangeable, so that a studio might collaborate. Let’s say that Universal might work with Sony -- collaborate on a picture -- because it’s so costly that neither one of them could do it themselves. Well, they have the infrastructure and the business process in place that enables that to happen. Meanwhile, an independent film like "Crash" comes out of nowhere last year and wins the Academy Award for best picture of the year, and it’s all possible because you now have this more accessible infrastructure. The moral of the story for SOA here, trying to bring this back on topic -- is that if you get the plumbing down right -- you don’t have to be an SAP to introduce a killer app anymore.
Kobielus: I’m going to take this movie industry metaphor out of the realm of metaphor into the actuality of, teams becoming very SOA-focused in terms of the actual production ... A mashup is reusing existing components of service -- content, -- whatever into new vehicles or new compositions. If you look at the content that’s being developed out there in the IT world, more of it's getting built through various types of mashups, which is very much an instantiation of the SOA paradigm into a different world -- not the software world so much as the normal cultural world that we all inhabit.
Gardner: It's interesting, because when we talk about movie production, it’s really content creation. It’s artistic. It’s creative, complex, and changeable. It has to fit a certain audience at a certain point in time. A movie that was made 30 years ago might not work today and vice versa. What’s also interesting here, looking through this lens of the movie industry, is that depending on the culture of the organization, it can adjust.
So, if you’re a big monolithic enterprise, you're like a command and control structure, you want to do it all your way. You’ve always done this. You’re sort of a traditional brick-and-mortar company. Then, you can perhaps approach IT like an MGM studio, and you can have your own infrastructure, all of your own developers. You can do things without much off-the-shelf or highly customized -- but you come out with a really great product if you’re willing to put that emphasis in. And, if you have such a great product that you dominate your industry, then the numbers will make sense. The investment you made in IT will pay off.
On the other hand, if you want to be a small company, a green-field software-as-a-service organization, you want to only be in an ecology play where you’re acquiring things, and you’re going to plug them in and then rip them out, reuse as much as you can. Then, you can be like a Miramax instead of an MGM. You’re going to do just independent films. You’re going to go in and spend a little bit of money, and maybe you’ll still come up with some great product that will be right fit for your market. The nice thing is, as we said earlier, you can slice and dice it either way.
Does this make sense -- that the culture of the company needs to be considered when we look at the way in which it approaches IT?
Kobielus: For sure. Some companies are not very partner friendly. Therefore, they’re not really the kinds of corporate cultures that lend themselves to outsourcing. First and foremost is managing the relationships. There are other companies that are very partner friendly, it's really quite often a culture issue.
Gardner: Right. I think we’re going to need to cut off our discussion there, but obviously we can take this further and I’m sure we will, but perhaps one of the takeaways that we’ve led to is that companies need to be introspective. They need to look at what kind of organization they are, what kind of organization they want to be, and get a sense of which IT approaches, particularly vis-à-vis SOA and software-as-a-service and outsourcing, make sense for them.
McKendrick: The question is: does management -- does the C level -- understand what SOA is about.
Gardner: I’m not sure we’ve even seen a lot of instances where companies would say, "Let’s take a look at our culture and define it, and then decide our IT approach as a result of that."
Garone: That’s true. They’re going to have to do that. Many of them have not, because they started out looking at SOA from an individual or a pilot project perspective. When it expands beyond there, that will change. It will have to.
Baer: Well, they look at SOA as a technology as opposed to a way of delivering their business services.
Garone: When the technology has been mastered, and has demonstrated its value, it will have to become more than that, and then you’ll see the cultural issues playing a bigger role. A lot of the IT managers and people that I've talked to who are involved in the implementations of SOA solutions, even on a pilot level, seem to understand the cultural issues. They’ve encountered them, and they’ve tried to deal with them. But you’re right. On the C level that hasn’t yet happened.
McKendrick: That goes back to the whole ROI argument we were having a bit earlier as well. The ROI is being seen in developer productivity, for example. If you want to generate hard numbers from an SOA project, developer productivity is the first place you look -- the ability to reuse applications or services without having to have the developers spend X-amount of hours reinventing the wheel.
Gardner: We need to have the equivalent of a Myers-Briggs test, where you actually can ask a series of questions, or do some research, and can define the temperament and the personality traits of a company. From there, you can back up and say, "Well, here’s what your IT should be."
You need to get into the DNA of how a company thinks and behaves organically in order to make the right fit with IT, because the companies that fail are companies where there IT culture and their business culture are at odds or don’t mesh. And they collapse. They can’t function.
Baer: No question about that, and the fact is that, Dana, what you were just saying could apply to almost any type of transformation, but I think this sounds very specific to SOA here. We’ve been providing a whole bunch of metaphors, dealing with other industries like entertainment business or the ability to partner, but if you think about the implications of SOA, it not only should erode barriers between you and your partners. It makes it technologically possible to erode the silos within your organization by going inside and looking at how many instances of the customer object we have? Do we really need 45 instances, when maybe five would do for a business of our breadth and diversity?
Gardner: Okay. Let’s leave it there and we’ll revisit this, but I think metaphors are essential in this period of SOA evolution, because there are so many variables, and because you can slice it and dice it in so many ways. If we can help companies look at ways in which they can reduce the number of variable, and help them get started and find some traction and therefore benefits. That’s an important thing to do.
Joining us in this discussion about the impact of outsourcing, software-as-a-service and how to measure success has been, Steve Garone, the former IDC group VP, founder of the AlignIT group and an independent analyst. Thanks for joining, Steve.
Garone: It’s been a pleasure, Dana.
Gardner: Joe McKendrick, research consultant, columnist at Database Trends, and blogger at ZDNet and ebizQ. Thanks, Joe.
McKendrick: Thanks, Dana, for the opportunity to join you guys.
Gardner: Tony Baer, principal at OnStrategies. Thanks, Tony.
Baer: Thanks, Dana, good way to kick off the New Year.
Gardner: Jim Kobielus, principal analyst at Current Analysis. Thanks for coming, and join us again sometimes, Jim.
Kobielus: Thanks, I’ll look forward to it.
Gardner: This is Dana Gardner, your producer, host and moderator here at BriefingsDirect SOA Insights Edition. Come and join us again next week. Thanks.
If any of our listeners are interested in learning more about BriefingsDirect B2B informational podcasts or to become a sponsor of this or other B2B podcasts, please fill free to contact me, Dana Gardner at 603-528-2435.
Listen to the podcast here.
Transcript of Dana Gardner’s BriefingsDirect SOA Insights Edition, Vol. 7. Copyright Interarbor Solutions, LLC, 2005-2007. All rights reserved.
Sunday, January 21, 2007
Sunday, January 14, 2007
Transcript of BriefingsDirect Podcast on the Future of Enterprise Search
Edited transcript of BriefingsDirect[TM] podcast on search advancements with host Dana Gardner, recorded Jan. 3, 2007.
Listen to the podcast here. Podcast sponsor: FAST Search & Transfer.
Dana Gardner: Hi, this is Dana Gardner, principal analyst at Interarbor Solutions, and you’re listening to BriefingsDirect. Today a sponsored BriefingsDirect podcast discussion on search, the role of search, how search relates to today’s mega trends of Web 2.0, Enterprise 2.0, Services Oriented Architecture (SOA) and even mobile lifestyles.
Joining us in this discussion are executives from FAST Search & Transfer, a global company based in Oslo, Norway. Joining us is John Markus Lervik, the CEO. Also, Bjorn Olstad, CTO, and Zia Zaman, the Senior Vice President of Strategic Marketing. Welcome to the show everyone.
John Markus Lervik: Good morning, Dana.
Gardner: While search has been around for many years, it seems like we’ve had somewhat of a philosophical shift in the last few years, in the sense that people are no longer thinking of information based on its origins, but really they’ve supplanted these data structures with how they can use it and how they can access it. That cuts across desktop, intranet, and Web; and it seems that access to information quickly and in the way that any individual wants it has really changed the notion of fleet and agile business.
In fact, business process is increasingly being driven by how people access content and then inject that into business goals. So, I want to first talk to you about business goals because many companies are very practical, of course. They’re looking at their top line and they’re looking at their bottom line. I want to throw this out perhaps to John Markus, how are companies using information discovery to further their business goals in a way now that’s perhaps different from ever before?
Lervik: Dana, I think that one way of looking at technology is always in the light of the business processes that they’re enabling. Many of our customers are thinking about ways in which they can achieve competitive advantage through productivity change, and they’re looking for something new. What we enable is for search to help facilitate communications between individuals.
We’ve all heard of Web 2.0 applied to the "Websphere," but in the enterprise, Enterprise 2.0 trends of social networking are also taking hold, allowing people to collaborate better and allowing individuals to streamline business processes. What's interesting for us in the future is search as an entry point into facilitating improved business processes, and business process improvement leads to improved business goals, as well as business-model innovations.
Gardner: Is there an impact here for applications themselves to start to be created based on what search can bring to the table rather than having search perhaps be an afterthought? What do you see as the future of search-driven applications?
Bjorn Olstad: This is Bjorn, and I'll have a go at that. The fundamental difference between search and such things as database, content management, and document management systems is that search starts with the user and then does reverse engineering -- what is necessary to realize this user's experience. It's starting with the content and then trying to deduce what should we do with this content. When you do that, you end up with a user-driven experience.
As you’re starting off, Dana, the user doesn’t care about the origin of the content. So, the ability to go into content sources and then chop that into information nuggets that can then be re-orchestrated and put into the context of what is useful for that specific individual user in what he’s trying to accomplish right now is a very powerful paradigm. It’s also a paradigm that can be used as the basis for building applications.
We see search more and more becoming the framework for the metaphor in how the user interface is being defined and also for how the application is actually defined by the source of the framework and then embedding functionalities in a source-stream framework.
Gardner: Is this really a shift of intent or are there some technologies that are newly arriving that empower this shift? Bjorn, can you tell us a little bit more about what’s happening technically that is allowing an expansion of the role of search?
Olstad: Let's go back a little bit. As you said, search used to be an afterthought. Typically, you had an application that was kind of hard-wired, done in HTML, or hard-coded down to the logic that usually resided on some type of database. So there was hard-wiring between the schemas and the data model and the database, and then the user interface.
Going forward, I think increasingly the user experience will be driven by algorithms and will be dynamic, so that you can actually optimize the user experience and the business efficiency. This is valid for both consumer-facing portals that you’ve seen on the Web but also on other applications internally. So then you get that framework and you say. "Let’s not hire system integrators to hard wire my content. Let the user decide, and when we deduce what the user likes, the algorithms can populate the user experience."
This will include both content and relevant services, and maybe a connection to people, so that you can find people to collaborate with. That becomes a framework, and search has matured from being merely access to documents to being this umbrella environment where you can provide access both to structured data and unstructured data and transform them into structure by putting additional metadata onto it. You can do the same with rich media, and search suddenly becomes that framework that can give you access to everything, as opposed to just the text that didn’t fit into the hard wire or the schema model.
Gardner: I suppose that with the increasing use of SOA, where applications are decomposed into services with the productivity coming from reuse as much as possible, and with governance providing an overlay for managing and organizing services, that search could have quite a powerful impact on how services are put together on the fly to create unique business processes. Do you see search moving in that direction as well?
Olstad: Absolutely. If you think about Web 1.0, Web 2.0 and Web 3.0, one way to interpret this is that Web 1.0 included these monolithic Web services. Then, in Web 2.0, it’s about the different services that make up atomic individual services and then having technologies to mash up and combine them to create new services on top of that, the same way as SOA. It seems nice at first because they are driven toward reuse, et cetera, but you move the complexity to the next level. How do you actually orchestrate? How do you put together these atomic content components and atomic service components? How do you actually do that and how to do that in a user-centric way, so you can gain the right user experience?
With search, it’s actually designed to do exactly that, to bring both components and services to atomic units and then bring the orchestration level that is able to recompile these content and service elements into what makes sense for the end user. Hence, it brings 3.0 integration levels on top of the atomic components that we have in Web 2.0.
Zia Zaman: This isn’t just our idea, if I can build upon that. I was talking to an information provider that provides information to financial services firms. What he asked for was the ability to atomize information, allowing the users the flexibility to consume it in their own way. So, it’s exactly what Bjorn was talking about. In Enterprise 2.0, we allow for the atomization. And then the re-organization around the business processes is what happens in the next phase.
Gardner: Zia, there is already quite a bit of difference of opinion on how to define Web 2.0, never mind Web 3.0, but let’s try to just get a handle on Web 3.0 for our audience. To me this is really getting towards the Semantic Web, where there is context on an automated basis, with much more rich metadata involved, and therefore the ability to apply algorithms more generally across the Internet. Is that your understanding at FAST about this Web 3.0 or how would you further that definition?
Zaman: Dana, I think you've got it right. Metadata is the key, but I'm going to pass this one over to Bjorn to talk a little bit more about Semantic Web.
Olstad: There are so many opinions about the Semantic Web. First of all, what I don’t like with the approach of Semantic Web is that you have to put in structure, and it’s not really useful until almost everybody has started to put structure into it.
So, how do you leap from that and actually make it useful before everybody has adopted this new scheme? Search can actually play a role, because search can auto-generate metadata and find ways to use that structure and to improve the discovery. Then, the allocation elements that the traditional Semantic Web talks about can be aimed at how to improve algorithms, as opposed to starting from scratch. In doing that, I think search has the opportunity to deliver on the premise of the Semantic Web, by applying algorithms as opposed to altering tools.
If you then move to Web 3.0, then using algorithms increasingly becomes a component of it. I think an immediate step, which is an opportunity that FAST is looking at right now, is how can you use search at this level to go from atomic services to the rich environment, where you effectively can develop and deploy services. That’s kind of the orchestration: how you put services and content components into meaningful context for the end users. That is what brings business value. It brings time to market value for companies that are trying either to develop Web 2.0 and Web 3.0 applications or move their current models to a more collaborative approach.
Gardner: I suppose one of the frustrations we’ve heard quite often from users is that they can find things easier on the World Wide Web, than on their own hard drive or their own enterprise network. We’ve come a long way to assuage those concerns, but it seems to me like perhaps we’re at a point where we can go and leapfrog the Web.
If we’re thinking about a Semantic Web, that’s something that requires a lot of standardization. It could take a number of years and a high level of complexity, but moving to the "Semantic Enterprise" seems something that’s quite a bit more attainable in the short term. Do you think, and I’ll throw this out to anyone, are we going to move to a point where we can apply these benefits to the Enterprise and then the Web will catch up to them or is the Web always going to be a bit easier to manage?
Olstad: Maybe I could start, and, Zia, you could give some business examples. This is an excellent question, Dana. First of all, just so that we’re on the same page about what the Web experience is all about, there are two types of queries. There is the type of query where you know what you’re looking for. So you search for some properties and what you’re actually looking for.
The other type of query is when you don’t know what's there, but you throw out a wide, open-ended query, and you want the data to talk to you. In Web search, there is no completeness requirement. Web search is defined when you type in one or two query terms, which is a nice starting point for your browsing. So you get a link to a Website where you can start. There is no completeness. There is no overview of the other elements that didn’t end on the first result page.
Web search has moved outside Web search and can be used inside the Enterprise for this semantic analysis. It can also identify facts inside documents, combine it with the information structure databases, and provide discovery. You can start to do analytics, so that you can relate concepts and enable people to make decisions and get completeness and overview. That is why you see a merger between business intelligence and search starting to appear.
Gardner: So, perhaps moving toward a business intelligence (BI) benefit vis-à-vis search, and of course the other tools that are available for BI, is it first a necessary step to the Semantic Enterprise?
Olstad: Yes, I think so. To give just one example of a company that is trying to use search in an innovative way, what about all communication internal in the company? What about instant messaging, email, Voice over IP (VOIP), Skype, et cetera, and how to improve the quality of that communication and also derive analytics from it.
For example, if I call you, Dana, I should call you as a conceptual entity and you should actually be the one who decides if that ends up as a Skype call or cell phone call. This company is doing that and using search to orchestrate all the communication, allowing you, as the recipient of the call, to configure how you want to be accessed. By having that system, you can also create analytics that understand what is really happening in the company: who are the experts, what’s happening, what are the trends, etc.
Zaman: I actually think that it could be a lot simpler than that. The expert location is something that Bjorn just touched upon, which may be one of the next killer apps in the enterprise. What we’re finding more and more when we talk to our customers is that information discovery isn’t just about connecting users to results. It’s about connecting users to answers, to analytics, and to services -- but most importantly, to people. Connecting people to other people, so that they can do their job better. To improve decision-making throughout the enterprise is a big part of what information discovery tools are about to enable.
One way of looking at it is, if I can find the expert in the organization or outside the organization who can help me best in the decision process that I'm currently in -- and I can find that expert in the shortest period of time, using insights about who this person might be by using metadata about this individual, by actually discovering this path that I didn’t know existed upfront -- then I have actually added value to the business. So, we believe that search as a matching service is really one of the killer applications.
Gardner: So, what we’re talking about here is bringing the tacit knowledge that’s available to any organization into play with the semantic knowledge that’s gathered across all modalities, all types of media and information, but allowing those to come together in real-time based on someone’s specific needs through a very simple query.
Zaman: That’s right. I often like to talk about Greek philosophers, and Socrates is one of my favorites. The reason is because of what he did with a seeker. He didn’t answer a question with an answer. Rather he asked another question, and that allowed the seeker to refine his or her question, until finally they got to what they were looking for. In many ways, technology -- whether it’s information discovery, search, and business intelligence, whatever it might be -- shouldn’t insult the intelligence of the seeker. Rather it should allow individuals to find the answer that they’re looking for, either through tacit information that’s stored in the enterprise, or semantic information, or structured information, whatever it might be. What we’re trying to do is mimic the type of dialogue that Socrates had.
Gardner: Sure, because people are quite good at peeling away the layers of an onion, as a metaphor, to find what it is they’re looking for. There is no beginning or end to this peeling, but we certainly want to get him or her close to where they can extract real value.
Zaman: You got it.
Gardner: All right. Now people have been talking about this for years. I deal with developers a lot. I think they understand a lot of these theories. They understand the value and the goals, but putting it into practice, putting it into execution seems to be where there is still a need for some work. When we go to developers, we try to encourage them to start exploiting these capabilities of changing the mindset of how an application, specifically the UI, is actually constructed to make the entry point into an application more amenable for search and for gathering of these layers within an organization’s assets and resources.
Are there tools? Are there APIs? Are there SDKs? Is there a mindset? How do we bring the developer into play here? I’ll throw this out to anyone.
Olstad: I think Gartner predicted that in 2012, 75 percent of the world's applications will use search as the dominant metaphor for how an application is being operated by the end-user. That is a very powerful statement. And it means that all those developers that you’re asking about, Dana, they have to adapt to that.
Simplicity in an application [means] that you have a powerful metaphor that you can use to deduce what you should expect the application to be doing. Search is exactly that. So, obviously, search is evolving to not only provide access to text, but to provide value to all types of elements.
Another change in search technology is that it handles content natively, and previously it did not. In Web search you type some queries and you get a URL link to the content. Now you can take XML or SQL databases, and you can get native content. You can search in native content with queries that emulate what you can do in the database or in an XML database. This means that you can build the same type of application with the freedom to access data in an ordinalistic way, and have a mechanism to prioritize and do exactly what Zia talked about with Socrates.
Technically, if you use an open-ended query like, "List the innovative people in my company," or something, you could get back kind of a menu of people that have been referred to in the documents or discussions where there is talk of innovation, and then get the facts related to these people. So it’s not coming back with an answer, but it is coming back with an analysis -- and giving you the opportunity to refine the query.
Gardner: Another thing with developers is that they’re interested in seeing examples. Are there folks out there doing this now who are really on the cutting-edge, who have been able to instantiate rather than theorize? Maybe, Zia, you could address this. Are there folks that are out there using FAST Search & Transfer technologies or other technologies who are actually realizing some of these benefits? And, who are they, and how are they doing it?
Zaman: All over the landscape of the economy, people are using search in interesting ways. I think that the message to the developer is: "Think about the app that you’re working on right now, and think about how the users are accessing either the application or the information."
Take a consumer perspective and you’ll definitely think that search plays an important part in whatever it is that you’re doing. So, get involved and talk to people about search. I think you will find that, even in your own sphere, there are people who are already working on search technologies. In one particular investment bank, 15 or 16 different applications of FAST Search have been put into place across the business to help with everything from getting access to research reports to improving customer service.
There are search policies at law firms or at pharmaceutical companies or retail or e-commerce searches for Christmas shopping. Or just B2B searches, so that one individual can find another professional in the shortest amount of time in the right region. All these applications are about the user consuming information in a way that is most effective for them -- and search technologies are enabling each of these examples. Maybe John or Bjorn, you want to talk about a European example and I can talk about an American example?
Olstad: Sure. Let me start off with our broad business goals, and there are some that have nice benefits for the businesses. We have many customers that don’t have a choice because if you look at media, entertainment, and telcos, there are so many disruptive changes going on. You have print transforming to electronic. You have advertising on the Internet, where search-based advertising is now bypassing traditional banner advertising. You have telcos, where telephone calls may soon be free because there are alternative ways to monetize it. And you already have Skype, Voice over IP, etc.
So, if what a telco provider, for example, is providing today is going to be free, then the name of the game is going to be who is able to create applications and to have service delivery platforms that use that customer base to derive new services. They’re not very well equipped for that. The key enabling-technology that actually helps with this has got to be the user-driven applications. And they see search from a monetization perspective and from the perspective of building the end-user experience. So for lot of customers that we work with from media, entrainment, and telcos -- it’s not that it's "nice to have," but it's "How do we survive?"
Zaman: An example of that in the back office is AutoTrader.com, which is a phenomenal example of how search was used to fundamentally rethink the way in which records were served up. The database administrator at AutoTrader says that they tripled their performance at half the cost, using a highly scalable search platform, which was very easy to deploy and a lot more dynamic than their traditional database architecture. So, even in the back office for applications as diverse as AutoTrader, we are seeing search being used to really disrupt the way in which you think about information delivery.
Gardner: It also seems that the younger generation, as they come into play here, are probably already automatically thinking "search." The people who have grown up with the Web around them start almost any activity with search. So, this is becoming imbued into the society and into the behavior and culture?
At the same time, these folks are also quite used to mobile devices and small handheld devices. Can you explore for me a little bit of how search plays into young people in mobile devices? Perhaps we’re seeing some activities in early-adopter environments, South Korea for example, where search has to be the only way in a small-form factor that vast amounts of information could be managed. Help me understand the impact of use and mobile search?
Olstad: That’s a very interesting question, Dana. As you said, most exploration starts with a search on the Web. You start with a search on a Web search engine, and then you come to the site of this company. Then that company has some options. Should we let the user continue with the search experience or should we force him to go back and understand the text and how we have orchestrated all our content? Then, he is not allowed to do anything meaningful before he has completed that education. That’s slowing down the progress of the users.
Moving to mobile, if we go back to 2001-2002, we were trying to educate mobile operators that they needed to use search. That was a difficult discussion, a difficult sell. In Europe and Asia, at least, it was very popular to download ringtones and wallpapers and things like that. The operators felt that it was sufficient to have a chart with the most popular items and have them available on the static page, allowing people to choose from that.
When they moved to search, what they experienced was that a good way to measure the UI or the usability for the user is how many operations it took to actually perform the action. One of our customers is Vodafone. They measured how many operations young people had to do, from coming into the portal until they could complete a transaction, which in this case was to download a ringtone or music, etc. That used to be 3.5-operations, or something like that, when it was based on the charts and navigation mechanisms. When they substituted search, it went down to 1.5, which means that almost always you go directly to something that you can download. And, once in a while, you have to do one iteration before you get it. That drove revenue and traffic tremendously.
Zaman: You’re right. Generation Y and the younger generation are very used to unfettered access to their data, their content. What does that mean in the Enterprise? This is something that I hear increasingly on Wall Street and in the City of London. I know how to deal with the managing director, who is in his 40s. It’s very easy. For their relationship with their data, they ask someone junior.
But for that younger investment banker who’s just grown up with unfettered access to their personal data, they keep asking why they can’t get access to enterprise data with the same facility. The information technology people who are serving this younger generation have to change. They don’t quite understand that search and other technologies, including Web 2.0 technologies, are forcing them to rethink the way in which they have access to their information, because that’s the way they can be creative and allow for them to get the data they need in order to make the right decisions. I think it’s having a cultural effect, even in more staid industries.
Olstad: Then, if you go to Japan or to Asia, you have also this competition between the telco providers, where the name of the game now is how do you add video services, and with extremely high bandwidth on the phones, how to do that? You see multimedia screening, where you are serving video quality down to the cell phones. So there is a tremendous interest in how to capitalize on new user experience to combine rich media assets, and then use this to build the community stuff you have seen in YouTube in the U.S. This is happening on the mobile side at a tremendous speed in Asia.
Gardner: Very interesting. So we’re really still in the early stages of search. It sounds like there is going to be a lot to discuss at your forthcoming FASTforward ’07 event. Perhaps as a little tease for our audience, is there anything you can tell us about this event in terms of some additional technologies, products or approaches? I will throw this out to anyone.
Olstad: If I could start from the technology side. There will be a lot of cool introductions and some new technology. We will look at how to interface search with repositories, from structured data to rich media and to traditional text. We think there is a game-changing opportunity in doing that and also to enable the Semantic Web in a more powerful way, as we talked about.
We also really believe that, as we have touched on here, that search is not just a feature, but it will become the framework for building search-driven, user-driven applications, to actually make that a reality and make it possible to build those applications. We are introducing new components to do that.
We see that search is driving the connective role in Web 2.0 in the evolution towards Web 3.0 by being a service-delivery platform. So we are introducing a Web 2.0 platform that uses search as the main orchestration -- both for monetization purposes and for understanding both content and the user. We'll be tracking the user and understanding his preferences, building personalized experiences, and creating this platform where you can take the atomic components of content and services and effectively roll that out as user-centric services. We’re building a service delivery platform around search for doing that and you will see applications that already have been built on top of that platform.
Zaman: I think it’s going to be a very hands-on conference in San Diego. There will be customers. There will be developers. There will be case studies, demos, and a lot of learning going on, interacting and sharing of ideas. Sure, there will be the predictions and the analysts and the advice that goes along with it. But really I think that the value of this search conference will be in the exchange of innovative ideas that takes place, not just from FAST to the rest of the community -- but also from the community to other members within the community. I think it's going to be great.
Gardner: Thanks. John Markus, just to return back to the business value, we’ve been discussing this at a fairly technical level -- developers and business intelligence folks -- but for CEOs, for the leadership at many enterprises, what are some of the salient messages that you think they need to understand in order to better appreciate search and why it makes a good deal of sense to invest in it?
Lervik: In general for most information-intensive companies -- media companies, financial services companies, telecom companies and pharmaceutical companies -- search is becoming both a key part of the infrastructure to enable these companies to make decisions, but also -- more importantly -- to drive and increase business online.
Because we see more and more of these companies relying on effectively connecting customers or end-users to their products, to their contents, and to their services. That’s where search comes in as a mission-critical application, or at least to enable these mission critical applications. At FASTforward, there will be a great number of speakers who will talk about how these have effectively implemented a solution that drive and increase business.
Gardner: I’m afraid we’re about out of time. We’ve been discussing search, the role of search, Enterprise 2.0, Web 3.0. Clearly the role and impact of search is increasing rapidly and will increase further over the coming years.
I’d like to thank our guests and sponsor, FAST Search & Transfer. We’ve been talking with John Markus Lervik, the CEO. Also, Bjorn Olstad, the CTO, and Zia Zaman, the Senior Vice President of Strategic Marketing. This is Dana Gardner, principal analyst at Interarbor Solutions. Thanks for joining us for this BriefingsDirect podcast.
Listen to the podcast here. Podcast sponsor: FAST Search & Transfer.
Transcript of Dana Gardner’s BriefingsDirect podcast on the future of enterprise search. Copyright Interarbor Solutions, LLC, 2005-2007. All rights reserved.
Listen to the podcast here. Podcast sponsor: FAST Search & Transfer.
Dana Gardner: Hi, this is Dana Gardner, principal analyst at Interarbor Solutions, and you’re listening to BriefingsDirect. Today a sponsored BriefingsDirect podcast discussion on search, the role of search, how search relates to today’s mega trends of Web 2.0, Enterprise 2.0, Services Oriented Architecture (SOA) and even mobile lifestyles.
Joining us in this discussion are executives from FAST Search & Transfer, a global company based in Oslo, Norway. Joining us is John Markus Lervik, the CEO. Also, Bjorn Olstad, CTO, and Zia Zaman, the Senior Vice President of Strategic Marketing. Welcome to the show everyone.
John Markus Lervik: Good morning, Dana.
Gardner: While search has been around for many years, it seems like we’ve had somewhat of a philosophical shift in the last few years, in the sense that people are no longer thinking of information based on its origins, but really they’ve supplanted these data structures with how they can use it and how they can access it. That cuts across desktop, intranet, and Web; and it seems that access to information quickly and in the way that any individual wants it has really changed the notion of fleet and agile business.
In fact, business process is increasingly being driven by how people access content and then inject that into business goals. So, I want to first talk to you about business goals because many companies are very practical, of course. They’re looking at their top line and they’re looking at their bottom line. I want to throw this out perhaps to John Markus, how are companies using information discovery to further their business goals in a way now that’s perhaps different from ever before?
Lervik: Dana, I think that one way of looking at technology is always in the light of the business processes that they’re enabling. Many of our customers are thinking about ways in which they can achieve competitive advantage through productivity change, and they’re looking for something new. What we enable is for search to help facilitate communications between individuals.
We’ve all heard of Web 2.0 applied to the "Websphere," but in the enterprise, Enterprise 2.0 trends of social networking are also taking hold, allowing people to collaborate better and allowing individuals to streamline business processes. What's interesting for us in the future is search as an entry point into facilitating improved business processes, and business process improvement leads to improved business goals, as well as business-model innovations.
Gardner: Is there an impact here for applications themselves to start to be created based on what search can bring to the table rather than having search perhaps be an afterthought? What do you see as the future of search-driven applications?
Bjorn Olstad: This is Bjorn, and I'll have a go at that. The fundamental difference between search and such things as database, content management, and document management systems is that search starts with the user and then does reverse engineering -- what is necessary to realize this user's experience. It's starting with the content and then trying to deduce what should we do with this content. When you do that, you end up with a user-driven experience.
As you’re starting off, Dana, the user doesn’t care about the origin of the content. So, the ability to go into content sources and then chop that into information nuggets that can then be re-orchestrated and put into the context of what is useful for that specific individual user in what he’s trying to accomplish right now is a very powerful paradigm. It’s also a paradigm that can be used as the basis for building applications.
We see search more and more becoming the framework for the metaphor in how the user interface is being defined and also for how the application is actually defined by the source of the framework and then embedding functionalities in a source-stream framework.
Gardner: Is this really a shift of intent or are there some technologies that are newly arriving that empower this shift? Bjorn, can you tell us a little bit more about what’s happening technically that is allowing an expansion of the role of search?
Olstad: Let's go back a little bit. As you said, search used to be an afterthought. Typically, you had an application that was kind of hard-wired, done in HTML, or hard-coded down to the logic that usually resided on some type of database. So there was hard-wiring between the schemas and the data model and the database, and then the user interface.
Going forward, I think increasingly the user experience will be driven by algorithms and will be dynamic, so that you can actually optimize the user experience and the business efficiency. This is valid for both consumer-facing portals that you’ve seen on the Web but also on other applications internally. So then you get that framework and you say. "Let’s not hire system integrators to hard wire my content. Let the user decide, and when we deduce what the user likes, the algorithms can populate the user experience."
This will include both content and relevant services, and maybe a connection to people, so that you can find people to collaborate with. That becomes a framework, and search has matured from being merely access to documents to being this umbrella environment where you can provide access both to structured data and unstructured data and transform them into structure by putting additional metadata onto it. You can do the same with rich media, and search suddenly becomes that framework that can give you access to everything, as opposed to just the text that didn’t fit into the hard wire or the schema model.
Gardner: I suppose that with the increasing use of SOA, where applications are decomposed into services with the productivity coming from reuse as much as possible, and with governance providing an overlay for managing and organizing services, that search could have quite a powerful impact on how services are put together on the fly to create unique business processes. Do you see search moving in that direction as well?
Olstad: Absolutely. If you think about Web 1.0, Web 2.0 and Web 3.0, one way to interpret this is that Web 1.0 included these monolithic Web services. Then, in Web 2.0, it’s about the different services that make up atomic individual services and then having technologies to mash up and combine them to create new services on top of that, the same way as SOA. It seems nice at first because they are driven toward reuse, et cetera, but you move the complexity to the next level. How do you actually orchestrate? How do you put together these atomic content components and atomic service components? How do you actually do that and how to do that in a user-centric way, so you can gain the right user experience?
With search, it’s actually designed to do exactly that, to bring both components and services to atomic units and then bring the orchestration level that is able to recompile these content and service elements into what makes sense for the end user. Hence, it brings 3.0 integration levels on top of the atomic components that we have in Web 2.0.
Zia Zaman: This isn’t just our idea, if I can build upon that. I was talking to an information provider that provides information to financial services firms. What he asked for was the ability to atomize information, allowing the users the flexibility to consume it in their own way. So, it’s exactly what Bjorn was talking about. In Enterprise 2.0, we allow for the atomization. And then the re-organization around the business processes is what happens in the next phase.
Gardner: Zia, there is already quite a bit of difference of opinion on how to define Web 2.0, never mind Web 3.0, but let’s try to just get a handle on Web 3.0 for our audience. To me this is really getting towards the Semantic Web, where there is context on an automated basis, with much more rich metadata involved, and therefore the ability to apply algorithms more generally across the Internet. Is that your understanding at FAST about this Web 3.0 or how would you further that definition?
Zaman: Dana, I think you've got it right. Metadata is the key, but I'm going to pass this one over to Bjorn to talk a little bit more about Semantic Web.
Olstad: There are so many opinions about the Semantic Web. First of all, what I don’t like with the approach of Semantic Web is that you have to put in structure, and it’s not really useful until almost everybody has started to put structure into it.
So, how do you leap from that and actually make it useful before everybody has adopted this new scheme? Search can actually play a role, because search can auto-generate metadata and find ways to use that structure and to improve the discovery. Then, the allocation elements that the traditional Semantic Web talks about can be aimed at how to improve algorithms, as opposed to starting from scratch. In doing that, I think search has the opportunity to deliver on the premise of the Semantic Web, by applying algorithms as opposed to altering tools.
If you then move to Web 3.0, then using algorithms increasingly becomes a component of it. I think an immediate step, which is an opportunity that FAST is looking at right now, is how can you use search at this level to go from atomic services to the rich environment, where you effectively can develop and deploy services. That’s kind of the orchestration: how you put services and content components into meaningful context for the end users. That is what brings business value. It brings time to market value for companies that are trying either to develop Web 2.0 and Web 3.0 applications or move their current models to a more collaborative approach.
Gardner: I suppose one of the frustrations we’ve heard quite often from users is that they can find things easier on the World Wide Web, than on their own hard drive or their own enterprise network. We’ve come a long way to assuage those concerns, but it seems to me like perhaps we’re at a point where we can go and leapfrog the Web.
If we’re thinking about a Semantic Web, that’s something that requires a lot of standardization. It could take a number of years and a high level of complexity, but moving to the "Semantic Enterprise" seems something that’s quite a bit more attainable in the short term. Do you think, and I’ll throw this out to anyone, are we going to move to a point where we can apply these benefits to the Enterprise and then the Web will catch up to them or is the Web always going to be a bit easier to manage?
Olstad: Maybe I could start, and, Zia, you could give some business examples. This is an excellent question, Dana. First of all, just so that we’re on the same page about what the Web experience is all about, there are two types of queries. There is the type of query where you know what you’re looking for. So you search for some properties and what you’re actually looking for.
The other type of query is when you don’t know what's there, but you throw out a wide, open-ended query, and you want the data to talk to you. In Web search, there is no completeness requirement. Web search is defined when you type in one or two query terms, which is a nice starting point for your browsing. So you get a link to a Website where you can start. There is no completeness. There is no overview of the other elements that didn’t end on the first result page.
Web search has moved outside Web search and can be used inside the Enterprise for this semantic analysis. It can also identify facts inside documents, combine it with the information structure databases, and provide discovery. You can start to do analytics, so that you can relate concepts and enable people to make decisions and get completeness and overview. That is why you see a merger between business intelligence and search starting to appear.
Gardner: So, perhaps moving toward a business intelligence (BI) benefit vis-à-vis search, and of course the other tools that are available for BI, is it first a necessary step to the Semantic Enterprise?
Olstad: Yes, I think so. To give just one example of a company that is trying to use search in an innovative way, what about all communication internal in the company? What about instant messaging, email, Voice over IP (VOIP), Skype, et cetera, and how to improve the quality of that communication and also derive analytics from it.
For example, if I call you, Dana, I should call you as a conceptual entity and you should actually be the one who decides if that ends up as a Skype call or cell phone call. This company is doing that and using search to orchestrate all the communication, allowing you, as the recipient of the call, to configure how you want to be accessed. By having that system, you can also create analytics that understand what is really happening in the company: who are the experts, what’s happening, what are the trends, etc.
Zaman: I actually think that it could be a lot simpler than that. The expert location is something that Bjorn just touched upon, which may be one of the next killer apps in the enterprise. What we’re finding more and more when we talk to our customers is that information discovery isn’t just about connecting users to results. It’s about connecting users to answers, to analytics, and to services -- but most importantly, to people. Connecting people to other people, so that they can do their job better. To improve decision-making throughout the enterprise is a big part of what information discovery tools are about to enable.
One way of looking at it is, if I can find the expert in the organization or outside the organization who can help me best in the decision process that I'm currently in -- and I can find that expert in the shortest period of time, using insights about who this person might be by using metadata about this individual, by actually discovering this path that I didn’t know existed upfront -- then I have actually added value to the business. So, we believe that search as a matching service is really one of the killer applications.
Gardner: So, what we’re talking about here is bringing the tacit knowledge that’s available to any organization into play with the semantic knowledge that’s gathered across all modalities, all types of media and information, but allowing those to come together in real-time based on someone’s specific needs through a very simple query.
Zaman: That’s right. I often like to talk about Greek philosophers, and Socrates is one of my favorites. The reason is because of what he did with a seeker. He didn’t answer a question with an answer. Rather he asked another question, and that allowed the seeker to refine his or her question, until finally they got to what they were looking for. In many ways, technology -- whether it’s information discovery, search, and business intelligence, whatever it might be -- shouldn’t insult the intelligence of the seeker. Rather it should allow individuals to find the answer that they’re looking for, either through tacit information that’s stored in the enterprise, or semantic information, or structured information, whatever it might be. What we’re trying to do is mimic the type of dialogue that Socrates had.
Gardner: Sure, because people are quite good at peeling away the layers of an onion, as a metaphor, to find what it is they’re looking for. There is no beginning or end to this peeling, but we certainly want to get him or her close to where they can extract real value.
Zaman: You got it.
Gardner: All right. Now people have been talking about this for years. I deal with developers a lot. I think they understand a lot of these theories. They understand the value and the goals, but putting it into practice, putting it into execution seems to be where there is still a need for some work. When we go to developers, we try to encourage them to start exploiting these capabilities of changing the mindset of how an application, specifically the UI, is actually constructed to make the entry point into an application more amenable for search and for gathering of these layers within an organization’s assets and resources.
Are there tools? Are there APIs? Are there SDKs? Is there a mindset? How do we bring the developer into play here? I’ll throw this out to anyone.
Olstad: I think Gartner predicted that in 2012, 75 percent of the world's applications will use search as the dominant metaphor for how an application is being operated by the end-user. That is a very powerful statement. And it means that all those developers that you’re asking about, Dana, they have to adapt to that.
Simplicity in an application [means] that you have a powerful metaphor that you can use to deduce what you should expect the application to be doing. Search is exactly that. So, obviously, search is evolving to not only provide access to text, but to provide value to all types of elements.
Another change in search technology is that it handles content natively, and previously it did not. In Web search you type some queries and you get a URL link to the content. Now you can take XML or SQL databases, and you can get native content. You can search in native content with queries that emulate what you can do in the database or in an XML database. This means that you can build the same type of application with the freedom to access data in an ordinalistic way, and have a mechanism to prioritize and do exactly what Zia talked about with Socrates.
Technically, if you use an open-ended query like, "List the innovative people in my company," or something, you could get back kind of a menu of people that have been referred to in the documents or discussions where there is talk of innovation, and then get the facts related to these people. So it’s not coming back with an answer, but it is coming back with an analysis -- and giving you the opportunity to refine the query.
Gardner: Another thing with developers is that they’re interested in seeing examples. Are there folks out there doing this now who are really on the cutting-edge, who have been able to instantiate rather than theorize? Maybe, Zia, you could address this. Are there folks that are out there using FAST Search & Transfer technologies or other technologies who are actually realizing some of these benefits? And, who are they, and how are they doing it?
Zaman: All over the landscape of the economy, people are using search in interesting ways. I think that the message to the developer is: "Think about the app that you’re working on right now, and think about how the users are accessing either the application or the information."
Take a consumer perspective and you’ll definitely think that search plays an important part in whatever it is that you’re doing. So, get involved and talk to people about search. I think you will find that, even in your own sphere, there are people who are already working on search technologies. In one particular investment bank, 15 or 16 different applications of FAST Search have been put into place across the business to help with everything from getting access to research reports to improving customer service.
There are search policies at law firms or at pharmaceutical companies or retail or e-commerce searches for Christmas shopping. Or just B2B searches, so that one individual can find another professional in the shortest amount of time in the right region. All these applications are about the user consuming information in a way that is most effective for them -- and search technologies are enabling each of these examples. Maybe John or Bjorn, you want to talk about a European example and I can talk about an American example?
Olstad: Sure. Let me start off with our broad business goals, and there are some that have nice benefits for the businesses. We have many customers that don’t have a choice because if you look at media, entertainment, and telcos, there are so many disruptive changes going on. You have print transforming to electronic. You have advertising on the Internet, where search-based advertising is now bypassing traditional banner advertising. You have telcos, where telephone calls may soon be free because there are alternative ways to monetize it. And you already have Skype, Voice over IP, etc.
So, if what a telco provider, for example, is providing today is going to be free, then the name of the game is going to be who is able to create applications and to have service delivery platforms that use that customer base to derive new services. They’re not very well equipped for that. The key enabling-technology that actually helps with this has got to be the user-driven applications. And they see search from a monetization perspective and from the perspective of building the end-user experience. So for lot of customers that we work with from media, entrainment, and telcos -- it’s not that it's "nice to have," but it's "How do we survive?"
Zaman: An example of that in the back office is AutoTrader.com, which is a phenomenal example of how search was used to fundamentally rethink the way in which records were served up. The database administrator at AutoTrader says that they tripled their performance at half the cost, using a highly scalable search platform, which was very easy to deploy and a lot more dynamic than their traditional database architecture. So, even in the back office for applications as diverse as AutoTrader, we are seeing search being used to really disrupt the way in which you think about information delivery.
Gardner: It also seems that the younger generation, as they come into play here, are probably already automatically thinking "search." The people who have grown up with the Web around them start almost any activity with search. So, this is becoming imbued into the society and into the behavior and culture?
At the same time, these folks are also quite used to mobile devices and small handheld devices. Can you explore for me a little bit of how search plays into young people in mobile devices? Perhaps we’re seeing some activities in early-adopter environments, South Korea for example, where search has to be the only way in a small-form factor that vast amounts of information could be managed. Help me understand the impact of use and mobile search?
Olstad: That’s a very interesting question, Dana. As you said, most exploration starts with a search on the Web. You start with a search on a Web search engine, and then you come to the site of this company. Then that company has some options. Should we let the user continue with the search experience or should we force him to go back and understand the text and how we have orchestrated all our content? Then, he is not allowed to do anything meaningful before he has completed that education. That’s slowing down the progress of the users.
Moving to mobile, if we go back to 2001-2002, we were trying to educate mobile operators that they needed to use search. That was a difficult discussion, a difficult sell. In Europe and Asia, at least, it was very popular to download ringtones and wallpapers and things like that. The operators felt that it was sufficient to have a chart with the most popular items and have them available on the static page, allowing people to choose from that.
When they moved to search, what they experienced was that a good way to measure the UI or the usability for the user is how many operations it took to actually perform the action. One of our customers is Vodafone. They measured how many operations young people had to do, from coming into the portal until they could complete a transaction, which in this case was to download a ringtone or music, etc. That used to be 3.5-operations, or something like that, when it was based on the charts and navigation mechanisms. When they substituted search, it went down to 1.5, which means that almost always you go directly to something that you can download. And, once in a while, you have to do one iteration before you get it. That drove revenue and traffic tremendously.
Zaman: You’re right. Generation Y and the younger generation are very used to unfettered access to their data, their content. What does that mean in the Enterprise? This is something that I hear increasingly on Wall Street and in the City of London. I know how to deal with the managing director, who is in his 40s. It’s very easy. For their relationship with their data, they ask someone junior.
But for that younger investment banker who’s just grown up with unfettered access to their personal data, they keep asking why they can’t get access to enterprise data with the same facility. The information technology people who are serving this younger generation have to change. They don’t quite understand that search and other technologies, including Web 2.0 technologies, are forcing them to rethink the way in which they have access to their information, because that’s the way they can be creative and allow for them to get the data they need in order to make the right decisions. I think it’s having a cultural effect, even in more staid industries.
Olstad: Then, if you go to Japan or to Asia, you have also this competition between the telco providers, where the name of the game now is how do you add video services, and with extremely high bandwidth on the phones, how to do that? You see multimedia screening, where you are serving video quality down to the cell phones. So there is a tremendous interest in how to capitalize on new user experience to combine rich media assets, and then use this to build the community stuff you have seen in YouTube in the U.S. This is happening on the mobile side at a tremendous speed in Asia.
Gardner: Very interesting. So we’re really still in the early stages of search. It sounds like there is going to be a lot to discuss at your forthcoming FASTforward ’07 event. Perhaps as a little tease for our audience, is there anything you can tell us about this event in terms of some additional technologies, products or approaches? I will throw this out to anyone.
Olstad: If I could start from the technology side. There will be a lot of cool introductions and some new technology. We will look at how to interface search with repositories, from structured data to rich media and to traditional text. We think there is a game-changing opportunity in doing that and also to enable the Semantic Web in a more powerful way, as we talked about.
We also really believe that, as we have touched on here, that search is not just a feature, but it will become the framework for building search-driven, user-driven applications, to actually make that a reality and make it possible to build those applications. We are introducing new components to do that.
We see that search is driving the connective role in Web 2.0 in the evolution towards Web 3.0 by being a service-delivery platform. So we are introducing a Web 2.0 platform that uses search as the main orchestration -- both for monetization purposes and for understanding both content and the user. We'll be tracking the user and understanding his preferences, building personalized experiences, and creating this platform where you can take the atomic components of content and services and effectively roll that out as user-centric services. We’re building a service delivery platform around search for doing that and you will see applications that already have been built on top of that platform.
Zaman: I think it’s going to be a very hands-on conference in San Diego. There will be customers. There will be developers. There will be case studies, demos, and a lot of learning going on, interacting and sharing of ideas. Sure, there will be the predictions and the analysts and the advice that goes along with it. But really I think that the value of this search conference will be in the exchange of innovative ideas that takes place, not just from FAST to the rest of the community -- but also from the community to other members within the community. I think it's going to be great.
Gardner: Thanks. John Markus, just to return back to the business value, we’ve been discussing this at a fairly technical level -- developers and business intelligence folks -- but for CEOs, for the leadership at many enterprises, what are some of the salient messages that you think they need to understand in order to better appreciate search and why it makes a good deal of sense to invest in it?
Lervik: In general for most information-intensive companies -- media companies, financial services companies, telecom companies and pharmaceutical companies -- search is becoming both a key part of the infrastructure to enable these companies to make decisions, but also -- more importantly -- to drive and increase business online.
Because we see more and more of these companies relying on effectively connecting customers or end-users to their products, to their contents, and to their services. That’s where search comes in as a mission-critical application, or at least to enable these mission critical applications. At FASTforward, there will be a great number of speakers who will talk about how these have effectively implemented a solution that drive and increase business.
Gardner: I’m afraid we’re about out of time. We’ve been discussing search, the role of search, Enterprise 2.0, Web 3.0. Clearly the role and impact of search is increasing rapidly and will increase further over the coming years.
I’d like to thank our guests and sponsor, FAST Search & Transfer. We’ve been talking with John Markus Lervik, the CEO. Also, Bjorn Olstad, the CTO, and Zia Zaman, the Senior Vice President of Strategic Marketing. This is Dana Gardner, principal analyst at Interarbor Solutions. Thanks for joining us for this BriefingsDirect podcast.
Listen to the podcast here. Podcast sponsor: FAST Search & Transfer.
Transcript of Dana Gardner’s BriefingsDirect podcast on the future of enterprise search. Copyright Interarbor Solutions, LLC, 2005-2007. All rights reserved.
Friday, January 12, 2007
Transcript of BriefingsDirect Podcast on Developer Productivity Metrics
Edited transcript of BriefingsDirect[TM] podcast on developer productivity with host Dana Gardner, recorded Nov. 29, 2006.
Listen to the podcast here. Podcast sponsor: 6th Sense Analytics.
Dana Gardner: Hi, this is Dana Gardner, principal analyst at Interarbor Solutions, and you’re listening to BriefingsDirect. Today, a sponsored podcast discussion about developer productivity. Interestingly enough, some 9.5 million developers around the world are toiling away, and their bosses -- either individually or collectively -- have rather small insight and limited information about what they do and how they do it.
We see requirements go in one end. We see code come out the other end. And there’s often a great veil, cloud, or fog over what detailed activities develop the solutions and the processes that create good code. To discuss this need for greater insight, analytics, and data around developer productivity -- joining us for the first time -- we have Greg Burnell. He is the chairman, co-founder and CEO of 6th Sense Analytics. Hi, Greg.
Greg Burnell: Thanks Dana, glad to be here.
Gardner: I’ve painted a portrait here about lack of insight. You’ve been in this business for going on 20 years. Tell us a little bit about the history of how development came about. Even though we’ve had tremendous productivity benefits and improvements, it’s still somewhat of a "dark art," if you will. Give us some context on the history of how we got to where we are today?
Burnell: I might even roll back to some of your opening comments, Dana. You characterized it from a management perspective. I'd suggest, too, that individual developers don’t have a lot of insight into the day-to-day conduct of their position. All stakeholders in the lifecycle have an equal visibility problem.
Application development is a knowledge-worker-centric thing. You’ve got very bright, very talented people, but they work in an isolated format, and in many instances there aren't a lot of tangible, visible outcomes to their work. We’ve always kept our hands off of application development. We’ve felt like it’s an area that we don’t want to manage too heavily or monitor -- or even measure, which is an absolutely forbidden concept.
We've lived in this world for many, many years, introducing tools, technologies and processes to try to help people do this better. The fact is that while software drives a lot of the growth and innovation throughout the world we do a very inadequate job of bringing visibility to this whole process. I think it’s because people work in such an insulated manner, and they feel that it’s not possible to measure it. We’re on a mission to change that.
Gardner: I suppose that the insight needs to happen on an individual basis for what the individual developer is up to, but also in a context of a collaborative environment -- with teams in one locale, and increasingly teams distributed all over the world.
Burnell: Absolutely. There are a couple of phenomena. You talked about globally distributed development. We all think we’re aware of the global nature of work that occurs today. Application development and IT support resources are being managed all over the world, but I think the challenge is that application teams have never really focused on providing the same data and visibility to all people in the process. I wrote something yesterday and said if the data isn’t available and accessible to all the people that are part of it, then it really doesn’t have any meaning.
Data that the individual doesn’t see is relatively useless, and data that individuals have -- but that never really makes its way to the manager in an aggregated way -- doesn’t have a lot of value. What we’re trying to do is bring to everyone the same level of visibility, so people can talk about the same things and understand themselves all in the context of the execution of the development project.
Gardner: This seems to be a long time in coming. I mentioned that there are 9.5 million developers around the world. That number's growing. Even the definition of a developer is growing -- business analysts can now be considered developers, given the level of tools and how processes are being brought together.
There have been tremendous insights brought to other types of activities. We’ve applied sciences and statistics, economics and theory to a lot of different activities in the last 10 or 15 years. Why do you think it is that development, while important in growing part of the economy, has not been looked at through a scientific lens?
Burnell: It’s a great question. I’m not sure I can answer why it hasn’t happened, to be honest with you.
I can tell you a couple of trends that we saw, that we think lent itself to the creation of a company and an opportunity like 6th Sense. In years past, there was just an enormous number of vendors out there providing application tooling -- tools for infrastructure to build software.
One thing that happened in the last decade is that there’s been a lot of consolidation in that industry. Also you have companies like Serena, IBM, Rational, Compuware, and Borland. They’ve kind of assembled what we call an SDLC, a Software Development Life Cycle -- all of the tooling for the end-to-end lifecycle app-dev project.
So, these consolidations help, because now you just don’t have a ridiculous number of technologies from which you try to cobble together the whole best-of-breed solution. You go out and buy some type of a lifecycle from somebody. The other thing that’s happened is that people have had to plug their own tools together. In order to do that they’ve had to build these robust connectors or APIs -- ways in which these tools can work together.
That migration and consolidation in the industry and the creation of all of this open infrastructure are what created the opportunity for 6th Sense. That’s why we can participate with those tools. We have access to those toolings, and we don’t have to worry about supporting 10,000 tools. We have to worry about supporting probably in the neighborhood of 100 to 200 tools, which is much more doable.
That transition in the industry has really created opportunity to do it in the way we do it. There have been other attempts, manual processes. I don’t think that people are reluctant to do it. I think they’ve tried to do it in many ways, but they’ve always found that, when push comes to shove, and the project delivery’s in jeopardy or the schedule is slipping, that the first thing that gets thrown out the widow is any type of management oversight. With the technology we put together, we’re unobtrusive. We don’t take away from the process. We empower the process, and I think that’s important.
Gardner: I see. Previously, many tools were involved, and it was difficult to get into each one and have insight into the individual idiosyncrasies that could then be relayed back to give a general view of productivity, habits, and work. At the same time, there was a great deal of dynamics and change in the marketplace, so the tool that was popular three years ago might not be very popular now.
It might not have made business sense for someone to go in and make an investment to have insight into a tool that actually might fall from favor in a fairly short amount of time. I bring this up, because as a hosted service, 6th Sense Analytics takes advantage of this software-as-a-service (SaaS) trend that also gives you insight into activities, but without being really embedded into the technology. Is that right?
Burnell: Absolutely. The software-as-a-service model reflects the fact that people are building software all over the place, and teams are distributed all over the place. Organizations have come to an awareness that they don’t want to continue to build out this IT infrastructure for no value received, when they can go to a vendor. We have security. We have all the permissioning and all the facility skill. We have a level of security equivalent to what they have in their IT shops. They’re looking using the software-as-a-service model, because it has a much lower cost infrastructure.
The other day, I was reading an article that was saying, “Is Eclipse the IDE Killer?” Eclipse is a phenomenon in our industry. It walked in and now dominates development at the IDE level. Few people saw that coming, and all of a sudden here it is. IBM certainly saw it, and that’s a huge trend that we get to leverage.
The two major tools in the industry are Microsoft Visual Studio and Eclipse, and everybody else is a distant number two in those spaces. Software-as-a-service is an interesting opportunity for organizations to have through the web, and through all the of the infrastructure that is in place from a high-speed perspective. Now, we can really distribute applications very quickly, and we can evolve our application very rapidly to benefit all of our customers at the same time.
Gardner: I suppose the development process has also been sort of a whipping boy, if you will. People like to point to low success rates when it comes to completion time and when it comes to full quality and requirements criteria that were set out. So, people often come away with this impression that development is a haphazard and losing-proposition of some kind.
I think that this propels the notion of development as a cost center -- rather than an innovation center -- and is really problematic. Part of the reason for that is just a lack of transparency into what’s going on. Before we get into the notion, can you tell us how your solution works? Help me better understand why development remains with sort of a tarnish to it. From what you’ve done to date with these analytics, do you really think that when we get in there that a different story emerges?
Burnell: We’re very optimistic about application development. I want to be very clear about that. We believe a lot of the perception of failure is because expectations were set incorrectly. If the expectation’s wrong or overly optimistic -- or even overly negative, to be honest with you -- the outcome is never going to meet the expectation. If you build a car, and your expectation was it’s going to have four wheels and it shows up with three, it’s very clear that it didn’t meet your expectation. With application software it’s not so easy to focus on one particular attribute of it.
One of the things we believe is that we’re going to start helping people to understand a level of effort and to set expectations in a more discrete manner. They’ll have a better idea that they’ve applied this level of effort and the outcomes that they have. They’ll have a better opportunity to view themselves successfully in the context of the outcome. If you start off on the wrong road, or if you start off too optimistically, you’re never going to get back on track.
This is a highly unstructured process involving knowledge workers. These are very bright, very talented, very creative people, and you’re trying to corral this energy and create something. It’s a unique management challenge, and requires unique approaches.
That’s why we’ve always approached it from the perspective that if we can be part of the process and not sit external to it like a process tool or a portfolio management tool -- if we can resonate with the process as it unfolds -- then we have a better opportunity for people to understand where they are relative to where they’re going. They have a better opportunity to set an expectation, based upon this effort to produce an outcome. So when they’re done, they can sit down and make a much better assessment of how they’ve done against their expectation. And, they can revise accordingly for the next time.
Gardner: Let’s help our readers out there understand what it is that we’re discussing, we’ve talked a little bit about the “why,” and what the problem set is and the issues. This is BriefingsDirect. I’m the analyst. You tell me, Greg, what is it that you bring to the table as a hosted service that gives insight into developer behavior?
Burnell: What we’re trying to do is say that software development can be measured. The singular or the collective effort of a development team can have associated with it: measurement, metrics, and analytics that the team, the individual, the project, the executive can rely upon. What we’ve done, as we talked about earlier, is create a technology that resides with the development technologies that you use: IDEs, SCM systems, defect trackers, test tools, requirement management systems. These are tools that you use inside of the development process.
As you utilize these tools it’s kind of like a hammer in construction. You have to use a hammer to build a house. In order to build software, you have to use technologies to do it. We sit there with these technologies -- plugins, addins, whatever kind of technology is most appropriate for the tool -- and as you take advantage of these tools, we simply collect data about the process, and then we give it back to you.
I like to say we don’t impart the process -- we reveal the process. We show you what you’ve done.
One of the neat things that’s important to know is that when a knowledge worker is most successful and there’s a concept called flow. I don’t think we’ll get to that in this call, but when knowledge workers are highly successful and highly focused on the task at hand, time becomes an irrelevant concept to them. They want to achieve the objective. One of the things we’re trying to do is to help people understand the relationship of time and effort to outcome, and in order to do that you have to be transparent behind the scenes, and you can’t take away from their efforts.
We plug in with these technologies -- Eclipse, Visual Studio, as I mentioned earlier -- and as the developers use these tools we collect data and we send it back to our host, the centralized server. We run it through an exciting analytics engine that takes this raw data and translates it into a picture of what has occurred in the software development process. Through a Web 2.0-style Web interface, utilizing Ajax and the like, we serve back to all the stake-holders in the process a very rich analytics engine for them to get a multidimensional view of the application-development process.
Gardner: Is this qualitative or quantitative? Determinations can be made, but are they specific, team-wide, or project-wide? Are they specific to a day? Can you give me some sense of what the data shows?
Burnell: There’s a lot of quantitative measurement. The base unit of measurement is called “active time.” It’s our unique offering. “Active time” is the amount of time you’re actively developing software.
We take that “active time,” that base unit of measurement, and we can roll it up. We know things like what your process is, your mix of edit time, view time, debug time, test time, and design time. We know that by your use of the tools, by the functions that you’re using inside the tool.
That’s very quantitative, but that’s also qualitative, because it reflects back on your process. We understand the tools and the technologies that you’re taking advantage of. We know the amount of time spent on tasks. At the end of the day, instead of just trying to sit down and say “I spent four hours on this and three hours on this,” we answer those questions for you.
It’s a rich set of both qualitative and quantitative metrics, and certainly the quantitative aspects of it are important. We measure the same way yesterday, today, and tomorrow. That’s important. We now have the opportunity, when change is imparted and we’ve changed either a process or a tool or a person, to sense that, because we understand what the dynamic is in the next day. We see it measured the same way and we can now start seeing the outcomes.
One of the things with software development is that we make a lot of changes as we go through the process. People come up with great ideas, read new books, and buy new tools, but no one has an idea whether they, in fact, improved the process or if they’ve had a beneficial effect from that. We think that we'll be able to help bring a lot of visibility to that.
Gardner: So, you’re leveling the playing field, not only within an individual activity, but perhaps across a multitude of activities. You can draw industry-wide inferences and perhaps start saying which tools are shelfware, what aspects of development are hang-ups that need to be focused on. That might bring knowledge to other vendors as to where they should devote their energy. This could be not only an individual, project, or even a company-wide benefit, but also an industry-wide benefit.
Burnell: That’s the other reason that we do it as a software-as-a-service solution, Dana. We gain the approval of our users to participate in our community of data. We do have that kind of visibility. We understand what the utilization of Eclipse is on a global basis for an installed community. We become a representative. Instead of looking at surveys and samples and calling a CIO, I know in my organization that 68.8 percent of the time spent in the last year was spent inside of Eclipse. I know that answer. I also know what my breakdown of my other tooling is.
You can look at it on a global basis. Application development is global, so you not only can look inside of your organization, but you should look outside of your organization and see what’s happening in the industry at large that you should be aware of and attuned to in your own track. Are you taking advantage of the things that are out there right now? We’re going to provide a lot of visibility to that.
This is all about decision making, and having the right things at your fingertips to make the critical decisions to ensure a successful outcome. The only thing anybody wants is software that’s on time, on budget, and meets the expectation. We’re going after those same challenges. That’s all we want to accomplish too. We’re going to help people get there faster and help people get there with a higher degree of confidence that they’re getting done what they need to get done.
Gardner: Those industry-wide metrics are extremely interesting to me. I think we’ve been fairly underserved, even in the analyst business, by the quality of the data we can get. We do surveys. We don’t know exactly who’s taking them, and how honest they are about the responses. People who are the busiest, the most productive, will be the least likely to take the survey. So, to me it’s very exciting to be able to get that visibility.
Let’s move into where this is going to be most relevant. You’ve mentioned offshoring and global issues quite a bit. Talk about lack of transparency and visibility. When I send off a set of requirements across the globe, and things come back from a variety of sources, how could I start measuring so that I know if I’m doing the right thing as a business person? What will your solution bring to the table that will help address this lack of insight into what’s going on in these offshore projects?
Burnell: My co-founder, Todd Olson, really is the architect of the solution we’re discussing. He said early on that the one commonality across application development worldwide is the desktop. People are working with the same tools. People are using Eclipse worldwide. They’re using Visual Studio worldwide.
So, we can basically isolate ourselves from all the cultural differences, the management differences, and the process differences. We can just eliminate those, and we can measure things the same way. What that means is that whether a person is managing a distributed team, an outsource situation or a captive situation doesn’t matter. They know that they’re looking at data that’s highly comparable between the locations. It speaks the same language. That’s the great thing about it.
The challenge with globalization is that you physically can’t be there all the time. So, you need to find ways to measure things. It goes well beyond just collaboration tools, meetings and phone calls. You’ve got to have some measurement that you rely upon. That gives you confidence that people are applying effort to the places that you want effort applied to, that the teams are conducting themselves in the manner that you wish them to. That’s what we do.
We basically give a high-degree of confidence that you can look at this information that we provide and ensure that your teams are all working together in the manner in which they’re supposed to. You can ensure that people are buying into the goals and the objectives, and they’re putting their time against it. That’s a key thing. People outsource for cost. I don’t want to take a radical view of outsourcing, but they do it for cost reasons. It’s not because they just have this desire to go global. They go out and try to find out lower-cost-basis resource. That does not diminish the talent and the genius of the educational systems and the people on a worldwide basis. They’re unbelievable.
We found that out after the fact, though. We went there for the cost and then we found out, hey, they’re very good. They’re extraordinary, actually. So now we said, let’s manage this as an opportunity. And, you have to have a measurement approach that is consistent and independent of all the other variables that exist in a global environment.
Gardner: So, we’ve got some value here: small teams, big teams, industry wide. We can address some of the issues about globalization. How about verticals? Are there specific sweet spots in the development world -- the type of applications, the type of environment -- perhaps it’s ones where the time-to-value or time-to-market is a really important element. Where’s the sweet spot for this stuff first?
Burnell: Anybody who makes their living from software should be worried about this. People who make their living on time to market, quality, and cost concerns should be all over this today. We’re going to tell them that you can now measure it, but beyond that, we really think that this is an opportunity for anyone to get a better grip on the overall process.
Organizations of all sizes are outsourcing now. Most boards of directors are looking at their companies and asking, “What’s your outsourcing strategy?” They just assume that there is some intrinsic benefit to outsourcing. We’re not so sure that’s true, but that’s not really the point. If your life depends upon your execution of your software process, this should be something that you would look at very closely. Beyond that, if you still have a need to manage a big team, manage people all over the world, manage across borders and times zones and language, this is also another thing you should be concerned about.
The interesting thing is, Dana, I’ve got eight or 10 people sitting in my development shop right now. We’re a relatively modest size organization from the development team point of view. I will tell you that as CEO of this company I have better visibility into the execution of my IT development resources than any other CEO on the planet. I know what we’re doing, and that’s an empowering position for any executive to be in from a decision-making perspective.
Gardner: You’ve got a development dashboard as a business person.
Burnell: I’ve got more than a dashboard -- I can see the alignment of the objectives that we set in our managements meetings to the execution of my development activities on a weekly and monthly basis. I know where my technologies are. I know where I need to go buy technologies.
If you’d called me up a year ago and said “Greg, you’re going to be spending 48 percent of your time building JavaScript,” I’d have told you that you were crazy. But that’s what we do here. I didn’t realize, and we didn’t realize going into this, the amount of investment we would have in the Web-side technologies like Java Scripting and Ajax.
We need tooling to support that. And I have to tell you, there’s not a lot out there. Now, we’re now talking to other people to build the tools. Instead of being reactive, we’re basically being proactive, looking for technologies that line up with our development requirements. And that’s a new place to be.
Gardner: Because you’re a hosted service, explain how the cost works. Do you charge per developer seat, per organization, per line of code? What’s the way that you can charge for this?
Burnell: It’s on a subscription basis, and we do it per subscriber; individual subscribers. Certainly, we understand enterprise pricing and the dynamics of organizations. As people come to us, we’re willing to sit down and look at the organization, because we believe this is the technology that needs to be implemented for all people in the process. It is a per-user basis at this time, but in any of our discussions with large corporate clients, we’re looking at the organization, the needs of the organization, and we can put a annual pricing in place for those organizations, if needed.
Gardner: Now let’s extrapolate a little bit and look into the future. If you can help define what that "flow" is -- when people are at their most productive and when they’ve used the tools that have been made available to them; when they’ve used their own creative juices -- you've found the efficiencies that they need to work smart, not necessarily work hard.
If you can quantify and qualify that for development, it seems to me that you could perhaps define that flow and apply your metrics and ability to gather information to make inferences about other creative activities. Help me understand that. Is this pie in the sky or do you think there’s something there?
Burnell: There is the old adage that you know who your good developers are. When we start showing you what the good developers do, what the most efficient developers do, you can now start looking at them and start prototyping activities and events and technologies against people that produce outcomes, and try to understand them a lot better.
To answer the second part of your question, this really is a platform for collection of data related to the knowledge-work process that involves a desktop technology. It even goes outside of application development. If you are managing a process, and the worker is using technology to do it, you can call it a human-based process.
I’m not really crazy about that term, because last time I checked, there aren’t any animals involved in the process. Nonetheless, a human being is sitting there interacting with a desktop technology to produce an outcome. So, we can extend it beyond software development, and we can look at other processes. What we’re talking about is trying to understand the dynamics of human interaction to a desktop technology to produce an outcome.
You mentioned manufacturing earlier. If Henry Ford didn’t figure out the assembly line, we’d still be building one-off cars at a $100,000 apiece. We have the same opportunity to really dig into and understand these human processes related to technology, and start evolving on a much faster basis our productivity and our output.
Gardner: That’s very interesting, Greg. Thanks for sharing all that. We’ve been talking about development productivity and a new approach from 6th Sense Analytics to gather data and provide analytics into how developers are the most productive, and how people can help them further that trend toward greater productivity.
Joining us for the call has been Greg Burnell, he's the chairman, co-founder and CEO of 6th Sense Analytics. And 6th Sense has also been the sponsor of this podcast. We appreciate their support. Thanks for joining, Greg.
Burnell: Dana, it’s my pleasure, and it’s great to talk you.
Listen to the podcast here.
Podcast sponsor: 6th Sense Analytics.
Transcript of Dana Gardner’s BriefingsDirect podcast on developer productivity metrics. Copyright Interarbor Solutions, LLC, 2005-2007. All rights reserved.
Listen to the podcast here. Podcast sponsor: 6th Sense Analytics.
Dana Gardner: Hi, this is Dana Gardner, principal analyst at Interarbor Solutions, and you’re listening to BriefingsDirect. Today, a sponsored podcast discussion about developer productivity. Interestingly enough, some 9.5 million developers around the world are toiling away, and their bosses -- either individually or collectively -- have rather small insight and limited information about what they do and how they do it.
We see requirements go in one end. We see code come out the other end. And there’s often a great veil, cloud, or fog over what detailed activities develop the solutions and the processes that create good code. To discuss this need for greater insight, analytics, and data around developer productivity -- joining us for the first time -- we have Greg Burnell. He is the chairman, co-founder and CEO of 6th Sense Analytics. Hi, Greg.
Greg Burnell: Thanks Dana, glad to be here.
Gardner: I’ve painted a portrait here about lack of insight. You’ve been in this business for going on 20 years. Tell us a little bit about the history of how development came about. Even though we’ve had tremendous productivity benefits and improvements, it’s still somewhat of a "dark art," if you will. Give us some context on the history of how we got to where we are today?
Burnell: I might even roll back to some of your opening comments, Dana. You characterized it from a management perspective. I'd suggest, too, that individual developers don’t have a lot of insight into the day-to-day conduct of their position. All stakeholders in the lifecycle have an equal visibility problem.
Application development is a knowledge-worker-centric thing. You’ve got very bright, very talented people, but they work in an isolated format, and in many instances there aren't a lot of tangible, visible outcomes to their work. We’ve always kept our hands off of application development. We’ve felt like it’s an area that we don’t want to manage too heavily or monitor -- or even measure, which is an absolutely forbidden concept.
We've lived in this world for many, many years, introducing tools, technologies and processes to try to help people do this better. The fact is that while software drives a lot of the growth and innovation throughout the world we do a very inadequate job of bringing visibility to this whole process. I think it’s because people work in such an insulated manner, and they feel that it’s not possible to measure it. We’re on a mission to change that.
Gardner: I suppose that the insight needs to happen on an individual basis for what the individual developer is up to, but also in a context of a collaborative environment -- with teams in one locale, and increasingly teams distributed all over the world.
Burnell: Absolutely. There are a couple of phenomena. You talked about globally distributed development. We all think we’re aware of the global nature of work that occurs today. Application development and IT support resources are being managed all over the world, but I think the challenge is that application teams have never really focused on providing the same data and visibility to all people in the process. I wrote something yesterday and said if the data isn’t available and accessible to all the people that are part of it, then it really doesn’t have any meaning.
Data that the individual doesn’t see is relatively useless, and data that individuals have -- but that never really makes its way to the manager in an aggregated way -- doesn’t have a lot of value. What we’re trying to do is bring to everyone the same level of visibility, so people can talk about the same things and understand themselves all in the context of the execution of the development project.
Gardner: This seems to be a long time in coming. I mentioned that there are 9.5 million developers around the world. That number's growing. Even the definition of a developer is growing -- business analysts can now be considered developers, given the level of tools and how processes are being brought together.
There have been tremendous insights brought to other types of activities. We’ve applied sciences and statistics, economics and theory to a lot of different activities in the last 10 or 15 years. Why do you think it is that development, while important in growing part of the economy, has not been looked at through a scientific lens?
Burnell: It’s a great question. I’m not sure I can answer why it hasn’t happened, to be honest with you.
I can tell you a couple of trends that we saw, that we think lent itself to the creation of a company and an opportunity like 6th Sense. In years past, there was just an enormous number of vendors out there providing application tooling -- tools for infrastructure to build software.
One thing that happened in the last decade is that there’s been a lot of consolidation in that industry. Also you have companies like Serena, IBM, Rational, Compuware, and Borland. They’ve kind of assembled what we call an SDLC, a Software Development Life Cycle -- all of the tooling for the end-to-end lifecycle app-dev project.
So, these consolidations help, because now you just don’t have a ridiculous number of technologies from which you try to cobble together the whole best-of-breed solution. You go out and buy some type of a lifecycle from somebody. The other thing that’s happened is that people have had to plug their own tools together. In order to do that they’ve had to build these robust connectors or APIs -- ways in which these tools can work together.
That migration and consolidation in the industry and the creation of all of this open infrastructure are what created the opportunity for 6th Sense. That’s why we can participate with those tools. We have access to those toolings, and we don’t have to worry about supporting 10,000 tools. We have to worry about supporting probably in the neighborhood of 100 to 200 tools, which is much more doable.
That transition in the industry has really created opportunity to do it in the way we do it. There have been other attempts, manual processes. I don’t think that people are reluctant to do it. I think they’ve tried to do it in many ways, but they’ve always found that, when push comes to shove, and the project delivery’s in jeopardy or the schedule is slipping, that the first thing that gets thrown out the widow is any type of management oversight. With the technology we put together, we’re unobtrusive. We don’t take away from the process. We empower the process, and I think that’s important.
Gardner: I see. Previously, many tools were involved, and it was difficult to get into each one and have insight into the individual idiosyncrasies that could then be relayed back to give a general view of productivity, habits, and work. At the same time, there was a great deal of dynamics and change in the marketplace, so the tool that was popular three years ago might not be very popular now.
It might not have made business sense for someone to go in and make an investment to have insight into a tool that actually might fall from favor in a fairly short amount of time. I bring this up, because as a hosted service, 6th Sense Analytics takes advantage of this software-as-a-service (SaaS) trend that also gives you insight into activities, but without being really embedded into the technology. Is that right?
Burnell: Absolutely. The software-as-a-service model reflects the fact that people are building software all over the place, and teams are distributed all over the place. Organizations have come to an awareness that they don’t want to continue to build out this IT infrastructure for no value received, when they can go to a vendor. We have security. We have all the permissioning and all the facility skill. We have a level of security equivalent to what they have in their IT shops. They’re looking using the software-as-a-service model, because it has a much lower cost infrastructure.
The other day, I was reading an article that was saying, “Is Eclipse the IDE Killer?” Eclipse is a phenomenon in our industry. It walked in and now dominates development at the IDE level. Few people saw that coming, and all of a sudden here it is. IBM certainly saw it, and that’s a huge trend that we get to leverage.
The two major tools in the industry are Microsoft Visual Studio and Eclipse, and everybody else is a distant number two in those spaces. Software-as-a-service is an interesting opportunity for organizations to have through the web, and through all the of the infrastructure that is in place from a high-speed perspective. Now, we can really distribute applications very quickly, and we can evolve our application very rapidly to benefit all of our customers at the same time.
Gardner: I suppose the development process has also been sort of a whipping boy, if you will. People like to point to low success rates when it comes to completion time and when it comes to full quality and requirements criteria that were set out. So, people often come away with this impression that development is a haphazard and losing-proposition of some kind.
I think that this propels the notion of development as a cost center -- rather than an innovation center -- and is really problematic. Part of the reason for that is just a lack of transparency into what’s going on. Before we get into the notion, can you tell us how your solution works? Help me better understand why development remains with sort of a tarnish to it. From what you’ve done to date with these analytics, do you really think that when we get in there that a different story emerges?
Burnell: We’re very optimistic about application development. I want to be very clear about that. We believe a lot of the perception of failure is because expectations were set incorrectly. If the expectation’s wrong or overly optimistic -- or even overly negative, to be honest with you -- the outcome is never going to meet the expectation. If you build a car, and your expectation was it’s going to have four wheels and it shows up with three, it’s very clear that it didn’t meet your expectation. With application software it’s not so easy to focus on one particular attribute of it.
One of the things we believe is that we’re going to start helping people to understand a level of effort and to set expectations in a more discrete manner. They’ll have a better idea that they’ve applied this level of effort and the outcomes that they have. They’ll have a better opportunity to view themselves successfully in the context of the outcome. If you start off on the wrong road, or if you start off too optimistically, you’re never going to get back on track.
This is a highly unstructured process involving knowledge workers. These are very bright, very talented, very creative people, and you’re trying to corral this energy and create something. It’s a unique management challenge, and requires unique approaches.
That’s why we’ve always approached it from the perspective that if we can be part of the process and not sit external to it like a process tool or a portfolio management tool -- if we can resonate with the process as it unfolds -- then we have a better opportunity for people to understand where they are relative to where they’re going. They have a better opportunity to set an expectation, based upon this effort to produce an outcome. So when they’re done, they can sit down and make a much better assessment of how they’ve done against their expectation. And, they can revise accordingly for the next time.
Gardner: Let’s help our readers out there understand what it is that we’re discussing, we’ve talked a little bit about the “why,” and what the problem set is and the issues. This is BriefingsDirect. I’m the analyst. You tell me, Greg, what is it that you bring to the table as a hosted service that gives insight into developer behavior?
Burnell: What we’re trying to do is say that software development can be measured. The singular or the collective effort of a development team can have associated with it: measurement, metrics, and analytics that the team, the individual, the project, the executive can rely upon. What we’ve done, as we talked about earlier, is create a technology that resides with the development technologies that you use: IDEs, SCM systems, defect trackers, test tools, requirement management systems. These are tools that you use inside of the development process.
As you utilize these tools it’s kind of like a hammer in construction. You have to use a hammer to build a house. In order to build software, you have to use technologies to do it. We sit there with these technologies -- plugins, addins, whatever kind of technology is most appropriate for the tool -- and as you take advantage of these tools, we simply collect data about the process, and then we give it back to you.
I like to say we don’t impart the process -- we reveal the process. We show you what you’ve done.
One of the neat things that’s important to know is that when a knowledge worker is most successful and there’s a concept called flow. I don’t think we’ll get to that in this call, but when knowledge workers are highly successful and highly focused on the task at hand, time becomes an irrelevant concept to them. They want to achieve the objective. One of the things we’re trying to do is to help people understand the relationship of time and effort to outcome, and in order to do that you have to be transparent behind the scenes, and you can’t take away from their efforts.
We plug in with these technologies -- Eclipse, Visual Studio, as I mentioned earlier -- and as the developers use these tools we collect data and we send it back to our host, the centralized server. We run it through an exciting analytics engine that takes this raw data and translates it into a picture of what has occurred in the software development process. Through a Web 2.0-style Web interface, utilizing Ajax and the like, we serve back to all the stake-holders in the process a very rich analytics engine for them to get a multidimensional view of the application-development process.
Gardner: Is this qualitative or quantitative? Determinations can be made, but are they specific, team-wide, or project-wide? Are they specific to a day? Can you give me some sense of what the data shows?
Burnell: There’s a lot of quantitative measurement. The base unit of measurement is called “active time.” It’s our unique offering. “Active time” is the amount of time you’re actively developing software.
We take that “active time,” that base unit of measurement, and we can roll it up. We know things like what your process is, your mix of edit time, view time, debug time, test time, and design time. We know that by your use of the tools, by the functions that you’re using inside the tool.
That’s very quantitative, but that’s also qualitative, because it reflects back on your process. We understand the tools and the technologies that you’re taking advantage of. We know the amount of time spent on tasks. At the end of the day, instead of just trying to sit down and say “I spent four hours on this and three hours on this,” we answer those questions for you.
It’s a rich set of both qualitative and quantitative metrics, and certainly the quantitative aspects of it are important. We measure the same way yesterday, today, and tomorrow. That’s important. We now have the opportunity, when change is imparted and we’ve changed either a process or a tool or a person, to sense that, because we understand what the dynamic is in the next day. We see it measured the same way and we can now start seeing the outcomes.
One of the things with software development is that we make a lot of changes as we go through the process. People come up with great ideas, read new books, and buy new tools, but no one has an idea whether they, in fact, improved the process or if they’ve had a beneficial effect from that. We think that we'll be able to help bring a lot of visibility to that.
Gardner: So, you’re leveling the playing field, not only within an individual activity, but perhaps across a multitude of activities. You can draw industry-wide inferences and perhaps start saying which tools are shelfware, what aspects of development are hang-ups that need to be focused on. That might bring knowledge to other vendors as to where they should devote their energy. This could be not only an individual, project, or even a company-wide benefit, but also an industry-wide benefit.
Burnell: That’s the other reason that we do it as a software-as-a-service solution, Dana. We gain the approval of our users to participate in our community of data. We do have that kind of visibility. We understand what the utilization of Eclipse is on a global basis for an installed community. We become a representative. Instead of looking at surveys and samples and calling a CIO, I know in my organization that 68.8 percent of the time spent in the last year was spent inside of Eclipse. I know that answer. I also know what my breakdown of my other tooling is.
You can look at it on a global basis. Application development is global, so you not only can look inside of your organization, but you should look outside of your organization and see what’s happening in the industry at large that you should be aware of and attuned to in your own track. Are you taking advantage of the things that are out there right now? We’re going to provide a lot of visibility to that.
This is all about decision making, and having the right things at your fingertips to make the critical decisions to ensure a successful outcome. The only thing anybody wants is software that’s on time, on budget, and meets the expectation. We’re going after those same challenges. That’s all we want to accomplish too. We’re going to help people get there faster and help people get there with a higher degree of confidence that they’re getting done what they need to get done.
Gardner: Those industry-wide metrics are extremely interesting to me. I think we’ve been fairly underserved, even in the analyst business, by the quality of the data we can get. We do surveys. We don’t know exactly who’s taking them, and how honest they are about the responses. People who are the busiest, the most productive, will be the least likely to take the survey. So, to me it’s very exciting to be able to get that visibility.
Let’s move into where this is going to be most relevant. You’ve mentioned offshoring and global issues quite a bit. Talk about lack of transparency and visibility. When I send off a set of requirements across the globe, and things come back from a variety of sources, how could I start measuring so that I know if I’m doing the right thing as a business person? What will your solution bring to the table that will help address this lack of insight into what’s going on in these offshore projects?
Burnell: My co-founder, Todd Olson, really is the architect of the solution we’re discussing. He said early on that the one commonality across application development worldwide is the desktop. People are working with the same tools. People are using Eclipse worldwide. They’re using Visual Studio worldwide.
So, we can basically isolate ourselves from all the cultural differences, the management differences, and the process differences. We can just eliminate those, and we can measure things the same way. What that means is that whether a person is managing a distributed team, an outsource situation or a captive situation doesn’t matter. They know that they’re looking at data that’s highly comparable between the locations. It speaks the same language. That’s the great thing about it.
The challenge with globalization is that you physically can’t be there all the time. So, you need to find ways to measure things. It goes well beyond just collaboration tools, meetings and phone calls. You’ve got to have some measurement that you rely upon. That gives you confidence that people are applying effort to the places that you want effort applied to, that the teams are conducting themselves in the manner that you wish them to. That’s what we do.
We basically give a high-degree of confidence that you can look at this information that we provide and ensure that your teams are all working together in the manner in which they’re supposed to. You can ensure that people are buying into the goals and the objectives, and they’re putting their time against it. That’s a key thing. People outsource for cost. I don’t want to take a radical view of outsourcing, but they do it for cost reasons. It’s not because they just have this desire to go global. They go out and try to find out lower-cost-basis resource. That does not diminish the talent and the genius of the educational systems and the people on a worldwide basis. They’re unbelievable.
We found that out after the fact, though. We went there for the cost and then we found out, hey, they’re very good. They’re extraordinary, actually. So now we said, let’s manage this as an opportunity. And, you have to have a measurement approach that is consistent and independent of all the other variables that exist in a global environment.
Gardner: So, we’ve got some value here: small teams, big teams, industry wide. We can address some of the issues about globalization. How about verticals? Are there specific sweet spots in the development world -- the type of applications, the type of environment -- perhaps it’s ones where the time-to-value or time-to-market is a really important element. Where’s the sweet spot for this stuff first?
Burnell: Anybody who makes their living from software should be worried about this. People who make their living on time to market, quality, and cost concerns should be all over this today. We’re going to tell them that you can now measure it, but beyond that, we really think that this is an opportunity for anyone to get a better grip on the overall process.
Organizations of all sizes are outsourcing now. Most boards of directors are looking at their companies and asking, “What’s your outsourcing strategy?” They just assume that there is some intrinsic benefit to outsourcing. We’re not so sure that’s true, but that’s not really the point. If your life depends upon your execution of your software process, this should be something that you would look at very closely. Beyond that, if you still have a need to manage a big team, manage people all over the world, manage across borders and times zones and language, this is also another thing you should be concerned about.
The interesting thing is, Dana, I’ve got eight or 10 people sitting in my development shop right now. We’re a relatively modest size organization from the development team point of view. I will tell you that as CEO of this company I have better visibility into the execution of my IT development resources than any other CEO on the planet. I know what we’re doing, and that’s an empowering position for any executive to be in from a decision-making perspective.
Gardner: You’ve got a development dashboard as a business person.
Burnell: I’ve got more than a dashboard -- I can see the alignment of the objectives that we set in our managements meetings to the execution of my development activities on a weekly and monthly basis. I know where my technologies are. I know where I need to go buy technologies.
If you’d called me up a year ago and said “Greg, you’re going to be spending 48 percent of your time building JavaScript,” I’d have told you that you were crazy. But that’s what we do here. I didn’t realize, and we didn’t realize going into this, the amount of investment we would have in the Web-side technologies like Java Scripting and Ajax.
We need tooling to support that. And I have to tell you, there’s not a lot out there. Now, we’re now talking to other people to build the tools. Instead of being reactive, we’re basically being proactive, looking for technologies that line up with our development requirements. And that’s a new place to be.
Gardner: Because you’re a hosted service, explain how the cost works. Do you charge per developer seat, per organization, per line of code? What’s the way that you can charge for this?
Burnell: It’s on a subscription basis, and we do it per subscriber; individual subscribers. Certainly, we understand enterprise pricing and the dynamics of organizations. As people come to us, we’re willing to sit down and look at the organization, because we believe this is the technology that needs to be implemented for all people in the process. It is a per-user basis at this time, but in any of our discussions with large corporate clients, we’re looking at the organization, the needs of the organization, and we can put a annual pricing in place for those organizations, if needed.
Gardner: Now let’s extrapolate a little bit and look into the future. If you can help define what that "flow" is -- when people are at their most productive and when they’ve used the tools that have been made available to them; when they’ve used their own creative juices -- you've found the efficiencies that they need to work smart, not necessarily work hard.
If you can quantify and qualify that for development, it seems to me that you could perhaps define that flow and apply your metrics and ability to gather information to make inferences about other creative activities. Help me understand that. Is this pie in the sky or do you think there’s something there?
Burnell: There is the old adage that you know who your good developers are. When we start showing you what the good developers do, what the most efficient developers do, you can now start looking at them and start prototyping activities and events and technologies against people that produce outcomes, and try to understand them a lot better.
To answer the second part of your question, this really is a platform for collection of data related to the knowledge-work process that involves a desktop technology. It even goes outside of application development. If you are managing a process, and the worker is using technology to do it, you can call it a human-based process.
I’m not really crazy about that term, because last time I checked, there aren’t any animals involved in the process. Nonetheless, a human being is sitting there interacting with a desktop technology to produce an outcome. So, we can extend it beyond software development, and we can look at other processes. What we’re talking about is trying to understand the dynamics of human interaction to a desktop technology to produce an outcome.
You mentioned manufacturing earlier. If Henry Ford didn’t figure out the assembly line, we’d still be building one-off cars at a $100,000 apiece. We have the same opportunity to really dig into and understand these human processes related to technology, and start evolving on a much faster basis our productivity and our output.
Gardner: That’s very interesting, Greg. Thanks for sharing all that. We’ve been talking about development productivity and a new approach from 6th Sense Analytics to gather data and provide analytics into how developers are the most productive, and how people can help them further that trend toward greater productivity.
Joining us for the call has been Greg Burnell, he's the chairman, co-founder and CEO of 6th Sense Analytics. And 6th Sense has also been the sponsor of this podcast. We appreciate their support. Thanks for joining, Greg.
Burnell: Dana, it’s my pleasure, and it’s great to talk you.
Listen to the podcast here.
Podcast sponsor: 6th Sense Analytics.
Transcript of Dana Gardner’s BriefingsDirect podcast on developer productivity metrics. Copyright Interarbor Solutions, LLC, 2005-2007. All rights reserved.
Subscribe to:
Posts (Atom)
