How SKYPAD and HPE Vertica Enable Luxury Retail Brands to Gain Rapid Insight into Consumer Sales Trends

Transcript of a BriefingsDirect discussion on how Sky I.T. has changed its platform and solved the challenges around variety, velocity, and volume for big data to make better insights available to retail users.

Listen to the podcast. Find it on iTunes. Get the mobile app. Download the transcript. Sponsor: Hewlett Packard Enterprise.

Dana Gardner: Hello, and welcome to the next edition of the HPE Discover Podcast Series. I'm Dana Gardner, Principal Analyst at Interarbor Solutions, your host and moderator for this ongoing discussion on IT innovation and how it’s making an impact on people’s lives.

Gardner

Our next big-data use case leadership discussion explores how retail luxury goods market analysis provider Sky I.T. Group has upped its game to provide more buyer behavior analysis faster and with more depth. We will see how Sky I.T. changed its data analysis platform infrastructure and why that has helped solve its challenges around data variety, velocity, and volume to make better insights available to its retail users.

Here to share how retail intelligence just got a whole lot smarter, we are joined by Jay Hakami, President of Sky I.T. Group in New York. Welcome, Jay.

Jay Hakami: Thank you very much. Thank you for having us.

Gardner: We're also here with Dane Adcock, Vice President of Business Development at Sky I.T. Group. Welcome Dane.

Dane Adcock: Thank you very much.

Gardner: And we're here with Stephen Czetty, Vice President and Chief Technology Officer at Sky I.T. Group. Welcome to BriefingsDirect, Stephen.

Sky I.T. Group
Retail Business-Intelligence Solutions
Get More Information

Stephen Czetty: Thank you, Dana, and I'm looking forward to the chance.

Gardner: What are the top trends that are driving the need for greater and better big-data analysis for retailers? Why do they need to know more, better, faster?

Adcock: Well, customers have more choices. As a result, businesses need to be more agile and responsive and fill the customer's needs more completely or lose the business. That's driving the entire industry into practices that mean shorter times from design to shelf in order to be more responsive.

It has created a great deal of gross marketing pressure, because there's simply more competition and more selections that a consumer can make with their dollar today.

Gardner: Is there anything specific to the retail process around luxury goods that is even more pressing when it comes to this additional speed? Are there more choices and  higher expectations of the end user?

Greater penalty

Adcock: Yes. The downside to making mistakes in terms of designing a product and allocating it in the right amounts to locations at the store level carries a much greater penalty, because it has to be liquidated. There's not a chance to simply cut back on the supply chain side, and so margins are more at risk in terms of making the mistake.

Ten years ago, from a fashion perspective, it was about optimizing the return and focusing on winners. Today, you also have to plan to manage and optimize the margins on your losers as well. So, it's a total package.

Gardner: So, clearly, the more you know about what those users are doing or what they have done is going to be essential. It seems to me, though, that we'rere talking about a market-wide look rather than just one store, one retailer, or one brand.

How does that work, Jay? How do we get to the point where we've been able to gather information at a fairly comprehensive level, rather than cherry-picking or maybe getting a non-representative look based on only one organization’s view into the market?

Hakami: With SKYPAD, what we're doing is collecting data from the supplier, from the wholesaler, as well as from their retail stores, their wholesale business, and their dot-com, meaning the whole omni channel. When we collect that data, we cleanse it to make sure its meaningful to the user.

Hakami

Now, we're dealing with a connected world where the retailer, wholesalers, and suppliers have to talk to one another and plan together for the buying season. So the partnerships and the insight that they get into the product performance is extremely important, as Dane mentioned, in terms of the gross margin and in terms of the software information. SKYPAD basically provides that intelligence, that insight, into this retail/wholesale world.

Gardner: Correct me if I'm wrong, but isn’t this also a case where people are opening up their information and making it available for the benefit of a community or recognizing that the more data and the more analysis that’s available, the better it is for all the participants, even if there's an element of competition at some point?

Hakami: Dana, that's correct. The retail business likes to share the information with their suppliers, but they're not sharing it across all the suppliers. They're sharing it with each individual supplier. Then, you have the market research companies who come in and give you aggregation of trends and so on. But the retailers are interested in sell-through. They're interested in telling X supplier, "This is how your products are performing in my stores."

If they're not performing, then there's going to be a mark down. There's going to be less of a margin for you and for us. So, there's a very strong interest between the retailer and a specific supplier to improve the performance of the product and the sell-through of those products on the floor.

Gardner: Before we learn more about the data science and dealing with the technology and business case issues, tell us a little bit more about Sky I.T. Group, how you came about, and what you're doing with SKYPAD to solve some of these issues across this entire supply chain and retail market spot.

Complex history

Hakami: I'll take the beginning. I'll give you a little bit of the history, Dana, and then maybe Dane and Stephen can jump in and tell you what we are doing today, which is extremely complex and interesting at the same time.

We started with SKYPAD about eight years ago. We found a pain point within our customers where they were dealing with so many retailers, as well as their own retail stores, and not getting the information that they needed to make sound business decisions on a timely basis.

We started with one customer, which was Theory. We came to them and we said, "We can give you a solution where we're going to take some data from your retailers, from your retail stores, from your dot-com, and bring it all into one dashboard, so you can actually see what’s selling and what’s not selling."

Fast forward, we've been able to take not only EDI transactions, but also retail portals. We're taking information from any format you can imagine -- from Excel, PDF, merchant spreadsheets -- bringing that wealth of data into our data warehouse, cleansing it, and then populating the dashboard.

So today, SKYPAD is giving a wealth of information to the users by the sheer fact that they don’t have to go out by retailer and get the information. That’s what we do, and we give them, on a Monday morning, the information they need to make decisions.

As these business intelligence (BI) tools have become more popular, the distribution of data coming from the retailers has gotten more ubiquitous and broader in terms of the metrics.

Dane, can you elaborate more on this as well?

Adcock: This process has evolved from a time when EDI was easy, because it was structured, but it was also limited in the number of metrics that were provided by the mainstream. As these business intelligence (BI) tools have become more popular, the distribution of data coming from the retailers has gotten more ubiquitous and broader in terms of the metrics.

But the challenge has moved from reporting to identification of all these data sources and communication methodologies and different formats. These can change from week to week, because they're being launched by individuals, rather than systems, in terms of Excel spreadsheets and PDF files. Sometimes, they come from multiple sources from the same retailer.

One of our accounts would like to see all of their data together, so they can see trends across categories and different geographies and markets. The challenge is to bring all those data sources together and align them to their own item master file, rather than the retailer’s item master file, and then be able to understand trends, which accounts are generating the most profits, and what strategies are the most profitable.

Visit the Sky BI Team
At The 2016 NRF Big Show!
Get More Information

It's been a shifting model from the challenge of reporting all this data together, to data collection. And there's a lot more of it today, because more retailers report at the UPC level, size level, and the store level. They're broadcasting some of this data by day. The data pours in, and the quicker they can make a decision, the more money they can make. So, there's a lot of pressure to turn it around.

Gardner: Let me understand, Dane. When you're putting out those reports on Monday morning, do you get queries back? Is this a sort of a conversation, if you will, where not only are you presenting your findings, but people have specific questions about specific things? Do you allow for them to do that, and is the data therefore something that’s subject to query?

Subject to queries

Adcock: It’s subject to queries in the sense that they're able to do their own discovery within the data. In other words, we put it in a BI tool, it’s on the web, and they're doing their own analysis. They're probing to see what their best styles are. They're trying to understand how colors are moving, and they're looking to see where they're low on stock, where they may be able to backfill in the marketplace, and trying to understand what attributes are really driving sales.

But of course, they always have questions about completeness of the data. When things don’t look correct, they have questions about it. That drives us to be able to do analysis on the fly, on-demand, and deliver some responses, "All your stores are there, all of your locations, everything looks normal." Or perhaps there seems to be some flaws or things in the data that don’t actually look correct.

Not only do we need to organize it and provide it to them so that they can do their own broad, flexible analysis, but they're coming back to us with questions about how their data was audited. And they're looking for us to do the analysis on the spot and provide them with satisfactory answers.

Gardner: Stephen Czetty, we've heard about the use case, the business case, and how this data challenge has grown in terms of variety as well as volume. What do you need to bring to the table from the architecture and the data platform to sustain this growth and provide for the agility that these market decision makers are demanding?

Czetty: We started out with an abacus, in a sense, but today we collect information from thousands of sources literally every single week. Close to 9,000 files will come across to us and we'll process them correctly and sort of them out -- what client they belong to and so forth, but the challenge is forever growing.

Czetty

We needed to go from older technology to newer technology, because our volumes of data are increasing and the amount of time that we need to consume to data in is static.

So we're quite aware that we have a time limit. We found Vertica as a platform for us to be able to collect the data into a coherent structure in a very rapid time as opposed to our legacy systems.

It allows us to treat the data in a truly vertical way, although that has nothing to do with the application or the database itself. In the past we had to deal with each client separately. Now we can deal with each retailer separately and just collect their data for every single client that we have. That makes our processes much more pipelined and far faster in performance.

The secret sauce behind that is the ability in our Vertica environment to rapidly sort out the data -- where it belongs, who it belongs to -- calculate it out correctly, put it into the database tables that we need to, and then serve it back to the front end that we're using to represent it.

That's why we've shifted from a traditional database model to a Vertica-type model. It's 100 percent SQL for us, so it looks the same for everybody who is querying it, but under the covers we get tremendous performance and compression and lots of cost savings.

Gardner: For some organizations that are dealing with the different sources and  different types of data, cleansing is one problem. Then, the ability to warehouse that and make it available for queries is a separate problem. You've been able to tackle those both at the same time with the same platform. Is that right?

Proprietary parsers

Czetty: That's correct. We get the data, and we have proprietary parsers for every single data type that we get. There are a couple of hundred of them at this point. But all of that data, after parsing, goes into Vertica. From there, we can very rapidly figure out what is going where and what is not going anywhere, because it’s incomplete or it’s not ours, which happens, or it’s not relevant to our processes, which happens.

We can sort out what we've collected very rapidly and then integrate it with the information we already have or insert new information if it's brand-new. Prior to this, we'd been doing this by hand to a large-scale, and that's not effective any longer with our number of clients growing.

Gardner: I'd like to hear more about what your actual deployment is, but before we do that, let’s go back to the business case. Dane and Jay, when Vertica came online, when Steve was able to give you some of these more pronounced capabilities, how did that translate into a benefit for your business? How did you bring that out to the market, and what's been the response?

Hakami: I think the first response was "wow." And I think the second response was "Wow, how can we do this fast and move quickly to this platform?"

Prior to this, we'd been doing this by hand to a large-scale, and that's not effective any longer with our number of clients growing.

Let me give you some examples. When Steve did the proof of concept (POC) with the folks from HP, we were very impressed with the statistics we had seen. In other words, going from a processing time of eight or nine hours to minutes was a huge advantage that we saw from the business side, showing our customers that we can load data much faster.

The ability to use less hardware and infrastructure as a result of the architecture of Vertica allowed us to reduce, and to continue to reduce, the cost of infrastructure. These two are the major benefits that I've seen in the evolution of us moving from our legacy to Vertica.

From the business perspective, if we're able to deliver faster and more reliably to the customer, we accomplished one of the major goals that we set for ourselves with SKYPAD.

Adcock: Let me add something there. Jay is exactly right. The real impact, as it translates into the business, is that we have to stop processing and stop collecting data at a certain point in the morning and start processing it in order for us to make our service-level agreements (SLAs) on reporting for our clients, because they start their analysis. The retail data comes in staggered over the morning and it may not all be in by the time that we need to shut that processing off.

One of the things that moving to Vertica has allowed us to do is to cut that time off later, and when we cut it off later, we have more data, as a rule, for a customer earlier in the morning to do their analysis. They don’t have to wait until the afternoon. That’s a big benefit. They get a much better view of their business.

Driving more metrics

The other thing that it has enabled us to do is drive more metrics into the database and do some processing in the database, rather than in the user tool, which makes the user tool faster and it provides more value.

For example, maybe for age on the floor, we can do the calculation in the background, in the database, and it doesn't impede the response in the front-end engine. We get more metrics in the database calculated rather than in our user tool, and it becomes more flexible and more valuable.

Sky I.T. Group
Retail Business-Intelligence Solutions
Get More Information

Gardner: So not only are you doing what you used to do faster, better, cheaper, but you're able to now do things you couldn't have done before in terms of your quality of data and analysis. Is there anything else that is of a business nature that you're able to do vis-à-vis analytics that just wasn't possible before, and might, in fact, be equivalent of a new product line or a new service for you?

Czetty: In the old model, when we got a new client we had to essentially recreate the processes that we'd built for other clients to match that new client, because they're collecting that data just for that client just at that moment.

In the current model, where we're centered on retailers, the only thing that will take us a long time to do in this particular situation is if there's a new retailer that we've never collected data from.

So 99 percent of it is the same as any other client, but one percent is always different, and it had to be built out. On-boarding a client, as we call it, took us a considerable amount of time -- we are talking weeks.

In the current model, where we're centered on retailers, the only thing that will take us a long time to do in this particular situation is if there's a new retailer that we've never collected data from. We have to understand their methodology of delivery, how it comes, how complex it is and so forth, and then create the logic to load that into the database correctly to match up with what we are collecting for others.

In this scenario, since we’ve got so many clients, very few new stores or new retailers show up, and typically it’s just our clients on retail chain, and therefore our on-boarding is just simplified, because if we are getting Nordstrom’s data from client A, we're getting the same exact data for client B, C, D, E, and F.

Now, it comes through a single funnel and it's the Nordstrom funnel. It’s just a lot easier to deal with, and on-boarding comes naturally.

Hakami: In addition to that, since we're adding more significant clients, the ability to increase variety, velocity, and volume is very important to us. We couldn't scale without having Vertica as a foundation for us. We'd be standing still, rather than moving forward and being innovative, if we stayed where we were. So this is a monumental change and a very instrumental change for us going forward.

Gardner: Steve, tell us a little bit about your actual deployment. Is this a single tenant environment? Are you on a single database? What’s your server or data center environment? What's been the impact of that on your storage and compression and costs associated with some of the ancillary issues?

Multi-tenant environment

Czetty: To begin with, we're coming from a multi-tenant environment. Every client had its own private database in the past, because in DB2, we couldn't add all these clients into one database and get the job done. There was not enough horsepower to do the queries and the loads.

We ran a number of databases on a farm of servers, on Rackspace as our hosting system. When we brought in Vertica, we put up a minimal configuration with three nodes, and we're still living with that minimal configuration with three nodes.

We haven't exhausted our capacity on the license by any means whatsoever in loading up this data. The compression is obscenely high for us, because at the end of the day, our data absolutely lends itself to being compressed.

Everything repeats over and over again every single week. In the world of Vertica, that means it only appears once in wherever it lives in the database, and the rest of it is magic. Not to get into the technology underneath it at this point, from our perspective, it's just very effective in that scenario.

With the three nodes, we've had zero problems with performance. It hasn't been an issue at all. We're just looking back and saying that we wish we had this a little sooner.

Also in our DB2 world, we're using quite costly large SAN configurations with lots of spindles, so that we can have the data distributed all across the spindles for performance on DB2, and that does improve the performance of that product.

However, in Vertica, we have 600 GB drives and we can just pop more in if we need to expand our capacity. With the three nodes, we've had zero problems with performance. It hasn't been an issue at all. We're just looking back and saying that we wish we had this a little sooner.

Vertica came in and did the install for us initially. Then, we ended up taking those servers down and reinstalling it ourselves. With a little information from the guide, we were able to do it. We wanted to learn it for ourselves. That took us probably a day and a half to two days, as opposed to Vertica doing it in two hours. But other than that, everything is just fine. We’ve had a little training, we’ve gone to the Vertica event to learn how other people are dealing with things, and it's been quite a bit of fun.

Now there is a lot of work we have to do at the back end to transform our processes to this new methodology. There are some restrictions on how we can do things, updates and so forth. So, we had to reengineer that into this new technology, but other than that, no changes. The biggest change is that we went vertical on the retail silos. That's just a big win for us.

Gardner: As you know, Vertica is cloud ready. Is there any benefit to that further down the road where maybe it’s around issues of a spike demand in holiday season, for example, or for backup recovery or business continuity? Any thoughts about where you might leverage that cloud readiness in the future?

Dedicated servers

Czetty: We're already sort of in the cloud with the use of dedicated servers, but in our business, the volume increases in the stores around holidays is not doubling the volume. It’s adding 10 percent, 15 percent, maybe 20 percent of the volume for the holiday season. It hasn’t been that big a problem in DB2. So, it’s certainly not going to be a problem in Vertica.

We've looked at virtualization in the cloud, but with the size of the hardware that we actually want to run, we want to take advantage of the speed and the memory and everything else. We put up pretty robust servers ourselves, and it turns out that in secure cloud environments like we're using right now at Rackspace, it's simply less expensive to do it as dedicated equipment. To spin up a machine, like another node for us at Rackspace, would take about same time it would take for virtual system setup and configure to a day or so. They can give us another node just like this on our rack.

We looked at the cloud financially every single time that somebody came around and said there was a better cloud deal, but so far, owning it seems to be a better financial approach.

Gardner: Before we close out, looking to the future, I suppose the retailers are only going to face more competition. They're going to be getting more demand from their end users or customers for user experience for information.

We looked at the cloud financially every single time that somebody came around and said there was a better cloud deal, but so far, owning it seems to be a better financial approach.

We're going to see more mobile devices that will be used in a dot-com world or even a retail world. We are going to start to see geolocation data brought to bear. We're going to expect the Internet of Things (IoT) to kick in at some point where there might be more sensors involved either in a retail environment or across the supply chain.

Clearly, there's going to be more demand for more data doing more things faster. Do you feel like you're in a good position to do that? Where do you see your next challenges from the data-architecture perspective?

Czetty: Not to disparage too much the industry of luxury, but at this point, they're not the bleeding edge on the data collection and analysis side, where they are on the bleeding edge on social media and so forth. We've anticipated that. We've got some clients who were collecting information about their web activities and we have done analysis for identifying customers who are presenting different personas through their different methods as they contact the company.

We're dabbling in that area and that’s going to grow as it becomes so tablet-oriented or phone-oriented as the interfaces go. A lot of sales are potentially going to go through social media and not just the official websites in the future.

We'll be capturing that information as well. We’ve got some experience with that kind of data that we’ve done in the past. So, this is something I'm looking forward to getting more of, but as of today, we’re only doing it for a few clients.

Well positioned

Hakami: In terms of planning, we're very well-positioned as a hub between the wholesaler and the retailer, the wholesaler and their own retail stores, as well as the wholesaler and their dot-coms. One of the things that we are looking into, and this is going to probably get more oxygen next year, is also taking a look at the relationships and the data between the retailer and the consumer.

As you mentioned, this is a growing area, and the retailers are looking to capture more of the consumer information so they can target-market to them, not based on segment but based on individual preferences. This is again a huge amount of data that needs to be cleansed, populated, and then presented to the CMOs of companies to be able to sell more, market more, and be in front of their customers much more than ever before.

Gardner: That’s a big trend that we are seeing in many different sectors of the economy -- that drive for personalization, and it really is a result of these data technologies to allow that to happen.

Last word to you, Dane. Any other thoughts about where the intersection of computer science capabilities and market intelligence demands are coming together in new and interesting ways?

Adcock: I'm excited about the whole approach to leveraging some predictive capabilities alongside the great inventory of data that we've put together for our clients. It's not just about creating better forecasts of demand, but optimizing different metrics, using this data to understand when product should be marked down, what types of attributes of products seem to be favored by different locations of stores that are obviously alike in terms of their shopper profiles, and bringing together better allocations and quantities in breadth and depth of products to individual locations to drive better, higher percentage of full-price selling and fewer markdowns for our clients.

So it’s a predictive side, rather than discovery using a BI tool.

Czetty: Just to add to that, there's the margin. When we talked to CEOs and CFOs five or six years ago and told them we could improve business by two, three, or four percent, they were laughing at us, saying it was meaningless to them. Now, three, four, or five percent, even in the luxury market, is a huge improvement to business. The companies like Michael Kors, Tory Burch, Marc Jacobs, Giorgio Armani, and Prada are all looking for those margins.

I'm excited about the whole approach to leveraging some predictive capabilities alongside the great inventory of data that we've put together for our clients.

So, how do we become more efficient with a product assortment, how do we become more efficient with distribution and all of these products to different sales channels, and then how do we increase our margins? How do we not over-manufacture and not create those blue shirts in Florida, where they are not selling, and create them for Detroit, where they're selling like hotcakes.

These are the things that customers are looking at and they must have that tool or tools in place to be able to manage their merchandising and by doing so become a lot more agile and a lot more profitable.

Gardner: Well, great. I'm afraid we will have to leave it there. We've been discussing how retail luxury goods and fashion market goods providers are using analysis from Sky I.T. Group and how Sky I.T. Group heads up its game through using HPE Vertica to provide more buyer behavior analysis faster, better, and cheaper.

And we’ve seen how Sky I.T. has changed its platform and solved the challenges around variety, velocity, and volume for that data to make those better insights available to those retail users, allowing them to become more data-driven across their entire market.

So please join me in thanking our guests. We have been talking with Jay Hakami,  President of Sky I.T. Group in New York. Thank you so much, Jay.

Visit the Sky BI Team
At The 2016 NRF Big Show!
Get More Information

Hakami: Thank you, Dana. I appreciate it very much.

Gardner: And we've also been talking with Dane Adcock, Vice President of Business Development at Sky I.T. Group. Thank you, Dane.

Adcock: It’s great to have the conversation. Thank you.
Gardner: I've enjoyed it myself. And lastly, a big thank you to Stephen Czetty, Vice-President and Chief Technology Officer there at Sky I.T. Group. Thank you, Stephen.

Czetty: You're very welcome, and I enjoyed the conversation. Thank you.

Gardner: And I’d also like to thank our audience as well for joining us for this big-data use case leadership discussion.

I'm Dana Gardner, Principal Analyst at Interarbor Solutions, your host for this ongoing series of Hewlett Packard Enterprise-sponsored discussions. Thanks again for listening, and come back next time.

Listen to the podcast. Find it on iTunes. Get the mobile app. Download the transcript. Sponsor: Hewlett Packard Enterprise.

Transcript of a BriefingsDirect discussion on how Sky I.T. has changed its platform and solved the challenges around variety, velocity, and volume for big data to make better insights available to retail users. Copyright Interarbor Solutions, LLC, 2005-2016. All rights reserved.

You may also be interested in:

• Redmonk analysts on best navigating the tricky path to DevOps adoption
• DevOps by design--A practical guide to effectively ushering DevOps into any organization
• Need for Fast Analytics in Healthcare Spurs Sogeti Converged Solutions Partnership Model
• HPE's composable infrastructure sets stage for hybrid market brokering role
• Nottingham Trent University Elevates Big Data's role to Improving Student Retention in Higher Education
• Forrester analyst Kurt Bittner on the inevitability of DevOps
• Agile on fire: IT enters the new era of 'continuous' everything
• Big data enables top user experiences and extreme personalization for Intuit TurboTax
• Feedback loops: The confluence of DevOps and big data
• IoT brings on development demands that DevOps manages best, say experts
• Big data generates new insights into what’s happening in the world's tropical ecosystems
• DevOps and security, a match made in heaven
• How Sprint employs orchestration and automation to bring IT into DevOps readiness

BriefingsDirect Analysts Unpack the Psychology of Project Management Via 'Pragmatic Enterprise 2.0' and SOA

Edited transcript of BriefingDirect Analyst Insights Edition podcast, Vol. 47 on new tools for measuring and building trust in technology adoption.

Listen to the podcast. Find it on iTunes/iPod and Podcast.com. Download the transcript. Charter Sponsor: Active Endpoints. Also sponsored by TIBCO Software.

Special offer: Download a free, supported 30-day trial of Active Endpoint's ActiveVOS at www.activevos.com/insight.

Dana Gardner: Hello, and welcome to the latest BriefingsDirect Analyst Insights Edition, Volume 47. I'm your host and moderator Dana Gardner, principal analyst at Interarbor Solutions.

This periodic discussion and dissection of IT infrastructure related news and events -- with a panel of industry analysts and guests -- comes to you with the help of our charter sponsor, Active Endpoints, maker of the ActiveVOS visual orchestration system, and through the support of TIBCO Software.

Our topic this week on BriefingsDirect Analyst Insights Edition, and it is the week of Nov. 9, 2009, centers on how to define, track and influence how people adapt to and adopt technology.

Any new information technology might be the best thing since sliced bread, but if people don’t understand the value or how to access it properly -- or if adoption is spotty, or held up by sub-groups, agendas, or politics -- then the value proposition is left in the dust.

A crucial element for avoiding and overcoming social and user dissonance with technology adoption is to know what you are up against, in detail. Yet, data and inferences on how people really feel about technology is often missing, incomplete, or inaccurate.

Today, we're going to hear from two partners who are working to solve this issue pragmatically. First, with regard to Enterprise 2.0 technologies and approaches. And, if my hunch is right, it could very well apply to service-oriented architecture (SOA) adoption as well.

I suppose you can think of this as a pragmatic approach to developing business intelligence (BI) values for people’s perceptions and their ongoing habits as they adopt technology in a business context.

So join fellow ZDNet bloggers, Michael Krigsman, president and CEO of Asuret, as well as Dion Hinchcliffe, founder and chief technology officer at Hinchcliffe & Co. as they explain how Pragmatic Enterprise 2.0 works. Together with our panel, we can plumb whether this could help with SOA adoption -- and maybe even other types of technology- or creative pursuit-adoptions as well.

Before we delve into this and hear more about Pragmatic Enterprise 2.0, please allow me to introduce our panel this week. We're joined by Joe McKendrick, prolific blogger and analyst. Welcome to the show, Joe.

Joe McKendrick: Thanks, Dana. Pleased to be here, and hello to everybody.

Gardner: We’re also joined by Miko Matsumura, vice president and chief strategist at Software AG. Welcome, Miko.

Miko Matsumura: Great. Good to be here.

Gardner: Ron Schmelzer, managing partner at ZapThink. Welcome back, Ron.

Ron Schmelzer: Hola. Bienvenido.

Gardner: Tony Baer, senior analyst at Ovum. Hi, Tony.

Tony Baer: Hey, Dana. Hi, everybody. Good to be here.

Gardner: We're also joined by Sandy Rogers, independent industry analyst. Welcome, Sandy.

Sandy Rogers: Great, Dana. Glad to be here.

Gardner: And, last but not the least, Jim Kobielus, senior analyst at Forrester Research. Welcome, Jim.

Kobielus: Selamat petang. There, I matched Ron Schmelzer in gratuitously using a foreign language to welcome everybody.

Gardner: Okay, let's hand this off to Dion Hinchcliffe. Tell me, Dion, about how you thought that you needed more pragmatism when it came to bringing Enterprise 2.0 technologies into practice, how you found Michael Krigsman, and how you two hooked up on this?

Social software

Dion Hinchcliffe: Absolutely, Dana, thanks for having us on the show. It's a real honor to be in front of such an august audience. As many of you know, we've been spending a lot of time over the last few years talking about how things like Web 2.0 and social software are moving beyond just what’s happening in the consumer space, and are beginning to really impact the way that we run our businesses.

More and more organizations are using social software, whether this is consumer tools or specific enterprise-class tools, to change the way they work. At my organization, we've been working with large companies for a number of years trying to help them get there.

This is the classic technology problem. Technology improves and gets better exponentially, but we, as organizations and as people, improve incrementally. So, there is a growing gap between what’s possible and what the technology can do, and what we are ready to do as organizations.

I've been helping organizations improve their businesses with things like Enterprise 2.0, which is social collaboration, using these tools, but with an enterprise twist. There are things like security, and other important business issues that are being addressed.

Businesses are about collaboration, team work, and people working together . . .

But, I never had a way of dealing with the whole picture. We find that that folks need a deep introduction to what the implications are when you have globally visible persistent collaboration using these very social models and the implications of the business.

We see organizations like the public sector getting more into this. Government 2.0 is one of the hottest topics. Increasingly, as these things become reality in large organizations, people are worried about what kind of control they're giving up and what kind of risk they're incurring?

Michael, of course, is famous for his work in IT project risk -- what it takes for projects to succeed and what causes them not to succeed. I saw this as the last leg of the stool for a complete way of delivering these very new, very foreign models, yet highly relevant models, to the way that organizations run their business.

Businesses are about collaboration, team work, and people working together, but we have used things like email, and models that people trust a lot more than these new tools.

Understanding projects

I've been working on big IT projects most of my life. I've been a solo architect and also focused on people in the enterprise. What Michael brings to the table is all his experience in terms of understanding. But, it requires understanding what’s really taking place inside projects where, one, the technology is not necessarily well-understood by the organization and, two, the implications on the business side are not well-understood.

There is usually a lot of confusion and uncertainty about what’s really taking place and what the expectations are. And Michael, with Asuret, brings something to the table. When we package it as a service that essentially brings these new capabilities, these new technologies and approaches, it manages the uncertainty about what the expectations are and what people are doing.

What we have designed is not specific to Enterprise 2.0 at all. It's really for when you are bringing in any new transformative technology with the same set of issues. I want Michael to speak more about what he is doing and how his side works.

Gardner: Sure. Michael, tell us a little bit about Asuret, and how your process works, maybe specifically with Enterprise 2.0, but with technology generally?

Michael Krigsman: Think about business transformation projects -- any type. This can be any major IT project, or any other type of business project as well. What goes wrong? If we are talking about IT, it's very tempting to say that the technology somehow screws up. If you have a major IT failure, a project is late, the project is over budget, or the project doesn’t meet expectations or plan, it's extremely easy to point the finger at the software vendor and say, "Well, the software screwed up."

If we look a little bit deeper, we often find the underlying drivers of the project that is not achieving its results. The underlying drivers tend to be things like mismatched expectations between different groups or organizations.

For example, the IT organization has a particular set of goals, objectives, restrictions, and so forth, through which they view the project. The line of business, on the other hand, has its own set of business objectives. Very often, even the language between these two groups is simply not the same.

As another example, we might say that the customer has a particular set of objectives and the systems integrator has its own objectives for the particular project. The customer wants to get it done as fast and as inexpensively as possible. The systems integrator is often -- and I shouldn’t make generalizations, but -- interested in maximizing their own revenue.

If we look inside each of these groups, we find that inside the groups you have divisions as well, and these are the expectation mismatches that Dion was referring to.

If we look at IT projects or any type of business transformation project, what’s the common denominator? It's the human element. The difficulty is how you measure, examine, and pull out of a project these expectations around the table. Different groups have different key performance indicators (KPIs), different measures of success, and so forth, which create these various expectations.

Amplifying weak signals

How do you pull that out, detect it inside the project, and then amplify what we might call these weak signals? The information is there. The information exists among the participants in the project. How do you then amplify that information, package it, and present it in a way that it can be shared among the various decision-makers, so that they have a more systematic set of inputs for making decisions consistently based on data, rather than anecdote? That’s the common thread.

Gardner: Michael, what you're doing here is providing this as a service, right? People don’t have to install this. They don’t have to do it themselves. You're offering a web interface-based approach to gathering inference from different players within this project, portfolio, what have you, implementation adoption pattern, sussing out what those folks are feeling, and then bringing that in a visual way to the attention of the project leaders. Is that right?

Krigsman: Yes. We offer a service. We're not selling software. We offer a service, and the service provides certain results. However, we've developed software, tools, methods, techniques, and processes that enable us to go through this process behind the scenes very efficiently and very rapidly.

Gardner: I had a chance to look at a demo of this. I haven’t tried it myself. I can’t vouch for it and I am not endorsing it, but what struck me as interesting is the fact that we're actually approaching the right part of the problem, when it comes to adoption.

Let's go to Sandy Rogers. Now, you headed up the SOA practice at IDC not too long ago. We kept encountering, in some of the discussions I had with you, this issue about why the projects stumble from time-to-time. What’s the hold up? It almost always came back to these people issues, and yet we had very little at our fingertips to apply to it. Does this sound like something that is a start for going in the right direction. Do you have any takeaways?

Rogers: What we discovered in our studies is that one of the fundamental needs in running any type of business project -- an SOA project or an Enterprise 2.0 IT project -- is the ability to share information and expose that visibility to all parties at levels that will resonate with what matters to them the most, but also bring them outside of their own domain to understand where dependencies exist and how one individual or one system can impact another.

One of the keys, however, is understanding that the measurements and the information need to get past system-level elements. If you design your services around what business elements are there and what matters to the business, then you can get past that IT-oriented view in bringing business stakeholders in aligned management and business goals to what transpires in the project.

Any way that you can get out -- web-based, easy-access dashboard with information -- and measure that regularly, you can allow that to proliferate through the organization. Having that awareness can help build trust, and that’s critical for these projects.

Gardner: Ron Schmelzer, we had a roundtable discussion in Boston several months ago, looking at the SOA topic and whether it was dead or not. I think some of the feelings from the panelists there were that it's not dead, but that it needs to be done better and differently. Is this what we're describing here with Michael and Dion? It sounds like it's moving in the right direction toward that balance of people, process, and technology.

More than technology

Schmelzer: Certainly, a lot of what people are doing with SOA is really just trying to do the things that people have established with enterprise architecture (EA). As we all know EA isn't about technology and buying things. It's about applying things and it's about people and process, much more than it’s about technology. That’s the hope.

The last thing you want to be doing is constantly scrambling and redoing your architecture because somebody somewhere in the organization has introduced some new technology. The wonderful paradise that we're trying to achieve is the stability of architecture, even though everything else is changing, the process is changing, and the technology is changing.

Given that, one of the things that we realized pretty early in this coverage of SOA, maybe about a decade ago, is that companies really need to manage their people, their governance, and their organization much more than they need to worry about buying the right tools.

As a matter of fact, you can buy the wrong tools, have great processes, and still have great outcomes. But if you buy the best tools, whatever that means, and you've got poor processes, you are guaranteed to have poor outcomes.

. . . It's mostly up to people and process to make sure the whole thing functions in a way that's returning value to the business.

It's like buying a pair of pants and putting them on. We have a complex system with a lot of moving parts, with a lot of interactions that are visible and hidden, and it's mostly up to people and process to make sure the whole thing functions in a way that's returning value to the business.

Gardner: Joe McKendrick, you’ve been covering SOA for some time, as a blogger on ebizQ and ZDNet. Often times, these topics around politics, fiefdoms, misunderstandings, and allowing people to communicate well come up again and again. Like the weather, we keep talking about it, but no one does anything about it.

It sounds as if Michael and Dion are trying to do something about it, at least for Enterprise 2.0. How does this strike you as to getting inside in data, into perceptions, and then being able to work with that? Is it a significant part of the problem and solution?

All about organization

McKendrick: Michael and Dion, I think you're on the right track with that. In fact, it's all about organization. It's all about the way IT is organized within the company and, vice-versa, the way the company organizes the IT department. I’ll quote Mike Hammer, the consultant, not the detective, "Automate a mess and you get an automated mess." That's what's been happening with SOA.

Upper management either doesn't understand SOA or, if they do, it's bits and pieces -- do this, do that. They read Enterprise Magazine. The governance is haphazard, islands across the organization, tribal. Miko talks about this a lot in his talks about the tribal aspect. They have these silos and different interest groups conflicting.

There's a real issue with the way the whole process is managed. One thing I always say is that the organizations that seem to be getting SOA right, as Michael and Dion probably see with the Enterprise 2.0 world, are usually the companies that are pretty progressive. They have a pretty good management structure and they're able to push a lot of innovations through the organization.

The companies that really could use these processes, the companies that really could use a good dose of service orientation, are the companies that just don't get it. It's a paradox.

Gardner: Miko Matsumura, as a supplier of software and services at Software AG, are you all looking for people like Dion and Michael to come up with these ways in which those tribal elements can be addressed? Is this something that intrigues you?

Matsumura: Absolutely. I had a wonderful conversation with Michael earlier and I appreciate his invite to come join this conversation. This type of an approach really reflects the evolution of the best practice of adoption. Some of the themes that we've been talking about today around this sharing of information, communication, and collaboration, are really are essential for success.

I do want to caution just a little bit. People talk about complexity and they create a linkage between complexity and failure. It's more important to try to look at, first of all, the source of the problem. Complexity itself is not necessarily indicative of a problem. Sure, it's correlated, but ice-cream consumption is correlated with the murder rate, just as a function of when temperatures get hot, both things happen to increase. So complexity is also a measure of success and scale.

I’d like to point to a different culprit, which I call "entropy" or "waste," and look at waste as being either over-complexity -- or over-simplicity in some cases. Over-simplicity can be as much of a villain as over-complexity. To me one of the biggest sources of complexity is tribalism and people fighting each other.

Providing a really transparent flow of measurements and metrics is obviously a tremendously important step. We have a methodology that we call the performance-driven organization that uses KPIs to increase organizational alignment. But, really, what you're doing is just shifting the fight. You're basically saying, "Let's not fight about one set of things. Let's fight about a set of so-called objective KPIs."

All about trust

The issue it comes down to for me is what Sandy said, which is that the word "trust," which is thrown in at the very end, turns out as extremely expensive. That alignment of organization and trust is actually a really important notion.

What happens with trust is that you can put things behind a service interface. Everything that's behind a service interface has suddenly gotten a lot less complex, because you're not looking at all that stuff. So, the reduction of complexity into manageability is completely dependent on this concept of trust and building it.

Gardner: The interesting thing you mentioned here is the metrics and the data. Having some kind of objective or constant way of evaluating what's going on and how that's changing over time, whether it's positive or negative, and then how to adapt, creates some sort of a positive feedback process loop.

Jim Kobielus, you deal with data analysis all the time. Tell me your impressions about bringing a data-analysis capability to how people react to something like implementing and adopting and adapting to Enterprise 2.0 or SOA.

Kobielus: A dashboard is so important when you are driving a vehicle, and that's what a consolidated view of KPIs and metrics provides. They are a dashboard in the BI sense, and that's what this is, project intelligence dashboard for very complex project or mega programs that are linked projects. In other words, SOA in all of its manifestations.

In organization, you have to steer your enterprise in a different direction. You need obviously to bring together many projects and many teams across many business domains. They all need to have a common view of the company as a whole -- its operations, various stakeholders, their needs, and the responsibilities internally on various projects of various people. That's highly complex. So, it’s critical to have a dashboard that's not just a one-way conduit of metrics, from the various projects and systems.

In the BI world, which I cover, most of the vendors now are going like crazy to implement more collaboration and work-flow and more social community-style computing capabilities into their environments. It's not just critical to have everybody on the same page in terms of KPIs, but to have a sideband of communication and coordination to make sure that the organization is continuing to manage collectively according to KPIs and objectives that they all ostensibly agree upon.

This is important. Social computing must come to the very heart of dashboarding to enable collaborative SOA project governance.

Gardner: But perhaps not just social from the gut, but social with some science, metrics, and real data.

Kobielus: Exactly. It has to be real data that's grounded in project objectives and in current status and delivering on those objectives.

Gardner: Tony Baer, what are we missing here? Is there some part of this equation that we're glossing over? Is there any cold water we should be pouring on here, just to be safe?

Recipe for tribalism


Baer: Oh, you read my mind on this one. I can quote from a project that my wife is currently involved with, which is basically a whole recipe for what you're talking about.

What Dion and Michael are talking about is an excellent idea in terms of that, in any type of environment where there is a lack of communication and trust, data is essential to really steer things. Data, and also assurances with risk management and protection of IT and all that. But, the fact is that there are some real clear hurdles, especially when you have what Miko talks about with tribes.

An example is this project that my wife is working on at the moment. She was brought in as a consultant to a consulting firm that's working for the client, and each of them have very different interests. This is actually in a healthcare-related situation. They're trying to do some sort of compliance effort, and whoever was the fount of wisdom there postponed the most complex part of this problem to the very end. At the very end, they basically did a Hail Mary pass bringing a few more bodies.

They didn't look for domain expertise or anything. Essentially it's like having eight women be pregnant and having them give birth to a baby in a month. That's essentially the push they are doing.

On top of that, there is also a fear among each tribe of the other coming up with a solution that makes the other tribes look bad. So, I can't tell exactly the feedback from this, but I do know that my wife came in as a process expert. She had a pretty clear view on how to untie the bottlenecks.

As soon as the project leader learned that she had this expertise, she was excluded from this, because this consulting firm was very afraid that her knowledge would make their firm look bad to the customer. In this case, they would rather risk complete failure of the project than have the firm be upstaged by someone who had been brought late in the process.

That pattern that you described there is essentially a factor about distribution of individual risk versus enterprise risk. The enterprise becomes a dumping ground for individual risk and it creates this kind of very large aggregate risk.

This is just an example of social and tribal challenges that you face. I very much agree that having a data-oriented approach and a risk management approach won't necessarily solve the problem. But, in case like this, that might be the only way out, provide cold, hard data from some neutral third-party.

Matsumura: I just want to jump in quickly and, first of all, applaud Tony Baer, the carrier of the cold water. That pattern that you described there is essentially a factor about distribution of individual risk versus enterprise risk. The enterprise becomes a dumping ground for individual risk and it creates this kind of very large aggregate risk.

Gardner: Let's take that point to Michael Krigsman. Michael, in what you're doing, are you allowing risk to be assigned? Are you be able to identify risk factors across different groups of people involved in a fairly large project? Is that part of what's going on here?

Essential elements

Krigsman: We gather a lot of data. The essential elements have been identified during this conversation. As Miko said, it's absolutely accurate to look at this tribally. Tony spoke about tribal divisions and the social tribal challenges.

The fundamental trick is how to convert this kind of trust information. Jim was talking about collaborative project governance. All of this relates to the fact that you've got various groups of people. They have their own issues, their own KPIs, and so forth. How do you service issues that could impact trust and then convert that to a form that can then be examined dispassionately. I'd love to use the word "objectively," but we all know that being objective is a goal and it's never outcome that you can ultimately reach.

At least you have a way to systematically and consistently have metrics that you can compare. And then, as Miko said, when you want to have a fight, at least you are fighting about KPIs, and you don't have people sitting in a conference room saying, "Well, my group thinks this. We believe the project 'blank.' If somebody says the same, my group thinks that." Well, let's have some common data that's collected across the various information silos and groups that we can then share and look at dispassionately.

Gardner: So, we want to get some objectivity about perception. It almost sounds like an oxymoron, but actually I think it's quite essential. Let's go back to Dion Hinchcliffe. Dion, you announced your Pragmatic Enterprise 2.0 initiative just a week or two ago. There is quite a bit of information about it on your website at Hinchcliffe & Co. Tell me a little bit about what the results are. When you bring this to bear, are they tangible results? Is there data about how well your data-driven process works?

Hinchcliffe: The way the process works is that we come into a client with an end-to-end service. Most organizations -- and this is going to be true of Enterprise 2.0 or SOA -- are looking at solving a problem. There's some reason why they think that this is going to help, but they're often not sure.

There are often a lot of unstated assumptions about how to apply technology to a business problem and what the outcome is going to be.

We start with this strategy piece that looks at the opportunity and tries to identify that for them and helps them correct the business case to understand what the return on investment (ROI) is going to be. To do that, you really have to understand what the needs of the organization are. So, one of the first things we do is bring Michael's process in, and we try and get ground truths.

There are often a lot of unstated assumptions about how to apply technology to a business problem and what the outcome is going to be. Particularly with SOA, you have so many borders that are typically involved. It's the whole concept around Conway's Law that the architecture tends to look back at the structure of the organization, because those are the boundaries in which everything runs.

One of the ways that we can assure that we have ground truth is by applying this dispassionate measurement process upfront to understand what people's expectations are, what their needs are, and what their concerns are. It's much more than just a risk-management approach. It's a way to get strategic project intelligence in a way that hasn't been possible before. We're really excited about it.

A lot of uncertainty

My specialty has always been focusing on emerging technology. There is always a lot of uncertainty, because people don't know necessarily what it is. They don't know what to expect. They have to have a way of understanding what that is, and you have an array of issues including the fact there are people who aren't willing to normally admit that they don't know things.

But, here is a way to safely and succinctly, on a regular basis, surface those issues and deal with them before they begin to have issues in the project. We then continue on through implementation and then regular assessments on the KPIs that can cause potential issues down the road. I think it's a valuable service. It's low impact, compared to another traditional interview process. This is something most organizations can afford to do on a regular basis.

Gardner: I'd like to go around our panel and get some more reaction to this.
Ron Schmelzer, the idea of this strategic project management caught my attention when Dion mentioned it. We've had lots of software products thrown at project management and portfolio management. Those don't seem to work. What's the difference between the project and portfolio management approach to some of these issues -- and what Michael Krigsman and Dion Hinchcliffe are doing with this more social inference gathering and measurement approach.

Schmelzer: I'm glad that you brought up the difference between project and portfolio management. This may be something unique in our perspective, or maybe it's becoming common. It's hard to tell when you talk to yourself a lot. We think that the whole idea of project management is just an increasing fallacy in IT anyway. There is no such thing now. It's really a discrete project.

Can you really say that some enterprise software that you maybe buying or building yourself or maybe even sourcing as a service is really completely disconnected from all the other projects that you have going on or the other technology? The answer is, they are not.

The enterprise is a collection of many different IT projects, some of which are ongoing, some of which may have been perceived to be dead or no longer in development, or maybe some are in the future.

So, it's very hard to do something like discrete project management, where you have defined set of requirements and a defined timeline and defined budget, and make the assumption or the premise, which is false, that you're not going to be impacting any of the other concurrently running projects.

We think of this like a game of pick-up sticks. The enterprise is a collection of many different IT projects, some of which are ongoing, some of which may have been perceived to be dead or no longer in development, or maybe some are in the future. The idea that you could take any one of those little projects, and manipulate them without impacting the rest of the pile is clearly becoming false.

In portfolio management you're basically managing a variety of ongoing concurrent tasks that either have budget or don't have budget and you're trying to achieve some sort of change with the least form of destruction within the business requirements and the money and the resources you have.

That's very different from this whole idea of, "Let's put together a Gantt chart. Let's throw a bunch of resources at it. Let's have some defined requirements. Let's build to it. Let's hope and pray that we're right." The industry, as a whole, is moving away from this idea of discrete IT project management.

Gardner: Joe McKendrick, thinking about discrete as something in the rear-view mirror, that means that we need to factor in cloud computing and software as a service (SaaS). They were not just going to have internal constituencies that need to be monitored and brought to some sort of a level set for understanding. We're going to have external influences, be they hosting organizations or applications that are being delivered and pulled across the wire.

How do you view the complexity of a project or portfolio management or enablement, when we're starting to bring in more and more parties to the process?

IT no longer internally focused

McKendrick: Dion, I'm an avid fan of your writings and, in particular, your ideas around web-oriented architecture (WOA), the next evolution of SOA, Enterprise 2.0, and those forces converging. I love the way you express it.

Dana is exactly right. IT is no longer an internally-focused effort. There are a lot of external factors at play. In the first stage, you have a lot of external business partners you need to expose interfaces to and you need to share information with. Right there, that dramatically increases the complexity of what you need to do.

Down the road, as you talk about cloud, you're talking about the sharing of services across enterprise borders. Everyone is going to be a producer, a publisher, or a creator of services, as well as consumers of services. It's going to be a two-way street.

There is a lot of discussion about cloud computing and the way these services will be consumed from the cloud. I don't think enough people are thinking about the services they will be producing and offering up to the cloud for others to consume. I'd be curious. Dion and Michael, do you address that in your model, in your web-based offering?

Hinchcliffe: Right now, we're going to validate some Enterprise 2.0 markets, looking at potential things as how they process. Then, of course, we'll be expanding particularly on next-generation SOA maturity. Enterprise 2.0 is getting very big right now, so that's our focus at the moment.

It needs to be much more federated, and a lot of companies, when they first took on SOA, tried to control things from a central unit.

Gardner: Sandy Rogers, another thought that I've had about this is how important governance, policy, and automation are in making SOA successful. If we have more inference information, a dashboard if you will, about the social landscape, about the buy-in or lack of buy-in from different participants in a adoption and/or execution phase of this sort of thing, can we take some of that information and then use it in the context of governance, policies, and management that are more traditional software-based SOA functions and features?

Rogers: One of the keys here is that it's a constant feedback loop of what you can automatically provide in what you are measuring and assessing, and then be able to look at that and change whether something should be standardized and should be collected.

It becomes this incremental cycle of building out that information. One of the keys that everyone is talking about here is this needs to be much more distributed. It needs to be much more federated, and a lot of companies, when they first took on SOA, tried to control things from a central unit.

When you start expanding SOA into the enterprise, especially with Enterprise 2.0, the idea of changing behaviors is something that has to start. This information that's distributed could help individuals gain knowledge and then be able to change their own behaviors.

Everyone realizes that people need to understand current state, before they can actually get to that next state, and then eventually to that ideal state. They also need to feel comfortable that in this federated approach. They may not want to share everything right-away, but incrementally contribute to the whole, and make it much more of a community.

Analysis benefit

Gardner: Michael Krigsman, we were using words like feelings and behavior. Is it fair to say that you're bringing some sort of an analysis benefit to an IT project or adoption pattern? Are we getting closer to a psychological participation project?

Krigsman: I am so hesitant to use the term psychological, because it has so many connotations associated with it. But, the fact is that we spoke about perception earlier, and there has been a lot of discussion of trust and community and collaboration. All of these issues fundamentally relate to how people work together. These are the drivers of success, and especially the drivers of lack of success on projects of every kind.

It therefore follows that, if we want our projects to be governed well and to succeed, one way or another we have to touch and look at these issues. That’s precisely what we're doing with Asuret and it’s precisely the application that we have taken with Dion into Pragmatic Enterprise 2.0. You have to deal with these issues.

Gardner: Jim Kobielus, this kind of reminds me of Star Trek: The Next Generation where there was this counselor. Deanna Troi was on the bridge with all the technicians, the leaders, drivers at warp speed, and the executive decision-makers. Is that what we need in IT, a virtual counselor along the way with us?

Kobielus: Virtual counselor? Hmm. I’ll answer that by tossing another metaphor. It really seems like the enterprise is becoming a cloud of stakeholders and interested parties that coalesces based on various needs and then scatters in the way that clouds tend to do.

The way open-source projects coalesce, certain people are first among equals, and they are the committers who defend, the general scope of common hopes and dreams.

The common denominator for getting things done in this new world is that responsibility needs to somehow precipitate out of the cloud and that certain individuals or teams take it upon themselves to get certain things done at certain times, because they recognize that things, results, need to happen.

So, virtual counselor ... I like that concept. The virtual counselor in this federated, distributed, or social-SOA governance environment. A virtual counselor is simply that person who takes command of or masters a set of channels or media -- Twitter, Facebook, blogging, and whatever else you have out there -- to be able to share all the KPIs and metrics to get others to wheedle them or cajole them to taking their responsibilities and their domains to get things done.

That person or persons will make sure there's one particular work stream within this broader project or program. This one person makes sure that certain things happen at certain times, and then gracefully, when necessary, hands off that virtual counselor post to others who pick up the baton. I'm extending metaphors here.

That’s absolutely what has to happen in a world of shifting alliances, shifting responsibilities, and shifting budgets across domains. It's like the open-source world. The way open-source projects coalesce, certain people are first among equals, and they are the committers who defend, the general scope of common hopes and dreams.

Gardner: Miko Matsumura, do you agree with my perception that this is a big step forward? In the context of a IT project or roll out, they're thinking about people’s feelings and behaviors and perceptions. It strikes me as a big step forward. Isn’t this long overdue?

Objectivity and rationality

Matsumura: About two months ago, I tweeted, "Enterprise does not need architecture. Enterprise needs psychiatrists." It does sort of preface some of this discussion. The reason I made such a point is that, the word "architecture" unfortunately implies this kind of objectivity and rationality that I, to some extent, resist when I hear words like data, rationality, objectivity, whatever.

The reason I rail against it is that the system aggregate in enterprise has in it a vicious cycle. It's not passively complex. It's not static complexity.

Ron was talking about the generation of the project management paradigm, the huge Gantt chart. Those huge Gantt charts are indicative of static complexity, and static complexity is actually not the paradigm. What Sandy was saying that I really appreciated is this notion of iteration, which is really critical.

When you get these “objective KPIs” to align organization, the next thing that happens is that organizations gain the hell out of KPIs, especially if you tie them to job review, performance evaluation, and, God forbid, bonuses.

You're going to ask people who are going to spend 40 hours a week, drilling away at ways to gain the KPIs to advantage themselves and maximize their own personal game, and it's not to say actively perverse, but essentially "to hell with everyone else."

. . . KPIs are well and good, but as soon as you institutionalize them, be ready to change them, because you will have unexpected outcomes.

The nature of the beast is such that when you encounter this kind of scenario, it's not merely this notion of lack of information or confusion. This is active perversity on an organizational and individual level. The point I'd make is that KPIs are well and good, but as soon as you institutionalize them, be ready to change them, because you will have unexpected outcomes.

Gardner: Tony Baer, last word to you. An important aspect of what Michael Krigsman and Dion are doing is that this can be anonymitized. The ability to draw inference, feelings, and perceptions from people can be done in a way that they feel empowered, that they can share their feelings without it becoming a political football or a hot potato perhaps by being anonymous. But, what you get is the insight into what the thinking is, the feelings are, the perceptions across the portfolio of participants in a project.

Does that strike you as an important factor? I want to ask you also about this counselor or analyst’s features. Do we need to bring a purple dinosaur into each SOA activity -- "I love you, you love me, let's talk about our feelings?" How do we stop being silly, but still get the benefit of this sharing going on?

Baer: I agree with you that basically that trust is really important. And, when I say trust here, it's trust in feeling that I can give information without it being used against me. No project can function in an atmosphere where everybody is just presenting basically what management wants. That eventually becomes an emperor’s new clothes situation. So obviously, I think that’s really essential.

All become counselors

I am a little cynical about the idea of a counselor, per se. I'm very much a fan of internalizing, so we all become counselors. I really like Sandy’s ideas of distributed governance, where Jim was talking about making this data-driven. I see this becoming a self-learning governance, because you can govern from the top based on assumptions that you make at the outset of a project that are totally oblivious to the conditions on the ground.

Therefore, you have to set this up so that you need to have an atmosphere of trust, where we can contribute information without fingers being pointed, and therefore names being given.

At the same time, we can then use this information to adapt. As Miko was saying, be prepared to change those KPIs, if those KPIs are not relevant. We should not be measuring to last week’s objectives, if, all of a sudden, the world has changed. So the short answer is, I agree that the anonymization is essential. I am leery about the idea of a counselor, but I am very much a very believer in everybody taking responsibility in this, and we all become counselors.

Gardner: Very good. I am afraid we’ll have to leave it there. I encourage folks to check this out. It really opened my thinking about how to make these projects more successful. It's a new dimension that I think needs to be brought in increasingly across a variety of different activities, and that would be at a business level, technology level, or a combination.

There is a lot more information available at the Hinchcliffe & Co., as well as Asuret, and of course. You can also find a lot more at the ZDNet blog that Michael Krigsman has been doing for several years now, the Project Failures blog.

I want to thank everyone for joining. We’ve been here with Dion Hinchcliffe, founder and chief technology officer at Hinchcliffe & Co. We’ve also been joined by Michael Krigsman, president and CEO of Asuret.

Please also join me in thanking our panel, Joe McKendrick, a prolific blogger and IT analyst. Thank you, Joe.

McKendrick: Thanks, Dana. Glad to be here.

Gardner: Miko Matsumura, vice president and chief strategist at Software AG. Thanks, Miko.

Matsumura: Thank you very much.

Gardner: Ron Schmelzer, managing partner at ZapThink.

Schmelzer: Muchas gracias.

Gardner: Tony Baer, senior analyst at Ovum.

Baer: Great, as always, Dana.

Gardner: Sandy Rogers, independent industry analyst. Thanks, Sandy.

Rogers: Thank you.

Gardner: Jim Kobielus, senior analyst at Forrester Research.

Kobielus: Great. I will sign off in a deep dose of alliterative English. I think it was a deep dose of domain expertise from SOA specialists.

Gardner: And I also want to thank the sponsors for the BriefingsDirect Analyst Insights Edition podcast series, Active Endpoints and TIBCO Software.

This is Dana Gardner, principal analyst at Interarbor Solutions. You've been listening to BriefingsDirect. Thanks, and come back next time.

Listen to the podcast. Find it on iTunes/iPod and Podcast.com. Download the transcript. Charter Sponsor: Active Endpoints. Also sponsored by TIBCO Software.

Special offer: Download a free, supported 30-day trial of Active Endpoint's ActiveVOS at www.activevos.com/insight.

Edited transcript of BriefingDirect Analyst Insights Edition podcast, Vol. 47 on new tools for measuring and building trust in technology adoption. Copyright Interarbor Solutions, LLC, 2005-2009. All rights reserved.

You may also be interested in:

• BriefingsDirect Analysts Discuss Business Commerce Clouds: Wave of the Future or Old Wine in a New Bottle?
• Linthicum's Latest Book: How SOA and Cloud Intersect for Enterprise Productivity Benefits
• Panel Discussion: Is Cloud Computing More or Less Secure than On-Premises IT?