Showing posts with label business intelligence. Show all posts
Showing posts with label business intelligence. Show all posts

Monday, July 22, 2013

HP Vertica Architecture Gives Massive Performance Boost to Toughest BI Queries for Infinity Insurance

Transcript of a BriefingsDirect podcast on how a major insurance company is using improved data architecture to gain a competitive advantage.

Listen to the podcast. Find it on iTunes. Download the transcript. Sponsor: HP.

Dana Gardner: Hello, and welcome to the next edition of the HP Discover Performance Podcast Series. I'm Dana Gardner, Principal Analyst at Interarbor Solutions, your moderator for this ongoing discussion of IT innovation and how it’s making an impact on people’s lives.

Gardner
Once again, we're focusing on how IT leaders are improving their services' performance to deliver better experiences and payoffs for businesses and end users alike, and this time we're coming to you directly from the HP Discover 2013 Conference in Las Vegas.

Our next innovation case study interview highlights how Infinity Insurance Companies in Birmingham, Alabama has been deploying a new data architecture to improve productivity for their analysis and business intelligence (BI). [Learn more about the upcoming Vertica conference in Boston Aug. 5.]

To learn more about how they are improving their performance and their results for their business activities, please join me in welcoming our guest, Barry Ralston, Assistant Vice President for Data Management at Infinity Insurance Companies. Welcome, Barry. [Disclosure: HP is a sponsor of BriefingsDirect podcasts.]

Barry Ralston: Thanks for having me, Dana.

Gardner: You're welcome. Tell me a bit about the need for change. What was it that you've been doing with your BI and data warehousing that prompted you to seek an alternative?

Ralston: Like many companies in our space, we have constructed an enterprise data warehouse deployed to a row-store technology. In our case, it was initially Oracle RAC and then, eventually, the Oracle Exadata engineered hardware/software appliance.

Ralston
We were noticing that analysis that typically occurs in our space wasn’t really optimized for execution via that row store. Based on my experience with Vertica, we did a proof of concept with a couple of other alternative and analytic store-type databases. We specifically chose Vertica to achieve higher productivity and to allow us to focus on optimizing queries and extracting value out of the data.

Gardner: Before we learn more about how that’s worked out for you, maybe you could explain for our listeners’ benefit, what Infinity Insurance Companies does. How big are you, and how important is data and analysis to you?

Ralston: We are billion-dollar property and casualty company, headquartered in Birmingham, Alabama. Like any insurance carrier, data is key to what we do. But one of the things that drew me to Infinity, after years of being in a consulting role, was the idea of their determination to use data as a strategic weapon, not just IT as a whole, but data specifically within that larger IT as a strategic or competitive advantage.

Vertica environment

Gardner: You have quite a bit of internal and structured data. Tell me a bit what happened when you moved into a Vertica environment, first to the proof of concept and then into production?

Ralston: For the proof of concept, we took the most difficult or worst-performing queries from our Exadata implementation and moved that entire enterprise data warehouse set into a Vertica deployment on three Dual Hex Core, DL380 type machines. We're running at the same scale, with the same data, with the same queries.

We took the top 12 worst-performing queries or longest-running queries from the Exadata implementation, and not one of the proof of concept queries ran less than 100 times faster. It was an easy decision to make in terms of the analytic workload, versus trying to use the row-store technology that Oracle has been based on.

Gardner: Let’s dig into that a bit. I'm not a computer scientist and I don’t claim to fully understand the difference between row store, relational, and the column-based approach for Vertica. Give us the quick "Data Architecture 101" explanation of why this improvement is so impressive? [Learn more about the upcoming Vertica conference in Boston Aug. 5.]

Ralston: The original family of relational databases -- the current big three are  Oracle, SQL Server and DB2 -- are based on what we call row-storage technologies. They store information in blocks on disks, writing an entire row at a time.

If you had a record for an insured, you might have the insured's name, the date the policy went into effect, the date the policy next shows a payment, etc. All those attributes were written all at the same time in series to a row, which is combined into a block.
It’s an optimal way of storing data for transaction processing.

So storage has to be allocated in a particular fashion, to facilitate things like updates. It’s an optimal way of storing data for transaction processing. For now, it’s probably the state-of-the-art for that. If I am running an accounting system or a quote system, that’s the way to go.

Analytic queries are fundamentally different than transaction-processing queries. Think of the transaction processing as a cash register. You ring up a sale with a series of line items. Those get written to that row store database and that works well.

But when I want to know the top 10 products sold to my most profitable 20 percent of customers in a certain set of regions in the country, those set-based queries don’t perform well without major indexing. Often, that relates back to additional physical storage in a row-storage architecture.

Column store databases -- Vertica is a native column store database -- store data fundamentally differently than those row stores. We might break down a record into an entire set of columns or store distinctly. This allows me to do a couple of different things from an architectural level.

Sort, compress, organize

First and foremost, I can sort, compress, and organize the data on disk much more efficiently. Compression has been recently added to row-storage architectures, but in a row-storage database, you largely have to compress at the entirety of a row.

I can’t choose an optimal compression algorithm for just a date, because in that row, I will have text, numbers, and dates. In a column store, I can apply specific compression algorithm to the data that's in that column. So date gets one algorithm, a monotone increasing key like a surrogate key you might have in a dimensional data warehouse, has a different encoding algorithm, etc.

This is sorting. How data gets retrieved is fundamentally different, another big point for row-storage databases at query time. I could say, "Tell me all the customers that bought a product in California, but I only want to know their last name."

If I have 20 different attributes, a row-storage database actually has to read all the attributes off of disk. The query engine eliminates the ones I didn’t ask for in the eventual results, but I've already incurred the penalty of the input-output (I/O). This has a huge impact when you think of things like call detail records in telecom which have a 144-some odd columns.

If I'm only asking against a column store database, "Give me all the people who have last names, who bought a product in California," I'm essentially asking the database to read two columns off disk, and that’s all that’s happening. My I/O factors are improved by an order of 10 or in the case of the CDR, 1 in 144.
The great question is what ends up being the business value.

Gardner: Fundamentally it’s the architecture that’s different. You can’t just go back and increase your I/O improvements in those relational environments by making it in-memory or cutting down on the distance between the data and the processing. That only gets you so far, and you can only throw hardware at it so much. Fundamentally, it’s about the architecture.

Ralston: Absolutely correct. You've seen a lot of these -- I think one of the fun terms around this is "unnatural acts with data," as to how data gets either scattered or put into a cache or other things. Every time you introduce one of these mechanisms, you're putting another bottleneck between near real-time analytics and getting the data from a source system into a user’s hands for analytics. Think of a cache. If you’re going to cache, you’ve got to warm that cache up to get an effect.

If I'm streaming data in from a sensor, real-time location servers, or something like that, I don’t get a whole lot of value out of the cache to start until it gets warmed up. I totally agree with your point there, Dana, that it’s all about the architecture.

Gardner: So you’ve gained on speed and scale, and you're able to do things you couldn’t do differently when it comes to certain types of data. That’s all well and good for us folks who are interested in computers. What about the people who are interested in insurance? What were you able to bring back to your company that made a difference for them and their daily business that’s now allowed you to move beyond your proof of concept into wider production?

Ralston: The great question is what ends up being the business value. In short, leveraging Vertica, the underlying architecture allows me to create a playfield, if you will, for business analysts. They don’t necessarily have to be data scientists to enjoy it and be able to relate things that have a business relationship between each other, but not necessarily one that’s reflected in the data model, for whatever reason.

Performance suffers

Obviously in a row storage architecture, and specifically within dimensional data warehouses, if there is no index between a pair of columns, your performance begins to suffer. Vertica creates no indexes and it’s self-indexing the data via sorting and encoding.

So if I have an end user who wants to analyze something that’s never been analyzed before, but has a semantic relationship between those items, I don’t have to re-architect the data storage for them to get information back at the speed of their decision.

Gardner: You've been able to apply the Vertica implementation to some of your existing queries and you’ve gotten some great productivity benefits from that. What about opening this up to some new types of data and/or giving your users the folks in the insurance company the opportunity to look to external types of queries and learn more about markets, where they can apply new insurance products and grow the bottom line rather than just repay cowpaths?

Ralston: That's definitely part of our strategic plan. Right now, 100 percent of the data being leveraged at Infinity is structured. We're leveraging Vertica to manage all that structured data, but we have a plan to leverage Hadoop and the Vertica Hadoop connectors, based on what I'm seeing around HAVEn, the idea of being able to seamlessly structured, non-structured data from one point.
Then, I’ve delivered what my CIO is asking me in terms of data as a competitive advantage.

Insurance is an interesting business in that, as my product and pricing people look for the next great indicator of risk, we essentially get to ride a wave of that competitive advantage for as long a period of time as it takes us to report that new rate to a state. The state shares that with our competitors, and then our competitors have to see if they want to bake into their systems what we’ve just found.

So we can use Vertica as a competitive hammer, Vertica plus Hadoop to do things that our competitors aren’t able to do. Then, I’ve delivered what my CIO is asking me in terms of data as a competitive advantage.

Gardner: Well, great. I'm afraid we will have to leave it there. We've been learning about how Infinity Insurance Companies has been deploying HP Vertica technology and gaining scale and speed benefits. And now also setting themselves up for perhaps doing types of queries that they hadn’t been able to do before.

I’d like to thank our guest for joining us, Barry Ralston, Assistant Vice President for Data Management at Infinity Insurance company. Thank so much, Barry. [Learn more about the upcoming Vertica conference in Boston Aug. 5.]

Ralston: Thank you very much.

Gardner: I’d like to thank our audience as well for joining us for this special HP Discover Performance Podcast, coming to you from the HP Discover 2013 Conference in Las Vegas.

I'm Dana Gardner, Principle Analyst at Interarbor Solutions, your host for this ongoing series of HP-sponsored discussions. Thanks again for listening, and come back next time.

Listen to the podcast. Find it on iTunes. Download the transcript. Sponsor: HP.

Transcript of a BriefingsDirect podcast on how a major insurance company is using improved data architecture to gain a competitive advantage. Copyright Interarbor Solutions, LLC, 2005-2013. All rights reserved.

You may also be interested in:

Tuesday, July 09, 2013

Want a Data-Driven Culture? Start Sorting Out the BI and Big Data Myths Now

Transcript of a BriefingsDirect podcast on current misconceptions about big data and how organizations should best approach a big-data project.

Listen to the podcast. Find it on iTunes. Download the transcript. Sponsor: Dell Software.

Dana Gardner: Hi, this is Dana Gardner, Principal Analyst at Interarbor Solutions and you're listening to BriefingsDirect.

Gardner
Today, we present a sponsored podcast discussion on debunking some major myths around big data. It used to be that data was the refuse of business applications, a necessary cleanup chore for audit and compliance sake.

But now, as analytics grow in importance for better running businesses and in knowing and predicting dynamic market trends and customer wants in real-time, data itself has become the killer application.

As the volumes and types of value data are brought to bear on business analytics, the means to manage and exploit that sea of data has changed rapidly, too. But that doesn't mean that the so-called big data is beyond the scale of mere business mortals or too costly or complex for mid-size companies to master.

So we're here to pose some questions -- many of them the stuff of myth -- and then find better answers to why making data and big data the progeny of innovative insight is critical for more companies.

To help identify and debunk the myths around big data so that you can enjoy the value of those analytics better, please join me in welcoming our guest, Darin Bartik, Executive Director of Products in the Information Management Group at Dell Software. Welcome, Darin. [Disclosure: Dell is a sponsor of BriefingsDirect podcasts.]

Darin Bartik: Thanks, Dana. Good to be with you.

Gardner: We seem to be at an elevated level of hype around big data. I guess a good thing about that is it’s a hot topic and it’s of more interest to more people nowadays, but we seem to have veered away from the practical and maybe even the impactful. Are people losing sight of the business value by getting lost in speeds and feeds and technical jargon? Is there some sort of a disconnect between the providers and consumers of big data?

Bartik: I'm sure we're going to get into a couple of different areas today, but you hit the nail on the head with the first question.  We are experiencing a disconnect between the technical side of big data and the business value of big data, and that’s happening because we’re digging too deeply into the technology.

Bartik
With a term like big data, or any one of the trends that the information technology industry talks about so much, we tend to think about the technical side of it. But with analytics, with the whole conversation around big data, what we've been stressing with many of our customers is that it starts with a business discussion. It starts with the questions that you're trying to answer about the business; not the technology, the tools, or the architecture of solving those problems. It has to start with the business discussion.

That’s a pretty big flip. The traditional approach to business intelligence (BI) and reporting has been one of technology frameworks and a lot of things that were owned more by the IT group. This is part of the reason why a lot of the BI projects of the past struggled, because there was a disconnect between the business goals and the IT methods.

So you're right. There has been a disconnect, and that’s what I've been trying to talk a lot about with customers -- how to refocus on the business issues you need to think about, especially in the mid-market, where you maybe don’t have as many resources at hand. It can be pretty confusing.

Part of the hype cycle

The other thing you asked is, “Are vendors confusing people?" Without disparaging the vendors like us, or anyone else, that’s part of the problem of any hype cycle. Many people jumped on the bandwagon of big data. Just like everyone was talking cloud. Everyone was talking virtualization, bring your own device (BYOD), and so forth.

Everyone jumps on these big trends. So it's very confusing for customers, because there are many different ways to come at the problem. This is why I keep bringing people back to staying focused on what the real opportunity is. It’s a business opportunity, not a technical problem or a technical challenge that we start with.

Gardner: Right. We don’t want to lose the track of the ends because the means seem to be so daunting. We want to keep our focus on the ends and then find the means. Before we go into our myths, tell me a little bit, Darin, about your background and how you came to be at Dell.

Bartik: I've been a part of Dell Software since the acquisition of Quest Software. I was a part of that organization for close to 10 years. I've been in technology coming up on 20 years now. I spent a lot of time in enterprise resource planning (ERP), supply chain, and monitoring, performance management, and infrastructure management, especially on the Microsoft side of the world.

Most recently, as part of Quest, I was running the database management area -- a business very well-known for its products around Oracle, especially Toad, as well as our SQL Server management capabilities. We leveraged that expertise when we started to evolve into BI and analytics.

I started working with Hadoop back in 2008-2009, when it was still very foreign to most people. When Dell acquired Quest, I came in and had the opportunity to take over the Products Group in the ever-expanding world of information management. We're part of the Dell Software Group, which is a big piece of the strategy for Dell over all, and I'm excited to be here.
It’s not a size issue. It's really a trend that has happened as a result of digitizing so much more of the information that we all have already.

Gardner: Great. Even the name "big data" stirs up myths right from the get-go, with "big" being a very relative term. Should we only be concerned about this when we have more data than we can manage? What is the relative position of big data and what are some of the myths around the size issue?

Bartik: That’s the perfect one to start with. The first word in the definition is actually part of the problem. "Big." What does big mean? Is there a certain threshold of petabytes that you have to get to? Or, if you're dealing with petabytes, is it not a problem until you get to exabytes

It’s not a size issue. When I think about big data, it's really a trend that has happened as a result of digitizing so much more of the information that we all have already and that we all produce. Machine data, sensor data, all the social media activities, and mobile devices are all contributing to the proliferation of data.

It's added a lot more data to our universe, but the real opportunity is to look for small elements of small datasets and look for combinations and patterns within the data that help answer those business questions that I was referencing earlier.

It's not necessarily a scale issue. What is a scale issue is when you get into some of the more complicated analytical processes and you need a certain data volume to make it statistically relevant. But what customers first want to think about is the business problems that they have. Then, they have to think about the datasets that they need in order to address those problems.

Big-data challenge

That may not be huge data volumes. You mentioned mid-market earlier. When we think about some organizations moving from gigabytes to terabytes, or doubling data volumes, that’s a big data challenge in and of itself.

Analyzing big data won't necessarily contribute to your solving your business problems if you're not starting with the right questions. If you're just trying to store more data, that’s not really the problem that we have at hand. That’s something that we can all do quite well with current storage architectures and the evolving landscape of hardware that we have.

We all know that we have growing data, but the exact size, the exact threshold that we may cross, that’s not the relevant issue.

Gardner: I suppose this requires prioritization, which has to come from the business side of the house. As you point out, some statistically relevant data might be enough. If you can extrapolate and you have enough to do that, fine, but there might be other areas where you actually want to get every little bit of possible data or information relevant, because you don't know what you're looking for. They are the unknown unknowns. Perhaps there's some mythology about all data. It seems to me that what’s important is the right data to accomplish what it is the business wants.

Bartik: Absolutely. If your business challenge is an operational efficiency or a cost problem, where you have too much cost in the business and you're trying to pull out operational expense and not spend as much on capital expense, you can look at your operational data.
There's a lot of variability and prioritization that all starts with that business issue that you're trying to address.

Maybe manufacturers are able to do that and analyze all of the sensor, machine, manufacturing line, and operational data. That's a very different type of data and a very different type of approach than looking at it in terms of sales and marketing.

If you're a retailer looking for a new set of customers or new markets to enter in terms of geographies, you're going to want to look at maybe census data and buying-behavior data of the different geographies. Maybe you want datasets that are outside your organization entirely. You may not have the data in your hands today. You may have to pull it in from outside resources. So there's a lot of variability and prioritization that all starts with that business issue that you're trying to address.

Gardner: Perhaps it's better for the business to identify the important data, rather than the IT people saying it’s too big or that big means we need to do something different. It seems like a business term rather than a tech term at this point.

Bartik: I agree with you. The more we can focus on bringing business and IT to the table together to tackle this challenge, the better. And it does start with the executive management in the organization trying to think about things from that business perspective, rather than starting with the IT infrastructure management team. 

Gardner: What’s our second myth?

Bartik: I'd think about the idea of people and the skills needed to address this concept of big data. There is the term "data scientist" that has been thrown out all over the place lately. There’s a lot of discussion about how you need a data scientist to tackle big data. But “big data” isn't necessarily the way you should think about what you’re trying to accomplish. Instead, think about things in terms of being more data driven, and in terms of getting the data you need to address the business challenges that you have. That’s not always going to require the skills of a data scientist.

Data scientists rare

I suspect that a lot of organizations would be happy to hear something like that, because data scientists are very rare today, and they're very expensive, because they are rare. Only certain geographies and certain industries have groomed the true data scientist. That's a unique blend between a data engineer and someone like an applied scientist, who can think quite differently than just a traditional BI developer or BI programmer.

Don’t get stuck on thinking that, in order to take on a data-driven approach, you have to go out and hire a data scientist. There are other ways to tackle it. That’s where you're going to combine people who can do the programming around your information, around the data management principles, and the people who can ask and answer the open-minded business questions. It doesn’t all have to be encapsulated into that one magical person that’s known now as the data scientist.

Gardner: So rather than thinking we need to push the data and analytics and the ability to visualize and access this through a small keyhole, which would be those scientists, the PhDs, the white lab coats, perhaps there are better ways now to make those visualizations and allow people to craft their own questions against the datasets. That opens the door to more types of people being able to do more types of things. Does that sum it up a bit?

Bartik: I agree with that. There are varying degrees of tackling this problem. You can get into very sophisticated algorithms and computations for which a data scientist may be the one to do that heavy lifting. But for many organizations and customers that we talk to everyday, it’s something where they're taking on their first project and they are just starting to figure out how to address this opportunity.

For that, you can use a lot of the people that you have inside your organization, as well potentially consultants that can just help you break through some of the old barriers, such as thinking about intelligence, based strictly on a report and a structured dashboard format.
Often a combination of programming and some open-minded thinking, done with a  team-oriented approach, rather than that single keyhole person, is more than enough to accomplish your objectives.

That’s not the type of approach we want to take nowadays. So often a combination of programming and some open-minded thinking, done with a  team-oriented approach, rather than that single keyhole person, is more than enough to accomplish your objectives.

Gardner: It seems also that you're identifying confusion on the part of some to equate big data with BI and BI with big data. The data is a resource that the BI can use to offer certain values, but big data can be applied to doing a variety of other things. Perhaps we need to have a sub-debunking within this myth, and that is that big data and BI are different. How would you define them and separate them?

Bartik: That's a common myth. If you think about BI in its traditional, generic sense, it’s about gaining more intelligence about the business, which is still the primary benefit of the opportunity this trend of big data presents to us. Today, I think they're distinct, but over time, they will come together and become synonymous.

I equate it back to one of the more recent trends that came right before big data, cloud. In the beginning, most people thought cloud was the public-cloud concept. What’s turned out to be true is that it’s more of a private cloud or a hybrid cloud, where not everything moved from an on-premise traditional model, to a highly scalable, highly elastic public cloud. It’s very much a mix.

They've kind of come together. So while cloud and traditional data centers are the new infrastructure, it’s all still infrastructure. The same is true for big data and BI, where BI, in the general sense of how can we gain intelligence and make smarter decisions about our business, will include the concept of big data.

Better decisions

So while we'll be using new technologies, which would include Hadoop, predictive analytics, and other things that have been driven so much faster by the trend of big data, we’ll still be working back to that general purpose of making better decisions.

One of the reasons they're still different today is because we’re still breaking some of the traditional mythology and beliefs around BI -- that BI is all about standard reports and standard dashboards, driven by IT. But over time, as people think about business questions first, instead of thinking about standard reports and standard dashboards first, you’ll see that convergence.

Gardner: We probably need to start thinking about BI in terms of a wider audience, because all the studies I've seen don't show all that much confidence and satisfaction in the way BI delivers the analytics or the insights that people are looking for. So I suppose it's a work in progress when it comes to BI as well.

Bartik: Two points on that. There has been a lot of disappointment around BI projects in the past. They've taken too long, for one. They've never really been finished, which of course, is a problem. And for many of the business users who depend on the output of BI -- their reports, their dashboard, their access to data -- it hasn’t answered the questions in the way that they may want it to.

One of the things in front of us today is a way of thinking about it differently. Not only is there so much data, and so much opportunity now to look at that data in different ways, but there is also a requirement to look at it faster and to make decisions faster. So it really does break the old way of thinking.
People are trying to make decisions about moving the business forward, and they're being forced to do it faster.

Slowness is unacceptable. Standard reports don't come close to addressing the opportunity in front us, which is to ask a business question and answer it with the new way of thinking supported by pulling together different datasets. That’s fundamentally different from the way we used to do it.

People are trying to make decisions about moving the business forward, and they're being forced to do it faster. Historical reporting just doesn't cut it. It’s not enough. They need something that’s much closer to real time. It’s more important to think about open-ended questions, rather than just say, "What revenue did I make last month, and what products made that up?" There are new opportunities to go beyond that.

Gardner: I suppose it also requires more discipline in keeping your eye on the ends, rather than getting lost in the means. That also is a segue to our next myth, which is, if I have the technology to do big data, then I'm doing big data, and therefore I'm done.

Bartik: Just last week, I was meeting with a customer and they said, "Okay, we have our Hadoop cluster set up and we've loaded about 10 terabytes of sample data into this Hadoop cluster. So we've started our big data project."

When I hear something like that, I always ask, "What question are you trying to answer? Why did you load that data in there? Why did you start with Hadoop? Why did you do all this?" People are starting with the technology first too often. They're not starting with the questions and the business problems first.

Not the endgame

You said as far as making sure that you keep your eye on the endgame, the endgame is not to spin up a new technology, or to try a new tool. Hadoop has been one of those things where people have started to use that and they think that they're off and running on a big-data project. It can be part of it, but it isn't where you want to start, and it isn’t the endgame.

The endgame is solving the business problem that you're out there trying to address. It’s either lowering costs inside the business, or it’s finding a new market, figuring out why this customer set loves our products and why some other customer set doesn’t. Answering those questions is the endgame, not starting a new technology initiative.

Gardner: When it comes to these technology issues, do you also find, Darin, that there is a lack of creativity as to where the data and information resides or exists and thinking not so much about being able to run it, but rather acquire it? Is there a dissonance between the data I have and the data I need. How are people addressing that?

Bartik: There is and there isn’t. When we look at the data that we have, that’s oftentimes a great way to start a project like this, because you can get going faster and it’s data that you understand. But if you think that you have to get data from outside the organization, or you have to get new datasets in order to answer the question that’s in front of us, then, again, you're going in with a predisposition to a myth.

You can start with data that you already have. You just may not have been looking at the data that you already have in the way that’s required to answer the question in front of you. Or you may not have been looking at it all. You may have just been storing it, but not doing anything with it.
Storing data doesn’t help you answer questions. Analyzing it does.

Storing data doesn’t help you answer questions. Analyzing it does. It seems kind of simple, but so many people think that big data is a storage problem. I would argue it's not about the storage. It’s like backup and recovery. Backing up data is not that important, until you need to recover it. Recovery is really the game changing thing.

Gardner: It’s interesting that with these myths, people have tended, over the years, without having the resources at hand,  to shoot from the hip and second-guess. People who are good at that and businesses that have been successful have depended on some luck and intuition. In order to take advantage of big data, which should lead you to not having to make educated guesses, but to have really clear evidence, you can apply the same principle. It's more how you get big data in place, than how you would use the fruits of big data.

It seems like a cultural shift we have to make. Let’s not jump to conclusions. Let’s get the right information and find out where the data takes us.

Bartik: You've hit on one of the biggest things that’s in front of us over the next three to five years -- the cultural shift that the big data concept introduces.

We looked at traditional BI as more of an IT function, where we were reporting back to the business. The business told us exactly what they wanted, and we tried to give that to them from the IT side of the fence.

Data-driven organization

But being successful today is less about intuition and more about being a data-driven organization, and, for that to happen, I can't stress this one enough, you need executives who are ready to make decisions based on data, even if the data may be counter intuitive to what their gut says and what their 25 years of experience have told them.

They're in a position of being an executive primarily because they have a lot of experience and have had a lot of success. But many of our markets are changing so frequently and so fast, because of new customer patterns and behaviors, because of new ways of customers interacting with us via different devices. Just think of the different ways that the markets are changing. So much of that historical precedence no longer really matters. You have to look at the data that’s in front of us.

Because things are moving so much faster now, new markets are being penetrated and new regions are open to us. We're so much more of a global economy. Things move so much faster than they used to. If you're depending on gut feeling, you'll be wrong more often than you'll be right. You do have to depend on as much of a data-driven decision as you can. The only way to do that is to rethink the way you're using data.

Historical reports that tell you what happened 30 days ago don't help you make a decision about what's coming out next month, given that your competition just introduced a new product today. It's just a different mindset. So that cultural shift of being data-driven and going out and using data to answer questions, rather than using data to support your gut feeling, is a very big shift that many organizations are going to have to adapt to.

Executives who get that and drive it down into the organization, those are the executives and the teams that will succeed with big data initiatives, as opposed to those that have to do it from the bottom up.
It's fair to say that big data is not just a trend; it's a reality. And it's an opportunity for most organizations that want to take advantage of it.

Gardner: Listening to you Darin, I can tell one thing that isn’t a product of hype is just how important this all is. Getting big data right, doing that cultural shift, recognizing trends based on the evidence and in real-time as much as possible is really fundamental to how well many businesses will succeed or not.

So it's not hype to say that big data is going to be a part of your future and it's important. Let's move towards how you would start to implement or change or rethink things, so that you can not fall prey to these myths, but actually take advantage of the technologies, the reduction in costs for many of the infrastructures, and perhaps extend and exploit BI and big data problems.

Bartik: It's fair to say that big data is not just a trend; it's a reality. And it's an opportunity for most organizations that want to take advantage of it. It will be a part of your future. It's either going to be part of your future, or it's going to be a part of your competition’s future, and you're going to be struggling as a result of not taking advantage of it.

The first step that I would recommend -- I've said it a few times already, but I don't think it can't be said too often -- is pick a project that's going to address a business issue that you've been unable to address in the past.

What are the questions that you need to ask and answer about your business that will really move you forward?" Not just, "What data do we want to look at?" That's not the question.

What business issue?

The question is what business issue do we have in front of us that will take us forward the fastest? Is it reducing costs? Is it penetrating a new regional market? Is it penetrating a new vertical industry, or evolving into a new customer set?

These are the kind of questions we need to ask and the dialogue that we need to have. Then let's take the next step, which is getting data and thinking about the team to analyze  it and the technologies to deploy. But that's the first step – deciding what we want to do as a business.

That sets you up for that cultural shift as well. If you start at the technology layer, if you start at the level of let's deploy Hadoop or some type of new technology that may be relevant to the equation, you're starting backwards. Many people do it, because it's easier to do that than it is to start an executive conversation and to start down the path of changing some cultural behavior. But it doesn’t necessarily set you up for success.

Gardner: It sounds as if you know you're going on a road trip and you get yourself a Ferrari, but you haven't really decided where you're going to go yet, so you didn’t know that you actually needed a Ferrari.

Bartik: Yeah. And it's not easy to get a tent inside a Ferrari. So you have to decide where you're going first. It's a very good analogy.
Get smart by going to your peers and going to your industry influencer groups and learning more about how to approach this.

Gardner: What are some of the other ways when it comes to the landscape out there? There are vendors who claim to have it all, everything you need for this sort of thing. It strikes me that this is more of an early period and that you would want to look at a best-of-breed approach or an ecosystem approach.

So are there any words of wisdom in terms of how to think about the assets, tools, approaches, platforms, what have you, or not to limit yourself in a certain way?

Bartik: There are countless vendors that are talking about big data and offering different technology approaches today. Based on the type of questions that you're trying to answer, whether it's more of an operational issue, a sales market issue, HR, or something else, there are going to be different directions that you can go in, in terms of the approaches and the technologies used.

I encourage the executives, both on the line-of-business side as well as the IT side, to go to some of the events that are the "un-conferences," where we talk about the big-data approach and the technologies. Go to the other events in your industry where they're talking about this and learn what your peers are doing. Learn from some of the mistakes that they've been making or some of the successes that they've been having.

There's a lot of success happening around this trend. Some people certainly are falling into the pitfalls, but get smart by going to your peers and going to your industry influencer groups and learning more about how to approach this.

Technical approaches

There are technical approaches that you can take. There are different ways of storing your data. There are different ways of computing and processing your data. Then, of course, there are different analytical approaches that get more to the open-ended investigation of data. There are many tools and many products out there that can help you do that.

Dell has certainly gone down this road and is investing quite heavily in this area, with both structured and unstructured data analysis, as well as the storage of that data. We're happy to engage in those conversations as well, but there are a lot of resources out there that really help companies understand and figure out how to attack this problem.

Gardner: In the past, with many of the technology shifts, we've seen a tension and a need for decision around best-of-breed versus black box, or open versus entirely turnkey, and I'm sure that's going to continue for some time.

But one of the easier ways or best ways to understand how to approach some of those issues is through some examples. Do we have any use cases or examples that you're aware of, of actual organizations that have had some of these problems? What have they put in place, and what has worked for them?
There are a lot of resources out there that really help companies understand and figure out how to attack this problem.

Bartik: I'll give you a couple of examples from two very different types of organizations, neither of which are huge organizations. The first one is a retail organization, Guess Jeans. The business issue they were tackling was, “How do we get more sales in our retail stores? How do we get each individual that's coming into our store to purchase more?”

We sat down and started thinking about the problem. We asked what data would we need to understand what’s happening? We needed data that helps us understand the buyer’s behavior once they come into the store. We don't need data about what they are doing outside the store necessarily, so let's look specifically at behaviors that take place once they get into the store.

We helped them capture and analyze video monitoring information. Basically it followed each of the people in the store and geospatial locations inside the store, based on their behavior. We tracked that data and then we compared against questions like did they buy, what did they buy, and how much did they buy. We were able to help them determine that if you get the customer into a dressing room, you're going to be about 50 percent more likely to close transactions with them.

So rather than trying to give incentives to come into the store or give discounts once they get into the store, they moved towards helping the store clerks, the people who ran the store and interacted with the customers, focus on getting those customers into a dressing room. That itself is a very different answer than what they might have thought of at first. It seems easy after you think about it, but it really did make a significant business impact for them in rather short order.

Now, they're also thinking about other business challenges that they have and other ways of analyzing data and other datasets, based on different business challenges, but that’s one example.

Another example is on the higher education side. In universities, one of the biggest challenges is having students drop out or reduce their class load. The fewer classes they take, or if they dropout entirely, it obviously goes right to the top and bottom line of the organization, because it reduces tuition, as well as the other extraneous expenses that students incur at the university.

Finding indicators

The University of Kentucky went on an effort to reduce students dropping out of classes or dropping entirely out of school. They looked at a series of datasets, such as demographic data, class data, the grades that they were receiving, what their attendance rates were, and so forth. They analyzed many different data points to determine the indicators of a future drop out.

Now, just raising the student retention rate by one percent would in turn mean about $1 million of top-line revenue to the university. So this was pretty important. And in the end, they were able to narrow it down to a couple of variables that strongly indicated which students were at risk, such that they could then proactively intervene with those students to help them succeed.

The key is that they started with a very specific problem. They started it from the university's core mission: to make sure that the students stayed in school and got the best education, and that's what they are trying to do with their initiative. It turned out well for them.

These were very different organizations or business types, in two very different verticals, and again, neither are huge organizations that have seas of data. But what they did are much more manageable and much more tangible examples  many of us can kind of apply to our own businesses.

Gardner: Those really demonstrate how asking the right questions is so important.
What we have today is a set of capabilities that help customers take more of a data-type agnostic view and a vendor agnostic view to the way they're approaching data and managing data.

Darin, we're almost out of time, but I did want to see if we could develop a little bit more insight into the Dell Software road map. Are there some directions that you can discuss that would indicate how organizations can better approach these problems and develop some of these innovative insights in business?

Bartik: A couple of things. We've been in the business of data management, database management, and managing the infrastructure around data for well over a decade. Dell has assembled a group of companies, as well as a lot of organic development, based on their expertise in the data center for years. What we have today is a set of capabilities that help customers take more of a data-type agnostic view and a vendor agnostic view to the way they're approaching data and managing data.

You may have 15 tools around BI. You may have tools to look at your Oracle data, maybe new sets of unstructured data, and so forth. And you have different infrastructure environments set up to house that data and manage it. But the problem is that it's not helping you bring the data together and cross boundaries across data types and vendor toolset types, and that's the challenge that we're trying to help address.

We've introduced tools to help bring data together from any database, regardless of where it may be sitting, whether it's a data warehouse, a traditional database, a new type of database such as Hadoop, or some other type of unstructured data store.

We want to bring that data together and then analyze it. Whether you're looking at more of a traditional structured-data approach and you're exploring data and visualizing datasets that many people may be working with, or doing some of the more advanced things around unstructured data and looking for patterns, we’re focused on giving you the ability to pull data from anywhere.

Using new technologies

We're investing very heavily, Dana, into the Hadoop framework to help customers do a couple of key things. One is helping the people that own data today, the database administrators, data analysts, the people that are the stewards of data inside of IT, advance their skills to start using some of these new technologies, including Hadoop.

It's been something that we have done for a very long time, making your C players B players, and your B players A players. We want to continue to do that, leverage their existing experience with structured data, and move them over into the unstructured data world as well.

The other thing is that we're helping customers manage data in a much more pragmatic way. So if they are starting to use data that is in the cloud, via Salesforce.com or Taleo, but they also have data on-prem sitting in traditional data stores, how do we integrate that data without completely changing their infrastructure requirements? With capabilities that Dell Software has today, we can help integrate data no matter where it sits and then analyze it based on that business problem.

We help customers approach it more from a pragmatic view, where you're  taking a stepwise approach. We don't expect customers to pull out their entire BI and data-management infrastructure and rewrite it from scratch on day one. That's not practical. It's not something we would recommend. Take a stepwise approach. Maybe change the way you're integrating data. Change the way you're storing data. Change, in some perspective, the way you're analyzing data between IT and the business, and have those teams collaborate.
But you don't have to do it all at one time. Take that stepwise approach.

But you don't have to do it all at one time. Take that stepwise approach. Tackle it from the business problems that you're trying to address, not just the new technologies we have in front of us.

There's much more to come from Dell in the information management space. It will be very interesting for us and  for our customers to tackle this problem together. We're excited to make it happen.

Gardner: Well, great. I'm afraid we'll have to leave it there. We've been listening to a sponsored BriefingsDirect podcast discussion on debunking some major myths around big data use and value. We've seen how big data is not necessarily limited by scale and that the issues around  it don't always have to supersede the end for your business goals.

We've also learned more about levels of automation and how Dell is going to be approaching the market. So I appreciate that. With that, we'll have to end it and thank our guest.

We've been here with Darin Bartik, Executive Director of Products in the Information Management Group at Dell Software. Thanks so much, Darin.

Bartik: Thank you, Dana, I appreciate it.

Gardner: This is Dana Gardner, Principal Analyst at Interarbor Solutions. Thanks also to our audience for joining and listening, and don't forget to come back next time.

Listen to the podcast. Find it on iTunes. Download the transcript. Sponsor: Dell Software.

Transcript of a BriefingsDirect podcast on current misconceptions about big data and how organizations should best approach a big-data project.  Copyright Interarbor Solutions, LLC, 2005-2013. All rights reserved.

You may also be interested in:

Thursday, April 15, 2010

Information Management Takes Aim at Need for Improved Business Insights From Complex Data Sources

Transcript of a sponsored BriefingsDirect podcast on how companies are leveraging information management solutions to drive better business decisions in real time.

Listen to the podcast. Find it on iTunes/iPod and Podcast.com. Download the transcript. Learn more. Sponsor: HP.

Get a free white paper on how Business Intelligence enables enterprises to better manage data and information assets:
Top 10 trends in Business Intelligence for 2010

Dana Gardner: Hi, this is Dana Gardner, principal analyst at Interarbor Solutions, and you’re listening to BriefingsDirect. Today's sponsored podcast discussion delves into how to better harness the power of information to drive and improve business insights.

We’ll examine how the tough economy has accelerated the progression toward more data-driven business decisions. To enable speedy proactive business analysis, information management (IM) has arisen as an essential ingredient for making business intelligence (BI) for these decisions pay off.

Yet IM itself can become unwieldy, as well as difficult to automate and scale. So managing IM has become an area for careful investment. Where then should those investments be made for the highest analytic business return? How do companies better compete through the strategic and effective use of its information?

We’ll look at some use case scenarios with executives from HP to learn how effective IM improves customer outcomes, while also identifying where costs can be cut through efficiency and better business decisions.

To get to the root of IM best practices and value, please join me in welcoming our guests, Brooks Esser, Worldwide Marketing Lead for Information Management Solutions at HP. Welcome, Brooks.

Brooks Esser: Hi, Dana. How are you today?

Gardner: I’m great. We’re also here with John Santaferraro, Director of Marketing and Industry Communications for BI Solutions at HP. Hello, John.

John Santaferraro: Hi Dana. I’m glad to be here, and hello to everyone tuning into the podcast.

Gardner: And also, we’re here with Vickie Farrell, Manager of Market Strategy for BI Solutions at HP. Welcome to the show.

Vickie Farrell: Hi, Dana, thanks.

Gardner: Let me take our first question out to John. IM and BI in a sense come together. It’s sort of this dynamic duo in this era of cost consciousness and cost-cutting. What is it about the two together that you think is the right mix for today’s economy?

Santaferraro: Well, it’s interesting, because the customers that we work with tend to have very complex businesses, and because of that, very complex information requirements. It used to be that they looked primarily at their structured data as a source of insight into the business. More recently, the concern has moved well beyond business intelligence to look at a combination of unstructured data, text data, IM. There’s just a whole lot of different sources of information.

Enterprise IM

The idea that they can have some practices across the enterprise that would help them better manage information and produce real value and real outcomes for the business is extremely relevant. I’d like to think of it as actually enterprise IM.

Very simply, first of all, it’s enterprise, right? It’s looking across the entire business and being able to see across the business. It’s information, all types of information as we identify structured, unstructured documents, scanned documents, video assets, media assets.

Then it’s the management, the effective management of all of those information assets to be able to produce real business outcomes and real value for the business.

Gardner: So the more information you can manage to bring into an analytics process, the higher the return?

Santaferraro: I don’t know that it’s exactly just "more." It’s the fact that, if you look at the information worker or the person who has to make decisions on the front line, if you look at those kinds of people, the truth is that most of them need more than just data and analysis. In a lot of cases, they will need a document, a contract. They need all of those different kinds of data to give them different views to be able to make the right decision.

Gardner: Brooks, tell me a little bit about how you view IM. Is this a life cycle we’re talking about? Is it a category? Where do we draw the boundaries around IM? Is HP taking an umbrella concept here?

Esser: We really are, Dana. We think of IM as having four pillars. The first is the infrastructure, obviously -- the storage, the data warehousing, information integration that kind of ties the infrastructure together. The second piece, which is very important, is governance. That includes things like data protection, master data management, compliance, and e-discovery.

The third, to John’s point earlier, is information processes. We start talking about paper-based information, digitizing documents, and getting them into the mix. Those first three pillars taken together really form the basis of an IM environment. They’re really the pieces that allow you to get the data right.

The fourth pillar, of course, is the analytics, the insight that business leaders can get from the analytics about the information. The two, obviously, go hand in hand. Rugged information infrastructure for your analytics isn’t any better than poor infrastructure with solid analytics. Getting both pieces of that right is very, very important.

Gardner: Vickie, if we take that strong infrastructure and those strong analytics and we do it properly, are we able to take the fruits of that out to a wider audience? Let’s say we are putting these analytics into the hands of more people that can take action.

Very important

Farrell: Yes, it is very important that you do both of those things. A couple of years ago, I remember, a lot of pundits were talking about BI becoming pervasive, because tools have gotten more affordable and easier to use. Therefore anybody with a smartphone or PDA or laptop computer was going to be able to do heavy-duty analysis.

Of course, that hasn’t happened. There is more limiting the wide use of BI than the tools themselves. One of the biggest issues is the integration of the data, the quality of the data, and having a data foundation in an environment where the users can really trust it and use it to do the kind of analysis that they need to do.

What we’ve seen in the last couple of years is serious attention on investing in that data structure -- getting the data right, as we put it. It's establishing a high level of data quality, a level of trust in the data for users, so that they are able to make use of those tools and really glean from that data the insight and information that they need to better manage their business.

Esser: We can’t overemphasize that, Dana. There's a great quote by Mark Twain, of all people, who said it isn’t what you don’t know that gets you into trouble -- it’s what you know for certain that just isn’t so. That really speaks to the point Vickie made about quality of data and the importance of having high-quality data in our analytics.

Gardner: We’re defining IM fairly broadly here, but how do we then exercise what we might consider due diligence in the enterprises -- security, privacy, making the right information available to people and then making sure the wrong people don’t have it? How do you apply that important governance pillar, when we’re talking about such a large and comprehensive amount of information, Brooks?

Esser: I think you have to define governance processes, as you’re building your information infrastructure. That’s the key to everything I talked about earlier -- the pillars of a solid IM environment. One of the key ones is governance, and that talks about protecting data, quality, compliance, and the whole idea of master data management -- limiting access and making sure that right people have access to input data and that data is of high-quality.

Farrell: In fact, we recently surveyed a number of data warehouse and BI users. We found that 81 percent of them either have a formal data governance process in place or they expect to invest in one in the next 12 months. There's a lot of attention on that, as Brooks was talking about.

Gardner: Now, as we also mentioned earlier, the economy is still tough. There is less discretionary spending than we’ve had in quite some time. How do you go to folks and get the rationale for the investment to move in this direction? Is it about cost-cutting? Is it about competitiveness? Is it about getting a better return on their infrastructure investments? John, do you have a sense of how to validate the market for IM?

Santaferraro: It’s really simple. By effectively using the information they have and further leveraging the investments that they’ve already made, there is going to be significant cost savings for the business. A lot of it comes out of just having the right insight to be able to reduce costs overall. There are even efficiencies to be had in the processing of information. It can cost a lot of money to capture data, to store it, and cleanse it.

Cleansing can be up to 70 percent of the cost of the data, trying to figure out your retention strategies. All of that is very expensive. Obviously, the companies that figure out how to streamline the handling and the management of their information are going to have major cost reductions overall.

Gardner: What about the business outcomes? Brooks, do we have a sense of what companies can do with this? If they do it properly, as John pointed out, how does that further vary the profitability, their market penetration, or perhaps even their dominance?

The way to compete

Esser: Dana, it’s really becoming the way that leading edge companies compete. I’ve seen a lot of research that suggests that CEOs are becoming increasingly interested in leveraging data more effectively in their decision-making processes. It used to be fairly simple. You would simply identify your best customers, market like heck to them, and try to maximize the revenue derived from your best customers.

Now, what we’re seeing is emphasis on getting the data right and applying analytics to an entire customer base, trying to maximize revenue from a broader customer base. We’re going to talk about a few cases today where entities got the data right, they now serve their customers better, reduced cost at the same time, and increased their profitability.

Gardner: We’ve talked about this at a fairly high level. I wonder if we could get a bit more specific. I’m curious about what is the problem that IM solves that then puts us in a position to leverage the analytics, put it in the hands of the right people, and then take those actions that cut the costs and increase the business outcome. I’m going to throw this out to anybody in our panel. What are the concrete problems that IM sets out to solve?

Esser: I’ll pick that up, Dana. Organizations all over the world are struggling with an expansion of information. In some companies, you’re seeing data doubling one year over the next. It’s creating problems for the storage environment. Managers are looking at processes like de-duplication to try to reduce the quantity of information.

Lots of information is still on paper. You’ve got to somehow get that into the mix, into your decision-making process. Then you have things like RFID tags and sensors adding to the expansion of information. There are legal requirements. When you think about the fact that most documents, even instant messages, are now considered business records, you’ve got to figure a way to capture that.

The challenge for a CIO is that you’ve got to balance the cost of IT, the cost of governance and risk issues involved in information, while at the same time, providing real insight to your business unit customer.



Then, you’re getting pressure from business leaders for timely and accurate information to make decisions with. So, the challenge for a CIO is that you’ve got to balance the cost of IT, the cost of governance and risk issues involved in information, while at the same time, providing real insight to your business unit customer. It’s a tough job.

Santaferraro: If I could throw another one in there, Dana, I recently talked to a couple of senior IT leaders, and both of them were in the same situation. They’ve been doing BI and IM for 10-plus years in their organization. They had fairly mature processes in place, but they were concerned with trying to take the insight that they had gleaned and turn it into action.

Along with all of the things that were just described by Brooks, there are a lot of companies out there that are trying to figure out how to get the data that last mile to the person on the front line who needs to make a decision. How do I get it to them in a very simple format that tells them exactly what they need to do?

So, it’s turning that insight into action, getting it to the teller in a bank, getting it to the clerk at the point of sale, or the ATM machine, or the web portal, when somebody is logging onto a banking system or a retail site.

Along with all of that, there is this new need to find a way to get the data that last mile to where it impacts a decision. For companies, that’s fairly complex, because that could mean millions of decisions every day, as opposed to just getting a report to an executive.

That whole world of the information worker and the need to use the information has changed as well, driving the need for IM.

Analyze the data

Farrell: Dana, you asked what the challenges are, and one that we see a lot is that people need to analyze the data. They'll traipse from data mart to data mart and pull data together manually. It’s time-consuming and it’s expensive. It’s fraught with error, and the fact that you have data stored in all these different data marts, just indicates that you’re going to have redundant data that’s going to be inconsistent.

Another problem is that you’ll end up with reports from different people and different departments, and they won’t match. They will have used different calculations, different definitions for business terms. They will have used different sources for the data. There is really no consistent reconciliation of all of this data and how it gets integrated.

This causes really serious problems for companies. That’s really what IM is going to help people overcome. In some cases, it doesn’t really cost as much as you’d think, because when you do IM properly, you're actually going to see some savings and correction of some of those things that I just talked about.

Gardner: It also seems to me, if you look at a historic perspective, that many of these information workers we're talking about didn’t even try to go after this sort of analytic information. They knew that it wasn’t going to be available to them. They’d probably have to wait in line.

But, if we open the floodgates and make this information available to them, it strikes me that they are going to want to start using it in new and innovative ways. That’s a good thing, but it could also tax the infrastructure and the processes that have been developed.

Without that close alignment between business and IT, a tie of the IT project to real business outcomes, and that constant monitoring by that group, it could easily get out of hand.



How do we balance an expected increase in the proactive seeking of this information? I guess we are starting to talk about the solution to IM. If we're good at it and people want it, how do we scale it? How do we ramp it up? What about that, John? How do we start in on the scaling and the automation aspect of IM?

Santaferraro: With our customers, some of the strategy and planning that we do up front helps them define IM practices internally and create things like an enterprise information competency center where the business is aligned with IT in a way that they are actually preparing for the growth of information usage. Without that close alignment between business and IT, a tie of the IT project to real business outcomes, and that constant monitoring by that group, it could easily get out of hand. The strategy and planning upfront definitely helps out.

Farrell: I'll add to that. The more effectively you bring together the IT people and the business people and get them aligned, the better the acceptance is going to be. You certainly can mandate use of the system, but that’s really not a best practice. That’s not what you want to do.

By making the information easily accessible and relevant to the business users and showing them that they can trust that data, it’s going to be a more effective system, because they are going to be more likely to use it and not just be forced to use it.

Esser: Absolutely, Vickie. When you think about it, it really is the business units within most enterprises that fund activities via a tax or however they manage to pay for these things. Doing it right means having those stakeholders involved from the very beginning of the planning process to make sure they get what they need out of any kind of an IT project.

Access a free white paper on how Business Intelligence enables enterprises to better manage data and information assets:
Top 10 trends in Business Intelligence for 2010

Gardner: It strikes me that we have a real virtuous cycle at work here, where the more people get access to better information, the more action they can take on, the more value is perceived in the information, the more demand for the information, the more that the IT folks can provide it and then so on and so forth.

Has anybody got an example of how that might show up in the real world? Do we have any use cases that capture that virtuous adoption benefit?

Better customer service

Farrell: Well, one comes to mind. It’s an insurance company that we have worked with for several years. It’s a regional health insurance company faced with competition from national companies. They decided that they needed to make better use of their data to provide better services for their members, the patients as well as the providers, and also to create a more streamlined environment for themselves.

And so, to bring the IT and business users together, they developed an enterprise data warehouse that would be a common resource for all of the data. They ensured that it was accurate and they had a certain level of data quality.

They had outsourced some of the health management systems to other companies. Diabetes was outsourced to one company. Heart disease was outsourced to another company. It was expensive. By bringing it in house, they were able to save the money, but they were also able to do a better job, because they could integrate the data from one patient, and have one view of that patient.

That improved the aggregate wellness score overall for all of their patients. It enabled them to share data with the care providers, because they were confident in the quality of that data. It also saved them some administrative cost, and they recouped the investment in the first year.

Gardner: Any other examples, perhaps examples that demonstrate how IM and HP’s approach to IM come together?

More real-time applications and more mission-critical applications are coming and there is not going to be the time to do the manual integration.



Farrell: Another thing that we're doing is working with several health organizations in states in the US. We did one project several years ago and we are now in the midst of another one. The idea here is to integrate data from many different sources. This is health data from clinics, schools, hospitals, and so on throughout the state.

This enables you to do many things like run programs on childhood obesity, for example, assess the effectiveness of the program, and assess the overall cost and the return on the investment of that program. It helps to identify classes of people who need extra help, who are at risk.

Doing this gives you the opportunity to bring together and integrate in a meaningful way data from all these different sources. Once that’s been done, that can serve not only these systems, but also some of the potential systems more real-time systems that we see coming down the line, like emergency surveillance systems that would detect terrorist threat, bioterrorism threats, pandemics, and things like that.

It's important to understand and be able to get this data integrated in a meaningful way, because more real-time applications and more mission-critical applications are coming and there is not going to be the time to do the manual integration that I talked about before.

Gardner: It certainly sounds like a worthwhile thing. It sounds like the return on investment (ROI) is strong and that virtuous adoption is very powerful. So, John Santaferraro, what is that HP does that could help companies get in the IM mode?

Obviously, this is not just something you buy and drop in. It's more than just methodologies as well. What are the key ingredients, and how does HP pull them together?

Bringing information together

Santaferraro: We find that a lot of our customers have very disconnected sets of intelligence and information. So, we look at how we can bring that whole world of information together for them and provide a connected intelligence approach. We are actually a complete provider of enterprise class industry-specific IM solutions.

There are a lot of areas where we drill down and bring in our expertise. We have expertise around several business domains like customer relationship management, risk, and supply chain. We go to market with specific solutions from 13 different industries. As a complete solution provider, we provide everything from infrastructure to financing.

Obviously, HP has all of the infrastructure that a customer needs. We can package their IM solution in a single finance package that hits either CAPEX or OPEX. We've got software offerings. We've got our consulting business that comes in and helps them figure out how to do everything from the strategy that we talked about upfront and planning to the actual implementation.

We can help them break into new areas where we have practices around things like master data management or content management or e-discovery.

Across the entire IM spectrum, we have offerings that will help our customers solve whatever their problems are. I like to approach our customers and say, "Give us your most difficult and complex information challenge and we would love to put you together with people who have addressed those challenges before and with technology that’s able to help you do it and even create innovation as a business."

Everyone in the IM market partners with other firms to some extent.



When we've come in and laid the IM foundation for our customers and given them a solid technology platform -- Neoview is a great example -- we find that they began to look at what they've got. It really triggers a whole lot of brand-new innovation for companies that are doing IM the right way.

Gardner: Given these vertical industries, I imagine there are some partners involved there, a specialist in specific regions as well as specific industries. Brooks, is there an ecosystem at work here as well, and how does that shape up?

Esser: Absolutely, Dana. Everyone in the IM market partners with other firms to some extent. We've chosen some strategic partners that complement our capabilities as well. For example, we team with Informatica for our data integration platform and SAP BusinessObjects and MicroStrategy for our BI platform.

We work with a company called Clearwell, and we leverage their e-discovery platform to deliver a solution that helps customers leverage the information in their corporate email systems. We work with Microsoft to deliver HP Enterprise Content Management Solution. So we really have an excellent group of go-to-market partners to leverage.

Gardner: We've talked about the context of the market, why the economy is important, and we looked at some of the imperatives from a business point of view, why this is essential to compete, what problems you need to overcome, and the solution.

So, in order to get towards this notion of a payback, it's important to know where to get started. There seem to be so many inception points, so many starting points. Let me take this to you, John. The holistic approach of being comprehensive, but at the same time, breaking this into parts that are manageable, how do you do that?


Best practices

Santaferraro: One of the things that we have done is made our best practices available and accessible to our customers. We actually operationalize them. A lot of consulting companies will come and plop a big fat manual on the desk and say we have a methodology.

We've created an offering called the methodology navigator which actually walks the customers through the entire project in an interactive environment, where depending on whatever step of the project they are in, they can click on a little box that represents that step and quickly access templates, accelerators, and best practices that are directly relevant to that particular step.

We look at this holistic approach, but we also break it down into best practices that apply to every single step along the way.

Gardner: This whole thing sounds like a no-brainer to me. I don’t know whether I am overly optimistic, but I can see applying more information to your personal life, your small business as well as your department and then of course, your comprehensive enterprise.

I think we're entering into a data-driven decade. The more data, the more better decisions, the more productivity. It's how you grow. Brooks, why do you think it’s a no-brainer? Am I overstating the case?

It's how leading edge companies are going to compete, particularly in a tough and the volatile economy.



Esser: I don’t think you are, Dana. It's how leading edge companies are going to compete, particularly in a tough and the volatile economy, as we have seen over the last 5, 7, 8 years. It's really simple. Better information about your customers can help you drive incremental revenue from your existing customer base. The cool part about it is that better information can help you prevent loss of customers that you already have. You know them better and know how to keep them satisfied.

Every marketer knows that it's a lot less expensive to keep a current customer than it is to go out and acquire a new one. So the ROI for IM projects can be phenomenal and, to your point, that makes it kind of a no-brainer.

Gardner: Vickie, we apply this to customers, we apply it to patients, payers, end-users, but are there other directions to point this at? Perhaps supply chain, perhaps thinking about cloud computing and multiple sources of finding social media metadata about processes, customers, suppliers. Are we only scratching the surface in a sense of how we apply IM?

Farrell: I think we probably are. I don’t know that there are any industries that can't make use of better organizing their data and better analyzing their data and making use of that insight that they’ve gained to make better decisions. In fact, across the board, one of the biggest issues that people have is making better decisions.

In some cases, it's providing information to humans through reports or queries, so that they can make the decisions. What we're going to be seeing -- and this gets to what you were talking about -- is that when data is coming in in real time from sensors and things like that, it has location context. It's very rich data, and it provides you with a lot of information and a lot of variables to make the best decisions based on all those variables that are taking place at that time.

Where once we were maybe developing a handful of possible scenarios and picking the closest one, we don’t have to do that anymore. We can really make use of all of that information and make the absolute best decision right then and there. I don’t really think that there are any industries or domains that can't make use of that kind of capability.

Capturing more data

Santaferraro: Dana, I love what we are doing in the oil and gas industry. We have taken the sensors from our printers, and they are some of the most sensitive sensors in the world, and we are doing a project with Shell Oil, where we are actually embedding our sensors at the tip of a drill head.

As it goes down, it's going to capture seismic data that is 100 times more accurate than anything that's been captured in the past. It's going to send it up through a thing called IntelliPipe which is a five-megabyte feed is this correct that goes up through the drill pipe and back up to the well head, where we will be capturing that in real time.

Seismic data tends to be dirty by nature. It needs to be cleansed. So, we're building a real-time cleansing engine to cleanse that data, and then we are capturing it on the back-end in our digital oil field intelligence offering. It's really fun to see as the world changes, there are all these new opportunities for collecting and using information, even in industries that tend to be a little more traditional and mechanical.

Gardner: That's a very interesting point that the more precise we get with instrumentation, the more data, the more opportunity to work with it and then to crunch that in real-time offers us the predictive aspect rather than a reactive aspect.

As I said, it's been compelling and a no-brainier for me. John, you mentioned an on ramp to this, that it's really the methodological approach. Are there any resources, are there places people can go to get more information, to start factoring where in their organization they will get their highest returns, perhaps focus there and then start working outward towards that more holistic benefit?

It's really up to the customers in terms of how they want to start out.



Let me go to you first, Brooks. Where can people go for more information?

Esser: Of course, I'm going to tell folks to talk to their HP reps. In the course of our discussion today, it's pretty obvious that IM projects are huge undertakings, and we understand that. So, we offer a group of assessment and planning services. They can help customers scope out their projects.

We have a couple of ways to get started. We can start with a business value assessment service. This is service that sets people up with a business case and tracks ROI, once they decide on a project. But, the interesting piece of that is they can choose to focus on data integration, master data management, what have you.

You look at the particular element of IM and build a project around that. This assessment service allows people to identify the element in their IM environment, their current environment, that will give them the best ROI. Or, we can offer them a master planning service which generates really comprehensive IM plan, everything from data protection and information quality to advanced analytics.

So, it's really up to the customers in terms of how they want to start out, taking a look at the element of their IM environment, or if they want us to come in and look at the entire environment, we can say, "Here's what you need to do to really transform the entire IM environment."

Obviously, you can get details on those services and our complete portfolio for that matter at www.hp.com/go/bi and www.hp.com/go/im.

Gardner: Vickie, any sense of where you would point people when they ask do I get started, where can I get more information?

Farrell: Well, I think Brooks covered it. All of our information is at www.hp.com/go/bi. We also have another site that's www.hp.com/go/neoview. There is some specific information about the Neoview Advantage enterprise data warehouse platform there.

Gardner: Very well. John Santaferraro, how about from professional services and solutions perceptive; any resources that you have in mind?

Santaferraro: Probably the hottest topic that I have heard from customers in the last year or so has been around the development of the BI competency center. Again if you go to our BI site, you will find some additional information there about the concept of a BICC.

And the other trend that I am seeing is that a lot of companies are wanting to move from just the BI space with that kind of governance. They want to create an enterprise information competency center, so expanding beyond BI to include all of IM.

We have got some great services available to help people set those up. We have customers that have been working in that kind of a governance environment for three or four years. The beautiful thing is that companies that have been doing this for three or four years are doing transformational things for their business.

They are really closely tied to business mission, vision, and objective, versus other companies that are doing a bunch of one-off projects. One customer recently had spent $11 million in a project over the last year, and they were still trying to figure out where they were going to get value out of the project.

Again, heading over to our BI website -- type in BICC, do a search -- there is some great documentation there I think that you will find to help set up some of the governance side.

Gardner: Well great. We've been talking about a natural progression towards data-driven business decisions and using IM to scale that and bring more types of data and content into play. I want to thank our guests for toady's podcast. We've been joined by Brooks Esser, Worldwide Marking Lead for Information Management Solutions at HP. Thank you, Brooks.

Esser: Thanks very much for having me, Dana.

Gardner: John Santaferraro. He is the Director of Marketing and Industry Communications for BI Solutions. Thank you, John.

Santaferraro: Thanks, Dana. Glad to be here.

Gardner: And also, Vickie Farrell, Manager of Market Strategy for BI Solutions. Thanks so much.

Farrell: Thank you, Dana. This is a pleasure.

Gardner: This is Dana Gardner, Principal Analyst at Interarbor Solutions. You’ve been listening to a sponsored BriefingsDirect podcast. Thanks for listening, and come back next time.

Listen to the podcast. Find it on iTunes/iPod and Podcast.com. Download the transcript. Learn more. Sponsor: HP.

Access a free white paper on how Business Intelligence enables enterprises to better manage data and information assets:
Top 10 trends in Business Intelligence for 2010

Transcript of a sponsored BriefingsDirect podcast on how companies are leveraging information management solutions to drive better business decisions in real time. Copyright Interarbor Solutions, LLC, 2005-2010. All rights reserved.

You may also be interested in: