Showing posts with label business intelligence. Show all posts
Showing posts with label business intelligence. Show all posts

Thursday, July 30, 2015

Full 360 Takes Big Data Analysis Cloud Services to New Business Levels

Transcript of a BriefingsDirect discussion on the benefits of joining data analysis and the cloud.

Listen to the podcast. Find it on iTunes. Get the mobile app for iOS or Android. Download the transcript. Sponsor: HP.

Dana Gardner: Hello, and welcome to the next edition of the HP Discover Podcast Series. I'm Dana Gardner, Principal Analyst at Interarbor Solutions, your host and moderator for this ongoing discussion on IT innovation.

Gardner
Our next cloud case study interview highlights how Full 360 uses big data and analytics to improve their financial operations. To learn how, we're joined by  Eric Valenzuela, Director of Business Development at Full 360, based in New York. Welcome, Eric.

Eric Valenzuela: Good morning. Thank you for having me.

Gardner: Tell us about Full 360 and the role it plays in the financial sector. [Register for the upcoming HP Big Data Conference in Boston on Aug. 10-13.]

Valenzuela: Full 360 is a consulting and services firm, and we purely focus on data warehousingbusiness intelligence (BI), and hosted solutions. We build and consult and then we do managed services for hosting those complex, sophisticated solutions in the cloud, in the Amazon cloud specifically.
Become a member of myVertica today
Register now
Gain access to the free HP Vertica Community Edition
Gardner: And why is cloud a big differentiator for this type of service in the financial sector?

Valenzuela: It’s not necessarily just for finance. It seems to be beneficial for any company that has a large initiative around data warehouse and BI. For us, specifically, the cloud is a platform that we can develop our scripts and processes around. That way, we can guarantee 100 percent that we're providing the same exact service to all of our customers.

Valenzuela
We have quite a bit of intellectual property (IP) that’s wrapped up inside our scripts and processes. The cloud platform itself is a good starting point for a lot of people, but it also has elasticity for those companies that continue to grow and add to their data warehousing and BI solutions.

Gardner: Eric, it sounds as if you've built your own platform as a service (PaaS) for your specific activities and development and analytics on top of a public cloud infrastructure. Is that fair to say?

Valenzuela: That’s a fair assumption.

Primary requirements

Gardner: So as you are doing this cloud-based analytic service, what is it that your customers are demanding of you? What are the primary requirements you fulfill for them with this technology and approach?

Valenzuela: With data warehousing being rather new, Vertica specifically, there is a lack of knowledge out there in terms of how to manage it, keep it up and running, tune it, analyze queries and make sure that they're returning information efficiently, that kind of thing. What we try to do is to supplement that lack of expertise.

Gardner: Leave the driving to us, more or less. You're the plumbers and you let them deal with the proper running water and other application-level intelligence?

Valenzuela: We're like an insurance policy. We do all the heavy lifting, the maintenance, and the management. We ensure that your solution is going to run the way that you expect it to run. We take the mundane out, and then give the companies the time to focus on building intelligent applications, as opposed to worrying about how to keep the thing up and running, tuned, and efficient.

Gardner: Given that Wall Street has been crunching numbers for an awfully long time, and I know that they have, in many ways, almost unlimited resources to go at things like BI -- what’s different now than say 5 or 10 years ago? Is there more of a benefit to speed and agility versus just raw power? How has the economics or dynamics of Wall Street analytics changed over the past few years?
We're like an insurance policy. We do all the heavy lifting, the maintenance, and the management.

Valenzuela: First, it’s definitely the level of data. Just 5 or 10 years ago, either you had disparate pieces of data or you didn’t have a whole lot of data. Now it seems like we are just managing massive amounts of data from different feeds, different sources. As that grows, there has to be a vehicle to carry all of that, where it’s limitless in a sense.

Early on, it was really just a lack of the volume that we have today. In addition to that, 8 or 10 years ago BI was still rather new in what it could actually do for a company in terms of making agile decisions and informed decisions, decisions with intent.

So fast forward, and it’s widely accepted and adopted now. It’s like the cloud. When cloud first came out, everybody was concerned about security. How are we going to get the data in there? How are we going to stand this thing up? How are we going to manage it? Those questions come up a lot less now than they did even two years ago. [Register for the upcoming HP Big Data Conference in Boston on Aug. 10-13.]

Gardner: While you may have cut your teeth on Wall Street, you seem to be branching out into other verticals -- gaming, travel, logistics. What are some of the other areas now to which you're taking your services, your data warehouse, and your BI tools?

Following the trends

Valenzuela: It seems like we're following the trends. Recently it's been gaming. We have quite a few gaming customers that are just producing massive amounts of data.

There's also the airline industry. The customers that we have in airlines, now that they have a way to -- I hate this term -- slice and dice their data, are building really informed, intelligent applications to service their customers, customer appreciation. It’s built for that kind of thing. Airlines are now starting to see what their competition is doing. So they're getting on board and starting to build similar applications so they are not left behind.

Banking was pretty much the first to go full force and adopt BI as a basis for their practice. Finance has always been there. They've been doing it for quite a long time.

Gardner: So as the director of business development, I imagine you're out there saying, "We can do things that couldn’t have been done before at prices that weren’t available before." That must give you almost an unlimited addressable market. How do you know where to go next to sell this?
At first, we were doing a lot of education. Now, it’s just, "Yes, we can do this."

Valenzuela: It’s kind of an open field. From my perspective, I look at the different companies out there that come to me. At first, we were doing a lot of education. Now, it’s just, "Yes, we can do this," because these things are proven. We're not proving any concepts anymore. Everything has already been done, and we know that we can do it.

It is an open field, but we focus purely on the cloud. We expect all of our customers will be in the Amazon cloud. It seems that now I am teaching people a little bit more -- just because it’s cloud, it’s not magic. You still have to do a lot of work. It’s still an infrastructure.
Become a member of myVertica today
Register now
Gain access to the free HP Vertica Community Edition
But we come from that approach and we make sure that the customer is properly aligned with the vision that this is not just a one- or two-month type commitment. We're not just going to build a solution, put it in our pocket, and walk away. We want to know that they're fully committed for 6-12 months.

Otherwise, you're not going to get the benefits of it. You're just going to spend the money and the effort, and you're not really going to get any benefits out of it if you're not going to be committed for the longer period of time. There still are some challenges with the sales and business development.

Gardner: Given this emphasis on selling the cloud model as much as the BI value, you needed to choose an analytics platform that was cloud-friendly and that was also Amazon AWS cloud-friendly. Tell me how Vertica and Amazon -- and your requirements -- came together.

Good timing

Valenzuela: I think it was purely a timing thing. Our CTO, Rohit Amarnath, attended a session at MIT, where Vertica was first announced. So he developed a relationship there.

This was right around the time when Amazon announced that they were offering its public cloud platform, EC2. So it made a lot of sense to look at the cloud as being a vision, looking at the cloud as a platform, looking at column databases as a future way of managing BI and analytics, and then putting the two together.

It was more or less a timing thing. Amazon was there. It was new technology, and we saw the future in that. Analytics was newly adopted. So now you have the column database that we can leverage as well. So blend the two together and start building some platform that hadn’t been done yet.
There are a lot of Vertica customers out there that are going to reach a limitation. That may require procuring more hardware, more IT staff. The cloud aspect removes all of that.

Gardner: What about lessons learned along the way? Are there some areas to avoid or places that you think are more valuable that people might appreciate? If someone were to begin a journey toward a combination of BI, cloud, and vertical industry tool function, what might you tell them to be wary of, or to double-down on?

Valenzuela: We forged our own way. We couldn’t learn from our competitors’ mistakes because we were the ones that were creating the mistakes. We had to to clear those up and learn from our own mistakes as we moved forward.

Gardner: So perhaps a lesson is to be bold and not to be confined by the old models of IT?

Valenzuela: Definitely that. Definitely thinking outside the box and seeing what the cloud can do, focus on forgetting about old IT and then looking at cloud as a new form of IT. Understanding what it cannot do as a basis, but really open up your mind and think about it as to what it can actually do, from an elasticity perspective.

There are a lot of Vertica customers out there that are going to reach a limitation. That may require procuring more hardware, more IT staff. The cloud aspect removes all of that.

Gardner: I suppose it allows you as a director of business development to go downstream. You can find smaller companies, medium-sized enterprises, and say, "Listen, you don’t have to build a data warehouse at your own expense. You can start doing BI based on a warehouse-as-a-service model, pay as you go, grow as you learn, and so forth."

Money concept

Valenzuela: Exactly. Small or large, those IT departments are spending that money anyway. They're spending it on servers. If they are on-premises, the cost of that server in the cloud should be equal or less. That’s the concept.

If you're already spending the money, why not just migrate it and then partner with a firm like us that knows how to operate that. Then, we become your augmented experts, or that insurance policy, to make sure that those things are going to be running the way you want them to, as if it were your own IT department.

Gardner: What are the types of applications that people have been building and that you've been helping them with at Full 360? We're talking about not just financial, but enterprise performance management. What are the other kinds of BI apps? What are some of the killer apps that people have been using your services to do?
I don’t know how that could be driven if it weren’t for analytics and if it weren’t for technology like Vertica to be able to provide that information.

Valenzuela: Specifically, with one of our large airlines, it's customer appreciation. The level of detail on their customers that they're able to bring to the plane, to the flight attendants, in a handheld device is powerful. It’s powerful to the point where you remember that treatment that you got on the plane. So that’s one thing.

That’s something that you don’t get if you fly a lot, if you fly other airlines. That’s just kind of some detail and some treatment that you just don’t get. I don’t know how that could be driven if it weren’t for analytics and if it weren’t for technology like Vertica to be able to provide that information.

Gardner: I'm afraid we'll have to leave it there. You've been learning about how Full 360 uses HP Vertica in the Amazon cloud to provide data warehouse and BI applications and services to its customers from Wall Street to the local airport. [Register for the upcoming HP Big Data Conference in Boston on Aug. 10-13.]

So join me in thanking Eric Valenzuela, Director of Business Development at Full 360 in New York. Thanks so much, Eric.
Become a member of myVertica today
Register now
Gain access to the free HP Vertica Community Edition
Valenzuela: Thank you.

Gardner: And I'd like to thank our audience as well for joining us for this IT innovation discussion. I'm Dana Gardner, Principal Analyst at Interarbor Solutions, your host for this ongoing series of HP-sponsored discussions. Thanks again for listening, and come back next time.

Listen to the podcast. Find it on iTunes. Get the mobile app for iOS or Android. Download the transcript. Sponsor: HP.


Transcript of a BriefingsDirect discussion on the benefits of joining data analysis and the cloud. Copyright Interarbor Solutions, LLC, 2005-2015. All rights reserved.

You may also be interested in:

Monday, December 01, 2014

Hortonworks Accelerates the Big Data Mashup between Hadoop and HP Haven

Transcript of a BriefingsDirect podcast on how companies are beginning to capture large volumes of data for past, present and future analysis capabilities.

Listen to the podcast. Find it on iTunes. Download the transcript. Sponsor: HP.

Dana Gardner: Hello, and welcome to the next edition of the HP Discover Podcast Series. I'm Dana Gardner, Principal Analyst at Interarbor Solutions, your host and moderator for this ongoing sponsored discussion on IT innovation and how it’s making an impact on people’s lives.

Gardner
Once again, we're focusing on how companies are adapting to the New Style of IT to improve IT performance, gain new insights, and deliver better user experiences — as well as better overall business results.

This time, we're coming to you directly from the recent HP Big Data 2014 Conference in Boston to learn directly from IT and business leaders alike how big data changes everything … for IT, for businesses and governments, as well as for you and me.

Our next innovation interview highlights how Hortonworks is now working with HP on the management of very large datasets. We'll hear how these two will integrate into more of the HP Haven family, but also perhaps into the cloud, and to make it easier for developers to access business intelligence (BI) as a service.
Fully experience the HP Vertica analytics platform...
Get the free HP Vertica Community Edition

Become a member of myVertica
To learn more about these ongoing big data trends, we are joined by Mitch Ferguson, Vice President of Business Development at Hortonworks. Welcome, Mitch.

Mitch Ferguson: Thank you, Dana. Pleasure to be here.

Gardner: We’ve heard the news earlier this year about HP taking a $50-million stake in Hortonworks, and Hortonworks' IPO plans. Please fill us in little bit about why Hortonworks and HP are coming together.

Ferguson: There are two core parts to that answer. One is that the majority of Hadoop came out of Yahoo. Hortonworks was formed by the major Hadoop engineers at Yahoo moving to Hortonworks. This was all in complete corporation with Yahoo to help evolve the technology faster. We believe the ecosystem around Hadoop is critical to the success of Hadoop and critical to the success of how enterprises will take advantage of big data.

Ferguson
If you look at HP, a major provider of technology to enterprises, both at the compute and storage level but the data management level, the analytics level, the systems management level, and the complimentary nature of Hadoop as part of the modern data architecture with the HP hardware and software assets provides a very strong foundation for enterprises to create the next generation modern data architecture.

Gardner: I'm hearing a lot about the challenges of getting big data into a single set or managing the large datasets.

Users are also trying to figure out how to migrate from SQL or other data stores into Hadoop and into HP Vertica. It’s a challenge for them to understand a roadmap. How do you see these datasets as they grow larger, and we know they will, in terms of movement and integration? How is that path likely to unfold?

Machine data

Ferguson: Look at the enterprises that have been adapting Hadoop. Very early adopters like eBay, LinkedIn, Facebook, and Twitter are generating significant amounts of machine data. Then we started seeing large enterprises, aggressive users of technology adopt it.

One of the core things is that the majority of data being created everyday in an enterprise is not coming from traditional enterprise resource planning (ERP) or customer relationship management (CRM) financial management systems. It's coming from websites like Clickstream, data, log data, or sensor, data. The reason there is so much interest in Hadoop is that it allows companies to cost effectively capture very large amounts of data.

Then, you begin to understand patterns across semi-structured, structured, and unstructured data to begin to glean value from that data. Then, they leverage that data in other technologies like Vertica, analytics technologies, or even applications or move the data back into the enterprise data warehouse.

As a major player in this Hadoop market, one of the core tenets of the company was that the ecosystem is critical to the success of Hadoop. So, from day one, we’ve worked very closely with vendors like Microsoft, HP, and others to optimize how their technologies work with Hadoop.

SQL has been around for a long time. Many people and enterprises understand SQL. That's a critical access mechanism to get data out of Hadoop. We’ve worked with both HP and Microsoft. Who knows SQL better than anyone? Microsoft. We're trying to optimize how SQL access to Hadoop can be leveraged by existing tools that enterprises know about, analytics tools, data management tools, whatever.

That's just one way that we're looking at leveraging existing integration points or access mechanisms that enterprises are used to, to help them more quickly adopt Hadoop.
The technology like Hadoop is optimized to allow an enterprise to capture very, very large amounts of that data.

Gardner: But isn’t it clear that what happens in many cases is that they run out of gas with a certain type of database and that they seek alternatives? Is that not what's driving the market for Hadoop?

Ferguson: It's not that they're running out of gas with an enterprise data warehouse (EDW) or relational database. As I said earlier, it's the sheer amount of data. By far, the majority of data is not coming from those traditional ERP,  CRM, or transactional systems. As a result, the technology like Hadoop is optimized to allow an enterprise to capture very, very large amounts of that data.

Some of that data may be relevant today. Some of that data may be relevant three months or six months from now, but if I don't start capturing it, I won't know. That's why companies are looking at leveraging Hadoop.

Many of the earlier adopters are looking at leveraging Hadoop to drive a competitive advantage, whether they're providing a high level of customer service, doing things more cost-effectively than their competitors, or selling more to their existing customers.

The reason they're able to do that is because they're now being able to leverage more data that their businesses are creating on a daily basis, understanding that data, and then using it for their business value.

More than size

Gardner: So this is an alternative for an entirely new class of data problem for them in many cases, but there's more than just the size. We also heard that there's interest in moving from a batch approach to a streaming approach, something that HP Vertica is very popular around.

What's the path that you see for Hortonworks and for Hadoop in terms of allowing it to be used in more than a batch sense, perhaps more toward this streaming and real-time analytics approach?

Ferguson: That movement is under way. Hadoop 1.0 was very batch-oriented. We're now in 2.0 and it's not only batch, but interactive and also real-time, and there's a common layer within Hadoop.  Hortonworks is very influential in evolving this technology. It's called YARN. Think of it as a data operating system that is part of Hadoop, and it sits on top of the file system.

Via YARN, applications or integration points, whether they're for batch oriented applications, interactive integration, or real-time like streaming or Spark, are access mechanisms. Then, those payloads or applications, when they leverage Hadoop, will go through these various batch interactive, real-time integration points.

They don't need to worry about where the data resides within Hadoop. They'll get the data via their batch real-time interactive access point, based on what they need. YARN will take advantage of moving that data in and out of those applications. Streaming is just one way of moving data into Hadoop. That's very common for sensor data. It’s also a way to move it out. SQL is a way, among others, to move data.
Fully experience the HP Vertica analytics platform...
Get the free HP Vertica Community Edition

Become a member of myVertica
Gardner: So this is giving us choice about how to manage larger scales of data. We're seeing choice about the way in which we access that data. There's also choice around the type of the underlying infrastructure to reduce costs and increase performance. I am thinking about in-memory or columnar.

What is there about the Hadoop community and Hortonworks, in particular, that allows you to throw the right horsepower at the problem?

Ferguson: It was very important, from Hortonworks perspective from day one, to evolve the Hadoop technology as fast as possible. We decided to do everything in open source to move the technology very quickly and leverage the community effective open-source, meaning lots of different individuals helping to evolve this technology fast.

The ability for the ecosystem to easily and optimally integrate with Hadoop is important. So there are very common integration points. For example, for systems management, there is the Ambari Hadoop services integration point.

Whether it's an HP OpenView or System Center in the Microsoft world, that allows it to leverage, manage, or monitor Hadoop along with other IT assets that those management technologies integrate with.

Access points

Then there's SQL's access via Hive, an access point to allow any technology that integrates or understands SQL to access Hadoop.

Storm and Spark are other access points. So, common open integration points well understood by the ecosystem are really designed to help optimize how various technologies at the virtualization layer, at the operating system layer, data movement, data management, access layer can optimally leverage Hadoop.

Gardner: One of the things that I hear a lot from folks who don't understand yet how things will unfold, is where data and analytics applications align with the creation of other applications or services, perhaps in a cloud setting like a platform as a service (PaaS).

It seems to me that, at some point, more and more application development will be done through PaaS with an associated or integrated cloud. We're also seeing a parallel trajectory here with the data, along the same lines of moving from traditional systems of record into relational, and now into big data and analytics in a cloud setting. It makes a lot of sense.
What a number of people are doing with this concept is called the data lake. They're provisioning large Hadoop clusters on prem, moving large amounts of data into this data lake.

I talked to lot of people about that. So the question, Mitch, is how do we see a commingling and even an intersection between the paths of PaaS in general application development and PaaS in BI services, or BI as a service, somehow relating?

Ferguson: I'll answer that question in two ways. One is about the companies that are using Hadoop today, and using it very aggressively. Their goal is to provide Hadoop as a service, irrespective of whether it's on premises or in the cloud.

Then we'll talk about what we see with HP, for example, with their whole cloud strategy, and how that will evolve into a very interesting hybrid opportunity and maybe pure cloud play.

When you think about PaaS in the cloud, the majority of enterprise data today is on premises. So there's a physics issue of trying to run all of my big data in the cloud. As a result, what a number of people are doing with this concept is called the data lake. They're provisioning large Hadoop clusters on premises, moving large amounts of data into this data lake.

That's providing data as a service to those business units that need data in Hadoop -- structured, semi-structured, unstructured for new applications, for existing analytics processes, for new analytics processes -- but they're providing effectively data as a service, capturing it all in this data lake that continues to evolve.

Think about how companies may want to leverage then a PaaS. It's the same thing on premises. If my data is on premises, because that's where the physics requires that, I can leverage various development tools or application frameworks on top of that data to create new business apps. About 60 percent of our initial sales at Hortonworks are new business applications by an enterprise. It’s business and IT being involved.

Leveraging datasets

Within the first five months, 20 percent of those customers begin to migrate to the data-lake concept, where now they are capturing more data and allowing other business entities within the company to leverage these datasets for additional applications or additional analytics processes. We're seeing Hadoop as a service on premises already. When we move to the cloud, we'll begin to see more of a hybrid model.

We are already starting to see this with one of Hortonworks large partners, where you put archive data from on premises to store in the cloud at low-cost storage. I think HP will have that same opportunity with Hadoop and their cloud strategy.

Already, through an initiative at HP, they're providing Hadoop as a service in the cloud for those entities that would like to run Hadoop in a managed service environment.
We're seeing Hadoop as a service on prem already. When we move to the cloud, we'll begin to see more of a hybrid model.

That’s the first step of HP beginning to provide Hadoop in a managed service environment off premises. I believe you'll begin to see that migrate to on-prem/off-prem integration in a hybrid opportunity in the some companies as their data moves off prem. They just want to run all of their big-data services or have Hadoop as a service running completely in HP cloud, for example.

Gardner: So, we're entering in an era now where we're going to be rationalizing how we take our applications as workloads, and continue to use them either on premises, in the cloud, or hybrid. At the same time, over on the side, we're thinking along the same lines architecturally with our data, but they're interdependent.

You can’t necessarily do a lot with the data without applications, and the applications aren’t as valuable without access to the analytics and the data. So how do these start to come together? Do you have a vision on that yet? Does HP have a vision? How do you see it?

Ferguson: The Hadoop market is very young. The vision today is that companies are implementing Hadoop to capture data that they're just letting fall on the floor. Now, they're capturing it. The majority of that data is on premises. They're capturing that data and they're beginning to use it in new a business applications or existing analytics processes.
Fully experience the HP Vertica analytics platform...
Get the free HP Vertica Community Edition

Become a member of myVertica
As they begin to capture that data, as they begin to develop new applications, and as vendors like HP working in combination with Hortonworks provide the ability to effectively move data from on premises to off premises and provide the ability to govern where that data resides in a secure and organized fashion, you'll begin to see much tighter integration of new business or big-data applications being developed on prem, off prem, or an integration of the two. It won't matter.

Gardner: Great. We've been learning quite a bit about how Hortonworks and Hadoop are changing the game for organizations as they seek to use all of their data and very massive datasets. We’ve heard that that aligns with HP Vertica and HP Haven's strategy around enabling more business applications for more types of data.

With that, I'd like to thank our guest, Mitch Ferguson, Vice President of Business Development at Hortonworks. Thank you, Mitch.

Ferguson: Thank you very much, Dana.

Gardner: This is Dana Gardner. I'd like our audience for joining us for a new style of IT discussion coming to you from the recent HP Big Data 2014 Conference in Boston. Thanks to HP for sponsoring our discussion, and don't forget to come back next time.

Listen to the podcast. Find it on iTunes. Download the transcript. Sponsor: HP.

Transcript of a BriefingsDirect podcast on how companies are beginning to capture large volumes of data for past, present and future analysis capabilities. Copyright Interarbor Solutions, LLC, 2005-2014. All rights reserved.

You may also be interested in:

Tuesday, November 11, 2014

After Cutting its Big Data Teeth on Wall Street, Vichara Technologies Grows the Market for Advanced Analytics

Transcript of a BriefingsDirect podcast on how one consultancy is looking to expand its horizons for big-data analytics using HP Vertica.

Listen to the podcast. Find it on iTunes. Download the transcript. Sponsor: HP.

Dana Gardner: Hello, and welcome to the next edition of the HP Discover Podcast Series. I'm Dana Gardner, Principal Analyst at Interarbor Solutions, your host and moderator for this ongoing sponsored discussion on IT innovation and how it’s making an impact on people’s lives.

Gardner
Once again, we're focusing on how companies are adapting to the new style of IT to improve IT performance and deliver better user experiences, as well as better business results.

This time, we're coming to you directly from the recent HP Big Data 2014 Conference in Boston. We're here to learn directly from IT and business leaders alike how big data, cloud, and converged infrastructure implementations are supporting their goals.
Fully experience the HP Vertica analytics platform...
Become a member of myVertica
Our next innovation case study interview highlights how Vichara Technologies in Hoboken, New Jersey is expanding its capabilities in big data from origins on Wall Street and in the financial sector into other areas, and thereby demonstrating the growing marketplace for big-data analytics services.

With that, please join me in welcoming our guest, Tim Meyer, Managing Director at Vichara Technologies. Welcome, Tim.

Tim Meyer: Thank you, glad to be here, Dana.

Gardner: Tell us how your organization evolved, and how big data has become such a large part of the marketplace for gaining insights into businesses.

Meyer: The company has its roots in analytics and risk modeling and for all sorts of instruments that are used on Wall Street for predicting prices and valuation of instruments. As the IT infrastructure grew from Excel to databases and eventually to very fast databases, such as Vertica, we realized that there were many problems that couldn't be solved before, and that required way too long a time to answer.

Meyer
Wall Street people measure time in seconds, not in hours. We've found that there's a great value in answering a lot of business intelligence (BI) questions -- especially around valuations and risk models, as well as portfolio management. These are very large portfolios and datasets that have to be analyzed. We think that this is a great use of big-data analytics.

Gardner: How long have you been using Vertica? How did it become a part of your portfolio of services?

Meyer: We've been using Vertica for at least for two years now. It’s one of the early ones, and we recognized it as being one of the very fastest databases. We try to use as many of these components as possible. We really like Vertica for its capabilities.

Risk assessment

Gardner: Tim, this whole notion of risk assessment is of interest to me. I think it's coming to bear on more industries. People are also interested in extending from knowing what has happened to being able to predict, and then better prescribe new efforts and new insights.

Tell me about predictive risk assessment. How do you go about that, and what should other companies understand about that?

Meyer: Risk assessment comes about from starting to look at how prices fluctuate and how interest rates move, and thus create changes in derivatives. What has happened most recently is that a lot of the banks and hedge funds have recognized this. Not only is [predictive risk assessment] a business imperative for them to have that half-percent hedge, but there are also compliance reasons for which they need to predict what their business is going to look like.

There are now more and more demands on stress testing, as well as demands from international banking regulations, such as Basel III, that require that businesses such as hedge funds and banks not just look behind, but ahead at how their business is going to look in a year. So this becomes really very important for a host of reasons even more than just how your business is doing.

Gardner: If I were a business and wanted to start taking advantage of what's now available through big-data analytics -- and at a more compelling price and higher performance than in the past -- what are some of the first steps?
Fully experience the HP Vertica analytics platform...
Become a member of myVertica
Do I need to think about the type of data or the type of risk? How do you go about of recognizing that you can now get the technology to do this at an analytics level, but there is still the needed understanding of how to do it at the process and methodological level?

Meyer: We work very closely with our customers and try to separate algorithmic work from the development work. A lot of our customers have more than a few Caltech and MIT PhDs who do the algorithmic definitions. But all of them still need the engine, the machine with its scripting, and fast capability to build those queries right into the system as quickly as possible.

We usually work with these kinds of people, and it is a bit of a team-work effort. We find that that’s a way to figure out what is our value, and what is the value of our customer. Together, it has turned out to be very good teamwork.

Gardner: And you are a consultancy, as well as a services provider? Do you extend into any hosting or do you have a cloud approach? How do you manage the technology for the consulting and services you offer?

Broader questions

Meyer: We expand from the core products and tools into broader questions for people who want a proof of concept (POC) into this new technology. We build those on an ongoing basis. People, as well, want to look at options such as different performances of clouds. They do vary.

So we take on those kinds of consulting work as well, not to mention that sometimes it expands into back-office compliance and sometimes into billing issues. They all relate to the core business of managing portfolios, but yet they are linked.

Very often, we've done those kinds of projects and we see even more of these possibilities as we see compliance as a bigger issue, such as Dodd-Frank as well as Basel III, in the financial world. But they are really no different than many regulations coming on the healthcare side for paperwork management, for example.

Gardner: So that raises the question of the verticals that you expect first. Where is predictive risk assessment and the analytics requirements for that likely to appear first?
They all relate to the core business of managing portfolios, but yet they are linked

Meyer: One thing we have learned from our experience in financial modeling and tools is that there is always a need for people who are totally unskilled in SQL or other query languages to quickly get answers. Although many people have different takes on this, we think we've found some tools that are unique. And we think that these tools will apply to other industries, most particularly to healthcare.

These are big problems, but we think the way we think of it is to start small with a POC or really defining a very small problem and solving it and not trying to take a bite of the entire elephant, so to speak. We find that to be a much better approach to going into new segments and we'll be looking at both insurance and healthcare as two examples.

Gardner: Back to the technology front. Are there any developments in the technology arena that give you more confidence that you can take on any number of data types, information types, and scale and velocity types?

I'm thinking of looking at either cloud or converged infrastructure support of in-memory or columnar architectures. Is there a sense of confidence that no matter what you go to bite off in the market, you have the technology, and the technology partner, to back you up?

Meyer: Dana, that's a great point. We're finding that there is much more maturity in a lot of database technologies that are now coming out.

There is always something new on the horizon, but there are, as you said, columnar architectures and so on. These are already here, and we're constantly experimenting with them.

To your point about cloud infrastructure and where that is going, it's the same thing. We see ParAccel, Amazon, and data warehouses such as Redshift showing us the way where a lot of the technology is becoming very prepackaged. The value-add is to talk to the customer and speed up that process of integration.
Fully experience the HP Vertica analytics platform...
Become a member of myVertica
Gardner: I'm afraid we will have to leave it there. We've been talking with Vichara Technologies about how they have gathered some expertise in the financial sector and are now looking to expand into such areas as insurance and healthcare.

So a big thank you to our guest, Tim Meyer, Managing Director at Vichara Technologies in Hoboken, New Jersey. Thank you, Tim.

Meyer: Thank you, Dana.

Gardner: This is Dana Gardner, Principal Analyst at Interarbor Solutions. A big thank you also to our audience for joining us in this special new style of IT discussion, coming to you directly from the recent HP Big Data 2014 Conference in Boston. Thanks again for listening, and come back next time.

Listen to the podcast. Find it on iTunes. Download the transcript. Sponsor: HP.

Transcript of a BriefingsDirect podcast on how one consultancy is looking to expand its horizons for big-data analytics using HP Vertica. Copyright Interarbor Solutions, LLC, 2005-2014. All rights reserved.

You may also be interested in:

Monday, March 10, 2014

HP Updates HAVEn Big Data Portfolio as Businesses Seek Transformation from More Data and Better Analysis

Transcript of a BriefingsDirect podcast on how HP is developing products and platforms to help businesses deal with the demands of big data in a competitive environment.

Listen to the podcast. Find it on iTunes. Download the transcript. Sponsor: HP.

Dana Gardner: Hello, and welcome to the next edition of the HP Big Data Podcast Series. I’m Dana Gardner, Principal Analyst at Interarbor Solutions, your moderator for this ongoing sponsored discussion on how data is analyzed and used to advance the way you live and work.

Gardner
Once again, we're showcasing thought-leaders and companies worldwide that are capturing myriad knowledge, gaining ever deeper analysis, and rapidly and securely making those insights available to more people on their own terms.

Our next big-data innovation discussion highlights how the latest version of HP HAVEn produces new business analytics value and strategic returns. So please now join me in welcoming Girish Mundada, Chief Technology Officer for HP HAVEn.

Girish Mundada: Thanks, Dana. Good to be here with you.

Gardner: And Dan Wood, Worldwide Solution Marketing Lead for Big Data at HP Software. Welcome, Dan.

Dan Wood: Hello, Dana,

Gardner: Dan, let me start with you. We’re in a fascinating time because analytics and big data are now top of mind. What was once relegated to a fairly small group of data scientists and analysts as reporting tools -- and I am thinking about business intelligence (BI) -- has really now become a comprehensive capability that’s proving essential to nearly any business strategy.

From your perspective, what’s behind this eagerness to gain big-data capabilities and exploit analytics so broadly?

Wood: You’re right, Dana, and it’s because we're starting to see some very clear quantification of the value and the benefits of big data. It’s fair to say that big data is probably the hottest topic in the industry.

Wood
There’s a lot of talk across all forms of media about big data right now, but what’s happened is that credible publications like the "Harvard Business Review," for example, have started to put solid numbers around the benefits that enterprises can get if they can get their hands around big-data analytics and apply it to business challenges.

For example, Harvard Business Review is saying that, on average, data-driven organizations will be five percent more productive and six percent more profitable than their competitors.

Worth chasing after

Think about that. A six-percent distinct profitability increase would double the stock price for a lot of organizations. So there really is a prize worth chasing after.

What we’re seeing, Dana, is much more widespread interest across the organization and not just within IT. We’re seeing line-of-business leaders understanding and, in many organizations, actually starting to benefit from big-data analytics.

They’re able to analyze the call logs in a call center, better understand the clickstreams on a website, and better understand how customers are using products. All of these are ways of analyzing large amounts of data and directly tying it to specific line-of-business problems.

That’s where we are right now. Industries around the world are going through transformational projects using big data to gain competitive advantage.

Gardner: It’s interesting too, Dan, that they’re not just taking these as individual data sets and handling them individually, but increasingly businesses are combining them, and finding new relationships, and doing things that they really couldn't have done before.

Wood: Absolutely. It’s the idea of 360-degree view of their internal operations, or of their external customer trends and needs -- and it’s come from combining data sets.
This industry label of big data is perhaps not the most helpful, because it’s not just the volume of data that is the challenge and the opportunity for the business.

For example, they’re combining social media analytics on customers with the call logs into the call center, with internal systems of record around the customer relationship management (CRM) and ongoing customer transactions. It’s by combining all those insights that the real big-data opportunity reveals itself.

Gardner: And the sources for those insights and data, of course, are across almost any type of information asset. It’s not a just structured data or data that your application standard is around -- it’s getting all the data all of the time.

Wood: That’s right. In some ways, this industry label of big data is perhaps not the most helpful, because it’s not just the volume of data that is the challenge and the opportunity for the business. It’s the variety of sources, as you’ve alluded to, and also the velocity at which that data is moving.

The business needs to get hold of these multiple sources of data and immediately be able to apply the analytics, get the insights, and make the business decisions. This is why still the vast majority of that data that’s available to an enterprise remains dark.

Unused and unexploited

It’s unused and unexploited. Organizations, with their traditional analytics systems, are struggling to get the meaning and insights from all these data types that we mentioned. These include unstructured information, such as social media sentiment, voice recordings, potentially even video recordings, and the structured and semi-structured things like log files and data center data. For many organizations, getting the information quickly enough out of their CRM and enterprise resource planning (ERP) systems is a challenge as well.

Gardner: So we see that there’s a great desire to do this, and there are great returns on being able to do this well. We talked about some of the general challenges. What specifically is holding people up?

Is this an issue of cost, complexity, or skills? Why aren’t companies able to move beyond this small fraction of the available information to which they could be applying such important insight and analytics?

Wood: It’s a complexity and a skills challenge, as you mentioned. The systems they have today, Dana, typically aren’t set up to able to analyze these vast amounts of unstructured information, and also to be able to analyze the structured data at a speed needed by the organization.
Typically, the analytic systems that organizations have had aren’t able to cope with that or with that unstructured human information.

Think about the need to analyze immediately a clickstream from an online shopping application or a pay-to-use application that an organization has. That is, a rapid-scale analysis of a large amount of structured data. Typically, the analytic systems that organizations have had aren’t able to cope with that or with the unstructured human information.

This is why HP has created the HAVEn Big Data Platform, and Girish will talk in more detail about this, and how it brings together the analytics engine needed to address these issues.

Just as importantly, there’s the ecosystem around HAVEn, which includes HP experts and services and services from partners, to bring together the skills needed to turn this data collection into useful information.

And there are skills around data scientists, as well -- skills around understanding the right questions the line of business needs to be asking, and understanding actually how to visualize and represent the data.

Gardner: Based on what we have talked about in terms of some of these serious challenges, Girish, what were some of the guiding principles that you were thinking of when HAVEn was being put together and refined?

Talking to customers

Mundada: HAVEn came together not by creating it in a dark room somewhere in the back office. It came together by talking to customers. On a regular basis, I meet with some of HP's largest customers worldwide, getting input from them. And they're telling us what their current problems are.

Mundada
Let me see if I can describe the landscape in a typical organization, and we can go from there. You'll see why we created HAVEn.

Let’s visualize four different waves of data. Back in early '60s,'70s, even part of the '80s, mainframes were the primary way to process data, and we used them for operationalizing certain parts of data processing, where data was extremely high-value. If you look at the cost of the systems, it was phenomenal.

Then came the next wave in the ‘80s, where we went into what I call client-server computing, and we already know several companies that were created in this space.

I’ve lived in Silicon Valley for almost 30 years now, and a whole bunch of new companies were born in this space. I worked for a company, Postgres, which became Illustra, then became Informix, and became IBM. If you look at that entire wave of OLTP technologies, we created data-processing technologies designed to solve basic business problems.

Application software was created: CRM, supplier relationship management (SRM), you name it. Many companies that did consulting around that were created, too. That was that second wave after the mainframe.
We’re talking about volumes that are growing exponentially. In the past, they were growing linearly.

Then came the third wave, where we took this data from all these transactional systems, brought them together to find out some basic analysis, which we now call business analytics, to find out "who is my most profitable customer, what are they buying, why are they buying," and things of that nature.

We created companies for that wave, too, and many technologies. Exadata, Teradata, Netezza, and a whole bunch of companies and applications were born in that space. That wave lasted for quite a while.

What we're seeing now is that from 2003 onward, something very fundamental has happened. At least, that’s the way I've been seeing this. If you look at the three Vs that Dan has described -- volume, velocity, and variety -- we’re talking about volumes that are growing exponentially. In the past, they were growing linearly. That creates a very different kind of requirement.

More importantly, if you look at the variety that Dan mentioned, that’s really the key driver in my mind. People are now routinely bringing in machine data, human data, and your traditional structured warehouses -- all of them together.

If you visualize a bar graph, you would see that 10 percent of the data that we now can monetize is coming from traditional sources, whereas 90 percent of the data that we need to monetize is now sitting in machine data and human data.

High velocity analytics

What we're trying to do with HAVEn is create a combined platform, where you can combine these three different data types and do very high-velocity analytics.

As a simple example, if you look at Apache Web Server logs, that data is used historically by the security people to see if anybody is breaking in. That data was being used by operational people to see if machines aren’t overloaded.

More importantly the digital marketing guys now want to look at that data to see who's coming to their website, what they’re buying, what they’re not buying, why they’re buying, and which geographies they’re coming from. Then, they want to combine all these data sets with their existing structured data to make sense out of it.

Today, it's a mess in the market. When we talk to our partners and customers, they’re saying that they have point solutions for each of these things, and if you want to combine that data, it’s really hard. That’s why we had to create HAVEn.

HAVEn is the fourth wave. HAVEn is specifically about big data, the fourth wave. If you look at HP’s portfolio, we sell products and services across each of these waves, and the fastest growing wave right now is the big-data wave. It’s growing at about 35 percent a year, according to Gartner, and that's why we're excited about it.
If you look at what’s required now to process big data in its entirety, one product no longer can do it all.

Gardner: Now we know why you created it and what it’s supposed to do. Tell us a little bit more about what’s included in HAVEn and why it is that you’ve been able to create a combination of product and platform that solves this very difficult task?

Mundada: If you look at what’s required now to process big data in its entirety, one product no longer can do it all. There is a very famous paper written by some university professors titled “One size does not fit all.” It proves that different data structures are able to solve different kinds of data problems far more efficiently.

One way to think about big data is to think of it as a pile of dirt. It’s a big pile. In that pile, there’s gold, silver, platinum, iron, and other metals you don’t even know. If the cost of mining that data is high, obviously you’re going to go after only the platinum and some known objects that you care about, because that’s all you can afford.

HAVEn is about bringing that cost of processing down to a very, very low level so you can go after more metals. That means you have to bring together a set of technologies to be able to solve this. If you look at the last three years, HP has made very significant amounts of investments in the big-data space.

Best of breed

We bought companies that were best of breed to try to solve specific problems. We bought Autonomy, Vertica, ArcSight, Fortify, TippingPoint, 3PAR Data, and Knightsbridge.

Now, we have a set of technologies to be able to combine them into a unique experience. Think of it almost like Microsoft Office. Before you had Microsoft Office, you would buy a word processor from one company, a spreadsheet from another company, and presentation software from a third company.

Let’s say you wanted to create a simple table. If you had created it in a word processor or even a spreadsheet, you couldn’t mix and match that. It was impossible to mix and match very different types.

Then, Microsoft came to the table and said, “Look, here’s a simplified solution.” If you want to create a table, go ahead and create it in PowerPoint. Or if you want to create more complicated thing, put it in Excel. Then, take that Excel and put it in PowerPoint. Or, you can put the whole thing into a Word document. That was the beauty of what Microsoft did.

We’re trying to do something similar for big data, make it very easy for people to combine all these different engines and the different data types and write simple applications on it.
Today you need to combine different sets of data techniques to solve different problems, and they have to work seamlessly.

Gardner: What also is going on, other than product acquisitions, is recognizing the industry standards and the H in HAVEn, being a representative of Hadoop, is an indication of that. Tell me, beyond the products, what is binding them together, and why being an open and standard space has its important role here, too?

Mundada: Let’s look at HAVEn as a platform. HAVEn is really two different concepts. There’s the HAVEn data platform, which we’ll talk about now, and there’s a HAVEn ecosystem, which I’ll mention in a minute.

HAVEn means Hadoop, Autonomy, Vertica, Enterprise Security, and “n” applications. That’s the acronym. So let’s look at one of these pieces, and why we need an architecture like this.

As I said, today you need to combine different sets of data techniques to solve different problems, and they have to work seamlessly. That’s what we did with HAVEn. I’ve been with HAVEn from day zero, before the project concept started, and I can tell you why and how we added these pieces and how we’re trying to integrate them better.

If you look at Hadoop as an ecosystem part of that HAVEn, our story with Hadoop at HP is that Hadoop is an integral part of HAVEn. We see a lot of our customers and partners betting on Hadoop and we think it’s a good thing to keep Hadoop open and non-proprietary.

Leading vendors

We also today work with all leading Hadoop vendors, so we have shipping appliances as well as reference architectures for both Cloudera and Hortonworks, and we’re working now with MapR to create similar infrastructure. That’s our Hadoop’s story.

We’ve also found that our customers are saying they want some flexibility in Hadoop. Today, they may want one vendor, and tomorrow, they may decide to go to another vendor for whatever business reasons they choose. They want to know if we can provide a simple management tool that works across multiple Hadoop distributions.

As an example, we had to extend our Business Service Management (BSM) portfolio, so we can manage Hadoop, Vertica, hardware, storage, and networking all from within one environment. This is simply operationalizing it. Having a standardized set of hardware that matches multiple Hadoop distributions was another thing we had to do. There are many such enterprise-class innovations that you’ll see coming from HP.

But more than that, we also found that Hadoop is really good for certain kinds of applications today, and obviously, the community will extend that. You will see more and more innovations coming from that community and ecosystem.
It’s an analytic database, and by that, I mean the underlying algorithms are completely designed from the ground up.

Today, there are several areas where there are holes in Hadoop, or maybe they’re not as strong as commercial products. One such area that you see is SQL. The SQL phase of Hadoop is going to be one of the key differentiators across the different Hadoop packaging.

In that area, we have a technology called Vertica, which is the V part of HAVEn, and you’ll see companies like Facebook, using a combination of both Hadoop and Vertica.

The classic use case we see is that people will bring all kinds of raw data, put it into Hadoop, and do some batch processing there. Hadoop is great as a file system, a batch processing environment. But then they’ll take pieces of that data and want to do deep analytics on it, like a regression analytics, and they will put it into Vertica.

Vertica is, is an analytic database platform, and I will break up those three words. It’s a database. It looks and feels like a database. It has SQL on it, open database connectivity (ODBC), and Java database connectivity (JDBC) connectivity. You can run all kinds of tools on it, the ones you are used to, Tableau, Pentaho, and Informatica. So from that perspective it’s a regular database.

What’s different is that it’s custom built for the fourth wave. It’s an analytic database, and by that, I mean the underlying algorithms are completely designed from the ground up. Michael Stonebraker who created the key products in the first wave and the second wave -- Ingres and Postgres -- also created this at MIT from the ground up.

Data today

The intuition was that if you look at the processing of data today, it’s gone from having 10 to 20 columns per row to possibly thousands of columns. A social media company, for example, might have 10,000 pieces of information on me, and while they do processing, it’s going more linear. It’s going regression-oriented in a sense. You might say “Girish, age x, lives here, and likes y. What’s the likelihood somebody else may like it?”

It’s meant for that kind of deep analytical processing, a column-oriented structure. In those kinds of applications, this database technology tends to be magnitudes faster -- tens of times faster. That’s one example of Hadoop and Vertica, and we can talk more about other pieces Autonomy and Enterprise Security with you.

Gardner: So we see that there’s a platform that you put together. There’s an ecosystem that’s supporting that. There are these binding standards that make the ecosystem and the platform more synergistic. But other people are doing the same thing. What’s making HAVEn different? What is it about HAVEn that you think is going to be a winner in the marketplace?

Mundada: There are two different answers to it. Let me talk about how we’ve taken just not the SQL piece of Hadoop, but how we extend it with other parts of HP that are unique to HAVEn. It’s the breadth of it. Let’s see how we extend this simple combination of Hadoop and Vertica.
With Vertica, we’re able to drop in other codes that are user defined and user written.

I said it’s an analytic database platform. If you look at that platform piece of it, with Vertica, we’re able to drop in other code that are user-defined and user-written. For example, you can drop in R language routines, Java, C++, or C language routines directly into the database. Now, we’re now able to combine that richness across our portfolio.

Autonomy, which is the A part of HAVEn, is a unique technology. It's one of a kind. Some of the largest governments and some of the largest organizations in the world, such as banks and financial institutions, have this in production in what it's meant for, human information processing, which is audio, video, and text.

As an example, you could take a video stream and ask simple questions. Tell me if an object is moving from point A to point B, or tell me what’s in the object. Is it a human? Is it a car? Can you read car number plates automatically?

And you could do some really sophisticated applications. Taking a car, we have cases where police cars have video cameras mounted on the side, and as they’re driving by in a parking lot, they can take photos of the number plates and compare it to stolen cars.


Crime detection

Imagine being able to take that technology and combining it automatically, through simple SQL-like or simple REST API-like commands with SQL, with your existing data and creating very sophisticated applications to understand your customer or for crime detection and things like that?

Now let’s bring in the third of part of the puzzle, the E part, which is Enterprise Security. That’s also unique. We have an entire portfolio, both for security as well as for operations management.

If you look at enterprise security and if you look at the Gartner Magic Quadrant, HP’s product set has been in the leader space for several years in a row. They are the number one vendor in that area.

Now, think about our portfolio of ArcSight, Fortify, Tipping Point, and other ESP products. Imagine being able to take the data-collection algorithms of those, bringing it into this common platform of HAVEn, combining it with other structured and unstructured data with just simple commands. That’s something we can do uniquely.

Operations management is another area where we have hundreds of these machine logs. We can collect them, break them open into modular pieces, and create new applications. You can go look at our website, Operations Analytics, where with a simple slider, you can go back and forth in time to millions of log files as if they were structured data.
With simple SQL, we can essentially write simple queries across structured and unstructured data.

We can do that uniquely, because we have that entire collection. Our BSM portfolio has been on the market for 30 years. It’s one of the leaders. This is the HP OpenView platform and this is one of the things we can do uniquely at HP, bring all these things together.

That’s the breadth of our portfolio, but it simply doesn’t stop at this platform level. Remember, I said that there are two concepts. There is a platform, and then there is the ecosystem. Let’s look at the platform level first.

We have the whole of HAVEn. We have the connectors, and we ship these 700 connectors out of the box. With simple commands, you can bring in social-media data in every language written. You can bring in machine logs and structured logs. That’s the platform.

Let’s extend it further into the ecosystem part. The next thing that people were saying was, “We want to use something very open. We have our own visualization tools. We have our own extract, transform, load (ETL) tools that we’re used to. Can you just make them work?" And we said, "Sure.”

That’s one of the things that we’re able to do now. With simple SQL, we can essentially write simple queries across structured and unstructured data. Using Tableau Software, or any other tool that you like, we can access this data through our connectors, but, more importantly, it let’s you hook in your existing ETL tools into this -- completely transparently.

Breadth and openness

So that’s the openness of the platform, the breadth and the openness of the platform. Breadth is not just about the software platform, but it’s about HP’s strength to bring together hardware, software, and services.

Even with the platform, the HAVEn components in the middle, the connectors, and being able to match them with matching hardware, our customers are asking, “Can you give us matching hardware for Hadoop, so we don’t have to spend time setting it up?” That’s one of things that HP can uniquely do, but more importantly we have appliances for Vertica, for example, which are standardized.

If you look at the other side, our customers are also saying, “We understand that HP wants to provide us all this, but we like openness and we like other partners.” So we said, “Fine, we’ll leave this entire ecosystem open.” Our software will work with HP hardware and we can optimize, but we also commit to working on everybody else’s hardware.
If you look at our visualization, we didn’t go and force a visualization technology on you. We kept it open.

Our cloud story is that we’ll work on Amazon, as well as OpenStack. For example, if you want to build a hybrid cloud, where part of your data resides on HP or your private environment using OpenStack, that’s fine. If you want to put it in Amazon or Rackspace, no problem. We’ll help you bridge all these. These are the kinds of enterprise-cloud innovations that HP is able to do, and we’re open to this.

So to answer your question very succinctly, if there were three things I would pick where HP is different, one is our breadth of our portfolio. We have very large breadth that we've brought together.

It’s the openness of the platform. HP is known to be a very open company. If you look our Hadoop story, we have an example. We didn’t create a proprietary Hadoop. We kept it open. If you look at our virtualization, we didn’t go and force a virtualization technology on you. We kept it open.

More importantly, if there is one key thing that you want to take home from what we've done with HAVEn, it's not about feeds and not about speeds. It's about business value.

The reason we created HAVEn was to create that iPhone-like environment or Android-like environment, where the vision is that you should be able to go to a website, say you have standardized on the HAVEn platform, and then, be able to point and click and download an application.

The end part of HAVEn is really the business value of it, and that’s how we see HAVEn as unique. There is nobody else, as far as we know, that has that end-vision, where you can build the applications yourself using standard tools -- SQL, ODBC, REST API, JDBC -- or you can buy ready-made software that HP Software has created.

We have packages across service, operations, and digital marketing. Or you can go with a partner. The partner could be HP Enterprise Services, Accenture, Capgemini, or any of those big partners. That’s something unique about the HP big-data ecosystem that doesn’t exist anywhere else today.

Applications

Gardner: Applications are something that take advantage of the platform, the capabilities, the breadth and depth of the data, and information.

I wonder if you could explain a little bit more about the application side of HAVEn, perhaps through examples of what people are already doing with these applications, and how they’re using them in their business setting?

Mundada: That’s actually one of the most exciting parts of my job. As I said, I meet literally 100 customers a month. I'm traveling across the continents, and the use-cases of big data that I see are truly phenomenal. It really keeps you very motivated to keep doing more.

Let's look at a very broad level of why these things matter. Big data is not just about monetary profits. It's really about what I call extended profits. It doesn’t have to be monetary. If you look at a simple example, we have medical companies using data, using our technologies, to dramatically speed up drug discovery hundreds of times more than they were able to with Hadoop.
HAVEn isn’t about speeds and feeds. It's about really creating business value in a hurry, so you get there before your competitors can.

That translates into just saving lives. At our Discover show, we saw that a very innovative organization is using our technology to look at bio-diversity and save wildlife in the Amazon.

That’s unique, but those are like edge cases. If you look at a regular enterprise, what they want to do at a very high level falls into three categories: Applications that HP itself is building, applications that partners are building, and applications that customers themselves are building.

Let's start with the ones that HP is building. Today HP is shipping several applications, and I’ll talk about a few of them. Even before I talk about these applications, let's look at why people generally want to do this. They’re saying that they want to either increase  revenues, so that’s affecting the top line, or they want to decrease costs, so they can increase the bottom line. Third is that they want to improve products and services. Those are really the three broad categories at a very, very high level.

As I said, HAVEn isn’t about speeds and feeds. It's about really creating business value in a hurry, so you get there before your competitors can.

From that perspective, there are three applications I’ll mention. In terms of increasing revenue, we have a product that we ship called Digital Marketing Hub, and it combines the power of Autonomy and Vertica to analyze all of your customer analytics.

You’re able to take your call center logs, your social media feeds, your emails, your phone interactions and find out what the customer is really is saying, what they want and don't want, and then, being able to optimize that interaction with the customer to create more revenue.

More precise answers

For example, when a customer calls knowing what they want, obviously you can tell them more precise things. That’s one example.

Let's look at another example, where you want to decrease your bottom line or decrease your costs. Operational Analytics is another software product we ship. We’re able to drive down costs of debugging network troubles by 80 percent by combining all these logs from machines on a very frequent basis.

We can look at this and say. "At this second, every machine was okay. A second later, machines have gone down." I can look exactly at the incremental logs that showed up, using a simple pen like a pointer, going through SQL-like data. That’s unique.

Those are the kinds of applications we’re able to create. It's not just these two. The other thing people want is improve products and services. We have something called Service Anywhere, where as you're calling or as you're typing in commands and saying you want to find information about that, the system is able to understand the meaning of what you’re saying.

Notice that this is not keyword search. This is meaning, where it's able to go through existing case reports from customers, look at existing resolutions, and then say, “Okay, this might solve your problem automatically.”
That’s the beauty of the HAVEn platform. On the same platform, you can buy HP built applications or you can build your own.

Imagine what that impacts. Your customers are happy, because the answers are quicker. We call this ticketless ID, but more important, look at some other interesting ways of how this affects a company.

For example, I was recently in Europe. I was talking to a very large telco there, and they said, “We have something like 20,000 call-center operators who are taking calls from customers. Each call volume might take six minutes and some of them are repeat calls. That’s really our problem.”

We worked out something that roughly could save them two minutes per call. That translates to about a $100 million net saving per year. That’s really phenomenal. Those are one kind of application that HP built.

Now imagine a customer wanting to build the same application themselves. That’s the beauty of the HAVEn platform. On the same platform, you can buy HP built applications or you can build your own.

Let's look at NASCAR as an example. They did something very similar for customer analytics. They are able to -- while the race is happening -- understand audio, television channels, radio, broadcast, and social media and bring that all together as if it's one unique piece of data.

Then, they’re able to use that data in really innovative ways to further their sport and to create more promotional dollars for just not themselves, but even the participants. That’s unique -- being able to analyze mass scale human data.

Looking to the future

Gardner: Well, we've learned a lot about the market, the demand, why big data makes so much sense. There is very large undertaking by HP around HAVEn, and what it’s getting in terms of openness, platforms, breadth, and these great examples of applications. But we also need to look to the future.

What's coming next in terms of HAVEn 2.0 or HAVEn 1.5? Dan, could you update us on how things are progressing, what you have in mind for the next versions of these products and, therefore, the whole increasing as sum of the parts increases?

Wood: Dana, we've just announced HAVEn 2.0. The way Girish explained HAVEn there in terms of the platform and the ecosystem and continuous innovation now is around both of those pieces. It's really important to us to be driving the ecosystem, as well as the platform. So I’ll speak to HAVEn 2.0 and one of the feature that’s the focus in driving HP forward.

In terms of the platform, there are the analytics engines that we have. Girish mentioned they were best in class at the time that HP acquired them, and we continue to invest in R and D across Autonomy, IDOL, Vertica, and the ArcSight Logger product. We recently announced new versions of all three of those, improving the analytics capability and the usability and, just as importantly, increasing the interoperability.
At the moment, on an early-access program, we’re making the IDOL engine available to developers as a cloud-based offering.

For example, we now have integration of the ArcSight Logger with the Autonomy IDOL engine for analyzing unstructured human information. A really great use case of this is Logger was previously enabling IT to understand data movements and potential threats and the risks in the organization.

For example, if I were sending 50 percent of my email to a competitor, you could combine that capability with the unstructured information analysis in Autonomy and understand by that the information layer exactly what’s in that email, 50 percent of which is going to a competitor.

Let’s start putting that together and getting a powerful view of what an individual is doing and whether it’s a risky individual in the organization, integrating those HAVEn engines and putting more effort on integrating it into the Hadoop environment as well.

For example, we have just announced integration Hadoop connectors for Autonomy. A lot of people are saying that they’re building this data lake with Hadoop and they want to have the capability of putting some analytics into the unstructured information that exists in that Hadoop data lake. Clearly, we’ve also got integration with Vertica in the Hadoop environment as well.

The other key thing within that on the engine is IDOL OnDemand. At the moment, on an early-access program, we’re making the IDOL engine available to developers as a cloud-based offering. This is to encourage the independent developer community to take components of IDOL with that social media analytics, whether it’s video or audio recognition, and start building that into their own applications.

We believe the power of HAVEn will come from the combination of HP-provided applications and also third-party applications on top.

Early-access program

We’re facilitating that with this initial early-access program on IDOL OnDemand, and also, we’re investing in developer programs to make the whole HAVEn development platform far easier for partners and independent developers to work with.

We’ve set up a HAVEn developer website, and stay tuned for some really fun events online and physical events, where we’ll be getting the developer community together.

In terms of those applications that make the whole HAVEn ecosystem come to life, Girish has mentioned some of them that we have announced over the last few weeks. So I’ll give you a quick recap on those.

We have the Operations Analytics and Service Anywhere apps, both aimed at the CIO. And we have the Digital Marketing Hub from HP aimed at marketing leaders in the organizations. These are three applications that HP has packaged on the HAVEn platform.

And along with the HAVEn 2.0 announcement, we’re really pleased that six of the leading SI partners -- Accenture, Capgemini, Deloitte, PwC, Accenture and Wipro -- themselves have put marketing applications on top of HAVEn. And those guys have gotten fascinating mixtures of very industry-specific analytics applications and more horizontal apps based on the priorities that they’re chasing after.
We’re populating our solutions and partner solutions to facilitate the whole commerce side of those applications taking off in the market.

So we’re really excited about that and expect to see many more announcements of partner applications over the next few months.

The final piece of HAVEn 2.0 to support this whole ecosystem thing is a marketplace that we’ve launched, where we’re populating our solutions and partner solutions to facilitate the whole commerce side of those applications taking off in the market.

Gardner: Just to flesh out that last point, when you say a marketplace, is this an app store? Will some of your partners that are able to create analytics-oriented applications on HAVEn then be able to sell them? Is this a commerce site or is it a community site only at this point?

Mundada: The original vision of HAVEn was to be able to make it essentially like how you buy applications on a mobile phone today. Once you have settled on a platform, the eventual vision is to be able to go there and just download these applications. As Dan said, they’ve launched this now and you will see much more stuff coming in this area.

Gardner: For those interested in learning more, those who might want to focus on one element of HAVEn, it’s not all-inclusive. You don’t have to buy it all at once. It comes in parts. There are on-ramps, and then you can expand. How do you get started? How do you learn about specific parts of HAVEn? Which combinations would work for you?

One-stop resource

Wood: The first place to go is hp.com/haven. That’s your one-stop resource for information on this platform, all of the engines that Girish alluded to. You can get the inspiration from some amazing customer case studies we have on there -- insights from experts like Girish and other people who are talking in depth about the individual engines.

And as you rightly say, Dana, it’s finding the right on-ramp for yourself.  You can look at the case studies we have, the use cases on big data in particular industries, and take a look at what the specific pain point you have today. That’s the hp.com/haven website, and that gives you all of that information.

You can also drill down from there, if you're a developer, and find the tools and resources that we’ve spoken about to enable you to start building apps on top of HAVEn. That’s one part.

The whole power of HP behind this HAVEn platform is in enabling, from an infrastructure and services point of view, to start building these big data analytics. A couple of key things here.

We started to build fully configured appliances around Hadoop and Vertica. So the Converged System’s team in HP has launched the ConvergedSystem 300, which enables you to have Vertica and Hadoop on a pre-configured appliance. That’s a great starting point for someone early on in the big-data analytics life cycle.
Those guys have data scientists and industry experts who can actually help customers go through the design phase for a big-data platform.

To expand on that, the Technology Services team is able to do full consulting on how to optimize the overall infrastructure from the point of view of processing, sharing, and storing this vast amount of information that all organizations are coping with today. That will then start to put in things like 3PAR storage systems and other innovations across the HP hardware business.

Another place where I see customers often needing some help to get started is in understanding exactly what the questions are that we need to be asking in terms of analytics and exactly what algorithms and analytics we need to put in place to get going. This is where the Big Data Discovery Experience Services from HP come in.

This is provided by the Enterprise Services Group (ESG). Those guys have data scientists and industry experts who can actually help customers go through the design phase for a big-data platform and than offer the HAVEn infrastructure supported by the ESG Services team.

Finally, Dana, come and see us on the road. We’ll be at HP Discover in Las Vegas June 10-12. We’re putting together several road shows and events across the main regions in Europe, the Americas, and in Asia Pacific, where we will be taking HAVEn on the road, too. Take a look at that hp.com/haven website, and details of the events will be found on there.

Key messages

Mundada: I wanted to add a couple of things that are a bit relevant. There are two key messages: big data is really important and it’s disrupting business. Your competitors are going to do it. You have a choice to either lead and do it yourself or you will be forced to follow. It’s one of those things that are disrupting industries worldwide.

Now, when you think of big data, don’t think of pieces and don’t think of piece parts. It’s not like you need a separate solution for human information, another for machine logs, and another for structured data. You almost have to think of it holistically, because there are many kinds of newer applications that I’m seeing regularly, where you have to bring all these data types together and create joint applications.

Whichever technologies that you choose and settle on, think of that Microsoft Office-like experience. You want to combine integrated solution across the entire stack and there aren’t that many available in the market today. So whoever you work with, make sure that you’re able to handle that entire piece as one giant puzzle.
You want to combine integrated solution across the entire stack and there aren’t that many available in the market today.

Gardner: Very good. I’m afraid we we'll have to leave it there. You’ve been listening to an executive-level discussion highlighting how the latest version of HP HAVEn produces new business analytics value and strategic return. We have seen how big-data capabilities and advanced business analytics have now really become essential to nearly any business activity.

This discussion marks the latest episode in the ongoing HP Big Data Podcast Series, where leading edge adopters of data-driven business strategies share their success stories and where the transformative nature of big data takes center stage.

Please join me now in thanking today’s guests, Girish Mundada, Chief Technology Officer for HP HAVEn. Thanks so much, Girish.

Mundada: Thanks.

Gardner: And also Dan Wood, Worldwide Solution Marketing Lead for Big Data at HP Software. Thank you, Dan.

Wood: Thanks, Dana.

Gardner: To learn more about how businesses anywhere can best capture knowledge, gain deeper analysis, and rapidly and securely make those insights available to more people on their terms, visit the HP HAVEn Resource Center at hp.com/haven.

I’m Dana Gardner, Principal Analyst at Interarbor Solutions, your moderator for this ongoing sponsored journey into how data is analyzed and used to advance the way we live and work. Thanks so much for listening and do come back next time.

Listen to the podcast. Find it on iTunes. Download the transcript. Sponsor: HP.

Transcript of a BriefingsDirect podcast on how HP is developing products and platforms to help businesses deal with the demands of big data in a competitive environment. Copyright Interarbor Solutions, LLC, 2005-2014. All rights reserved.

You may also be interested in: