California Natural Resources Agency Gains Agility Through Software-Defined Data Center Strategy

Transcript of a BriefingsDirect discussion on how a large state agency harnesses broad virtualization to do more with less in IT while remaining agile and efficient.

Listen to the podcast. Find it on iTunes. Download the transcript. Sponsor: VMware.

Dana Gardner: Hello, and welcome to the special BriefingsDirect podcast series coming to you directly from the recent VMworld 2014 Conference. I'm Dana Gardner, Principal Analyst at Interarbor Solutions, your host throughout this series of BriefingsDirect IT Strategy Discussions.

Gardner

We’re in San Francisco to explore the latest developments in hybrid cloud computing, end-user computing, software-defined data center (SDDC), and virtualization infrastructure management.

Our next innovator case study interview focuses on the California Natural Resources Agency (CNRA) in Sacramento. They have a large purview, overseeing some 25 different agencies. They've set up a SDDC and are deep into the process of maturing its value and utility.

To learn more about how the CNRA gains agility from a SDDC strategy, we welcome Tony Morshed, Chief Technology Officer for the California Resources Data Center in the California Department of Water Resources. Welcome, Tony.

Tony Morshed: Thanks.

Gardner: We’re also here with Michael Hom, Data Center Chief in the IT Infrastructure Services Branch for the California Department of Water Resources. Welcome, Michael.

Michael Hom: Good morning.

Gardner: First, gentlemen, help us understand a little bit about the size of your organization. This is a large state government department, but you're really a department of departments. Help me understand the breadth of your agency.

Morshed: Our department, Water Resources, consists of 3,500 people that are part of the agency that comprises many departments. The bigger ones are Parks and Rec, Cal Fire, and Fish and Wildlife. There are about 28 agencies and conservancies and 25,000 people onboard.

Morshed

Gardner: So in order to support all these people and all these different agencies, a common infrastructure is important, but also it sounds like you need to have differentiation and customization for their specific needs. How do you accomplish both the goal of a common infrastructure for efficiency, but also still be able to meet all your requirements for all those different people?

Morshed: When we started our consolidation effort, we decided to transform ourselves more to an ISP-style setup. We know that most departments have their own IT shops, and you know there is still that trust thing. So we just built the common infrastructure. We let them share the infrastructure, but they have their own security posture. We segregate all their traffic, so each department can still feel like they're autonomous, but yet we all share the infrastructure, in which case we all share the savings.

Mandate to consolidate

Hom: With that, we had a mandate to also consolidate. First, the State of California is really about cost savings. Each of the 25-plus organizations mostly had their own IT shops. By combining the infrastructure as a service (IaaS) in our multitenancy data center, they're able to reap the benefits of cost savings for infrastructure, but also concentrate each department's specific needs and applications.

Hom

Gardner: It sounds as if you have a private-cloud approach, multi-tenancy, and trying to get that elasticity and efficiency going. It also sounds like you’re well on the way to an  SDDC, but you know that means different things to different people. I'd like to get your take on what SDCC means.

Also, are you exploring software-defined storage, software-defined networking, or more on the workload support for the servers. How does it shape up in your mind and in your particular implementation?

Morshed: When the software-defined stuff came out, for me, one of the big things was disaster recovery (DR). If I could stretch my data center into another facility, DR becomes a non-issue, because those workloads can shift between sites without any trouble with automation.

That was the next piece for us -- automation. We realize that we’re part-way there, but to get all the way there, we need to do fuller automation. This means that we need to quit tinkering with the network and storage every time we want to do something new.

Those were big driving factors for going to SDDC. To us, it means that we’re obfuscating the hardware. The hardware’s there. It’s just running. We’re working and tweaking everything at the software layer -- so we could be a lot more agile.

Gardner: Michael, SDDC, how do you define that and to what degree are you into that journey?

Separating the physical

Hom: For the SDDC we really wanted to provide a logical data center to each of our organizations. We wanted to separate the physical, which allows our folks to support more of a logical infrastructure, where they still have autonomy, but the physical layer is basically one and the same.

Today, from a functional point of view, they get what they’ve had before, but without the overhead of physical support. We've used the VMware vSphere and vCloud Suite to provide that software-defined queuing. Right now, we're embarking on a software-defined networking, using VMware NSX and third-party vendors to support that.

We’re looking to use automation soon to help us decrease overhead, and pass those savings on to each of the organizations.

Gardner: Given that you have a common set of infrastructure and, I imagine, lot of common data, are you well into the VMware Virtual SAN adoption for software-defined storage as well? How does that shape up?

Morshed: For Virtual SAN, we're looking at these cases right now and we see some. We’re not quite there yet, and so we’ll focus on the software-defined networking or automation as well. Prior to Virtual SAN, which will probably come later, we’re still evaluating and determining what our needs are in that area.

For me, the greater organizational problems and the structure of your processes become the harder things, because we’re not as used to dealing with those things.

Gardner: It’s complicated, right? There are lot of interdependencies. Certain things happen that ripple across others, and so it’s a crawl-walk-run. Yet the benefits come from a whole greater than the sum of the parts, if I could use a couple of clichés, but it is an interesting time in the business.

What are some of the challenges you have in terms of getting closer to a full SDDC? Are these technology, culture, process, or all of the above?

Morshed: At first it was technology, but the culture and the organizational mindset are the bigger challenge. You can find solutions and work through technology. We're IT people, and we’re used to the technology problems. For me, the greater organizational problems -- and the structure of your processes -- become the harder things, because we’re not as used to dealing with those things.

Gardner: Now, Michael, part of the adoption of a complex, long-term journey on IT is to show results early and get buy-in. Are there instances where you look at that? Maybe, it's the DR, where you can have a sense of better dependability on your resources? Any way to describe what you’ve done early on that has lead to a greater emphasis to adopt more aggressively?

Better service levels

Hom: Definitely. One of the key things with early wins is providing a better service level for provisioning, and that’s something that everybody has been struggling with. With the cloud infrastructure, we've been able to provision within days, if not a day. And that typically beats most of the service tools that each of the organizations have had. So that was an early win.

It’s things like that where we decrease overhead and make IT more accessible for business. That makes it a win, and starts to have the ball rolling as far as other features, such as DR, greater capacity, and things that would be tough for each individual organization to do on their own.

Gardner: Of course, when you do well in a state agency, it’s apparent, right? They're representing services to the public, and you’re the largest US state. You have a large bureaucracy, probably the size of what many countries have. Is there something different about the public sector in terms of accountability or response? How do you have a different set of requirements as a public organization and perhaps enterprises, too?

Morshed: We do have a difference. Our procurement is much harder. Getting people is much harder. We live within a lot of constraints that the private sector doesn’t realize. We have a hard time adjusting our work levels. Can we get more people now? No. It takes forever to get more people, if you can ever get them.

We live within a lot of constraints that the private sector doesn’t realize.

Gardner: So it’s doing more with less over and over again.

Morshed: Constantly doing more with less. Part of this virtualization is survivability. We would never be able to survive or give our business the tools they need to do their business without this. We would just be a sinking ship.

Gardner: So the whole philosophy, Michael, of SDDC and virtualization, is of doing more with less, of automating, of boiling out the manual processes and going to more real-time responsive technology driven infrastructure makes total sense for your organization?

Hom: Definitely. To go with what Tony says, we really don’t have much overhead when we need to respond to future projects. When there's an uptick in activity, there is no way to have more resources available. So we need to build that into our infrastructure to allow for that dynamic bandwidth to happen from a personal level.

Gardner: We’re here of course at VMworld 2014. There’s lot of news going on. Anything in particular piquing your interest, perhaps with the OpenStack support in the EVO Hyper-Converged Infrastructure? What is now on your agenda after hearing some news to reach those goals as you describe them?

EVO looks pretty nice

Morshed: There are a couple of things. EVO looks pretty nice. I was out on the floor and looking at it yesterday and talking with the CIO and I see it as something that we might be able to use for some of our outlying offices, where we have around 100 to 150 people. We can drop something like that in, put virtual desktop infrastructure (VDI) on it, and deliver VDI services to them locally, so they don't have to worry about that traffic going over the wide area network (WAN).

The other piece is the acquisition made recently with CloudVolumes and looking at how we can use that to leverage our VDI structure. We're using another product right now in that space, but again with CloudVolumes it’s been a part of VMworld. It’s more interesting, because we know that the chances of all the software being upgraded and updated in at the same time in interoperability is greater if it’s a VMware product.

For us, it’s been a real struggle to make sure that all the products that we use, interact and as there’s an upgrade, everything upgrades at the same time. To me, those are the two biggest things that I'm getting out of the announcements.

The business could come up with more dollars, but to be able to be more agile and more flexible is where it really pays off.

Gardner: Right and it’s like building a virtuous cycle of adoption benefit because when you do the SDDC built on virtualization that provides a public-cloud benefit. Then, you can start realizing those end-user computing benefits like VDI. So, there’s really this snowball effect.

Is that something that you’ve been able to demonstrate? Do you have any metrics of success that you can point to?

Morshed: We do have some tangible benefits. We have reduced our CAPEX by somewhere around 40 percent and our OPEX around 32 percent. I don’t have the numbers, but we have deployed VDI in the Department of Water Resources and we already virtualized about 600 to 800 desktops. Not only is it helping us save costs there; it’s also used as a strategy for a remote access as a strategy to help protect our server infrastructure by using VDI for admins.

So there are those tangible things that you can reach out and measure and those intangible things, where it’s allowing us to do something easier and more flexible. That, for me, is the bigger win. The business could come up with more dollars, but to be able to be more agile and more flexible is where it really pays off.

Gardner: So we get productivity, we've got DR, which reduces your risk, and we've got some hard savings and economics. It's pretty compelling. Michael, any thoughts about how those fit together, and which ones are more important to you?

More flexible

Hom: Definitely, this allows us to be more flexible, as Tony said, and there are some things that we're trying to do that we would never imagine without a SDDC. So they increased security, greater capacity, capabilities to our business.

Gardner: How about VMware specifically? Is there some differentiator in terms of how they produce this products that has allowed you to follow this journey? Is this more of a partnership than a procurement relationship? It sounds like the track that VMware takes in its strategy very much aligns with yours.

Morshed: It’s very much a partnership. In fact, we basically only want to work with business partners. We don’t want to work with vendors, because we don’t need someone to sell us something and walk away. VMware has been hand-in-hand with us for this whole journey.

When we look at other products for the mix, we look for the deep partners with VMware because we know virtual machines (VMs) are core. So, when we look in the storage partners and we’re looking at networking partner, we’re making sure that those partners are partners of VMware.

We don’t want to work with vendors, because we don’t need someone to sell us something and walk away. VMware has been hand-in-hand with us for this whole journey.

One of the things that we find is the inoperability once everything has been virtualized. Everything has to connect, and it’s not a single stack. So if one thing gets upgraded, we need to make sure that everything across a stack can accept that upgrade. Otherwise, we lose the ability to take the advantage of the upgrade until everybody else catches up.

Early on, we were in that position and we’re doing everything we can to remove ourselves from that position.

Gardner: Michael, any thoughts on the nature of the VMware relationship that you could point to in terms of legacy in an approach that others might want to consider.

Hom: Definitely. We consider VMware a strategic partner. A couple of things that illustrate that is that we’ve been involved with the VMware’s Excellware and Velocity program and that's been two-fold. For the velocity side, we have marked up a fully working SDDC with SX, with virtualized automation, operations in business as a stack.

Gardner: One of the things we've heard here at VMworld is be bold, be brave, be a little bit aggressive. Go out there and do these things. Any thoughts for other organizations that are just dipping their toes into the water? Is it higher risk than reward to be bold and brave in getting early? Or is it perhaps something that allows you to then be a differentiator and be better in your own environment, whether it’s a public or private sector?

Set in stone

Morshed: The first thing is always question what you’ve got that's set in stone, because most of it is not set in stone. We've all heard a lot of things that you can't do. You can’t virtualize Oracle, but you can. You can't do this, you can't do that, you can't get the network focused at the top of the storage. That's all that stuff that you actually can do.

You have to really look at it, peel it back, make sure that "you can't" is an actual thing, and then figure out how to get around it. The way I see it is that, as the world turns, things morph, and if you don’t move into this virtualization space, you're going to be left behind. You're going to be the guy making buggy whips. There are no buggy whips running around. There’s no use for them.

We’re all being asked to do so much more with the same resources or fewer resources. We're all being pushed to keep up with how the demand is going out there. Technology is just jumping, and this is the only way on the infrastructure side to keep up with that.

We’re all being asked to do so much more with the same resources or fewer resources. We're all being pushed to keep up with how the demand is going out there.

Gardner: Michael, any other thoughts in terms of 20/20 hindsight on your experience and why being aggressive and being bold has paid off?

Hom: Virtualization is definitely up and running, at least in state organizations. It’s probably something that we might do that or we might use as a toolset, but from looking at VMware this week, virtualization is the industry standard.

If you don’t take it on, then you really won’t be able to respond to business needs. What happens is that when the official IT organization becomes obsolete, there are going to be ad-hoc IT organizations and those would become the norm. If you want to be relevant, you have to use every tool set that you can to provide the business needs.

Gardner: Very good. I'm afraid we’ll have to leave it there. We've been learning from the California Natural Resources Agency in Sacramento how they’ve been embarking and benefiting from a SDDC strategy. I'd like to thank our guests, Tony Morshed, Chief Technology Officer for the California Natural Resources Data Center in the California Department of Water Resources. Thank you so much, Tony.

Morshed: No problem. It’s been a pleasure.

Gardner: And we've also been joined by Michael Hom, Data Center Chief in the IT infrastructure services branch for the California Department of Water Resources. Thanks so much, Michael.

Hom: Thank you.

Gardner: And also a big thank you to our audience for joining this special podcast series coming to you directly from the recent 2014 VMworld Conference in San Francisco. I'm Dana Gardner, Principal Analyst at Interarbor Solutions, your host throughout this series of VMware-sponsored BriefingsDirect IT strategy discussions. Thanks again for listening, and come back next time.

Listen to the podcast. Find it on iTunes. Download the transcript. Sponsor: VMware.

Transcript of a BriefingsDirect discussion on how a large state agency harnesses broad virtualization to do more with less in IT while remaining agile and efficient. Copyright Interarbor Solutions, LLC, 2005-2015. All rights reserved.

You may also be interested in:

• Creative Solutions in Healthcare improves client services and saves money using VMware vCloud Air hybrid cloud
• University of New Mexico delivers efficient IT services by centralizing on secure, managed cloud automation
• A Gift That Keeps Giving, Software-Defined Storage Now Demonstrates Architecture-Wide Benefits
• Nimble Storage leverages big data and cloud to produce data performance optimization on the fly
• Navicure Gains IT Capacity Optimization and Performance Monitoring Using VMware vCenter Operations Manager
• Big data should eclipse cloud as priority for enterprises
• Different paths to cloud and SaaS enablement yield similar major benefits for Press Ganey and Planview
• Time to give storage virtualization a top place at IT efficiency table

Networks: The New Model for Business

Transcript of a discussion on how business networks are fast emerging as trusted, efficient hubs for cloud-based and data-driven commerce.

Listen to the podcast. Find it on iTunes. Download the transcript. Sponsor: SAP

Dana Gardner: Hi, this is Dana Gardner, Principal Analyst at Interarbor Solutions, and you're listening to BriefingsDirect. Our discussion today explores the role and impact of business networks, the often virtual assemblages of interrelated business services, processes, and data that are transforming how companies and consumers conduct commerce.

Gardner

These business networks are unlocking the ability for companies to extend processes and insights broadly and affordably to customers, suppliers, and other partners. Therefore, they're better able to engage with the participants across these networks in new and innovative ways.

We'll look at the historical record for how open markets and communities are rapidly changing business platforms. We'll see how today's consumer business models -- exemplified by Amazon, Uber, and Airbnb -- are extending to business-to-business (B2B) commerce, allowing buyers and sellers to find and know each other openly and accelerate B2B transactions and commerce efficiencies.

To learn more about the trends that are making business networks more powerful and more important than ever, please join me now in welcoming our guests.

We're here with Marshall Van Alstyne, Professor at Boston University School of Management and Research Scientist at the MIT Center for Digital Business. Welcome to BriefingsDirect, Marshall.

Marshall Van Alstyne: It’s a pleasure to be here, thanks for having us on-board.

Gardner: We're also here with Tim Minahan, Chief Marketing Officer SAP Cloud and Line of Business. Welcome back, Tim.

Tim Minahan: Thanks, Dana, it’s great to be here.

Gardner: Professor Van Alstyne, we've seen a great deal of network effects in business over the past decades. Yet nowadays, the confluence of cloud, mobile, and social and an emphasis on data-driven business processes seems to be accelerating and empowering these shifts. Some organizations call this the Third Platform. How does your research define business platforms, and how are the impacts from these Third Platform technology advancements newly impacting businesses?

Van Alstyne: I emphasize the network effect as one of the driving forces. Indeed, if we can create a positive feedback loop throughout the network effects, that’s where you see the efficiency in the scale happening so quickly.

Van Alstyne

In terms of a definition, we focus on two elements of the platform. The first is an open architecture that third parties can build upon.

The second is the governance rules. How is it that people can participate? Why would they participate? How do you share the profits? How do you resolve conflict? You think about it as a nexus of rules and architecture. If you can put those two things together, you can probably grow an ecosystem that helps to foster and stimulate some of those network effects.

Gardner: Tim Minahan, SAP has been a pioneer inside the four walls of the enterprise over the years with enterprise resource planning (ERP) and other business applications. Now, it seems as if you're recognizing what Professor Van Alstyne has been describing with these network effects and extending your value and business insight and processes across multiple boundaries, outside the four walls of any given enterprise into entire ecosystems.

Next productivity wave

Minahan: Absolutely, Dana. At SAP, we truly believe that the next wave of business productivity is not going to come just within enterprises, but between them. Forty years ago, when SAP arguably invented the whole concept of ERP, businesses were operating much, much differently.

Minahan

We showed them a new way to automate their internal information and process flows, but they were organized in a much more vertically oriented fashion. The employees would graduate from college, spend 40 years with the company, get the gold watch, and retire.

Companies owned most of their infrastructure, their manufacturing facilities, their inventory, their shipping fleets, but certainly this is not your father's business environment anymore. In part, this was accelerated by the recession that we're still emerging from. Companies are less vertically integrated than they were in the past.

They've adopted more variable operating models. They've outsourced everything from manufacturing to customer service, and they need to reach and compete with companies across the street and on the other side of the world. And this is creating new opportunities, as well as new challenges, for businesses today, and it’s increasing demands and expectations on individual functions of their teams.

You're seeing it everywhere. If you have an iPhone, look on the back. It’s designed in California by Apple, but it's built, shipped, and serviced by someone else entirely. Even beyond the physical device, Apple makes most of its revenue from network-based services. iTunes relies heavily on an ecosystem of mobile carriers and artists and studios.

Now, we're seeing this move into the business world, in which companies need to rapidly organize this virtual enterprise, all these resources of employees, manufacturing capacity, logistics, delivery capacity, and customer service to take advantage of certain market opportunity. Or, they need to adapt very quickly to certain market changes, and the only real way to do that is through a digitally connected network of partners, customers, and supply chain.

Gardner: Professor Van Alstyne, this is a tricky time for companies. It seems that they want to retain what works, the business models that have been tried and proven for them. They like having big margins, of course, but in order to grow and to be part of the future they need to expose themselves to these networks, take some risks, maybe lose margin in the process, but perhaps get scale and automation as a payback.

How do you view that? How do you see companies adjusting? Is this a cultural thing where some companies will take this plunge and others don't? It does seem to be a perilous time for companies. I hope they're not just freezing in the headlights.

Van Alstyne: It’s a great question. There are a couple of elements and they vary based on the where the company is currently. If it’s a relatively virgin market, then it’s fairly straightforward for them to invite others in, create the platform, and expand out in that direction. If it’s an existing market for them, they really have to worry about managing the cannibalization question.

I know SAP has also had a very interesting example of that, as you move from on-premise services to hosted services. They're doing a nice job of managing that migration. It’s a little bit tricky in terms of how much you expand the market, but you really need to. You have to realize that the scale, the innovation, the customer engagement happens on these business platforms. Long-term, one really doesn't have a choice.

Platforms vs. products

One of the arguments that we typically make is that even weak platforms tend to beat very strong products. You can look at any number of examples, whether you take a look at the Blackberry, the Sony Personal PlayStation gaming device, or the Garmin device for GPS.

All of these functions are effectively absorbed into the platform. If you manage a platform ecosystem, where third parties can add value on your behalf, you'll grow faster. The platforms almost always will be products. So, in the long-term, companies don't have a choice. They have to move in this direction.

Gardner: So if it’s inevitable that you have to change, it sounds to me from what I've seen at SAP, that they're recognizing this. They're dealing with it themselves as a company, but they're trying to put together a safe path for enterprises to expand into the networked economy. At the same time, they can have trusted partners for automating a lot of the behind-the-scenes activity and allowing them to still function within their business verticals to know what their intellectual property is and to extend to it.

Let’s go back to Tim. Am I reading that right, Tim, that SAP is trying to be, in a sense, the arbiter between risk and exploration when it comes to the networked economy?

Minahan: That’s an accurate depiction, Dana. Think about our personal lives. Whether we're engaging family and friends on Facebook, buying a book or a blender on Amazon, or trying to capture transportation services to get downtown during rush hour on Uber, we're experiencing the scale and simplicity and convenience of personal networks.

At SAP, we believe that solving this inter-enterprise collaboration challenge is one of the biggest opportunities of our era.

We run our daily lives on them now. Unfortunately the business world traditionally has been optimized within the four walls of the enterprise. Companies have invested billions over the past 20 to 30 years in re-engineering their processes and investing in systems to really automate those internal processes and information flows.

They have created what have become islands of efficiency that work very well, and continue to work well, for those that are highly vertically integrated, but very few are, as we talked about earlier.

At SAP, we believe that solving this inter-enterprise collaboration challenge is one of the biggest opportunities of our era. We feel that we're well positioned to do that and have been assembling some of these business networks. We've had the acquisition of Ariba and Fieldglass in the area of contingent workforce and, with Concur, now in the area of travel and expense.

We're complementing that with network extensions of our own, both through the addition of things like the product sustainability network, which leverages the existing connections within the network to help companies better perform tracing and trackability of their products, and the financial services network, which really facilitates and aids payment.

What we're looking at is an opportunity to extend existing IT infrastructure and business process outside the four walls of the enterprise in the most scalable and efficient way possible, no matter what systems a particular company or their trading partners use, all through a single integration point.

Integration adapters

Think about Amazon in your personal lives. You don’t worry about integrating tier trading partners or how you are going to sell that. Amazon takes care of that for you. That’s the same metaphor that we're attempting to carry through into the business world by providing single standard integration adapters or on-ramps to the network that allow you to manage this virtual enterprise in a highly transparent and highly efficient manner.

Gardner: Professor, you mentioned something about the very nature or definition of an enterprise changing. As we look to cloud computing and to these network effects, the ability to outsource so much of what a company does is based on the best value. If doing it internally makes more sense, you do it internally. If going external makes the most sense, you go external. We see this with computing. Hybrid cloud is pretty much about that.

We're also seeing a change in the workforce with contingency and part-time work. What is the new corporation about? It seems it’s mostly rules, relationships, collaboration and management. Let’s go to Tim first. As the very nature of corporations change, it's really about relationships, data, and feedback loops. The data-driven organization, is it really about that. Are we losing something, are we gaining something, or both, Tim, as we seek this new definition of a corporation.

Minahan: Yeah, I think as Professor Van Alstyne said, we're entering an age where the borders between enterprises are being taken down. Companies are moving toward a model where they're managing pools of resources, whether that’s pools of talent around expertise, as you just indicated Dana.

A third of a typical workforce is no longer on the company payroll. It's contingent, statement of work (SOW) workers. In some industries, it’s already more than half. This is fastest growing part of the workforce. HR executives, and I talk with many of them, are beginning to rethink what constitutes the workforce and are looking at pools of talent.

A third of a typical workforce is no longer on the company payroll. It's contingent, statement of work (SOW) workers.

They need to understand where the skill sets lie, not necessarily what roles someone plays today, what skills they have had in the past and be able to, when a particular opportunity or project arises, assemble that expertise very quickly to address that particular project, and disassemble them just as fast, but retain the knowledge within the enterprise for the next time that comes up.

The same thing is true if you're organizing a supply chain and need to be able to serve a new market like China. Where do you put your manufacturing? How do you address distribution, value-added taxes, and customer support. Traditionally, the model would have been to go and establish your own manufacturing facilities, build your own local agents, but no longer.

Now you can quickly assemble and address, or test, a particular market or test a particular product in any given market. Should it work, scale it up. Should it not, scale it down and move on. Networks allow you to achieve this.

I wanted to go back to something that Professor Van Alstyne said that's critically important. I fully agree that the networks go through phases. The first phase is to connect all the various parties, whether they be people, businesses, merchants, banks or all of the above.

The second part is to automate their existing processes. What gets really exciting, once you've automated these processes, once you have these parties collaborating or transacting its scale, are the new insights and entirely new services you could enable.

Transactional information

Once you have these millions of companies or people transacting at scale, you can see the transactional or relationship information. It could be the generated content that helps all members of the community make more informed decisions whether it's about buying or whether it’s about, should I bid on a particular bit of business as a seller or as a bank, mitigating risk in lending to allow them to understated who the buyer is, who the seller is and what their traditional history is.

That is the ultimate big data opportunity, when you have these networks operating at scale. We're beginning to deliver this networked intelligence in the form of insight services to help our members of the communities make important buying, selling, and financing decisions in ways that they couldn’t before.

Van Alstyne: Dana, let me jump in for a second. One of the things that Tim just said is quite important. One of the most interesting elements of the platform is the extent for new business services and new products to emerge. One of the Silicon Valley descriptions of the platform is that you know you have one when your community takes it in a direction you didn't expect.

One of the most interesting elements of the platform is the extent for new business services and new products to emerge.

You need to have made it possible for that. The underlying architecture needs the support the ability to develop something new that wasn't expected, but that’s one of the ways the platform adapts to create new value.

The communities start to add new value and new services in ways that the platform meets the needs of the ecosystem, so it’s this ability to turn out new sources of value based on the underlying architecture. This is one of the key distinctions of platforms that really do add value.

Minahan: I totally agree with that. We've only just entered it into this networked economy or networked era. One of the most exciting things is that it allows you to begin to entirely rethink traditional business models that were organized in an era where, to use an economic term, transaction costs were extremely high.

Look at Uber, what Uber has done, and the challenges we're now seeing around challenging the traditional medallion livery service. That was organized out of a very real concern around safety and issues, but over time, that model matured and unfortunately got very costly.

What you saw were the medallions being aggregated in the hands of a small few who could afford them. That obviously had some implications on the level of service and cost of service to employers. Now we've removed all of the transaction costs and could add up efficiently match demand -- i.e. you as the traveler -- and supply literally anyone that is a card-carrying member of the Uber service.

That’s an entirely new business model that is fundamentally challenging hundreds of old rules and thoughts about what it means to hail a cab. So let me toss in one additional principle that’s often used for design. I'm thinking exactly of the Uber example.

One of the best ways to view a platform is that you have the best platform and the transaction cost are the lowest. If you can get those lowest transactions, you're going to get more business taking place on that platform. So do whatever you can to see if you can lower those transaction costs to get the business going.

Looking for signs

Gardner: So we've taken a look at the inevitability of these networks. We've seen them already very prominent in the business to consumer (B2C) space, consumer activities, and commerce. We’ve recognized that openness is important. So we have innovation. We also recognized the importance of governance and management.

So how do we know when we've done this correctly? Is there a sign? Professor Alstyne, you've mentioned a few that describe powerful and successful networks. Do enterprises have to view themselves differently? Do they need to look at participants in their network as a metric rather than just margin and net in gross revenues and incomes? Is there a way to be successful?

Van Alstyne: Platform businesses behave differently than traditional businesses. Silicon Valley had been using lot of these metrics for engagement. How many new users do you get, and how engaged are they with the platform? It’s a wonderful place to start. Let me give you three rules that we like to use for platform design that actually help get the system running smoothly.

One of them is "frictionless entry." You would like to make it as easy as possible for people to get onboard your platform. It doesn’t matter if that user is on the developer side. You want folks to be able to enter the platform as easily as possible.

If you're bringing in apps in your ecosystem, your users are going to get a bad experience if they are low-quality apps.

The next one is that you need to manage "riskless quality." If anyone can participate, there's a danger that folks who actually get onto the platform don't necessarily add value or they may try to siphon off value. You may worry about lower quality. Atari fell apart as a platform when it got low-quality games on it. Uber has to worry about low-quality drivers. If you're bringing in apps in your ecosystem, your users are going to get a bad experience if they are low-quality apps. So you still need to have riskless quality.

The first principle is frictionless entry, and you need to manage riskless quality from the users on that side.

The third one is "permissionless innovation." You don't want your developers to necessarily have to come to you to get permission. There is always this danger because you own the platform. You have enormous power over them and you could simply take that idea and run with it. You need the ecosystem partners to be able to run with an idea and create something novel on their own and let them have that value. They don’t need to get permission first.

These are three rules that we use for design -- frictionless entry, riskless quality, and permissionless innovation. Those are really good guidelines and are helping to get these ecosystems to grow quickly, get more users onboard, and get your value add from third party participation.

Gardner: Tim Minahan at SAP Cloud, tell us a bit about what you're doing at SAP, some of your acquisitions, besides your cloud, this ability to be frictionless and help people come on the network easily. You recently finished up the Concur acquisition, one of your largest ever. Explain how you're growing the size of your network?

Application agnostic

Minahan: What Professor Van Alstyne just talked about are principles that we subscribe to. In a business network sense, it also requires you to be open and application agnostic and largely agnostic to the on-ramps. That’s part of the frictionless entry.

So regardless of what system you are using, whether it’s SAP, Ariba, Concur, Oracle, PeopleSoft Info, etc., you need to be able to attach the systems to the network, those demand systems that allow you to connect and collaborate through the network to extend that business process and engage with your customers, suppliers, and other partners.

Think about our personal lives, whether it’s Uber or Amazon, those networks that are most powerful and most impactful on our personal lives allow a seamless process. You don’t even think about the process, but it is end to end.

In the case of Amazon you don’t think about the buying process -- how am I going to connect to those individual merchants? They're already connected for you. Ultimately, you believe you're buying from Amazon, but you might be buying from an independent provider and they are still delivering to you.

Those networks that are most powerful and most impactful on our personal lives allow a seamless process.

Likewise, you don't think about, gee, now that I have placed the order, do I have to call my bank to settle out? No, that’s handled for you, SAP has been using these guiding principles to go out and make sure that we're building the network appropriately, both in our organic means through innovation and the introduction of new services, like payments, financing, and dynamic discounting, both independently with other members and financial institutions of the network, as well as inorganically through acquisitions.

Gardner: As we close out, we've determined that the number of participants and the value of the commerce is super important in these networks. Several times we've also touched on this feedback loop in the data. So as we look to the future, we might have  competing networks. If we assume that those networks are going to have some frictionless ability to move on and off of them, then the best network is where people will go.

Is the best network the one that provides the best insights in data? Can we close out our discussion by looking at the importance of shared data and analysis and the ability to counter that analysis up from these transactions as a differentiator going forward that will pick winners and losers in open commerce network environment, if you will.

Let’s go to Professor Van Alstyne first. Who is going to win in this network environment and is the data and openness and availability of analytics going to be a major determinant of that?

Van Alstyne: I am going to argue the best platform is the one that creates the most value over time and that probably means that the data analytics, those that can use the data to create these data-driven feedback loops, will be the winners.

One of the things that I want to emphasize is that frictionless entry and the ability of the movement of data doesn’t necessarily mean that switching costs are going to be low or that it’s going to be easy to necessarily change networks. Network effects do create winner-take-all markets, they do create these behemoths. Google Search has 67 percent market share in the US and 90 percent in Europe. Facebook has 1.3 billion users. I think Amazon web services has a huge proportion of the cloud services.

We need to be careful if we think we're going to be able to switch networks easily. There are going to be some very substantial winner-take-all networks and some concentration at the top. Cloud and data is going to be an integral part of that, as the data creates these data-driven feedback loops that support these network effects.

Data and analytics

Gardner: Tim, our last word to you on the role of data and analytics as the determinant of the most valuable network.

Minahan: I agree with the professor that the key litmus test of who wins is the platform or the network that creates the most value, and I think value comes in a few flavors.

Number one is relevancy. Are my trading partners there? At SAP, typically about half of any given company's trading partners are already connected in transacting. That makes the frictionless entry that much easier. Think about Facebook. Why would you join any other personal network when most of your friends and family are already there.

The second is the aspect of value. Can I manage most of my collaborations in a single environment or do I need to join multiple networks in order to complete a transactional process? The more capabilities you can layer in to make it more convenient for all members of the network to collaborate, the more value add.

The more capabilities you can layer in to make it more convenient for all members of the network to collaborate, the more value add.

And third, I believe that we've only scratched the surface on these insights. I wouldn’t even say that it’s a two-sided model. It’s a multi-sided model, where once you get these parties collaborating at scale, the transactional relationship and community-generated content can deliver new insights to help folks make more informed decisions, whether it's, which trading partners to do business with or which areas of your existing supply chain might be presented with risk in the future and you need to adapt quickly or which financial settlement options you have to settle out to help you optimize cash flow.

These are new insights that were previously impossible with traditional on-premise and point-to-point integration models and it can only be accomplished in a network model.

Gardner: Very good. I'm afraid we'll have to leave it there. You've been listening to a sponsored BriefingsDirect podcast discussion on business networks. You've heard how open markets and communities are rapidly changing business platforms, allowing sellers and buyers to find and know each other openly and therefore accelerating B2B transactions and commerce efficiencies.

And we have heard how companies can exploit business networks to automate and analyze how they transact commerce in new innovative ways.

So a big thank you to our guests, Marshall Van Alstyne, Professor at Boston University School of Management and Research Scientist at the MIT Center for Digital Business. Thank you so much, Marshall.

Van Alstyne: Dana, thanks so much for allowing us to participate.

Gardner: We also like to remind our listeners that there will be a new book called "Platform Strategies" out in 2015 from Professor Alstyne and co-authors.

Also a big thank you to Tim Minahan, Chief Marketing Officer, SAP Cloud and Line of Business. Thank you, Tim.

Minahan: Thank you, Dana, a great conversation.

Gardner: And a thanks to our audience as well for joining us for this discussion. I'm Dana Gardner, Principal Analyst at Interarbor Solutions. Don't forget to come back next time for more BriefingsDirect.

Listen to the podcast. Find it on iTunes. Download the transcript. Sponsor: SAP.

Transcript of a discussion on how business networks are fast emerging as trusted, efficient hubs for cloud-based and data-driven commerce. Copyright Interarbor Solutions, LLC, 2005-2015. All rights reserved.

You may also be interested in:

• Big Data Meets the Supply Chain -- SAP's Supplier InfoNet and Ariba Network Enable Companies to Predict and Manage Supplier Risk
• Ariba's Product Roadmap for 2014 Points to Instant, Integrated and Data-Rich Business Cloud Services
• Q&A with Marshall Van Alstyne, Professor, Boston University School of Management and Research Scientist MIT Center for Digital Business
• Modern supply chains — How innovative sellers engage customers in entirely new ways
• Arlington Computer Products Simplifies and Speeds its Billing and Payments Using New AribaPay
• The Future of Business Success Hinges on How Well Innovation Drives Supply Chain and Procurement Advantages
• Workforce of the Future -- and Why Preparing Means Rethinking Human Resources Now
• Spot buying automated by Ariba gives start-up KooZoo a means to shop more efficiently
• Combining big data and cloud capabilities for ecommerce matches buyers and sellers like never before
• Here's Why Healthcare Businesses Must Efficiently Manage Their Suppliers, Purchases, and Processes

HP Vertica Enables Rapid Matching of Consumer Inferences to Ads at Huge Scale for adMarketplace

Transcript of a BriefingsDirect podcast on how big data and data analytics combine to instantly match search users with ads appropriate to their interests.

Listen to the podcast. Find it on iTunes. Download the transcript. Sponsor: HP.

Dana Gardner: Hello, and welcome to the next edition of the HP Discover Podcast Series. I'm Dana Gardner, Principal Analyst at Interarbor Solutions, your host and moderator for this ongoing sponsored discussion on IT innovation and how it’s making an impact on people’s lives.

Once again, we're focusing on how companies are adapting to the new style of IT to improve IT performance and deliver better user experiences, as well as better business results.

Become a member of myVertica today
and gain access to the
FREE HP Vertica Community Edition

This time, we're coming to you directly from the recent HP Big Data 2014 Conference in Boston. We're here to learn directly from IT and business leaders alike how big data, cloud, and converged infrastructure implementations are supporting their goals.

Gardner

Our next innovation case study interview highlights how adMarketplace is using big data to improve its search advertising capabilities for its customers. [See part one of this two-part series.] We'll learn more about search for advertising and how big data plays a role in that in our discussion with our guest, Raj Yakkali, Director of Data Infrastructure at adMarketplace in New York. Welcome, Raj.

Raj Yakkali: Hello, Dana. Very nice to be here, and thank you.

Gardner: Good to have you here. Tell us a little bit about adMarketplace. What do you do, and why is big data such a big part of that?

Yakkali: adMarketplace is a leading advertising platform for search intent. We provide the advertisers with the consumer space where they can project their ads. The benefit of the adMarketplace comes into play where we provide a data platform that can match those ads with the right user intent.

Yakkali

When user searches for a certain keyword, they're directly telling us what they want to see, and we match it perfectly well with our ads. The relationship that we have with our advertisers is that we match them well and make it accessible in exactly what the user is thinking. We do some predictive analytics on top of what the user is saying. We add that dimension to our user search and provide ads aptly.

Gardner: I'm all for getting better ads based on lot of things I already get. Do you have more than just keywords in terms of how you can draw inference, and what sort of scale of data are we talking about when it comes to all that inference information about an intent on behalf of the consumer?

15 dimensions

Yakkali: Keyword search is one side or one dimension of the user search. There are also category campaigns that the advertisers are running. At the same time, there's a geospatial analysis to it as well. There are 15 dimensions that we go through to provide an ad that is perfectly fit for the advertiser and for the consumer to see and take advantage of to meet their needs. With some of the ads, we are trying to serve the user’s requirements and needs.

Gardner: With all these variables, this sounds like you're going to be gathering an awful lot of information. You also need to reply back with your results very fast or you lose the opportunity for that consumer to get the ad and then even click through and make a decision. Tell me about scale and speed.

Yakkali: You're right on with that question. In this business, latency is your enemy. If you look into the certain metrics, there are almost a half a billion requests that we're receiving every day and we have to match all of those ads with a sub-second performance. We have internal proprietary datasets, which we take care of before matching these ads. And there are two platforms that we've built internally.

One is called Bid Smart. That performs the analysis between the user intent and the traffic sources that the user search is coming from. At the same time, the price of that ad goes to the publisher. There are the pricing strategies, the traffic sources, and the user intent of the search. All of these things are put together. That predictive analytics system gathers all this information and emits the right ad towards the consumer.

With the partnership with Vertica, we’re able to take the dataset, derive analytics about it, and provide our marketers with all that information.

On top of it, if you look into the amount of data, those half a billion requests that are coming into our system, it generates around two terabytes per hour. At certain times, we can't store all of it for analytics. There is a lot of data that's not inside the database. Now, with the partnership with Vertica, we’re able to take the dataset, derive analytics about it, and provide our marketers with all that information. Bid Smart is the one that does the pricing and matching.

The other thing is Advertiser 3D, which provides that detailed analytics into all these dimensions on the metrics. That provides a very good insight. Now, when it comes to the competition or the opportunity to deliver the right ad at the right time, that's where data work flows make a difference.

We utilize Vertica to directly stream all this click data into it, rather than going into certain other locations and then doing it in a batch format. We directly live-stream that data into Vertica, so that it is readily available for analytics. Our Bid Smart System makes use of that dataset. That's where we get the opportunity to deliver much better ads, with price tags, and the right user intent matched.

Gardner: It sounds very complex. There's an awful lot going on for just serving up an ad. I suppose people don’t appreciate that, but the economics here are very compelling, the more refined and appropriate an ad can be, the more likely the consumer is to buy, but there are a lot of resources that don't get wasted in the meantime. Do you have any sense of what the payoff is, either in business, financial, or technical terms for when you can really accomplish this goal of targeted advertising?

Conversion rate

Yakkali: So our conversion rate is a major key performance indicator (KPI) when it comes to understanding how well we are doing. When we see higher conversion rates, that gives us the sense that we've done the best job and user is happy with what they are searching and what they are getting.

At the at the same time, the publishers, as well as the advertisers, are happy, because the user is coming to us again and again to get that similar, beautiful experience. The advertisers are able to sell more products that meet the needs of the user. And the users are able to get the product that really caters to their needs. We're in the middle of all these things, trying provide the facilitation to the advertisers, as well as the users and the publishers' space.

Gardner: I daresay this is the very future of advertising. Now for you to accomplish these goals and create those positive KPIs, are you housing Vertica in your own data center, do you use cloud, hybrid cloud? Given that you have different platforms, different datasets, how do you manage this technically?

Yakkali: On that end, we started with testing cloud two or three years ago, but again, it turned out that because of so many unknowns and troubleshooting, we had to go with our data centers. Now, we host all our systems in our own data centers and we manage it.

We have our own hardware to deal with. Our system is a 24/7, and we have to be able to deliver the sub-second latency performance. Having your own infrastructure, you have the controlled environment where you can tweak and tune your system to get the best performance out of it.

Become a member of myVertica today
and gain access to the
FREE HP Vertica Community Edition

Considering that it is a 24/7, there are fewer excuses that you can get away with in not delivering it. For that, we do innovation in terms the data flows and the process of how we ingest the data, how we process the data, how we emit the data, and how we clean up the data when we don’t really need it.

All these things have to come together, and it really helps us having that control on all of our infrastructure and all elements in the data pipeline, starting from the user intent and user search, until we provide the data and the results.

Gardner: How long have you been using Vertica in this regard and how did you go about making that decision?

Yakkali: We've been using Vertica for four to five years and our data pipeline was not on Vertica to start with, but as Vertica came into the picture and we saw the great beauty and the powerful features that it brings to capitalize our ability.

That really helped us. With Vertica in place, we have been migrating our mechanics slowly to use it for the real-time analysis and real-time bidding and all those beautiful features that make us do what we can do better. So it’s been a great partnership with Vertica and we see many more features coming in with the new version. Our Bid Smart mechanism is also improving, and with that, algorithmic capabilities are increasing. So it’s progressing.

Feedback loop

Gardner: Tell us a little bit about where your business is heading. In addition to speed, complexity, and scale, where do you see the ability to create this feedback loop? It’s very rapid feedback loop between a lot of incoming data and an action like streaming up an ad. It seems like this could be applied to either other marketing or advertising chores or perhaps even have an ancillary business-development direction. You’ve got this platform and these data centers. Is there something else that you're gearing up for?

Yakkali: At this point, we're in the business of connecting the advertisers, the publishers, and the users. But that is an untapped business to what it can accomplish. The market has started its pathway towards the level of reaching that epitome. If we take a step back and try to understand it, initially, when search started, there was no Google or anything. It was more about curated search.

So the publishers put out all this content together and then projected it out to the user. They didn't know what user wanted. At the same time, when the user looked at this content, they didn't know whether they want it or whether it catered to their needs.

Then, Google came along and user search started. What that directly told was "I want this piece of information. I want to use this piece of information. And I want to see this ad that is relevant to my needs." That’s a very powerful thing. When you hear that part, you're able to analyze that piece and match it properly with the advertisers. But then again, it started to fragment.

At this point, we're in the business of connecting the advertisers, the publishers, and the users.

Now, it’s not only Google. There is Yahoo, Bing, there is mobile, and there are certain apps. There are many apps in the mobile space and each one has its own search. So not all the searches are going to Google, Yahoo, or Bing. Search is already fragmented.

We tap all those pieces. The market that is beyond Google. Yahoo-Bing is stronger and it is growing. So there is a lot of market that needs to be tapped into. We come into the place connecting the advertisers to tap that untapped marketplace.

We've been improving our internal Bid Smart algorithm that came out in the last year. Then, we also launched Advertiser 3D last year as well. Those two products have been providing tremendous growth in our revenue, and the retention rates have been stellar.

The top 60 percent of Google’s top spenders are working with us to complement their business. At the same time, we're also able to provide 50 percent increase in year-over-year revenues. It's additional revenue for them, and even our revenues are increasing based on that fact.

Gardner: It seems like you have an awful lot of runway ahead of you in terms of where search could be applied, and analytics can be drawn from that to augment these services and explode that market.

Is Vertica being used just for the intercept between the incoming data and the outgoing ad, or you are also analyzing what goes on within these marketplace so that you better appreciate, whether you can offer reports, audit trails, and that sort of thing? Is this an inclusive platform, or do you use different analytics platforms for different aspects of what you are doing?

End to end

Yakkali: We do almost everything. It is an end-to-end platform. As part of the business we look into the operational metrics of the whole thing, starting from the user search until the ad is delivered. Then, from that end, there is always that analytics piece that comes onto play, which provides insights to the marketers.

Our market base is filled with the very data-savvy marketers, and they look into each and every data dimension to understand their return on investment (ROI). We give them transparency through our Advertise 3D System and utilizing that, they're able to navigate through the space and aptly tune their campaigns to get the best out of it and to deliver the best to the customer.

Gardner: Any thoughts about other organizations that are also facing significant challenges around speed, scale, also perhaps with a big runway, in terms of knowing that more and more business could be coming their way therefore more data? What would you advise them in terms of the data architecture or the planning in order to accomplish the goals?

Yakkali: When we look at the industries and the market, the ad industry still is untapped. The healthcare industry is just getting into the business of doing much more with analytics. It’s all about the speed and the latency and the insights as well. One, at the operational level and the other, at the insight level to do more innovation on top of it.

Our market base is filled with the very data-savvy marketers, and they look into each and every data dimension to understand their return on investment (ROI).

The ability to listen to the customer depends on how fast you can capture all that feedback, and you tighten that loop of feedback so that you're able to do something with it and make a better product out of it.

So it’s all about taking a look at the datasets very closely as to what they mean, what the user is asking us, what do they want to see, and how you are listening to the customer. Those two aspects really make the difference.

You want to listen to the customer, what they really want. Are you providing it and are you able to guess what they want for tomorrow for that predictive, and going into prescriptive analytics, phase later on. You're telling them what they need to do even before they tell you.

That's the stage that the market is going towards. We're not even scratching the surface of prescriptive analytics. The wave has not yet started towards that route. We're still at the predictive analytics phase, and there is still a lot more to go within that space. Get the foundation stronger, drive towards prescriptive analytics, and listening to your customer, are the three aspects that would make any industry. Those three would be the key foundational pieces for making innovation.

Become a member of myVertica today
and gain access to the
FREE HP Vertica Community Edition

Gardner: Thanks so much. We've been learning about how adMarketplace is using big data to perform some very complex marketing activities for their advertisers to match intent from a customer with an ad that suits their needs based on ever-growing amount of data and inference. [See part one of this two-part series.] I'd like to thank our guest. We've been joined Raj Yakkali, Director of Data Infrastructure at adMarketplace in New York. Thanks so much.

Yakkali: Thank you very much, Dana. It was a pleasure talking to you.

Gardner: And I'd like to thank our audience as well, for joining us for the special new style of IT discussion, coming to you directly from the recent HP Big Data 2014 Conference in Boston.

I'm Dana Gardner, Principal Analyst at Interarbor Solutions, your host for this ongoing series of HP sponsored discussions. Thanks again for listening, and come back next time.

Listen to the podcast. Find it on iTunes. Download the transcript. Sponsor: HP.

Transcript of a BriefingsDirect podcast on how big data and data analytics combine to instantly match search users with ads appropriate to their interests. Copyright Interarbor Solutions, LLC, 2005-2015. All rights reserved.

You may also be interested in:

• adMarketplace solves search intent challenge with HP Vertica big data warehouse
• HP launches Haven OnDemand to deliver big data services suite in the cloud
• Hortonworks accelerates the big data mashup between Hadoop and HP Haven
• HP simplifies Foundation Care Services to deliver just-in-time, pan-IT tech support
• HP Analytics Blazes New Trails in Examining Business Trends from Internal Data
• Vichara Technologies grows the market for advanced analytics after cutting its big data teeth on Wall Street
• Big Data Analysis Provides New Degree of Real-Time Financial Position Insights to Large Russian Bank
• A practical guide to rapid IT Service Management as a foundation for overall business agility
• Journey to SAP quality — Home Trust builds center of excellence with HP ALM tools
• ITSM adoption forces a streamlined IT operations culture at Desjardins, paves the way to cloud
• Cloud services brokerages add needed elements of trust and oversight to complex cloud deals