How the Data Science Profession is Growing in Value and Impact Across the Business World

Transcript of a discussion on how the role of the data scientist in the enterprise is expanding in both importance and influence that warrants a new level of business analysis professional certification.

Listen to the podcast. Find it on iTunes. Download the transcript. Sponsor: The Open Group.

Dana Gardner: Hi, this is Dana Gardner, Principal Analyst at Interarbor Solutions, and you’re listening to BriefingsDirect. Our next business trends panel discussion explores how the role of the data scientist in the enterprise is expanding in both importance and influence.

Gardner

Data scientists are now among the most highly sought-after professionals, and they are being called on to work more closely than ever with enterprise strategists to predict emerging trends, optimize outcomes, and create entirely new kinds of business value.

To learn more about modern data scientists, how they operate, and why a new level of business analysis professional certification has been created by The Open Group, we are now joined by our panel, Martin Fleming, Vice President, Chief Analytics Officer, and Chief Economist at IBM. Welcome, Martin.

Martin Fleming: Thanks, Dana.

Gardner: We are also joined by Maureen Norton, IBM Global Data Scientist Professional Lead, Distinguished Market Intelligence Professional, and author of Analytics Across the Enterprise. Welcome, Maureen.

Maureen Norton: Thank you very much, pleasure to be here.

Gardner: And we’re also here with George Stark, Distinguished Engineer for IT Operations Analytics at IBM. Welcome, George.

George Stark: Thank you, Dana.

Gardner: We are now characterizing the data scientist as a profession. Why have we elevated the role to this level, Martin? 

Data scientists add value 

Fleming

Fleming: The benefits we have from the technology that’s now available allow us to bring together the more traditional skills in the space of mathematics and statistics with computer science and data engineering. The technology wasn't as useful just 18 months ago. It’s all about the very rapid pace of change in technology.

Gardner: Data scientists used to be behind-the-scenes people; sneakers, beards, white lab coats, if you will. What's changed to now make them more prominent?

Norton: Today’s data scientists are consulting with the major leaders in each corporation and enterprise. They are consultants to them. So they are not in the back room, mulling around in the data anymore. They're taking the insights they're able to glean and support with facts and using them to provide recommendations and to provide insights into the business.

Gardner: Most companies now recognize that being data-driven is an imperative. They can’t succeed in today's world without being data-driven. But many have a hard time getting there. It's easier said than done. How can the data scientist as a professional close that gap?

Stark

Stark: The biggest drawback in integration of data sources is having disparate data systems. The financial system is always separate from the operational system, which is separate from the human resources (HR) system. And you need to combine those and make sure they're all in the same units, in the same timeframe, and all combined in a way that can answer two questions. You have to answer, “So what?” And you have to answer, “What if?” And that’s really the challenge of data science.

Gardner: An awful lot still has to go on behind the scenes before you get to the point where the “a-ha” moments and the strategic inputs take place.

Martin, how will the nature of work change now that the data scientist as a profession is arriving – and probably just at the right time?

Fleming: The insights that data scientists provide allow organizations to understand where the opportunities are to improve productivity, of how they can help to make workers more effective, productive, and to create more value. This enhances the role of the individual employees. And it’s that value creation, the integration of the data that George talked about, and the use of analytic tools that's driving fundamental changes across many organizations.

Captain of the data team

Gardner: Is there any standardization as to how the data scientist is being organized within companies? Do they typically report to a certain C-suite executive or another? Has that settled out yet? Or are we still in a period of churn as to where the data scientist, as a professional, fits in?

Norton

Norton: We're still seeing a fair amount of churn. Different organizing approaches have been tried. For example, the centralized center of excellence that supports other business units across a company has a lot of believers and followers.

The economies of scale in that approach help. It’s difficult to find one person with all of the skills you might need. I’m describing the role of consultant to the presidents of companies. Sometimes you can’t find all of that in one individual -- but you can build teams that have complimentary skills. We like to say that data science is a team sport.

Gardner: George, are we focusing the new data scientist certification on the group or the individual? Have we progressed from the individual to the group yet?

Stark: I don’t believe we are there yet. We’re still certifying at the individual level. But as Maureen said, and as Martin alluded to, the group approach has a large effect on how you get certified and what kinds of solutions you come up with.

Gardner: Does the certification lead to defining the managerial side of this group, with the data scientist certified in organizing in a methodological, proven way that group or office?

Learn How to Become
Certified as a

Data Scientist

Fleming: The certification we are announcing focuses not only on the technical skills of a data scientist, but also on project management and project leadership. So as data scientists progress through their careers, the more senior folks are certainly in a position to take on significant leadership and management roles.

And we are seeing over time, as George referenced, a structure beginning to appear. First in the technology industry, and over time, we’ll see it in other industries. But the technology firms whose names we are all familiar with are the ones who have really taken the lead in putting the structure together.

Gardner: How has the “day in the life” of the typical data scientist changed in the last 10 years?

Stark: It’s scary to say, but I have been a data scientist for 30 years. I began writing my own Fortran 77 code to integrate datasets to do eigenvalues and eigenvectors and build models that would discriminate among key objects and allow us to predict what something was.

The difference today is that I can do that in an afternoon. We have the tools, datasets, and all the capabilities with visualization tools, SPSS, IBM Watson, and Tableau. Things that used to take me months now take a day and a half. It’s incredible, the change.

Gardner: Do you as a modern data scientist find yourself interpreting what the data science can do for the business people? Or are you interpreting what the business people need, and bringing that back to the data scientists? Or perhaps both?

Collaboration is key

Stark: It’s absolutely both. I was recently with a client, and we told them, “Here are some things we can do today.” And they said, “Well, what I really need is something that does this.” And I said, “Oh, well, we can do that. Here’s how we would do it.” And we showed them the roadmap. So it’s both. I will take that information back to my team and say, “Hey, we now need to build this.”

Gardner: Is there still a language, culture, or organizational divide? It seems to me that you’re talking apples and oranges when it comes to business requirements and what the data and technology can produce. How can we create a Rosetta Stone effect here?

Norton: In the certification, we are focused on supporting that data scientists have to understand the business problems. Everything begins from that.

In the certification, we are focused on supporting that data scientists have to understand the business problems. Everything begins from that. Knowing how to ask the right questions, to scope the problem, and be able to then translate is essential.

Knowing how to ask the right questions, to scope the problem, and be able to then translate [is essential]. You have to look at the available data and infer some, to come up with insights and a solution. It's increasingly important that you begin with the problem. You don't begin with your solution and say, “I have this many things I can work with.” It's more like, “How we are going to solve this and draw on the innovation and creativity of the team?”

Gardner: I have been around long enough to remember when the notion of a chief information officer (CIO) was new and fresh. There are some similarities to what I remember from those conversations in what I’m hearing now. Should we think about the data scientist as a “chief” something, at the same level as a chief technology officer (CTO) or a CIO?

Chief Data Officer defined 

Fleming: There are certainly a number of organizations that have roles such as mine, where we've combined economics and analytics. Amazon has done it on a larger scale, given the nature of their business, with supply chains, pricing, and recommendation engines. But other firms in the technology industry have as well.

We have found that there are still three separate needs, if you will. There is an infrastructure need that CIO teams are focused on. There are significant data governance and management needs that typically chief data officers (CDOs) are focused on. And there are substantial analytics capabilities that typically chief analytics officers (CAOs) are focused on.

It's certainly possible in many organizations to combine those roles. But in an organization the size of IBM, and other large entities, it's very difficult because of the complexity and requirements across those three different functional areas to have that all embodied in a single individual.

Gardner: In that spectrum you just laid out – analytics, data, and systems -- where does The Open Group process for a certified data scientist fit in?

Fleming: It's really on the analytics side. A lot of what CDOs do is data engineering, creating data platforms. At IBM, we use the term Watson Data Platform because it's built on a certain technology that's in the public cloud. But that's an entirely separate challenge from being able to create the analytics tools and deliver the business insights and business value that Maureen and George referred to.

Gardner: I should think this is also going to be of pertinent interest to government agencies, to nonprofits, to quasi-public-private organizations, alliances, and so forth.

Given that this has societal-level impacts, what should we think about in improving the data scientists’ career path? Do we have the means of delivering the individuals needed from our current educational tracks? How do education and certification relate to each other?

Academic avenues to certification

Fleming: A number of universities have over the past three or four years launched programs for a master’s degree in data science. We are now seeing the first graduates of those programs, and we are recruiting and hiring.

I think this will be the first year that we bring in folks who have completed a master’s in data science program. As we all know, universities change very slowly. It's the early days, but demand will continue to grow. We have barely scratched the surface in terms of the kinds of positions and roles across different industries.

That growth in demand will cause many university programs to grow and expand to feed that career track. It takes 15 years to create a profession, so we are in the early days of this.

Norton: With the new certification, we are doing outreach to universities because several of them have master’s in data analytics programs. They do significant capstone-type projects, with real clients and real data, to solve real problems.

We want to provide a path for them into certification so that students can earn, for example, their first project profile, or experience profile, while they are still in school.

Gardner: George, on the organic side -- inside of companies where people find a variety of tracks to data scientist -- where do the prospects come from? How does organic development of a data scientist professional happen inside of companies?

Stark: At IBM, in our group, Global Services, in particular, we've developed a training program with a set of badges. They get rewarded for achievement in various levels of education. But you still need to have projects you've done with the techniques you've learned through education to get to certification.

Having education is not enough. You have to apply it to get certified.

Gardner: This is a great career path, and there is tremendous demand in the market. It also strikes me as a very fulfilling and rewarding career path. What sorts of impacts can these individuals have?

Learn How to Become
Certified as a

Data Scientist

Fleming: Businesses have traditionally been managed through a profit-and-loss statement, an income statement, for the most part. There are, of course, other data sources -- but they’re largely independent of each other. These include sales opportunity information in a CRM system, supply chain information in ERP systems, and financial information portrayed in an income statement. These get the most rigorous attention, shall we say.

We're now in a position to create much richer views of the activity businesses are engaged in. We can integrate across more datasets now, including human resources data. In addition, the nature of machine learning (ML) and artificial intelligence (AI) are predictive. We are in a position to be able to not only bring the data together, we can provide a richer view of what's transpiring at any point in time, and also generate a better view of where businesses are moving to.

It may be about defining a sought-after destination, or there may be a need to close gaps. But understanding where the business is headed in the next 3, 6, 9, and 12 months is a significant value-creation opportunity.

Gardner: Are we then thinking about a data scientist as someone who can help define what the new, best business initiatives should be? Rather than finding those through intuition, or gut instinct, or the highest paid person's opinion, can we use the systems to tell us where our next product should come from?

Pioneers of insight

Norton: That's certainly the direction we are headed. We will have systems that augment that kind of decision-making. I view data scientists as pioneers. They're able to go into big data, dark data, and a lot of different places and push the boundaries to come out with insights that can inform in ways that were not possible before.

It’s a very rewarding career path because there is so much value and promise that a data scientist can bring. They will solve problems that hadn't been addressed before.

It's a very exciting career path. We’re excited to be launching the certification program to help data scientists gain a clear path and to make sure they can demonstrate the right skills.

It's a very rewarding career path because there is so much value and promise that a data scientist can bring. They will solve problems that hadn't been addressed before.

Gardner: George, is this one of the better ways to change the world in the next 30 years?

Stark: I think so. If we can get more people to do data science and understand its value, I'd be really happy. It's been fun for 30 years for me. I have had a great time.

Gardner: What comes next on the technology side that will empower the date scientists of tomorrow? We hear about things like quantum computing, distributed ledger, and other new capabilities on the horizon?

Future forecast: clouds

Fleming: In the immediate future, new benefits are largely coming because we have both public cloud and private cloud in a hybrid structure, which brings the data, compute, and the APIs together in one place. And that allows for the kind of tools and capabilities that necessary to significantly improve the performance and productivity of organizations.

Blockchain is making enormous progress and very quickly. It's essentially a data management and storage improvement, but then that opens up the opportunity for further ML and AI applications to be built on top of it. That’s moving very quickly.

Quantum computing is further down the road. But it will change the nature of computing. It's going to take some time to get there but it nonetheless is very important and is part of that what we are looking at over the horizon.

Gardner: Maureen, what do you see on the technology side as most interesting in terms of where things could lead to the next few years for data science?

Norton: The continued evolution of AI is pushing boundaries. One of the really interesting areas is the emphasis on transparency and ethics, to make sure that the systems are not introducing or perpetuating a bias. There is some really exciting work going on in that area that will be fun to watch going forward.

Gardner: The data scientist needs to consider not just what can be done, but what should be done. Is that governance angle brought into the certification process now, or something that it will come later?

Stark: It's brought into the certification now when we ask about how were things validated and how did the modules get implemented in the environment? That’s one of the things that data scientists need to answer as part of the certification. We also believe that in the future we are going to need some sort of code of ethics, some sort of methods for bias-detection and analysis, the measurement of those things that don't exist today and that will have to.

Gardner: Do you have any examples of data scientists doing work that's new, novel, and exciting?

Rock star potential

Fleming: We have a team led by a very intelligent and aggressive young woman who has put together a significant product recommendation tool for IBM. Folks familiar with IBM know it has a large number of products and offerings. In any given client situation the seller wants to be able to recommend to the client the offering that's most useful to the client’s situation.

And our recommendation engines can now make those recommendations to the sellers.  It really hasn't existed in the past and is now creating enormous value -- not only for the clients but for IBM as well.

Gardner: Maureen any examples jump to mind that illustrate the potential for the data scientist?

Norton: We wrote a book, Analytics Across the Enterprise, to explain examples across nine different business units. There have been some great examples in terms of finance, sales, marketing, and supply chain.

Learn How to Become
Certified as a

Data Scientist

Gardner: Any use-case scenario come to mind where the certification may have been useful?

Norton: Certification would have been useful to an individual in the past because it helps map out how to become the best practitioner you can be. We have three different levels of certification going up to the thought leader. It's designed to help that professional grow within it.

Stark: A young man who works for me in Brazil built a model for one of our manufacturing clients that identifies problematic infrastructure components and recommends actions to take on those components. And when the client implemented the model, they saw a 60 percent reduction in certain incidents and a 40,000-hour-a-month increase in availability for their supply chain. And we didn't have a certification for him then -- but we will have now.

Gardner: So really big improvement. It shows that being a data scientist means you're impactful and it puts you in the limelight.

Stark: And it was pretty spectacular because the CIO for that company stood up in front of his whole company -- and in front of a group of analysts -- and called him out as the data scientist that solved this problem for their company. So, yeah, he was a rock star for a couple days.

Gardner: For those folks who might be more intrigued with a career path toward certification as a data scientist, where might they go for more information? What are the next steps when it comes to the process through The Open Group, with IBM, and the industry at large?

Where to begin

Norton: The Open Group officially launched this in January, so anyone can go to The Open Group website and check under certifications. They will be able to read the information about how to apply. Some companies are accredited, and others can get accredited for running a version of the certification themselves.

IBM recently went through the certification process. We have built an internal process that matches with The Open Group. People can apply either directly to The Open Group or, if they happen to be within IBM or one of the other companies who will certify, they can apply that way and get the equivalent of it being from The Open Group.

Gardner: I’m afraid we’ll have to leave it there. You have been listening to a sponsored BriefingsDirect discussion on how the role of the data scientist in the enterprise is expanding both in importance and influence. And we learned how data scientists -- especially those with this new certification -- are being called on to work more closely than ever with enterprise strategies to predict emerging trends, optimize outcomes, and create entirely new kinds of business value.

IBM has built an internal process that matches with The Open Group. Other companies are getting accredited for running a version of the certification themselves, too.

So please join me in thanking our guests, Martin Fleming, Vice President, Chief Analytics Officer, and Chief Economist at IBM. Thank you, sir.

Fleming: My pleasure, Dana.

Gardner: We also have been here with Maureen Norton, IBM Global Data Scientist Profession Lead, Distinguished Market Intelligence Professional, and author of Analytics Across the Enterprise. Thank you so much, Maureen.

Norton: Thank you, Dana. It’s been a lot of fun.

Gardner: And lastly, we have been here with George Stark, Distinguished Engineer for IT Operations Analytics at IBM. Thank you, sir.

Stark: Thank you, Dana.

Gardner: And a big thank you as well to our audience for joining this BriefingsDirect modern digital business innovation discussion. I’m Dana Gardner, Principal Analyst at Interarbor Solutions, your host throughout the series of BriefingsDirect discussions sponsored by The Open Group.

Thanks again for listening. Please pass this on to your IT community, and do come back next time.

Listen to the podcast. Find it on iTunes. Download the transcript. Sponsor: The Open Group.

Transcript of a discussion on how the role of the data scientist in the enterprise is expanding in both importance and influence that warrants a new level of business analysis professional certification. Copyright Interarbor Solutions, LLC, 2005-2019. All rights reserved.

You may also be interested in:

• The Open Group panel explores ways to help smart cities initiatives overcome public sector obstacles
• The Open Group digital practitioner effort eases the people path to digital business transformation
• How The Open Group Healthcare Forum and Health Enterprise Reference Architecture cures process and IT ills
• Why government agencies could lead the way in demanding inter-public cloud interoperability and standardization
• Panel explores how the IT4IT Reference Architecture acts as a digital business enabler
• The UNIX evolution: A history of innovation reaches an unprecedented 20-year milestone
• The Open Group president, Steve Nunn, on the inaugural TOGAF User Group and new role of EA in business transformation
• A Tale of Two IT Departments, or How Cloud Governance is Essential in the Bimodal IT Era
• Securing Business Operations and Critical Infrastructure: Trusted Technology, Procurement Paradigms, Cyber Insurance

Confluence of Global Trends Ups Ante for Improved IT Governance to Prevent Costly Business 'Glitches'

Transcript of a sponsored BriefingDirect podcast on the growing danger from faulty software and how to overcome it.

Listen to the podcast. Find it on iTunes/iPod and Podcast.com. Download the transcript. Sponsor: WebLayers.

Dana Gardner: Hi, this is Dana Gardner, principal analyst at Interarbor Solutions, and you’re listening to BriefingsDirect.

Today, we present a sponsored podcast discussion on the nature of, and some possible solutions for, a growing parade of enterprise-scale glitches. The headlines these days are full of big, embarrassing corporate and government "gotchas."

These complex snafus cost a ton of money, severely damage a company’s reputation, and most importantly, can hurt or even kill people.

From global auto recalls to bank failures, and the cyber crime that can uproot the private information from millions of users, the scale and damage that technology-accelerated glitches can inflict on businesses and individuals has probably never been higher. So what is at the root?

Is it a technology run amok problem, or a complexity spinning out of control issue -- and why is it seemingly worse now?

A new book is coming out this summer that explores the relationship between glitches and technology, specifically the role of software use and development in the era of cloud computing.

It turns out the role and impact of governance over people, process, and technology comes up again and again in the new book.

We have with us here today the author of the book as well as a software expert from IBM to delve into the causes and effects of glitches and how governance relates to the problem and fixes.

Please join me in welcoming our guests, Jeff Papows, President and CEO of WebLayers, and the author of Glitch: The Hidden Impact of Faulty Software. Welcome to the show, Jeff.

Jeff Papows: Thanks, Dana. Thanks for having us on.

Gardner: We're also here with Kerrie Holley, IBM fellow and Chief Technology Officer for IBM’s SOA Center of Excellence. Welcome to the show, Kerrie.

Kerrie Holley: Thank you, very much.

Gardner: Jeff, let me start with you. Now, the general trends around these complex issues are affecting business and probably affecting just about everyone’s lives. How do these seem to be something that’s different? Is there an inflection point? Is there something different now that 20 years ago in terms of the intersection of business with technology?

Papows: There is. I’ve done a lot of research in the past 10 months and what we're actually seeing is the confluence of three primary factors that are creating an information technology perfect storm of sorts. Some of these are obvious, but it’s the convergence of the three that’s creating problems on the scale that you are describing here.

The first is a loss of intellectual capital. For the first time in our careers -- the three of us have all been at this for a long time now -- we saw, between 2000 and 2007, the first drop in computer science graduates. That's the other side of the dot-com implosion.

Mainframe adoption patterns

While it’s not always popular or glamorous to talk about, 70 percent of the world’s critical infrastructure still runs on IBM mainframes. Yet, the focus of most of our new computer science graduates and early life professionals is on Java, XML, and the open and more modern development languages.

For the first time in our lifetimes and careers, the preponderance of that COBOL-based analytical community is retiring and/or -- God forbid -- aging and dying. That’s created a significant problem, concurrent with a time where the merger and consolidation activity -- the other side of the recession of 2008 -- have created this massive complexity in these giant mash-ups and critical back-office systems. For example, the mergers between Bank of America and Countrywide, and on and on.

The third factor is just the sheer ubiquity of the technological complexity curve. It’s the magnitude of technology that’s now part of our social fabric, whether it’s literally one million transistors that now exist for every human being on the planet or the six billion network devices that exist in the world today, all of which are accessing the same critical, in many cases, back-office structures.

It's reached the point, Dana, from a consumer standpoint, where 60 percent of the value of our automobiles now consists of networked electronic components -- not the drive trains, engines, and the other things. Look at the recent glitches you have seen at places like Toyota.

You take those three meta-level factors and put them together and we're making the morning broadcast news cycles now on a daily basis with, as you said, more and more of these embarrassing things coming to light. They're not just inconvenient, but there are monumental economic consequences -- and we're killing people.

Gardner: Kerrie Holley, we've looked at some of these issues -- society issues, organizational issues, and the technology behind them -- but technology has also been part of the solution or the ability to scale and manage and automate. I think service oriented architecture (SOA) has a major impact on that.

So, are we at a point where the ability of technology to keep up with the rate of growth is out of whack? What do you sense is behind some of this and why hasn't the technology been there to fix it along the way?

Holley: Jeff brought up some excellent points, which are spot-on. The other thing that we see is that we've had this growth of distributed computing. The easy stuff we've actually accomplished already.

If we look at a lot of what businesses are trying to accomplish today, whether it’s a new business model, differentiation, or whatever they're trying to do compete, what we are finding is that the complexity of that solution is pretty significant.

It's something that we obviously can do. If we look at a lot of technologies that are out in the market place, unfortunately, in many cases they are siloed. They repair or they help with a part of the problem, but perhaps they're not holistic in dealing with the whole life-cycle that is necessary to create some of this value.

Secondly -- this is a point-in-time statement -- we're seeing rapid improvements in the technology to solve this. With Jeff's company and other organizations, we are seeing that today. It hasn’t caught up, but I think it will. In summary, Jeff brought up several points in terms of the fact that we have ubiquitous devices and a tremendous amount of computing power. We have programming available to the masses. We have eight-year-olds, grandmothers, and everyone in between, writing software.

Connecting devices

We have a tremendous need to connect mobile devices and front-ends. We have 3D Internet. We just have an explosion of technologies that we have to integrate. Along with that comes some of the challenges in terms of how we make this agile, and how we make it such that it doesn't break. How do we make sure that we actually get the value propositions that we see? Clearly, SOA is a part of the solution, but it's certainly not the end-all in terms of how we repair and how we get better.

Gardner: One of the things that intrigues me about SOA is the emphasis on governance. To get the best out of a distributed services-orientation, you need to think at the very beginning and throughout the process about how to manage, automate, and reuse, as well as the feedback loops into the process -- all on an ongoing basis.

It strikes me that if that works for SOA, it probably also works for management and organizations, and it works for the relationship between workers and customers. Let me take this back to you, Jeff. Is governance also in catch-up mode? Do we have a sense of how to govern the technology, but not necessarily the process? Is that what's behind some of it?

Papows: You're right, Dana. There's a cultural maturation process here. Let's look at a couple of the broad economic planks that have affected how we got here, because I've been in the software industry for 30 years now. Remember that the average computer scientist, at least in North America, on average, makes 32 percent more than the mean average in the U.S. economy. And, software, computer services and infrastructure has accounted for about 37 percent of the growth in the gross domestic product in the United States and Asia in the last decade.

So the economic impact and success of our industry almost can’t be overstated. Because of that, we've grown up for decades now where we just threw more and more bodies at the problem, as
the technological curve grew.

All that means is automating those best practices and turning them inward, so that we’re governing ourselves as an industry the way that we would automate or govern many things.

There was always this never-ending economic rosy horizon, where you would just add more IT professionals and you would acquire and you’d merge systems, but rarely would you render
portions of those workforces redundant.

In 2008, the economic malaise that we’re managing our way through changed all of that. Now, the only way out of this complexity curve that we’ve created, to use Kerrie's terms, is turning the innovation that has been the hallmark of our industry back on ourselves.

That means automating and codifying all of the best practices and human capital that’s been in-place and learning for decades in the form of active policy management and inference engines in what we typically think of as SOA and design-time governance.

Really, all that means is automating those best practices and turning them inward, so that we’re governing ourselves as an industry in the same way that we would automate or govern many things. But now it’s no longer a "nice to have." I would argue that it’s critical, because the complexity curve and the economics have crossed and there is no way to put this genie back in the bottle. There is no way to go backward.

Gardner: Kerrie, any thoughts about what’s perhaps now a critical role for governance, perhaps governance up and down the technology spectrum, design time, runtime, but also governance in terms of how the people and processes come together?

Holley: Absolutely. One of the nice things that the attention to SOA has brought to our marketplace is the recognition that we do need to focus on governance. I don’t know of a single client who’s got an SOA implementation who has not, as a minimum, thought about governance. They may not be doing everything they want to do or should be doing, but governance is clearly on the attention span of everyone in terms of recognizing that it needs to be done.

So, when we look at governance and when we look at it around SOA, IT governance is something that we’ve had for a long time. SOA governance is a subset, you could say. It complements, but at the same time, it focuses our attention on, what some of the deltas have brought to the marketplace that require improved governance.

Services lifecycles

That governance is not only around the technology. It’s not only around the life-cycle of services. It’s not only around the use of addressing processes and addressing application development. Governance also focuses on the convergence that’s required between business and IT.

The synergistic relationship that we seek will be promoted through the use of governance. Change management specifically brings about a pretty significant focus, meaning that there will be a focus on the part of the business and the IT organizations and teams to bring about the results that are sought.

Examples of problems

Gardner: Jeff, in your book you identify some examples. Are there any that really stand out I that we can trace back to some root cause in the software lifecycle?

Papows: There are, and it’s unfortunate. The ones that make the greatest memory points and often the national headlines, characteristically are the ones that affect the consumer broadly as opposed to the corporate ones.

Obviously, Toyota is in the headlines everyday now. Actually, there was another news cycle recently about Toyota’s Lexus vehicles. The new models apparently have a glitch in the software that controls the balance system.

The ones that make the greatest memory points and often the national headlines, characteristically are the ones that affect the consumer broadly as opposed to the corporate ones.

One of the most heartbreaking things in the research for the book was on software that controls the radiation devices in our hospitals for cancer treatment. I ran across a bunch of research where, because of some software glitches and policy problems in terms of the way those updates were distributed, people with fairly nominal cancers received massive overdoses in radiation.

The medical professionals running these machines -- like much of our culture, because something is computerized -- just assume that it’s infallible. Because of the problems in governance or lack of governance policy, people were being over-radiated. Instead of targeting small tumors in a very targeted way, people’s entire upper torsos, and unfortunately, in one case, head and neck were targeted.

There are lots of examples like that in the book that may not be as ubiquitous as Toyota, but there are many cases of widespread health, power, energy, and security risks as a consequence of the lack of policy management or governance that Kerrie was speaking to just a few minutes ago.

Gardner: Well, these examples certainly are very poignant and clearly something to avoid. I wonder if these are also perhaps just the tip of the iceberg. In addition to things that are problematic at a critical level, is there also a productivity hit? Are large aspects of work in process not nearly as optimal as they could be or are plagued by mistakes that drag down the process?

I want to take this over to Kerrie. IBM has its Smarter Planet approach. I think they're talking about the issue that we're just not nearly as efficient as we could be. What makes the headlines are these terrible issues, but what we're really talking about is a tremendous amount of waste. Aren’t we?

Things we could do better

Holley: We are. That’s exactly what inefficiency is. It speaks to a lot of waste and a lot of things we could do better. A lot of what we’ve been talking about from a Smarter Planet standpoint is actually the exact issues that Jeff has talked about, which is that the world is getting more instrumented. There are more sensors. There is a convergence of a lot of different technology, SOA, business process management, mobile computing, and cloud computing.

Clearly, on one end of the spectrum, it’s increasing the complexity. On the other end of the spectrum, it’s adding tremendous value to businesses, but it mandates this attention to governance.

Gardner: Jeff, in your book do you offer up some advice or solutions about what companies ought to be doing in this governance arena to deal with these glitches?

Papows: We do. We talk about what I call the IT Governance Manifesto, for lack of another catchy phrase. I make the argument that it’s almost reached the point now where we need to lobby for legislation that requires more stringent reporting of software glitches in cases where there is human health and life at stake. Or, alternately, that we impose fines upon individuals or organizations responsible for cover-ups that put people at risk. Or, we simply require a level of IT governance at organizations that produce products that directly affect productivity and quality of life issues.

Kerrie said this really well, Dana. Remember that about 70 percent of our computer scientists in a given year are basically contending with maintaining the existing application inventories that run all of our financial transactions in core sub-systems and topologies. So, 70 percent of our human capital is there to basically keep the stuff that’s in place running.

So, 70 percent of our human capital is there to basically keep the stuff that’s in place running.

Concurrently, we have this smarter planet, where we’ve got billions of RFID tags in motion and 64-bit microprocessors have reached a price point where they are making the way into our dishwashers. We’ve got this plethora of hand-held devices and applications that’s exploding.

All of that is against the backdrop of this more difficult economy, where we can’t just hire more people without automation. We haven't a prayer keeping our noses about water here.

So, God forbid that we ask the federal government, which moves at a dinosaur’s pace relative to Internet speed, to intercede and insist on some of the stuff. But, if we don’t police our own industry, if we don’t get more serious about this governance, whether it’s IBM or WebLayers or some other technological help, we run the risk of seeing the headlines we’re seeing today become completely ubiquitous.

Gardner: Kerrie, I understand that you’re also penning a book, and it’s focused on SOA. First, could you tell us about it, but then are there any aspects of it that address this issue of governance, maybe from a self-help perspective and of not waiting for some legislation or external direction on it.

Holley: The book that’s going to be out later this year is 100 SOA Questions: Asked and Answered. What my co-author [Ali Arsanjani] and I are trying to accomplish in the book, which distinguishes us from other SOA books in the marketplace, is based on thousands of questions that we’ve experienced over the decade in hundreds of projects where we’ve had first-hand roles in as consultants, architects, and developers. We provide the audience with a hands-on, prescriptive understanding of some of the more difficult questions, and not just have platitudes as answers, but really give the reader an answer they can act on.

We’ve organized the content in a way that you can go by domain. If you’re a business stakeholder, you can go to particular areas. That gets back to your question, because business clearly has a big role to play here. The convergence or the relationship between business and IT has a big role to play.

You can go directly into those sections. We do talk about governance. The book is not about governance, but a good percentage of the questions are on governance. What we try to do is help organizations, clients, practitioners, and executives understand what works what doesn’t work.

Always a choice

One of the examples, a small example, is that we always have a choice when we do a project. We can do it in multitude of ways, but we have a lot of evidence that when governance is not applied, when it’s not automated, when it’s not thought about upfront, the expense on the back-end side is enormous. That expense could be the cost of not having the agility that you foresaw.

The expense could be not having the cost reduction that you foresaw. The expense could be the defects that Jeff has spoken about -- the glitches. There is a tremendous downside to not focusing on governance on the front-side, not looking at it in the beginning. The book really tries to ask and answer the toughest SOA questions that we’ve seen in the marketplace over the last decade.

Gardner: We’ll certainly look forward to that. Back to you Jeff. When we think about governance, it has a bit of a siloed history itself. There's the old form of management, the red-light, green-light approach to IT management. We’ve seen design-time governance, but it seems to be somewhat divorced from, even on a different plane than, runtime or operational governance.

What needs to happen in order to make governance more holistic, more end-to-end?

Papows: It’s a good question, Dana. It’s like everything else in our industry. We’re sometimes our own worst enemy and we get hung up on language, and God forbid, we create yet another acronym headache.

There's an old expression, "Everybody wants governance, but nobody wants to be governed." We run the risk, and I think we’ve tripped over it several times, where we get to the point where developers don’t want to be slowed down. There is this Big Brother-connotation at times to governance. We’ve got to explore a different cultural approach to it.

Governance, whether it’s design time or run time, is really about automating and codifying best practices.

Governance, whether it’s design-time or run-time, is really about automating and codifying best practices, and it’s not done generically as was once taught. It can be, in my experience, very specific. The things we see Ford Motor Co. doing are very different. They're germane to their IT culture and organization, and very different than what we see the Bank of America do, as an example.

To Kerrie’s point about the cost of a lack of automated best practices, if we can use the new verb, it isn’t always quantitative. Look at the brand damage to a bank when they shut customers out of their ATM network, the other side of turning the switch when they merged back-office systems. Look at the number of people whose automated payment systems and whatnot were knocked out of kilter.

The brand damage affecting major corporations is a consequence of having these inane debates about whether SOA is alive or dead, whether you need design-time governance or run-time governance. What you need is a way to automate what you are doing, so that your best practices are enforced throughout the development lifecycle.

Kerrie answered your question well when he said it really is about waste. It’s not just about wasted human capital or wasted productivity or cycles. It’s about wasted go-to-market opportunity. Remember, we're now living in the era of market-facing systems. For almost every major business enterprise, our digital footprint is directly accessible in the marketplace, whether it’s an ATM network or a hand-held device. The line between our back-office infrastructure and our consumer experience is being obliterated.

I'd argue that rather than making distinctions between design and run-time governance, companies simply, one way or another, need to automate their best practices. The business mandates of the corporations need to be reflected in an automated way that makes it manageable across the information technology life-cycle -- or you exist at your own peril.

Gardner: Kerrie, any thoughts on this concept of governance and how we make it more ubiquitous and more enforced as the pain and the problems grow evident? The solution at a high level seems pretty clear. It seems to be the implementation where we stumble.

Governance mindset

Holley: You hit it on the head, and Jeff made the point as well. A lot of people think governance is onerous, that it’s a structure that forces people to do things a certain way. They look at it as rigid, inflexible, unforgiving. They think it just gets in the way.

That’s a mindset that people find themselves in, and it’s a reason not to do something. But when you think about the goals that you're seeking, most goals have something to do with efficiency, lower cost, customers, and making the company more agile. When you think about this, pretty much everybody in the marketplace knows that you don’t get those goals for free. There is some cultural change that’s often necessary to bring those goals about, some organizational change.

There's automation. You don’t start with automation. You actually start with the problem, the processes, and picking the right tool. But, automation has to be a part of that solution. One end of the spectrum, we’ve got to address this mindset that governance gets in the way, that it’s overhead, and that it’s unnecessary.

We know that organizations that are very successful, that are achieving many of their goals, when we peel the onion back, we see them focused on governance. One advice that we all know is that you shouldn’t boil the ocean, that you should do incremental change. We also need to do this in governance.

We need to have these incremental successes, where we are focused on automation holistically and looking at the life-cycle, not just looking at the part-of-the-problem space.

Looking for automation as a way out of the hole that has been created is a consequence of the industry’s own success.

Gardner: Jeff, it sounds like governance needs a makeover. Is there an opportunity? You are going to be discussing this book at the IBM Impact Conference 2010, their SOA conference? Is this a good opportunity? You have a lot of IT executive and software executives from the variety of enterprises on hand, but what would you tell them in terms of how to make governance a bit more attractive?

Papows: We all need to say, "I am a computer science professional. We have reached a point in the complexity curve where I no longer scale." You have to start with an admission of fact. And the reality is that the demands placed on today's IT organizations, the magnitude of the existing infrastructure that needs to continue to be cared for, the magnitude of application demands for new systems and access points from all of this new technology, simply is not going to correlate without a completely different highly automated approach.

Kerrie is right. You can't boil the ocean and you can’t do it at once, but you have to start with an honest self-assessment that, as an industry, we can't continue to go forward at the rate and pace that we have grown, given everything we know and that we see, without finally eating our own cooking.

Looking for automation as a way out of the hole that has been created is a consequence of the industry’s own success. We didn't get here because we failed to be fair to all of those developers in the audience. They're going to listen to this and say, "Why am I the bad guy?" They're not the bad guys.

The reality is, as I said, that we're responsible for the greatest percentage of growth in the gross domestic product. We're responsible for the greatest percentage workforce productivity. We've changed the way civilization lives and works. We've dealt with a quantum leap -- and the texture of human existence is a consequence of this technology.

It's time that we simply admit that we need to turn back on ourselves in order to continue to manage this or we, literally, I believe, are on the precipice of that digital equivalent of a Pearl Harbor, and the economic and productivity consequences of failing are extreme.

Gardner: Well, we'll have to leave it there. We're about out of time. We've been discussing how glitches in business have highlighted a possible breakdown in the continuity of technology and that governance is an important factor in making technology continue on its productivity curve, without falling at some degree under its own weight.

I want to thank our guests. We have been joined today by Jeff Papows, President and CEO of WebLayers, and the author of the new book, Glitch: The Hidden Impact of Faulty Software. Thank you so much, Jeff.

Papows: Thank you, Dana, and thank you, Kerrie.

Gardner: And, we have been joined also by Kerrie Holley, an IBM Fellow as well as the CTO for IBM’s SOA Center of Excellence. Thanks for your input, and we will look forward to your book as well.

Holley: Thank you, Dana, and thank you, Jeff.

Gardner: This is Dana Gardner, principal analyst at Interarbor Solutions. You've been listening to a sponsored BriefingsDirect podcast. Thanks for listening and come back next time.

Listen to the podcast. Find it on iTunes/iPod and Podcast.com. Download the transcript. Sponsor: WebLayers.

Transcript of a sponsored BriefingDirect podcast on the growing danger from faulty software and how to overcome it. Copyright Interarbor Solutions, LLC, 2005-2010. All rights reserved.