The Open Group Panel Explains How the ArchiMate Modeling Language and The Open Group Architecture Framework Impact Such Trends as Big Data and Cloud

Transcript of a BriefingsDirect podcast on the role of enterprise and business architecture in helping enterprises exploit and manage technology and business transformation.

Listen to the podcast. Find it on iTunes. Download the transcript. Sponsor: The Open Group.

Dana Gardner: Hello, and welcome to a special BriefingsDirect thought leadership interview series coming to you in conjunction with The Open Group Conference recently held in Newport Beach, California.

Cardner

I'm Dana Gardner, Principal Analyst at Interarbor Solutions, and I'll be your host and moderator throughout these business transformation discussions. The conference itself is focused on "big data -- the transformation we need to embrace today."

We recently assembled a panel of experts to explore new trends and developments in enterprise architecture (EA) as businesses grapple with such issues as big data, cloud computing, security, and overall IT transformation. We'll learn more on how EA is evolving and specifically how the TOGAF® framework and the ArchiMate® modeling language are playing increased roles worldwide.

With that, please join me in welcoming our panel: Chris Forde, General Manager for Asia-Pacific and Vice President of Enterprise Architecture at The Open Group; Iver Band, Vice Chair of The Open Group ArchiMate Forum and Enterprise Architect at The Standard, a diversified financial services company; Mike Walker, Senior Enterprise Architecture Adviser and Strategist at HP and former Director of Enterprise Architecture at Dell; Henry Franken, the Chairman of The Open Group ArchiMate Forum and Managing Director at BIZZdesign, and Dave Hornford, Chairman of the Architecture Forum at The Open Group and Managing Partner at Conexiam. [Disclosure: The Open Group and HP are sponsors of BriefingsDirect podcasts.]

Gardner: Chris, at the conference you me with a lot of folks, and there is a lot of activity in socializing and whatnot. Is there something about the role of the enterprise architect that you sense is shifting, or are people, maybe even trying to project their roles differently in their organizations?

Consistent theme

Forde: At these conferences, generally there is a fairly consistent theme. It goes from "We're having difficulty defining our role in the context that makes it relevant and useful to the business," to "We're having a great opportunity with our business partners to drive business transformation." It really goes across the spectrum.

Forde

What I'm hearing in the conference, not just based on the themes, is a lot of discussion about that transformation topic and the role of the enterprise architect in moving the organization along. That's a very, very typical conversation to hear in the hallways.

Gardner: When it's a dynamic environment, lots of change, lots of movement, the enterprise architects' value can go up. If things were slow, constant and predictable, perhaps their value wouldn't be as high. Any thoughts about that?

Franken: Well sure. What you see is that the challenge within large organizations on business transformation is increasing and the number of good enterprise architects is small, so their value increases. It's simple mathematics.

Gardner: Mike Walker, how do you see EA and the role of the architect changing, vis-à-vis your experiences?

Walker: I’ll provide the perspective of the an EA leader and practitioner in the trenches of not only my company but also talking with colleagues in other companies as well. I see a lot of what was referred to from Henry and Chris. To add to that, there is more and more focus on reinvigorating the EA practices. There is less of a focus on the traditional things we come to think of EA such as standards, governance and policies, but rather into emerging areas such as the soft skills, business architecture, and strategy.

Walker

To this end I see a lot in the realm of working directly with the executive chain to understand the key value drivers for the company and rationalize where they want to go with their business. So we're moving into a business-transformation role in this practice.

At the same time, we've got to be mindful of the disruptive external technology forces coming in as well. EA can’t just divorce from the other aspects of architecture as well. So the role that enterprise architects play becomes more and more important and elevated in the organization.

Two examples of this disruptive technology that are being focused on at the conference are big data and cloud computing. Both are providing impacts to our businesses not because of some new business idea but because technology is available to enhance or provide new capabilities to our business. The EA’s still do have to understand these new technology innovations and determine how they will apply to the business.

To Henry's point around the need to get really good enterprise architects, it’s difficult to find good ones. There is a shortage right now especially given that a lot of focus is being put on the EA department to really deliver sound architectures.

Not standalone

Gardner: We've been talking a lot here about big data, but usually that's not just a standalone topic. It's big data and cloud, cloud, mobile and security.

So with these overlapping and complex relationships among multiple trends, why is EA and things like the TOGAF framework and the ArchiMate modeling language especially useful? Iver?

Band: One of the things that has been clear for a while now is that people outside of IT don't necessarily have to go through the technology function to avail themselves of these technologies any more. Whether they ever had to is really a question as well.

Band

One of things that EA is doing, and especially in the practice that I work in, is using approaches like the ArchiMate modeling language to effect clear communication between the business, IT, partners and other stakeholders. That's what I do in my daily work, overseeing our major systems modernization efforts. I work with major partners, some of which are offshore.

I'm increasingly called upon to make sure that we have clear processes for making decisions and clear ways of visualizing the different choices in front of us. We can't always unilaterally dictate the choice, but we can make the conversation clearer by using frameworks like the TOGAF standard and the ArchiMate modeling language, which I use virtually every day in my work.

Gardner: And so the more moving parts and the more complexity, the less likely that you can wing this or use traditional, linear tools. You need something that's a bit more up to the task. Dave, help us understand how these tools can grapple better with these multiple levels of complexity and then also bridge some of these communication gaps among different constituencies in these large organizations.

Hornford: The fundamental benefit of the tools is the organization realizing its capability and strategy. I just came from a session where a fellow quoted a Harvard study, which said that around a third of executives thought their company was good at executing on its strategy. He highlighted that this means that two-thirds are not good at executing on their strategy.

Hornford

If you're not good at executing on your strategy and you've got big data, mobile, consumerization of IT and cloud, where are you going? What's the correct approach? How does this fit into what you were trying to accomplish as an enterprise?

An enterprise architect that is doing their job is bringing together the strategy, goals and objectives of the organization. Also, its capabilities with the techniques that are available, whether it's offshoring, onshoring, cloud, or big data, so that the organization is able to move forward to where it needs to be, as opposed to where it's going to randomly walk to.

Forde: One of the things that has come out in several of the presentations is this kind of capability-based planning, a technique in EA to get their arms around this thing from a business-driver perspective. Just to polish what Dave said a little bit, it's connecting all of those things. We see enterprises talking about a capability-based view of things on that basis.

Gardner: Because we're here with a couple of the chairpeople from these forums, where a lot of the development and direction for these tools comes about, let's get a quick update. The TOGAF framework, where are we and what have been the highlights from this particular event?

Minor upgrade

Hornford: In the last year, we've published a minor upgrade for TOGAF version 9.1 which was based upon cleaning up consistency in the language in the TOGAF documentation. What we're working on right now is a significant new release, the next release of the TOGAF standard, which is dividing the TOGAF documentation to make it more consumable, more consistent and more useful for someone.

Today, the TOGAF standard has guidance on how to do something mixed into the framework of what you should be doing. We're peeling those apart. So with that peeled apart, we won't have guidance that is tied to classic application architecture in a world of cloud.

What we find when we have done work with the Banking Industry Architecture Network (BIAN) for banking architecture, Sherwood Applied Business Security Architecture (SABSA) for security architecture, and the TeleManagement Forum, is that the concepts in the TOGAF framework work across industries and across trends. We need to move the guidance into a place so that we can be far nimbler on how to tie cloud with my current strategy, how to tie consumerization of IT with on-shoring?

Franken: The ArchiMate modeling language turned two last year, and the ArchiMate 1.0 standard is the language to model out the core of your EA. The ArchiMate 2.0 standard added two specifics to it to make it better aligned also to the process of EA.

Franken

According to the TOGAF standard, this is being able to model out the motivation, why you're doing EA, stakeholders and the goals that drive us. The second extension to the ArchiMate standard is being able to model out its planning and migration.

So with the core EA and these two extensions, together with the TOGAF standard process working, you have a good basis on getting EA to work in your organization.

Gardner: Let’s also go back to the big data concepts that are driving this conference. I've been interested in this notion of the information architecture, data architecture and how that relates to the TOGAF framework. Mike, you've been doing some interesting writing on this subject. Fill us in on some of your thoughts about the role of information architecture vis-à-vis the larger business architect and enterprise architect roles.

Walker: Information architecture is an interesting topic in that it hasn’t been getting a whole lot of attention until recently.

Information architecture is an aspect of enterprise architecture that enables an information strategy or business solution through the definition of the company's business information assets, their sources, structure, classification and associations that will prescribe the required application architecture and technical capabilities.

Information architecture is the bridge between the business architecture world and the application and technology architecture activities.

The reason I say that is because information architecture is a business-driven discipline that details the information strategy of the company. As we know, and from what we’ve heard at the conference keynotes like in the case of NASA, big data, and security presentations, the preservation and classification of that information is vital to understanding what your architecture should be.

Least matured

From an industry perspective, this is one of the least matured, as far as being incorporated into a formal discipline. The TOGAF standard actually has a phase dedicated to it in data architecture. Again, there are still lots of opportunities to grow and incorporate additional methods, models and tools by the enterprise information management discipline.

Enterprise information management not only it captures traditional topic areas like master data management (MDM), metadata and unstructured types of information architecture but also focusing on the information governance, and the architecture patterns and styles implemented in MDM, big data, etc. There is a great deal of opportunity there.

From the role of information architects, I’m seeing more and more traction in the industry as a whole. I've dealt with an entire group that’s focused on information architecture and building up an enterprise information management practice, so that we can take our top line business strategies and understand what architectures we need to put there.

This is a critical enabler for global companies, because oftentimes they're restricted by regulation, typically handled at a government or regional area. This means we have to understand that we build our architecture. So it's not about the application, but rather the data that it processes, moves, or transforms.

We didn’t have to treat information as a first-class citizen. Times have changed, though.

Gardner: Up until not too long ago, the conventional thinking was that applications generate data. Then you treat the data in some way so that it can be used, perhaps by other applications, but that the data was secondary to the application.

But there's some shift in that thinking now more toward the idea that the data is the application and that new applications are designed to actually expand on the data’s value and deliver it out to mobile tiers perhaps. Does that follow in your thinking that the data is actually more prominent as a resource perhaps on par with applications?

Walker: You're spot on, Dana. Before the commoditization of these technologies that resided on premises, we could get away with starting at the application layer and work our way back because we had access to the source code or hardware behind our firewalls. We could throw servers out, and we used to put the firewalls in front of the data to solve the problem with infrastructure. So we didn’t have to treat information as a first-class citizen. Times have changed, though.

Information access and processing is now democratized and it’s being pushed as the first point of presentment. A lot of times this is on a mobile device and even then it’s not the corporate’s mobile device, but your personal device. So how do you handle that data?

It's the same way with cloud, and I’ll give you a great example of this. I was working as an adviser for a company, and they were looking at their cloud strategy. They had made a big bet on one of the big infrastructures and cloud-service providers. They looked first at what the features and functions that that cloud provider could provide, and not necessarily the information requirements. There were two major issues that they ran into, and that was essentially a showstopper. They had to pull off that infrastructure.

The first one was that in that specific cloud provider’s terms of service around intellectual property (IP) ownership. Essentially, that company was forced to cut off their IP rights.

Big business

As you know, IP is a big business these days, and so that was a showstopper. It actually broke the core regulatory laws around being able to discover information.

So focusing on the applications to make sure it meets your functional needs is important. However, we should take a step back and look at the information first and make sure that for the people in your organization who can’t say no, their requirements are satisfied.

Gardner: Data architecture is it different from EA and business architecture, or is it a subset? What’s the relationship, Dave?

Hornford: Data architecture is part of an EA. I won’t use the word subset, because a subset starts to imply that it is a distinct thing that you can look at on its own. You cannot look at your business architecture without understanding your information architecture. When you think about big data, cool. We've got this pile of data in the corner. Where did it come from? Can we use it? Do we actually have legitimate rights, as Mike highlighted, to use this information? Are we allowed to mix it and who mixes it?

When we look at how our business is optimized, they normally optimize around work product, what the organization is delivering. That’s very easy. You can see who consumes your work product. With information, you often have no idea who consumes your information. So now we have provenance, we have source and as we move for global companies, we have the trends around consumerization, cloud and simply tightening cycle time.

If we look at data in isolation, I have to understand how the system works and how the enterprise’s architecture fits together.

There was a very interesting thing that came out of a PricewaterhouseCoopers CEO summary, which said there has historically been cycles where the CEOs were focusing on innovation or cost. What they have observed over the last few surveys is much tightening of those cycles. We used to be a bit worried about cost for a few years. Then, we would worry about innovation for a few years. Now, it’s worrying about it for a year. What came out in the last survey? Both are rated number one.

How do we in global, tightly connected, information-rich environment manage? Do we have access to the information? Our competitors may, our customers do and our suppliers probably do. How do we fit into that? If we look at data in isolation, I have to understand how the system works and how the enterprise’s architecture fits together.

Gardner: Of course, the end game for a lot of the practitioners here is to create that feedback loop of a lifecycle approach, rapid information injection and rapid analysis that could be applied. So what are some of the ways that these disciplines and tools can help foster that complete lifecycle? Let’s go to Iver.

Band: The disciplines and tools can facilitate the right conversations among different stakeholders. One of the things that we're doing at The Standard is building cadres equally balanced between people in business and IT.

We're training them in information management, going through a particular curriculum, and having them study for an information management certification that introduces a lot of these different frameworks and standard concepts.

Creating cadres

We want to create these cadres to be able to solve tough and persistent information management problems that affect all companies in financial services, because information is a shared asset. The purpose of the frameworks is to ensure proper stewardship of that asset across disciplines and across organizations within an enterprise.

Gardner: If they add to the fostering of that nirvana of a full lifecycle that it cuts across different disciplines in the organization.

Hornford: The core is from the two standards that we have, The ArchiMate standard and the TOGAF standard. The TOGAF standard has, from its early roots, focused on the components of EA and how to build a consistent method of understanding of what I'm trying to accomplish, understanding where I am, and where I need to be to reach my goal.

When we bring in the ArchiMate standard, I have a language, a descriptor, a visual descriptor that allows me to cross all of those domains in a consistent description, so that I can do that traceability. When I pull in this lever or I have this regulatory impact, what does it hit me with, or if I have this constraint, what does it hit me with?

If I don’t do this, if I don’t use the framework of the TOGAF standard, or I don’t use the discipline of formal modeling in the ArchiMate standard, we're going to do it anecdotally. We're going to trip. We're going to fall. We're going to have a non-ending series of surprises, as Mike highlighted.

The businesses value of TOGAF is that they get a repeatable and a predictable process for building out our architectures that properly manage risks and reliably produces value.

"Oh, terms of service. I am violating the regulations. Beautiful. Let’s take that to our executive and tell him right as we are about to go live that we have to stop, because we can't get where we want to go, because we didn't think about what it took to get there." And that’s the core of EA in the frameworks.

Walker: To build on what Dave has just talked about and going back to your first question Dana, the value statement on TOGAF from a business perspective. The businesses value of TOGAF is that they get a repeatable and a predictable process for building out our architectures that properly manage risks and reliably produces value.

The TOGAF framework provides a methodology to ask what problems you're trying to solve and where you are trying to go with your business opportunities or challenges. That leads to business architecture, which is really a rationalization in technical or architectural terms the distillation of the corporate strategy.

From there, what you want to understand is information -- how does that translate, what information architecture do we need to put in place? You get into all sorts of things around risk management, etc., and then it goes on from there, until what we were talking about earlier about information architecture.

If the TOGAF standard is applied properly you can achieve the same result every time, That is what interests business stakeholders in my opinion. And the ArchiMate modeling language is great because, as we talked about, it provides very rich visualizations so that people cannot only show a picture, but tie information together. Different from other aspects of architecture, information architecture is less about the boxes and more about the lines.

Gardner: All right, thank you Mike. Chris, anything to add?

Quality of the individuals

Forde: Building on what Dave was saying earlier and also what Iver was saying is that while the process and the methodology and the tools are of interest, it’s the discipline and the quality of the individuals doing the work.

Iver talked about how the conversation is shifting and the practice is improving to build communications groups that have a discipline to operate around. What I am hearing is implied, but actually I know what specifically occurs, is that we end up with assets that are well described and reusable.

And there is a point at which you reach a critical mass that these assets become an accelerator for decision making. So the ability of the enterprise and the decision makers in the enterprise at the right level to respond is improved, because they have a well disciplined foundation beneath them.

A set of assets that are reasonably well-known at the right level of granularity for them to absorb the information and the conversation is being structured so that the technical people and the business people are in the right room together to talk about the problems.

This is actually a fairly sophisticated set of operations that I am discussing and doesn't happen overnight, but is definitely one of the things that we see occurring with our members in certain cases.

There is a point at which you reach a critical mass that these assets become an accelerator for decision making.

Hornford: I want to build on that what Chris said. It’s actually the word "asset." While he was talking, I was thinking about how people have talked about information as an asset. Most of us don’t know what information we have, how it’s collected, where it is, but we know we have got a valuable asset.

I'll use an analogy. I have a factory some place in the world that makes stuff. Is that an asset? If I know that my factory is able to produce a particular set of goods and it’s hooked into my supply chain here, I've got an asset. Before that, I just owned a thing.

I was very encouraged listening to what Iver talked about. We're building cadres. We're building out this approach and I have seen this. I'm not using that word, but now I'm stealing that word. It's how people build effective teams, which is not to take a couple of specialists and put them in an ivory tower, but it’s to provide the method and the discipline of how we converse about it, so that we can have a consistent conversation.

When I tie it with some of the tools from the Architecture Forum and the ArchiMate Forum, I'm able to consistently describe it, so that I now have an asset I can identify, consume and produce value from.

Business context

Forde: And this is very different from data modeling. We are not talking about entity relationship, junk at the technical detail, or third normal form and that kind of stuff. We're talking about a conversation that’s occurring around the business context of what needs to go on supported by the right level of technical detail when you need to go there in order to clarify.

Gardner: Thank you Chris. I believe we'll have to leave it there. We're about out of time. We've been talking about the enterprise architect’s role, how it's evolving, and how TOGAF and ArchiMate are playing increased roles worldwide.

We've seen how EA is being creatively employed as businesses grapple with such issues as cloud computing, security, big data, and overall IT transformation.

We're talking about a conversation that’s occurring around the business context of what needs to go on.

This special BriefingsDirect discussion comes to you in conjunction with The Open Group Conference in Newport Beach, California.

I want to extend a big thank you to our panel: Chris Forde, the General Manager Asia-Pacific and Vice President of Enterprise Architecture at The Open Group; Iver Band, Vice Chair of The Open Group ArchiMate Forum and Enterprise Architect at The Standard; Mike Walker, Senior Enterprise Architecture Adviser and Strategist at HP and former Director of Enterprise Architecture at Dell; Henry Franken, Chairman of The Open Group ArchiMate Forum and Managing Director at BIZZdesign, and Dave Hornford, Chairman of the Architecture Forum at The Open Group and a Managing Partner at Conexiam. Thanks to you all.

This is Dana Gardner, Principal Analyst at Interarbor Solutions, your host and moderator through these thought leadership interviews. Thanks again for listening, and come back next time.

Listen to the podcast. Find it on iTunes. Download the transcript. Sponsor: The Open Group.

Transcript of a BriefingsDirect podcast on the role of enterprise and business architecture in helping enterprises exploit and manage technology and business transformation. Copyright The Open Group and Interarbor Solutions, LLC, 2005-2013. All rights reserved.

You may also be interested in:

• The Open Group Conference Panel Explores How the Big Data Era Now Challenges the IT Status Quo
• Using the Cloud for Big-Data Requires a New Recipe
• Big Data Success Depends on Better Risk Management Practices Like FAIR, Say The Open Group Panelists
• The Open Group Keynoter Sees Big-Data Analytics Bolstering Quality, Manufacturing, Processes
• The Open Group Trusted Technology Forum is Leading the Way to Securing GLobal IT Supply Chains
• Corporate Data, Supply Chains Remain Vulnerable to Cyber Crime Attacks Says Open Group Conference Speaker
• Open Group Conference Speakers Discuss the Cloud: Higher Risk or Better Security?

The Open Group Conference Panel Explores How the Big Data Era Now Challenges the IT Status Quo

Transcript of a BriefingsDirect podcast from The Open Group Conference in January on how big data forces changes in architecting the enterprise.

Listen to the podcast. Find it on iTunes. Watch the video. Download the transcript. Sponsor: The Open Group.

Dana Gardner: Hello, and welcome to a special BriefingsDirect thought leadership interview series coming to you in conjunction with The Open Group Conference recently held in Newport Beach, California.

Gardner

I'm Dana Gardner, Principal Analyst at Interarbor Solutions, and I'll be your host and moderator throughout these business transformation discussions. The conference itself is focusing on "big data -- he transformation we need to embrace today." [Disclosure: The Open Group is a sponsor of this and other BriefingsDirect podcasts.]

We assembled a panel of experts to explore how big data changes the status quo for architecting the enterprise. We'll learn how large enterprises should anticipate the effects and impacts of big data, as well the simultaneous impacts of cloud computing and mobile.



It’s been an interesting thread throughout the conference for me to factor where big data begins and plain old data, if you will, ends. Of course, it's going to vary quite a bit from organization to organization.

But Chris Gerty from NASA provided a good example: It’s when you run out of gas with your old methods, and your ability to deal with the data -- and it's not just the size of the data itself.

When an enterprise architect and the business architect looked at data a few years ago, they might not have been as aware of these boundaries and the importance of data. They perhaps were thinking that the database administrators and the business intelligence (BI) folks would take care of that, and they just had to manage the fruits of the data vis-à-vis applications and integration points.

I don’t think that’s the case anymore, and one of the points we're going to get into now is where the enterprise architect needs to be factoring the impacts of big data.

Furthermore, there seems to be the need to do things differently, not just to manage the velocity and the volume and the variety of the data, but to really think about data fundamentally and differently. For many companies, data is now a product itself. That data can be monetized.

The analysis from the data becomes important to more and more people in the company, so that your employees, your partners, and those in your supply chain will be interacting with your data -- and the analysis from your data -- more than before.

So I think we need to also think about data differently. And, we need to think about security, risk and governance. If it's a "boundaryless organization" when it comes your data, either as a product or service or a resource, that control and management of which data should be exposed, which should be opened, and which should be very closely guarded all need to be factored, determined and implemented.

Expert panel

With that, let me now introduce our expert panel: Robert Weisman, CEO and Chief Enterprise Architect at Build The Vision; Andras Szakal, Vice President and CTO of IBM's Federal Division; Jim Hietala, Vice President for Security at The Open Group, and Chris Gerty, Deputy Program Manager at the Open Innovation Program at NASA.

Chris, let’s start with you. You mentioned that big data to you is not a factor of the size, because NASA's dealing with so much. It’s when you run out of steam, as it were, with the methodologies. Maybe you could explain more. When do you know that you've actually run out of steam with the methodologies?

Chris Gerty: When we collect data, we have some sort of goal in minds of what we might get out of it. When we put the pieces from the data together, it either maybe doesn't fit as well as you thought or you are successful and you continue to do the same thing, gathering archives of information.

Gerty

At that point, where you realize there might even something else that you want to do with the data, different than what you planned originally, that’s when we have to pivot a little bit and say, "Now I need to treat this as a living archive. It's a 'it may live beyond me' type of thing." At that point, I think you treat it as setting up the infrastructure for being used later, whether it’d be by you or someone else. That's an important transition to make and might be what one could define as big data.

Gardner: Andras, does that square with where you are in your government interactions -- that data now becomes a different type of resource, and when you are not able to execute or avail yourself of its value, then you know you need to do things differently?

Andras Szakal: The importance of data hasn’t changed. The data itself, the veracity of the data, is still important. Transactional data will always need to exist. The difference is that you have certainly the three or four Vs, depending on how you look at it, but the importance of data is in its veracity, and your ability to understand or to be able to use that data before the data's shelf life runs out.

Szakal

Some data has a shelf life that's long lived. Other data has very little shelf life, and you would use different approaches to being able to utilize that information. It's ultimately not about the data itself, but it’s about gaining deep insight into that data. So it’s not storing data or manipulating data, but applying those analytical capabilities to data.

Gardner: Bob, we've seen the price points on storage go down so dramatically. We've seem people just decide to hold on to data that they wouldn’t have before, simply because they can and they can afford to do so. That means we need to try to extract value and use that data. From the perspective of an enterprise architect, how are things different now, vis-à-vis this much larger set of data and variety of data, when it comes to planning and executing as architects?

Robert Weisman: One of the major issues is that normally organizations are holding two orders of magnitude more data then they need. It’s an huge overhead, both in terms of the applications architecture that has a code basis, larger than it should be, and also from the technology architecture that is supporting a horrendous number of servers and a whole bunch of technology stuff that they don't need.

The issue for the architect is to figure out as what data is useful, institute a governance process, so that you can have data lifecycle management, have a proper disposition,  focus the organization on information data and knowledge that is basically going to provide business value to the organization, and help them innovate and have a competitive advantage.

Can't afford it

And in terms of government, just improve service delivery, because there's waste right now on information infrastructure, and we can’t afford it anymore.

Gardner: I suppose big data is part of the problem, dealing with so much in redundancy and duplication through the lifecycle of data and what have you, but the data is also part of the solution in terms of getting the knowledge about what you should or shouldn't be doing as a business. So it's difficult to know what to keep and what not to keep.

I've actually spoken to a few people lately who want to keep everything, just because they want to mine it, and they are willing to spend the money and effort to do that. Jim Hietala, when people do get to this point of trying to decide what to keep, what not to keep, and how to architect properly for that, they also need to factor in security. It shouldn't become later in the process. It should come early. What are some of the precepts that you think are important in applying good security practices to big data?

Jim Hietala: One of the big challenges is that many of the big-data platforms weren’t built from the get-go with security in mind. So some of the controls that you've had available in your relational databases, for instance, you move over to the big data platforms and the access control authorizations and mechanisms are not there today.

Hietala

Planning the architecture, looking at bringing in third-party controls to give you the security mechanisms that you are used to in your older platforms, is something that organizations are going to have to do. It’s really an evolving and emerging thing at this point.

Gardner: There are a lot of unknown unknowns out there, as we discovered with our tweet chat last month. Some people think that the data is just data, and you apply the same security to it. Do you think that’s the case with big data? Is it just another follow-through of what you always did with data in the first place?

Hietala: I would say yes, at a conceptual level, but it's like what we saw with virtualization. When there was a mad rush to virtualize everything, many of those traditional security controls didn't translate directly into the virtualized world. The same thing is true with big data.

When you're talking about those volumes of data, applying encryption, applying various security controls, you have to think about how those things are going to scale? That may require new solutions from new technologies and that sort of thing.

Gardner: Chris Gerty, back to your experiences at NASA. You've taken the approach of keeping as much of that data and information as open as you can, fostering more research and the ability for people to do things with the data that you may never have been visioned yourselves. When it comes to that governance, security, and access control, are there any lessons that you've learned that you are aware of in terms of the best of openness, but also with the ability to manage the spigot?

Gerty: Spigot is probably a dangerous term to use, because it implies that all data is treated the same. The sooner that you can tag the data as either sensitive or not, mostly coming from the person or team that's developed or originated the data, the better.

Kicking the can

Once you have it on a hard drive, once you get crazy about storing everything, if you don't know where it came from, you're forced to put it into a secure environment. And that's just kicking the can down the road. It’s really a disservice to people who might use the data in a useful way to address their problems.

We constantly have satellites that are made for one purpose. They send all the data down. It’s controlled either for security or for intellectual property (IP), so someone can write a paper. Then, after the project doesn’t get funded or it just comes to a nice graceful close, there is that extra step, which is almost a responsibility of the originators, to make it useful to the rest of the world.

Gardner: Let’s look at big data through the lens of some other major trends right now. Let’s start with cloud. You mentioned that at NASA, you have your own private cloud that you're using a lot, of course, but you're also now dabbling in commercial and public clouds. Frankly, the price points that these cloud providers are offering for storage and data services are pretty compelling.

So we should expect more data to go to the cloud. Bob, from your perspective, as organizations and architects have to think about data in this hybrid cloud on-premises off-premises, moving back and forth, what do you think enterprise architects need to start thinking about in terms of managing that, planning for the right destination of data, based on the right mix of other requirements?

Weisman: It's a good question. As you said, the price point is compelling, but the security and privacy of the information is something else that has to be taken into account. Where is that information going to reside? You have to have very stringent service-level agreements (SLAs) and in certain cases, you might say it's a price point that’s compelling, but the risk analysis that I have done means that I'm going to have to set up my own private cloud.

Weisman

Right now, everybody's saying is the public cloud is going to be the way to go. Vendors are going to have to be very sensitive to that and many are, at this point in time, addressing a lot of the needs of some of the large client basis. So it’s not one-size-fits-all and it’s more than just a price for service. Architecture can bring down the price pretty dramatically, even within an enterprise.

Gardner: Andras, there's this mash up of cloud and big-data trends, the in-memory approaches, where we are no longer taking batches of data, cleansing it, and deduping it and bringing it into a warehouse, going through batch. We're still doing that' of course, but it seems that for a number of different applications of data and analytics, in-memory technology particularly, if you can control that in a cloud environment, private cloud or otherwise, it’s starting to change the game for that fast, real-time feedback loop benefit.

It's a roundabout way of asking if the cloud and big data come together in a way that’s intriguing to you and in what ways?

Szakal: Actually it’s a great question. We could take the rest of the 22 minutes talking on this one question. I helped lead the President’s Commission on big data that Steve Mills from IBM and -- I forget the name of the executive from SAP -- led. We intentionally tried to separate cloud from big data architecture, primarily because we don't believe that, in all cases, cloud is the answer to all things big data. You have to define the architecture that's appropriate for your business needs.

However, it also depends on where the data is born. Take many of the investments IBM has made into enterprise market management, for example, Coremetrics, several of these services that we now offer for helping customers understand deep insight into how their retail market or supply chain behaves.

Born in the cloud

All of that information is born in the cloud. But if you're talking about actually using cloud as infrastructure and moving around huge sums of data or constructing some of these solutions on your own, then some of the ideas that Bob conveyed are absolutely applicable.

I think it becomes prohibitive to do that and easier to stand up a hybrid environment for managing the amount of data. But I think that you have to think about whether your data is real-time data, whether it's data that you could apply some of these new technologies like Hadoop to, Hadoop MapReduce-type solutions, or whether it's traditional data warehousing.

Data warehouses are going to continue to exist and they're going to continue to evolve technologically. You're always going to use a subset of data in those data warehouses, and it's going to be an applicable technology for many years to come.

Gardner: So suffice it to say, an enterprise architect who is well versed in both cloud infrastructure requirements, technologies, and methods, as well as big data, will probably be in quite high demand. That specialization in one or the other isn’t as valuable as being able to cross-pollinate between them as it were.

Szakal: Absolutely. It's enabling our architects and finding deep individuals who have this unique set of skills, analytics, mathematics, and business. Those individuals are going to be the future architects of the IT world, because analytics and big data are going to be integrated into everything that we do and become part of the business processing.

Gardner: Well, that’s a great segue to the next topic that I am interested in, and it's around mobility as a trend and also application development. The reason I lump them together is that I increasingly see developers being tasked with mobile first.

When you create a new app, you have to remember that this is going to run in the mobile tier and you want to make sure that the requirements, the UI, and the complexity of that app don’t go beyond the ability of the mobile app and the mobile user. This is interesting to me, because data now has a different relationship with apps.

We used to think of apps as creating data and then the data would be stored and it might be used or integrated. Now, we have applications that are simply there in order to present the data and we have the ability now to present it to those mobile devices in the mobile tier, which means it goes anywhere, everywhere all the time.

Let me start with you Jim, because it’s security and risk, but it's also just rethinking the way we use data in a mobile tier. If we can do it safely, and that’s a big IF, how important should it be for organizations to start thinking about making this data available to all of these devices and just pour out into that mobile tier as possible?

Hietala: In terms of enabling the business, it’s very important. There are a lot of benefits that accrue from accessing your data from whatever device you happen to be on. To me, it is that question of "if," because now there’s a whole lot of problems to be solved relative to the data floating around anywhere on Android, iOS, whatever the platform is, and the organization being able to lock down their data on those devices, forgetting about whether it’s the organization device or my device. There’s a set of issues around that that the security industry is just starting to get their arms around today.

Mobile ability

Gardner: Chris, any thoughts about this mobile ability that the data gets more valuable the more you can use it and apply it, and then the more you can apply it, the more data you generate that makes the data more valuable, and we start getting into that positive feedback loop?

Gerty: Absolutely. It's almost an appreciation of what more people could do and get to the problem. We're getting to the point where, if it's available on your desktop, you’re going to find a way to make it available on your device.

That same security questions probably need to be answered anyway, but making it mobile compatible is almost an acknowledgment that there will be someone who wants to use it. So let me go that extra step to make it compatible and see what I get from them. It's more of a cultural benefit that you get from making things compatible with mobile.

Gardner: Any thoughts about what developers should be thinking by trying to bring the fruits of big data through these analytics to more users rather than just the BI folks or those that are good at SQL queries? Does this change the game by actually making an application on a mobile device, simple, powerful but accessing this real time updated treasure trove of data?

Gerty: I always think of the astronaut on the moon. He's got a big, bulky glove and he might have a heads-up display in front of him, but he really needs to know exactly a certain piece of information at the right moment, dealing with bandwidth issues, dealing with the environment, foggy helmet wherever.

It's very analogous to what the day-to-day professional will use trying to find out that quick e-mail he needs to know or which meeting to go to -- which one is more important -- and it all comes down to putting your developer in the shoes of the user. So anytime you can get interaction between the two, that’s valuable.

Gardner: Bob?

Weisman: From an enterprise architecture point of view my background is mainly defense and government, but defense mobile computing has been around for decades. So you've always been dealing with that.

The main thing is that in many cases, if they're coming up with information, the whole presentation layer is turning into another architecture domain with information visualization and also with your security controls, with an integrated identity management capability.

It's like you were saying about astronaut getting it right. He doesn't need to know everything that’s happening in the world. He needs to know about his heads-up display, the stuff that's relevant to him.

So it's getting the right information to person in an authorized manner, in a way that he can visualize and make sense of that information, be it straight data, analytics, or whatever. The presentation layer, ergonomics, visual communication are going to become very important in the future for that. There are also a lot of problems. Rather than doing it at the application level, you're doing it entirely in one layer.

Governance and security

Gardner: So clearly the implications of data are cutting across how we think about security, how we think about UI, how we factor in mobility. What we now think about in terms of governance and security, we have to do differently than we did with older data models.

Jim Hietala, what about the impact on spurring people towards more virtualized desktop delivery, if you don't want to have the date on that end device, if you want solve some of the issues about control and governance, and if you want to be able to manage just how much data gets into that UI, not too much not too little.

Do you think that some of these concerns that we’re addressing will push people to look even harder, maybe more aggressive in how they go to desktop and application virtualization, as they say, keep it on the server, deliver out just the deltas?

Hietala: That’s an interesting point. I’ve run across a startup in the last month or two that is doing is that. The whole value proposition is to virtualize the environment. You get virtual gold images. You don't have to worry about what's actually happening on the physical device and you know when the devices connect. The security threat goes away. So we may see more of that as a solution to that.

Gardner: Andras, do you see that that some of the implications of big data, far fetched as it may be, are propelling people to cultivate their servers more and virtualize their apps, their data, and their desktop right up to the end devices?

Szakal: Yeah, I do. I see IBM providing solutions for virtual desktop, but I think it was really a security question you were asking. You're certainly going to see an additional number of virtualized desktop environments.

Ultimately, our network still is not stable enough or at a high enough bandwidth to really make that useful exercise for all but the most menial users in the enterprise. From a security point of view, there is a lot to be still solved.

And part of the challenge in the cloud environment that we see today is the proliferation of virtual machines (VMs) and the inability to actually contain the security controls within those machines and across these machines from an enterprise perspective. So we're going to see more solutions proliferate in this area and to try to solve some of the management issues, as well as the security issues, but we're a long ways away from that.

Gardner: Okay, I am going to put you on the spot a little bit, because I want you to provide to us some examples of how you think big data is being used in a way that's fundamentally different than traditional data.

If you don't have permission to name these people don't, but you can just describe the use case. Let's just start with you Chris. You probably have quite a few in your own organization, but are there any ways that you're aware of that people are using big data that illustrate how fundamentally different and powerful this is going to be?

Most compelling

Gerty: We have several small projects that have come out of the events that we’ve worked on. The International Space Apps Challenge I mentioned before. These are mostly in the visualization realm, but it's the problems that go beyond those events that are really the most compelling. I’ll briefly touch on one.

A challenge that we’ve put out in the last Space Apps Challenge was to write an app that would allow someone to use NASA data to allow a farmer anywhere in the world to have an iPhone app or iPad app and say. "I live here. What should I grow? What could make me the most money and help my village the most?"

The team that worked on it quickly realized that even great satellite data didn't work for their application. There are too many other factors. There was the local economy, the runoff levels, and things that they just didn't have access to from the NASA data. So they decided that this was more than a just weekend project and they wanted to build that data set that they needed, so that they could finally make the product.

They found other collaboration mechanisms to continue the project after the Spaces Apps Challenge. They’ll be returning this year to the second one that we do in April with an entirely different view on the world, because they actually have some data sets now that they've been building up. They made some mechanism to capture it from the local environment.

Gardner: So that’s a great reminder that we’re not just talking about big data, but we’re talking about multiple big data and which ones you can pull together -- joined or otherwise -- to collate and produce big-data analysis results for something very, very interesting.

Gerty: Big data, by itself, isn't magical. It doesn't have the answers just by being big. If you need more, you need to pry deeper into it. That’s the example. They realized early enough that they were able to make something good.

Gardner: Chris, that’s a very good cause, but in a purely commercial sense, as we see more companies doing cloud ecosystem and partnership activities, when they start to share their data with that big "if" of secured and provisioned properly with other people in their markets, in their businesses, very powerful and interesting things can happen. Jim Hietala, any thoughts about examples that illustrate where we’re going and why this is so important.

Hietala: Being a security guy, I tend to talk about scare stories, horror stories. One example from last year that struck me. One of the major retailers here in the U.S. hit the news for having predicted, through customer purchase behavior, when people were pregnant.

They could look and see, based upon buying 20 things, that if you're buying 15 of these and your purchase behavior has changed, they can tell that. The privacy implications to that are somewhat concerning.

An example was that this retailer was sending out coupons related to somebody being pregnant. The teenage girl, who was pregnant hadn't told her family yet. The father found it. There was alarm in the household and at the local retailer store, when the father went and confronted them.

Privacy implications

There are privacy implications from the use of big data. When you get powerful new technology in marketing people's hands, things sometimes go awry. So I'd throw that out just as a cautionary tale that there is that aspect to this. When you can see across people's buying transactions, things like that, there are privacy considerations that we’ll have to think about, and that we really need to think about as an industry and a society.

Gardner: Just because you can do something, doesn't necessarily mean you should.

Allen Brown: Can I put some of the questions in and see how you can do with them? The first one is more of a bit of a security question, but also concerns things like thoughts on self-protecting data, like the Jericho Forum issues, and another one that says, in terms of security, that big data may not have strong confidentiality and availability requirements, but for collaboration, doesn't integrity nearly always need to considered. Other examples are that there is no integrity requirement.

Gardner: Jim, I think it’s best directed to you to start. These are issues about controlled managements. Any thoughts?

Hietala: I'll get straight to the integrity piece. The integrity of the data, whether it’s on older platforms or big data, is certainly an issue. When folks are using big data, that data has to have integrity, and there has to be adequate controls protect the data. So I think that is kind of a fundamental thing for big data as well.

Gardner: Anyone else on these issues of protection?

Gerty: It’s not only a matter of data protection. It's what we do with the data. Big data is a term that is kind of heading towards the end of its usefulness, because it's not the data and how large it is that's useful. It's actually how we apply these deep analytics solutions, for example Watson. You saw the Watson win on Jeopardy, but now Watson is a product that’s being used to help some customers diagnose disease and work with the insurance companies.

How you actually utilize that data to derive value through this deep analytics solution is through a new set of artificial-intelligence applications called cognitive computing. So cognitive computing, how you drive all of this information, and how you apply it in the context of its usefulness to privacy and security is going to be huge in the following years.

Gardner: Allen, other questions from the audience or online?

Brown: Interoperability is the focus of a couple of questions. One is asking if you can address the expected interoperability issues across semantics of big data. The other part of it asks what’s the unique challenge or problems that unstructured, big data from Twitter, Facebook, and so on present?

Gardner: This might be an area where the concepts work for traditional data, and it might still be the case that is we have to pull all these different data types, structured and unstructured, together to work in some holistic fashion. Bob, any thoughts about big data, correlating of different data is that different from the past? Is there something new?

Weisman: I'm looking at techniques that were pioneered 20-30 years ago on the artificial intelligence, knowledge base system side, and are still is relevant today. As a matter of fact they're more relevant than they've ever been. There is lot opportunity, but it doesn’t forego having a good interoperability architecture, understanding where your contacts are, and being able to integrate data. Right now most of analytics is kiboshed, because they spent all their time doing data integration, versus analytics, and it’s a great waste of a lot of people's times.

So if you architect this from the get-go, get the proper metadata, which will address some of the integrity, and understand the concept of data quality which is what’s coming through, that will go a long way to resolving some of these issues, but the architecture is going to be key, as is rigorous planning.

More usable

Gardner: Andras, same question. Is there something new or different about treating data in order to make it more useable?

Szakal: Big data is coming to us in all sorts of forms and formats. It’s coming from different sources. We don't really know the validity. The validity is determined by the application of the analytics solution. You'll have to have some internal process, some governance process, to determine whether you're getting the validity of the data that you expect.

When I was working as a graduate student for the psychology department as the SPSS programmer, people would bring their work to me. They would try to apply analytics to make any point they possibly could. It's the old story about making statistics mean anything you want. But you have to be very careful about how you do that, because it’s going to have a huge impact on your business.

Gardner: Jim, in the realm of privacy and security, any thoughts about what types of unstructured content you may or may not want to bring in? Is this something now that you need to consider, picking and choosing of data types with an overview or lens towards security and privacy issues?

Hietala: In terms of unstructured content, there’s a whole lot of work to be done there to understand the growth of that stuff in average enterprise and what's really in unstructured content stores. A lot of that is ending up in collaboration platforms today, and most organizations don’t have a great understanding of what’s really in there.

It’s the regulated data in there, sensitive data in there. That’s an area where there’s work to be done by most enterprises to understand that unstructured content and the risk that it represents to the business.

Gardner: We haven’t got into it,, but another factor is the whole social sphere of data, and information that is being generated constantly.

Brown: The next question is a concern about whether it's causing a disruption to object orientation. Object-oriented data is encapsulated by the application, and making big data shared seems to break this approach. What are your thoughts on that?

Gardner: All right, from an architectural standpoint we're treating data a little bit differently, separating it entirely from an application or service.

Hietala: We just did a study that of this exact same question and problem. We found that there's no official programming model of the big-data world or in the cloud, although it is all about the client and integration with services. But there are all sorts of programming models out there. I would say that you apply the one that’s got the best and most appropriate approach.

Information centric

Weisman: It’s starting to put the emphasis back on the information syllable and information technology. Object orientation was meant to basically support an information-centric approach, and now it’s being used much more as a service-centric approach. Now we’re going to go back to a much more information-centric, information-engineering approach and a lot of the architecture enabled by big data.

Gardner: Maybe you could just expand that a little bit for me? Does that mean we have a different type of application? That is to say that data is the application? What were the implications of what you just said?

Weisman: When object orientation first came out, the idea was to take the data and build services around it. Now, we have services that pass data back and forth. Most organizations have hundreds of applications with encapsulated data within them, and they can’t share it. Often the same information is found in hundreds of applications, which causes a huge security headache. Now we should be looking at getting much more information centric which is the core of information technology, information related technology.

Gardner: So really it's a flip architecturally, when you think about maintaining a pool resource of information, and applications are either newly built to expose and leverage, or all your existing applications also have to bring into and connect to and integrate. Fair enough?

Weisman: I think it’s a separation between process-centric services and information-centric services and harmonizing those. That will probably be the best bang for the buck.

Gardner: So now we're into IT transformation and business transformation, and you have to rethink your data center and your entire apparatus for supporting your storage. People are going to get into that anyway for some of the reasons we talked about, but again, we could look at big data and say this is an accelerator to some of those transformation efforts.

Brown: Something that has been troubling me is around the data architecture. Mike Walker, now at Dell, on the live stream, is asking what specific guidance and best practices can you give to enterprise data architects to properly architect their information architectures.

Weisman: We're talking that this afternoon. There’s going to be an entire track or two tracks on data architecture, which will be providing the guidance and it’s big-data centric.

Gerty: You're still going to be able to identify the service that provides the authoritative source for a set of data and marry that with other information, as necessary, whether it be sentiment analysis or what not, but you're always going to have to be able to point to that authoritative source.

Brown: Well, data architectures can be highly structured and big data can be somewhat unstructured. How do you marry the two?

Authoritative records

Gerty: How do you marry the two? Transactional systems are still very important. You have to be able to identify the authoritative records. Big data usually comes in multiple sources from multiple, different venues. The best example of the use of big data is around sentiment analysis, taking feeds from Twitter, Facebook, and these multiple sources, and then being able to analyze the information to the context of the authoritative sources. So your analytics have to take all of this into consideration.

Brown: Okay, we are just out of time. I just want to get a quick comment on these two other live streams. How are companies dealing with the shortage of big data scientists? Are they training current employees?

Gardner: A key question is who is actually spearheading this? Who is in the best position to be qualified? Under whose auspices do these big data initiatives fall? Let’s start with you Chris. Any insight as to how you've done it at NASA?

Gerty: I would draw a parallel from when I was in Mission Control and pretty highly trained. They wipe your brain and fill it up with everything you need to know, but we weren't really enabled to make those decisions, until we went through the data, page by page, and looked at each individual blip. If you can automate those, then you need less of whomever it is who's doing the job.

Automation there would have helped us immensely to make those decisions on the fly, rather than going over pages and pages of data from our batteries charging. It's not maybe that you need more data scientists, but you need the right data scientists. Then you need to be able to leverage off of other people’s data scientists. That's why open source is so attractive to us. You only need to do it once and then you can go off of it.

Gardner: Jim Hietala, the people that should be doing this, their qualification certification, organizational structure, any thoughts?

Hietala: It's way too early to certify people in this category right now. We really need individuals who went to graduate school to understand the proper application of analytics and mathematics. Those individuals would be highly valuable and prized, especially as they learn to how to apply that knowledge to your business.

Gardner: It’s tough to find the people who have deep and the wide expertise. Last word you, Bob?

Weisman: We have to take a look at career development within the CIO ranks. Making sense of data requires good business knowledge and too many people are being isolated within the CIO rank. They should be circulating throughout the companies, so they know what the company is doing, and then come back in. It's much more valuable.

There are some programs now that are joint ventures between the computer science departments and the business schools, and I think those are at the graduate level. As Andras was saying, they could provide people in their early 30s that can really do a fantastic job, and we really start taking advantage of this.

Brown: That's all we have time for. I think you've done a marvelous job, thank you very much.

Gardner: We’ve been talking with a panel of experts on how big data changes the status quo for architecting the enterprise. We've heard how large enterprises should better anticipate and prepare for the effects and impacts of big data, as well the simultaneous impacts of cloud computing and mobile.

This special BriefingsDirect discussion comes to you in conjunction with The Open Group Conference in Newport Beach, California. I'd like to thank our panel: Robert Weisman, CEO and Chief Enterprise Architect at Build The Vision; Andras Szakal, Vice President and CTO of IBM's Federal Division; Jim Hietala, Vice President for Security at The Open Group, and Chris Gerty, Deputy Program Manager at the Open Innovation Program at NASA.

This is Dana Gardner, Principal Analyst at Interarbor Solutions, your host and moderator through these thought leadership interviews. Thanks again for listening, and come back next time.

Listen to the podcast. Find it on iTunes. Watch the video. Download the transcript. Sponsor: The Open Group.

Transcript of a BriefingsDirect podcast from The Open Group Conference in January on how big data forces changes in architecting the enterprise. Copyright The Open Group and Interarbor Solutions, LLC, 2005-2013. All rights reserved.

You may also be interested in:

• Using the Cloud for Big-Data Requires a New Recipe
• Big Data Success Depends on Better Risk Management Practices Like FAIR, Say The Open Group Panelists
• The Open Group Keynoter Sees Big-Data Analytics Bolstering Quality, Manufacturing, Processes
• The Open Group Trusted Technology Forum is Leading the Way to Securing GLobal IT Supply Chains
• Corporate Data, Supply Chains Remain Vulnerable to Cyber Crime Attacks Says Open Group Conference Speaker
• Open Group Conference Speakers Discuss the Cloud: Higher Risk or Better Security?
• Capgemini's CTO on Why Cloud Computing Exposes the Duality Between IT and Business 

MIT's Ross on How Enterprise Architecture and IT More Than Ever Lead to Business Transformation

Transcript of a BriefingsDirect podcast in conjunction with The Open Group Conference in San Francisco on how enterprise architecture can lead to greater efficiency and agility.

Register for The Open Group Conference
Jan. 30 - Feb. 3 in San Francisco.

Listen to the podcast. Find it on iTunes/iPod. Download the transcript. Sponsor: The Open Group.

Dana Gardner: Hello, and welcome to a special BriefingsDirect thought leadership interview series coming to you in conjunction with The Open Group Conference this month in San Francisco. I'm Dana Gardner, Principal Analyst at Interarbor Solutions and I will be your host throughout these discussions.

The conference will focus on how IT and enterprise architecture support enterprise transformation. Speakers in conference events will also explore the latest in service oriented architecture (SOA), cloud computing, and security.

Today, we're here with one of the main speakers at the conference, Jeanne Ross, Director and Principal Research Scientist at the MIT Center for Information Systems Research. Jeanne studies how firms develop competitive advantage through the implementation and reuse of digitized platforms.

She is also the co-author of three books: IT Governance: How Top Performers Manage IT Decision Rights for Superior Results, Enterprise Architecture As Strategy: Creating a Foundation for Business Execution, and IT Savvy: What Top Executives Must Know to Go from Pain to Gain.

As a lead-in to her Open Group presentation on how adoption of enterprise architecture (EA) leads to greater efficiencies and better business agility, Jeanne and I will now explore how enterprise architects have helped lead the way to successful business transformations.

Please join me now in welcoming Jeanne Ross, Director and Principal Research Scientist at the MIT Center for Information Systems Research. Welcome back to BriefingsDirect, Jeanne. [Disclosure: The Open Group is a sponsor of BriefingsDirect podcasts.]

Jeanne Ross: Thank you, Dana. Nice to be here.

Gardner: Your upcoming presentation will describe how enterprise architecture has contributed to success for such companies as Campbell Soup and Southwest Airlines, but before we go into that, it has been typically difficult to concretely link things like IT productivity and general business success. I wonder, then, how you measure or determine that enterprise architects and their practices are intrinsic to successful business transformations? How do we link the two?

Ross: That’s a great question. Today, there remains kind of a leap of faith in recognizing that companies that are well-architected will, in fact, perform better, partly because you can be well-architected and perform badly. Or if we look at companies that are very young and have no competitors, they can be very poorly architected and achieve quite remarkably in the marketplace.

But what we can ascribe to architecture is that when companies have competition, then they can establish any kind of performance target they want, whether it’s faster revenue growth or better profitability, and then architect themselves so they can achieve their goals. Then, we can monitor that.

We do have evidence in repeated case studies of companies that set goals, defined an architecture, started to build the capabilities associated with that architecture, and did indeed improve their performance. We have wonderful case study results that should be very reaffirming. I accept that they are not conclusive.

Architectural maturity

We also have statistical support in some of the work we've done that shows that high performers in our sample of 102 companies, in fact, had greater architecture maturity. They had deployed a number of practices associated with good architecture.

So we do have evidence. It’s just that if you really don’t want to believe it, you could poke holes in it. There still is a certain amount of faith attached to the link between performance and architecture.

Gardner: I certainly get your point that repeatability would be a chief indicator, that if you intend to do something repeatedly, you can point to the ways in which you would carry that out. How about the intent from the perspective of wanting to transform in a certain way that you haven’t done before? Is there something that being an architect allows that’s different from the past? Is there something that’s new about this, rather than just trying to reengineer something?

Ross: Yes, the thing we're learning about enterprise architecture is that there's a cultural shift that takes place in an organization, when it commits to doing business in a new way, and that cultural shift starts with abandoning a culture of heroes and accepting a culture of discipline.

Nobody wants to get rid of the heroes in their company. Heroes are people who see a problem and solve it. But we do want to get past heroes sub-optimizing. What companies traditionally did before they started thinking about what architecture would mean, is they relied on individuals to do what seemed best and that clearly can sub-optimize in an environment that increasingly is global and requires things like a single face to the customer.

Nobody wants to get rid of the heroes in their company. Heroes are people who see a problem and solve it.

What we're trying to do is adopt a culture of discipline, where there are certain things that people throughout an enterprise understand are the way things need to be done, so that we actually can operate as an enterprise, not as individuals all trying to do the best thing based on our own experience.

The fundamental difference of being an architected firm is that there is some underlying discipline. I'll caution you that what tends to happen is great architects really embrace the discipline. They love the discipline. They understand the discipline, and there is a reluctance to accept that that’s not the only thing we need in our organization. There are times when ad hoc behaviors enable us to be much more innovative and much more responsive and they are exactly what we need to be doing.

So there is a cultural shift that is critical to understanding what it is to be architected. That’s the difference between a successful firm that’s successful because it hasn’t gotten into a world of really tough competition or restrictions on spending and things like that and an organization that is trying to compete in a global economy.

Gardner: It’s interesting to me that we're focusing not so much on the individual, the enterprise architect, but more the office of the enterprise architect.

Ross: Right. Would you like me to speak to an architect instead? Would that help?

Cultural phenomenon

Gardner: No, the point is that the champion that is important is not just an individual. It’s that putting into place a repeatable office of the enterprise architect that is a cultural phenomenon, rather than a charismatic one.

Ross: Yes.

Gardner: What then is the role of the architect, if this isn’t just about a champion, but really about change that’s repeatable and that’s culturally inculcated? What, then, is the role and what should they do?

Ross: The architect plays a really critical role in representing the need for this discipline, for some standards in the organization, and for understanding the importance of shared definitions for data. The architect should be able to create a very constructive tension in the organization, and that’s the tension between individuality, innovativeness, local responsiveness, and the need for enterprise thinking, standardization, and discipline.

Normally, in most companies, the architect’s role will be the enforcer of discipline, standardization and enterprise thinking. The tension will be created by all kinds of people who are saying, "Wait, I'm different. I need this. My customer insists on that." When the tension is working effectively, you get just enough architecture.

One thing we've learned over the years, as we've studied architecture, is that’s actually what we want. We don’t want to be a tightly architected organization, because tomorrow we're going to wake up and the world is going to change, and we have to be ready for that. We want to be architected enough to be efficient, to be able to reuse those things we need to reuse, to be agile, but we don’t want to start embracing architecture for architecture’s sake or discipline for discipline’s sake.

We don’t want to be a tightly architected organization, because tomorrow we're going to wake up and the world is going to change, and we have to be ready for that.

We really just need architecture to pull out unnecessary cost and to enable desirable reusability. And the architect is typically going to be the person representing that enterprise view and helping everyone understand the benefits of understanding that enterprise view, so that everybody who can easily or more easily see the local view is constantly working with architects to balance those two requirements.

Gardner: Let’s take a contextual view here. It’s 2012 already and there's a lot happening in IT with disruption in the form of cloud computing trends, an emphasis on mobile computing, big data, and the ability to harness analytics in new and interesting ways, all sort of churning together. We're also still faced with a difficult environment, when it comes to the economy. Is this a particularly good time, from your vantage point, to undertake enterprise architecture, or is this perhaps not the best time?

Ross: It’s a great time for most companies. There will be exceptions that I'll talk about in a minute. One thing we learned early on in the research is that companies who were best at adopting architecture and implementing it effectively had cost pressures. What happens when you have cost pressures is that you're forced to make tough decisions.

If you have all the money in the world, you're not forced to make tough decisions. Architecture is all about making tough decisions, understanding your tradeoffs, and recognizing that you're going to get some things that you want and you are going to sacrifice others.

If you don't see that, if you just say, "We're going to solve that by spending more money," it becomes nearly impossible to become architected. This is why investment banks are invariably very badly architected, and most people in investment banks are very aware of that. It’s just very hard to do anything other than say, "If that’s important to us, let’s spend more money and let’s get it." One thing you can't get by spending more money is discipline, and architecture is very tightly related to discipline.

Register for The Open Group Conference
Jan. 30 - Feb. 3 in San Francisco.

Tough decisions

In a tough economy, when competition is increasingly global and marketplaces are shifting, this ability to make tough decisions is going to be essential. Opportunities to save costs are going to be really valued, and architecture invariably helps companies save money. The ability to reuse, and thus rapidly seize the next related business opportunity, is also going to be highly valued.

The thing you have to be careful of is that if you see your markets disappearing, if your product is outdated, or your whole industry is being redefined, as we have seen in things like media, you have to be ready to innovate. Architecture can restrict your innovative gene, by saying, "Wait, wait, wait. We want to slow down. We want to do things on our platform." That can be very dangerous, if you are really facing disruptive technology or market changes.

So you always have to have that eye out there that says, "When is what we built that’s stable actually constraining us too much? When is it preventing important innovation?" For a lot of architects, that’s going to be tough, because you start to love the architecture, the standards, and the discipline. You love what you've created, but if it isn’t right for the market you're facing, you have to be ready to let it go and go seize the next opportunity.

Gardner: Perhaps this environment is the best of all worlds, because we have that discipline on the costs which forces hard decisions, as you say. We also have a lot of these innovative IT trends that would almost force you to look at doing things differently. I'm thinking again of cloud, mobile, the big data issues, and even social-media types of effects. So is that the case from your perspective?

Ross: Absolutely. We should all look at it that way and say, "What a wonderful world we live in." One of the companies that I find quite remarkable in their ability to, on the one hand, embrace discipline and architecture, and on the other hand, constantly innovate, is USAA. I'm sure I'll talk about them a little bit at the conference.

This is a company that just totally understands the importance of discipline around customer service. They're off the charts in their customer satisfaction.

This is a company that just totally understands the importance of discipline around customer service. They're off the charts in their customer satisfaction.

They're a financial services institution. Most financial services institutions just drool over USAA’s customer satisfaction ratings, but they've done this by combining this idea of discipline around the customer. We have a single customer file. We have an enterprise view of that customer. We constantly standardize those practices and processes that will ensure that we understand the customer and we deliver the products and services they need. They have enormous discipline around these things.

Simultaneously, they have people working constantly around innovation. They were the first company to see the need for this deposit with your iPhone. Take a picture of your check and it’s automatically deposited into your account. They were nearly a year ahead of the next company that came up with that service.

The way they see it is that for any new technology that comes out, our customer will want to use it. We've got to be there the day after the technology comes out. They obviously haven't been able to achieve that, but that’s their goal. If they can make deals with R&D companies that are coming up with new technologies, they're going to make them, so that they can be ready with their product when the thing actually becomes commercial.

So it's certainly possible for a company to be both innovative and responsive to what’s going on in the technology world and disciplined and cost effective around customer service, order-to-cash, and those other underlying critical requirements in your organization. But it's not easy, and that's why USAA is quite remarkable. They've pulled it off and they are a lesson for many other companies.

Gardner: And as you pointed out, being able to repeat this is really essential. So that gets back to that discipline. But you've mentioned that you've got ongoing research, and you've mentioned a company, USAA that you're working with and you're familiar with. I suppose this gives us a chance then to step back and take a look at what the MIT Center for Information Systems Research is and does and your role there.

Value from IT

Ross: The Center for Information Systems Research is part of the Sloan School of Management. We were formed in 1974 to study how companies get value from information technology.

In 1974, we were studying mainframes and IT directors. There was no such thing as a CIO yet, but we have certainly gone through the stages of the increasing importance of IT in organizations. We went through the end-user computing. We went through enterprise resource planning (ERP) and e-business. We've followed, and hopefully led, thinking around how IT adds value in organizations.

You mentioned this is a good time to be introducing architecture. This is a good time to be at the Center for Information Systems Research, because IT is so central now to business success, and many companies that didn't start as digital companies are really struggling to understand what it means to transform for the digital economy, and that's exactly what we study.

Gardner: You've mentioned one company, USAA. Let’s take a look at a number of companies. I know you're going to be mentioning several during your presentation. Are there any salient lessons that are common among them? Are they all different and therefore you can't draw such common denominators, or are there a couple that jump out?

Ross: Well, our established research on this, and this is the work that appeared in the Enterprise Architecture as Strategy book. We find that the things we learned as we prepared that book are still very true. Companies indeed go through stages, and they're very predictable -- we've not yet seen an exception to this -- and they're hard.

You have to respond to the marketplace. You have to do whatever it takes.

Stage one is the stage of, don't worry about the discipline, just have fun, learn how to use IT, apply it to any strategic need where it makes sense, and go out there and do your thing, but eventually all of that will lead to a fairly messy legacy environment.

We saw, when we studied these stages, that as companies understood these stages, they would avoid stage one, but it turns out that, if you are a fast growing innovative company, you can't avoid that stage. You actually don't know how you're going to make money. You have to respond to the marketplace. You have to do whatever it takes. Then, as you get really good at things, you start to establish yourself in what is often now a new industry.

You've created an industry. That's how you succeeded. But because you're making money, you're going to attract competitors. When you get to the stage that you actually have competitors, then you look at what you created and you say, "Oh no, we really have to clean up some of this legacy." That’s really what stage two is about. It's the underlying technology.

Now, we're learning how to not make quite as big a mess, but there is still this stage of, "Okay, let's refrain from kind of the crazy innovation and be more disciplined about what we put in and how we reuse" and all that kind of thing.

In the third stage, we get much more emphasis on building platforms that wire in those core processes that enable us to do high-volume transactions. These are things around order-to-cash, human resources (HR), or finance. There will be some of that in the earlier stages, but we really worry about scale in this third stage, scaling up so that we can manage large volume transactions.

We think this third stage is going to look different in a world of software as a service (SaaS) and cloud, because in the past, third stage often meant you put in Oracle, SAP, or something like that. Nowadays, it's much more about piecing together some cloud services. It does look different. It goes in faster, but it's still pretty tricky. If you're not architected well, you can really create a mess in stage three.

Working smarter

Stage four is really about working smarter on this platform, learning how to innovate off the platform. And companies are struggling to get there, because once you get in this platform, it takes a while to really make it solid and learn how to use it well. We've been studying that for some time, and companies get there.

This is the story of Campbell Soups and the Southwest Airlines. They're trying to use the platforms they've created, even though the process of putting them in takes a very long time. So you're still putting them in, while you are trying to learn to get good at using them. It's a challenging world out there.

Gardner: So I shouldn’t reach the conclusion that the enterprise architecture kicks in, in stage three and four. It should be something that would be there and useful throughout these stages.

Ross: That's correct. What happens is that in stage one you don't think a lot about architecture. If you don’t think at all, you are going to regret it. But you just can't predict what are going to be the critical capabilities in your organization. When you can't predict the critical capabilities in your organization, it limits how much you can architect.

You can bet on some things. There are some things around finance and HR that are pretty predictable even in stage one. But that early stage is where you're really defining yourself as a company, and that can last for some years, as you grow. As long as you're under $500 million in sales or at least, let's say, $200 million in sales, you've got some leverage there, because you can only create so big of a mess.

The Open Group is great for me, because there is so much serious thinking in The Open Group about what architecture is, how it adds value, and how we do it well.

If you start growing beyond that, you're going to need more architecture. That’s when you really get into stage two and start seriously defining your standards and the processes that enable you to get them in and recognize when you need exceptions and when they're out of date and that kind of thing.

Gardner: So even as we have had this evolution in these stages that happen within these enterprises, we have also had historical evolution in the definition, standardization, and certification around the architects themselves. Where are we there? Is there a stage three or four that we are at with the architects?

Ross: I think we'll be constantly tweaking the certification processes for architects. We get smarter about what they need to know and what they need to be good at, but I don’t know that I would so much call it stages for the architect certification as just getting smarter and smarter about what great architects will excel at. We have the basics in place. I haven't been involved a lot in certification programs, but I think there is a good sense of the basics that are required.

Gardner: We certainly seem to be well into a professionalization phase and we've got a number of different groups within The Open Group that are working on that across different disciplines. So I'm curious. Is The Open Group a good forum for your message and your research, and if so, why?

Ross: The Open Group is great for me, because there is so much serious thinking in The Open Group about what architecture is, how it adds value, and how we do it well. For me to touch base with people in The Open Group is really valuable, and for me to touch base to share my research and hear the push back, the debate, or the value add is perfect, because these are people who are living it every day.

Major themes

Gardner: Are there any other major themes that you'll be discussing at the conference coming up that you might want to share with us? Did we cover them all? What did we leave out?

Ross: Well, we're still doing the analysis on our latest survey. So I'm not exactly sure what the key findings will be that I'll be sharing. One thing we have observed in our cases that is more and more important to architects is that the companies are struggling more than we realized with using their platforms well.

I'm not sure that architects or people in IT always see this. You build something that’s phenomenally good and appropriate for the business and then you just assume, that if you give them a little training, they'll use it well.

That’s actually been a remarkable struggle for organizations. One of our research projects right now is called "Working Smarter on Your Digitized Platform." When we go out, we find there aren't very many companies that have come anywhere close to leveraging their platforms the way they might have imagined and certainly the way an architect would have imagined.

It's harder than we thought. It requires persistent coaching. It's not about training, but persistent coaching. It requires enormous clarity of what the organization is trying to do, and organizations change fast. Clarity is a lot harder to achieve than we think it ought to be.

We find there aren't very many companies that have come anywhere close to leveraging their platforms the way they might have imagined and certainly the way an architect would have imagined.

The message for architects would be: here you are trying to get really good at being a great architect. To add value to your organization, you actually have to understand one more thing: how effectively are people in your company adopting the capabilities and leveraging them effectively? At some point, the value add of the architecture is diminished by the fact that people don't get it. They don’t understand what they should be able to do.

We're going to see architects spending a little more time understanding what their leadership is capable of and what capabilities they'll be able to leverage in the organization, as opposed to which on a rational basis seem like a really good idea.

We've been studying companies, and the easiest ones to study are ones like 7-Eleven Japan and Protection One, which is a security company. These are companies that have replicated models. You look at one branch or one store and you say, "How are you doing this?" Then you say, "Okay, here is the best one. How are we going to make sure that everybody uses our technology and the information that's coming from it? How are we going to do that throughout the company?"

That’s even harder than designing and implementing an architecture. Architects are going to have to be well aware of that, because if companies are not driving value from what they have built, you may as well stop spending the money. That’s a tough thing for an architect to admit, because there’s so much you can do just on a rational basis to make the company look better. But if they are not using it, it's not worth anything.

Gardner: That might explain some of the attention that’s been given to things like cloud and mobile, because there is a sense of an organic adoption going on, and if the workers, the managers, the departments, specific functional groups like marketing, for example, are going to SaaS, cloud, mobile for "bring your own device," or consumerization of IT benefits, perhaps there's an opportunity to take advantage of that, learn from it, and then standardize it and implement as a platform. Is that somewhere close to what you are seeing?

Ross: Yes, absolutely.

Getting started

Gardner: Before we segue out, let's consider advice about getting started. When you're an organization and you've decided that you do want to be a level three or four maturity, that you want to transform and take advantage of unique opportunities for either technical disruption or market discipline, how do you go about getting more structure, more of an architecture?

Ross: That's idiosyncratic to some extent, because in your dream world, what happens is that the CEO announces, "This is what we are going to be five years from now. This is how we are going to operate and I expect everyone to get on board." The vision is clear and the commitment is clear. Then the architects can just say, and most architects are totally capable of this, "Oh, well then, here are the capabilities we need to build. Let’s just go build them and then we'll live happily ever after."

The problem is that’s rarely the way you get to start. Invariably, the CEO is looking at the need for some acquisitions, some new markets, and all kinds of pressures. The last thing you're getting is some clarity around the vision of an operating model that would define your critical architectural capabilities.

What ends up happening instead is architects recognize key business leaders who understand the need for, reused standardization, process discipline, whatever it is, and they're very pragmatic about it. They say, "What do you need here to develop an enterprise view of the customer, or what’s limiting your ability to move into the next market?"

And they have to pragmatically develop what the organization can use, as opposed to defining the organizational vision and then the big picture view of the enterprise architecture.

When they see real demand and real leadership around certain enterprise capabilities, they focus their attention on addressing those.

So in practice, it's a much more pragmatic process than what we would imagine when we, for example, write books on how to do enterprise architecture. The best architects are listening very hard to who is asking for what kind of capability. When they see real demand and real leadership around certain enterprise capabilities, they focus their attention on addressing those, in the context of what they realize will be a bigger picture over time.

They can already see the unfolding bigger picture, but there’s no management commitment yet. So they stick to the capabilities that they are confident the organization will use. That’s the way they get the momentum to build. That is more art than science and it really distinguishes the most successful architects.

Gardner: We'll be looking forward to learning more through your research and through the examples that you provide.

We've been talking with Jeanne Ross, the Director and Principal Research Scientist at the MIT Center for Information Systems Research. Jeanne and I have been exploring how enterprise architects have helped lead the way to successful business transformations as a lead-in to her upcoming Open Group presentation.

This special BriefingsDirect discussion comes to you in conjunction with The Open Group’s Conference, which is January 30 to February 3 in San Francisco. You'll hear more from Jeanne and many other global leaders on the ways that IT and enterprise architecture support enterprise transformation.

So thank you, Jeanne, for joining us in this fascinating discussion. I really had a good time.

Ross: Thanks so much, Dana, I enjoyed it.

Gardner: And I look forward to your presentation in San Francisco and I encourage our listeners and readers to attend the conference, if they're able. There’s more information available on our website and through this content.

This is Dana Gardner, Principal Analyst at Interarbor Solutions, your host and moderator throughout this Thought Leader Interview Series. Thanks again for listening, and come back next time.

Listen to the podcast. Find it on iTunes/iPod. Download the transcript. Sponsor: The Open Group.

Transcript of a BriefingsDirect podcast in conjunction with The Open Group Conference in San Francisco on how enterprise architecture can lead to greater efficiency and agility. Copyright Interarbor Solutions, LLC, 2005-2012. All rights reserved.

Register for The Open Group Conference
Jan. 30 - Feb. 3 in San Francisco.

You may also be interested in:

• Overlapping Criminal and State Threats Pose Growing Cyber Security Threat to Global Internet Commerce, Says Open Group Conference Speaker
  
• Exploring Business-IT Alignment: A 20-Year Struggle Culminating in the Role and Impact of Business Architecture
• Enterprise Architects Increasingly Leverage Advanced TOGAF 9 for Innovation, Market Response, and Governance Benefits
• Open Group Cloud Panel Forecasts Cloud s Spurring Useful Transition Phase for Enterprise Architecture
• The Open Group's Cloud Work Group Advances Understanding of Cloud-Use Benefits for Enterprises
• Exploring the Role and Impact of the Open Trusted Technology Forum to Ensure Secure IT Products in Global Supply Chains