Oracle fills another gap in its big data offering

This guest post comes courtesy of Tony Baer’s OnStrategies blog. Tony is a senior analyst at Ovum.

By Tony Baer

When we last left Oracle’s big data plans, there was definitely a missing piece. Oracle’s Big Data Appliance as initially disclosed at last fall’s OpenWorld was a vague plan that appeared to be positioned primarily as an appliance that would accompany and feed data to Exadata. Oracle did specify some utilities, such as an enterprise version of the open source R statistical processing program that was designed for multithreaded execution, plus a distribution of a NoSQL database based on Oracle’s BerkeleyDB as an alternative to Apache Hive. But the emphasis appeared to be extraction and transformation of data for Exadata via Oracle’s own utilities that were optimized for its platform.

With Oracle’s announcement of general availability of the big data appliance, it is filling in the blanks.

As such, Oracle’s plan for Hadoop was competition, not for Cloudera (or Hortonworks), which featured a full Apache Hadoop platform, but EMC, which offered a comparable, appliance-based strategy that pairs Hadoop with an Advanced SQL data store; and IBM, which took a different approach by emphasizing Hadoop as an analytics platform destination enhanced with text and predictive analytics engines, and other features such as unique query languages and file systems.

Oracle’s initial Hadoop blueprint lacked explicit support of many pieces of the Hadoop stack such as HBase, Hive, Pig, Zookeeper, and Avro. No more. With Oracle’s announcement of general availability of the big data appliance, it is filling in the blanks by disclosing that it is OEM’ing Cloudera’s CDH Hadoop distribution, and more importantly, the management tooling that is key to its revenue stream. For Oracle, OEM’ing Cloudera’s Hadoop offering fully fleshes out its Hadoop distribution and positions it as a full-fledged analytic platform in its own right; for Cloudera, the deal is a coup that will help establish its distribution as the reference. It is fully consistent with Cloudera’s goal to become the Red Hat of Hadoop as it does not aspire to spread its footprint into applications or frameworks.

Question of acquisition

Of course, whenever you put Oracle in the same sentence as OEM deal, the question of acquisition inevitably pops up. There are several reasons why an Oracle acquisition of Cloudera is unlikely.

1. Little upside for Oracle. While Oracle likes to assert maximum control of the stack, from software to hardware, its foray into productizing its own support for Red Hat Enterprise Linux has been strictly defensive; its offering has not weakened Red Hat.
   
   
2. Scant leverage. Compare Hadoop to MySQL and you have a Tale of Two Open Source projects. One is hosted and controlled by Apache, the other is hosted and controlled by Oracle. As a result, while Oracle can change licensing terms for MySQL, which it owns, it has no such control over Hadoop. Were Oracle to buy Cloudera, another provider could easily move in to fill the vacuum. The same would happen to Cloudera if, as a prelude to such a deal, it began forking from the Apache project with its own proprietary adds-ons or substitutions.

OEMs deals are a major stage of building the market. Cloudera has used its first mover advantage with Hadoop well with deals Dell, and now Oracle. Microsoft in turn has decided to keep the “competition” honest by signing up Hortonworks to (eventually) deliver the Hadoop engine for Azure.

The final piece of the trifecta will be commitments from the Accentures and Deloittes of the world to develop practices based on specific Hadoop platforms.

OEM deals are important for attaining another key goal in developing the Hadoop market: defining the core stack – as we’ve ranted about previously. Just as Linux took off once a robust kernel was defined, the script will be identical for Hadoop. With IBM and EMC/MapR forking the Apache stack at the core file system level, and with niche providers like Hadapt offering replacement for HBase and Hive, there is growing variability in the Hadoop stack. However, to develop the third party ecosystem that will be vital to the development of Hadoop, a common target (and APIs for where the forks occur) must emerge. A year from now, the outlines of the market’s decision on what makes Hadoop Hadoop will become clear.

The final piece of the trifecta will be commitments from the Accentures and Deloittes of the world to develop practices based on specific Hadoop platforms. For now they are still keeping their cards close to their vests.

This guest post comes courtesy of Tony Baer’s OnStrategies blog. Tony is a senior analyst at Ovum.

You may also be interested in:

• Game on: Oracle Joins big data market with appliance
• Oracle's big data appliance brings focus to bundled approach
• Big data consolidation race enters home stretch, as Teradata buys Aster Data
• Big data: The next frontier for innovation, competition, and productivity

MIT's Ross on how enterprise architecture and IT more than ever lead to business transformation

Listen to the podcast. Find it on iTunes/iPod. Read a full transcript or download a copy. Sponsor: The Open Group.

This BriefingsDirect thought leadership interview comes in conjunction with The Open Group Conference this month in San Francisco.

The conference will focus on how IT and enterprise architecture support enterprise transformation. Speakers in conference events will also explore the latest in service oriented architecture (SOA), cloud computing, and security.

We’re now joined by one of the main speakers, Jeanne Ross, Director and Principal Research Scientist at the MIT Center for Information Systems Research. Jeanne studies how firms develop competitive advantage through the implementation and reuse of digitized platforms.

She is also the co-author of three books: IT Governance: How Top Performers Manage IT Decision Rights for Superior Results, Enterprise Architecture As Strategy: Creating a Foundation for Business Execution, and IT Savvy: What Top Executives Must Know to Go from Pain to Gain.

As a lead-in to her Open Group presentation on how adoption of enterprise architecture (EA) leads to greater efficiencies and better business agility, Ross explains how enterprise architects have helped lead the way to successful business transformations. The interview is moderated by Dana Gardner, Principal Analyst at Interarbor Solutions. [Disclosure: The Open Group is a sponsor of BriefingsDirect podcasts.]

Here are some excerpts:

Gardner: How you measure or determine that enterprise architects and their practices are intrinsic to successful business transformations?

Ross: That’s a great question. Today, there remains kind of a leap of faith in recognizing that companies that are well-architected will, in fact, perform better, partly because you can be well-architected and perform badly. Or if we look at companies that are very young and have no competitors, they can be very poorly architected and achieve quite remarkably in the marketplace.

But what we can ascribe to architecture is that when companies have competition, then they can establish any kind of performance target they want, whether it’s faster revenue growth or better profitability, and then architect themselves so they can achieve their goals. Then, we can monitor that.

We do have evidence in repeated case studies of companies that set goals, defined an architecture, started to build the capabilities associated with that architecture, and did indeed improve their performance. We have wonderful case study results that should be very reaffirming. I accept that they are not conclusive.

Architectural maturity

We also have statistical support in some of the work we've done that shows that high performers in our sample of 102 companies, in fact, had greater architecture maturity. They had deployed a number of practices associated with good architecture.

Gardner: Is there something that’s new about this, rather than just trying to reengineer something?

Ross: Yes, the thing we're learning about enterprise architecture is that there's a cultural shift that takes place in an organization, when it commits to doing business in a new way, and that cultural shift starts with abandoning a culture of heroes and accepting a culture of discipline.

Nobody wants to get rid of the heroes in their company. Heroes are people who see a problem and solve it. But we do want to get past heroes sub-optimizing. What companies traditionally did before they started thinking about what architecture would mean, is they relied on individuals to do what seemed best and that clearly can sub-optimize in an environment that increasingly is global and requires things like a single face to the customer.

Nobody wants to get rid of the heroes in their company. Heroes are people who see a problem and solve it.

What we're trying to do is adopt a culture of discipline, where there are certain things that people throughout an enterprise understand are the way things need to be done, so that we actually can operate as an enterprise, not as individuals all trying to do the best thing based on our own experience.

The fundamental difference of being an architected firm is that there is some underlying discipline. I'll caution you that what tends to happen is great architects really embrace the discipline. They love the discipline. They understand the discipline, and there is a reluctance to accept that that’s not the only thing we need in our organization. There are times when ad hoc behaviors enable us to be much more innovative and much more responsive and they are exactly what we need to be doing.

So there is a cultural shift that is critical to understanding what it is to be architected. That’s the difference between a successful firm that’s successful because it hasn’t gotten into a world of really tough competition or restrictions on spending and things like that and an organization that is trying to compete in a global economy.

Gardner: What then is the proper role of the architect?

Ross: The architect plays a really critical role in representing the need for this discipline, for some standards in the organization, and for understanding the importance of shared definitions for data. The architect should be able to create a very constructive tension in the organization, and that’s the tension between individuality, innovation, local responsiveness, and the need for enterprise thinking, standardization, and discipline.

Normally, in most companies, the architect’s role will be the enforcer of discipline, standardization and enterprise thinking. ... We want to be architected enough to be efficient, to be able to reuse those things we need to reuse, to be agile, but we don’t want to start embracing architecture for architecture’s sake or discipline for discipline’s sake.

We don’t want to be a tightly architected organization, because tomorrow we're going to wake up and the world is going to change, and we have to be ready for that.

We really just need architecture to pull out unnecessary cost and to enable desirable reusability. And the architect is typically going to be the person representing that enterprise view and helping everyone understand the benefits of understanding that enterprise view, so that everybody who can easily or more easily see the local view is constantly working with architects to balance those two requirements.

Gardner: Is this a particularly good time, from your vantage point, to undertake enterprise architecture?

Ross: It’s a great time for most companies. There will be exceptions that I'll talk about in a minute. One thing we learned early on in the research is that companies who were best at adopting architecture and implementing it effectively had cost pressures. What happens when you have cost pressures is that you're forced to make tough decisions.

If you have all the money in the world, you're not forced to make tough decisions. Architecture is all about making tough decisions, understanding your tradeoffs, and recognizing that you're going to get some things that you want and you are going to sacrifice others.

If you don't see that, if you just say, "We're going to solve that by spending more money," it becomes nearly impossible to become architected. This is why investment banks are invariably very badly architected, and most people in investment banks are very aware of that. It’s just very hard to do anything other than say, "If that’s important to us, let’s spend more money and let’s get it." One thing you can't get by spending more money is discipline, and architecture is very tightly related to discipline.

Register for The Open Group Conference
Jan. 30 - Feb. 3 in San Francisco.

Tough decisions

In a tough economy, when competition is increasingly global and marketplaces are shifting, this ability to make tough decisions is going to be essential. Opportunities to save costs are going to be really valued, and architecture invariably helps companies save money. The ability to reuse, and thus rapidly seize the next related business opportunity, is also going to be highly valued.

The thing you have to be careful of is that if you see your markets disappearing, if your product is outdated, or your whole industry is being redefined, as we have seen in things like media, you have to be ready to innovate. Architecture can restrict your innovative gene, by saying, "Wait, wait, wait. We want to slow down. We want to do things on our platform." That can be very dangerous, if you are really facing disruptive technology or market changes.

So you always have to have that eye out there that says, "When is what we built that’s stable actually constraining us too much? When is it preventing important innovation?" For a lot of architects, that’s going to be tough, because you start to love the architecture, the standards, and the discipline. You love what you've created, but if it isn’t right for the market you're facing, you have to be ready to let it go and go seize the next opportunity.

Gardner: Perhaps this environment is the best of all worlds, because we have that discipline on the costs which forces hard decisions, as you say. We also have a lot of these innovative IT trends that would almost force you to look at doing things differently. I'm thinking again of cloud, mobile, the big data issues, and even social-media types of effects.

Ross: Absolutely. We should all look at it that way and say, "What a wonderful world we live in." One of the companies that I find quite remarkable in their ability to, on the one hand, embrace discipline and architecture, and on the other hand, constantly innovate, is USAA. I'm sure I'll talk about them a little bit at the conference.

This is a company that just totally understands the importance of discipline around customer service. They're off the charts in their customer satisfaction.

This is a company that just totally understands the importance of discipline around customer service. They're off the charts in their customer satisfaction.

They're a financial services institution. Most financial services institutions just drool over USAA’s customer satisfaction ratings, but they've done this by combining this idea of discipline around the customer. We have a single customer file. We have an enterprise view of that customer. We constantly standardize those practices and processes that will ensure that we understand the customer and we deliver the products and services they need. They have enormous discipline around these things.

Simultaneously, they have people working constantly around innovation. They were the first company to see the need for this deposit with your iPhone. Take a picture of your check and it’s automatically deposited into your account. They were nearly a year ahead of the next company that came up with that service.

The way they see it is that for any new technology that comes out, our customer will want to use it. We've got to be there the day after the technology comes out. They obviously haven't been able to achieve that, but that’s their goal. If they can make deals with R&D companies that are coming up with new technologies, they're going to make them, so that they can be ready with their product when the thing actually becomes commercial.

So it's certainly possible for a company to be both innovative and responsive to what’s going on in the technology world and disciplined and cost effective around customer service, order-to-cash, and those other underlying critical requirements in your organization. But it's not easy, and that's why USAA is quite remarkable. They've pulled it off and they are a lesson for many other companies.

Gardner: Is The Open Group a good forum for your message and your research, and if so, why?

Ross: The Open Group is great for me, because there is so much serious thinking in The Open Group about what architecture is, how it adds value, and how we do it well. For me to touch base with people in The Open Group is really valuable, and for me to touch base to share my research and hear the push back, the debate, or the value add is perfect, because these are people who are living it every day.

Major themes
Gardner: Are there any other major themes that you'll be discussing at the conference coming up that you might want to share with us?

Ross: One thing we have observed in our cases that is more and more important to architects is that the companies are struggling more than we realized with using their platforms well.

I'm not sure that architects or people in IT always see this. You build something that’s phenomenally good and appropriate for the business and then you just assume, that if you give them a little training, they'll use it well.

That’s actually been a remarkable struggle for organizations. One of our research projects right now is called "Working Smarter on Your Digitized Platform." When we go out, we find there aren't very many companies that have come anywhere close to leveraging their platforms the way they might have imagined and certainly the way an architect would have imagined.

It's harder than we thought. It requires persistent coaching. It's not about training, but persistent coaching. It requires enormous clarity of what the organization is trying to do, and organizations change fast. Clarity is a lot harder to achieve than we think it ought to be.

We find there aren't very many companies that have come anywhere close to leveraging their platforms the way they might have imagined and certainly the way an architect would have imagined.

The message for architects would be: here you are trying to get really good at being a great architect. To add value to your organization, you actually have to understand one more thing: how effectively are people in your company adopting the capabilities and leveraging them effectively? At some point, the value add of the architecture is diminished by the fact that people don't get it. They don’t understand what they should be able to do.

We're going to see architects spending a little more time understanding what their leadership is capable of and what capabilities they'll be able to leverage in the organization, as opposed to which on a rational basis seem like a really good idea.

Getting started

Gardner: When you're an organization and you've decided that you do want to transform and take advantage of unique opportunities for either technical disruption or market discipline, how do you go about getting more structure, more of an architecture?

Ross: That's idiosyncratic to some extent, because in your dream world, what happens is that the CEO announces, "This is what we are going to be five years from now. This is how we are going to operate and I expect everyone to get on board." The vision is clear and the commitment is clear. Then the architects can just say, and most architects are totally capable of this, "Oh, well then, here are the capabilities we need to build. Let’s just go build them and then we'll live happily ever after."

The problem is that’s rarely the way you get to start. Invariably, the CEO is looking at the need for some acquisitions, some new markets, and all kinds of pressures. The last thing you're getting is some clarity around the vision of an operating model that would define your critical architectural capabilities.

What ends up happening instead is architects recognize key business leaders who understand the need for, reused standardization, process discipline, whatever it is, and they're very pragmatic about it. They say, "What do you need here to develop an enterprise view of the customer, or what’s limiting your ability to move into the next market?"

And they have to pragmatically develop what the organization can use, as opposed to defining the organizational vision and then the big picture view of the enterprise architecture.

When they see real demand and real leadership around certain enterprise capabilities, they focus their attention on addressing those.

So in practice, it's a much more pragmatic process than what we would imagine when we, for example, write books on how to do enterprise architecture. The best architects are listening very hard to who is asking for what kind of capability. When they see real demand and real leadership around certain enterprise capabilities, they focus their attention on addressing those, in the context of what they realize will be a bigger picture over time.

They can already see the unfolding bigger picture, but there’s no management commitment yet. So they stick to the capabilities that they are confident the organization will use. That’s the way they get the momentum to build. That is more art than science and it really distinguishes the most successful architects.

Listen to the podcast. Find it on iTunes/iPod. Read a full transcript or download a copy. Sponsor: The Open Group.

Register for The Open Group Conference
Jan. 30 - Feb. 3 in San Francisco.

You may also be interested in:

• Overlapping Criminal and State Threats Pose Growing Cyber Security Threat to Global Internet Commerce, Says Open Group Conference Speaker
• Exploring Business-IT Alignment: A 20-Year Struggle Culminating in the Role and Impact of Business Architecture
• Enterprise Architects Increasingly Leverage Advanced TOGAF 9 for Innovation, Market Response, and Governance Benefits
• Open Group Cloud Panel Forecasts Cloud s Spurring Useful Transition Phase for Enterprise Architecture
• The Open Group's Cloud Work Group Advances Understanding of Cloud-Use Benefits for Enterprises
• Exploring the Role and Impact of the Open Trusted Technology Forum to Ensure Secure IT Products in Global Supply Chains

Genuitec's MobiOne eases way for Windows development of iOS apps

Genuitec, LLC has revamped its MobiOne development tool to allow Windows operating system users to design and build App Store-ready iOS apps -- native apps for iPhone, iPad and iPod Touch -- without using a Mac. This means there is no longer an additional expense to buy a Mac machine or learn Objective-C to design apps that operate natively on iOS devices.

Previously, the Flower Mound, Tex. company's MobiOne supported a webapp-only model that allowed design of webapps that run on iOS devices. Now, users can design native apps or webapps with the same design files, using AppCenter, a cloud technology that Genuitec engineered, that allows app designers to test their native and webapps in a private Genuitec cloud. [Disclosure: Genuitec is a sponsor of BriefingsDirect podcasts.]

“By removing the barriers to entry for iOS app design and building, MobiOne is truly at the forefront of making mobile technologies accessible to the masses," said Wayne Parrott, vice president of product development. "If a Windows users has enough skill to design a PowerPoint slide, they can design and build iPhone and iPad apps with ease. Web developers with HTML5 and CSS3 skills will see even greater productivity.”

MobiOne is truly at the forefront of making mobile technologies accessible to the masses.

MobiOne is designed for web developers, marketing departments, business consultants, and anyone who wants to create and build App Store-ready iOS applications and webapps. MobiOne uses drag-and-drop functionality similar to stringing together a PowerPoint presentation, but has a powerful engine that allows users to build iOS apps or webapps from the same code base.

That engine is the AppCenter technology, which allows for easy testing of apps and webapps over the air using iOS 4+ or through iTunes. Testing links can be shared via email or SMS for multiple device testing and previews.

To learn more about the MobiOne Studio, go to http://www.genuitec.com/mobile/docs/highlights/current/. A 15-day free trial is available at: http://www.genuitec.com/mobile/download.html. After the free trial, the cost is $99 per license.

You may also be interested in:

• Embarcadero brings self-service app store model to enterprises to target PCs and their universe of software
• App stores - They're not just for consumers any more, as more enterprises adopt the model to support mobile work
• Partnerpedia enables enterprise app store to build better applications marketplace for ISVs, service providers, mobile business ecosystems
• Why HTML5 enables more businesses to deliver more apps to more mobile devices with greater ease

Travel giant TUI Group leverages virtualization management tools to drastically improve IT performance troubleshooting

Listen to the podcast. Find it on iTunes/iPod. Read a full transcript or download a copy. Sponsor: VMware.

Better managing virtualized IT workloads and private clouds is a top concern for IT leaders going into 2012. They may want to follow the lead of global travel and tourism giant TUI Group. The IT organization there, TUI InfoTec, has found ways to manage highly virtualized IT operations better, especially in mixed environments like hybrid clouds.

The critical need to better identify performance issues and outages prompted TUI InfoTec to find ways to cut time to troubleshooting, resulting in a 50 percent reduction in the time needed to identify the causes of such problems.

To learn more about better systems management in heterogeneous cloud environments and in virtualized environments, BriefingsDirect interviewed Christian Rudolph, Infrastructure Architect at TUI InfoTec in Hanover, Germany. The discussion is moderated by Dana Gardner, Principal Analyst at Interarbor Solutions. [Disclosure: VMware is a sponsor of BriefingsDirect podcasts.]

Here are some excerpts:

Rudolph: We're a very silo-based environment. We have dedicated network storage and a server team responsible for resolving issues in our infrastructure.

What we've seen in the past were a lot of problems in getting these people together. Everybody had different management tools from the different vendors and nobody had an over-all view about the infrastructure.

We're also starting to take a look at how, from a cost perspective, we can do the best for our customers.

We’re 60 percent in the Windows environment, and 20 percent in the UNIX environment, which is virtualized, and we're currently planning to go further -- to 80 percent virtualization in the total landscape. That's our current state, and we’ve driven more and more to a virtualized infrastructure for all the mission-critical systems.

Normally when we have performance issues, our responsibilities are not very clear -- this is a server problem, a network problem, an OS system problem, or this is only the end-user who has a problem. He feels that the application isn't fast enough. In the past, we had a large problem getting information all together.

This is where we evaluated VMware vCenter Operations to get an over-all overview about our infrastructure and to get a deep dive into our infrastructure to take a look at how can we solve problems faster and how this could help us in the normal process.

Now we have vCenter Operations on a single pane of glass that can roll down to the storage network and also the infrastructure CPU memory resources to have a clear overview of what could be the first root cause of an issue or performance for the end user. We've tried to figure out how can we bring it better together, and for us vCenter Operations, it’s a single pane of glass.

We currently use the vCenter Operations 1.0 Standard version, but we're in the beta program currently for 5.0. It's a new version, which comes out [in 2012] with vCenter Operations 5.0. These version give us the ability to do capacity planning and also performance analysis in one view so that we can adapt the things we have discovered in normal business hours for the system and also to do capacity planning for the future.

Gardner: Tell me a little bit about TUI, and TUI InfoTec.

Rudolph: TUI InfoTec is an external IT provider for the TUI AG Group. The TUI AG Group is a European leading company in travel and tourism. They're very large in Germany, in the UK, and also in other European countries. They’re not presently doing a lot of business in the US.

We started as an internal IT organization from TUI Germany, and moved in 2006 to an external service provider for the TUI AG and other companies. We're a joint venture company with Sonata Software Ltd., which holds about 50 percent of the company. We're responsible for all the business-critical IT for TUI AG group like the booking systems, the access planning system, and all the other systems related to the business of the TUI AG group.

If it comes to an outage of the IT systems we lose a lot of money. So we have to take care that everything is working and running in the infrastructure.

Gardner: How is your landscape for cloud?

Rudolph: We’re currently thinking about planning our private cloud for our development team. We're also starting to take a look at how, from a cost perspective, we can do the best for our customers. Maybe we can include peak trading for some of the systems. We have a great opening for producing catalogs for the customer, so that they're able to connect our internal cloud over to external clouds and have the hybrid clouds then in place.

Gardner: Okay. How has that beta with vCenter Operations 5.0 worked out? Are some of these features something that you think will be of value to you?

A good overview

Rudolph: We have two or three good cases there. This has really helped us in the normal business. We've been running with the beta for two months and what we've detected is that we have a good overview, because we have some multi-vCenter environments. We have, in total, three productive vCenters and we need to discover all of them. We had a problem, because we can't use Linked Mode for the vCenters. We had no central view for all the systems to get a performance overview of the system.

And there is a second step. We didn't have the capacity in the same view. So we weren't able to do capacity planning, until we manually got all the information from the different vCenters to have a consolidated planning view. For us, this is one of the most important things that we can do for planning in one place for all our vCenters and also know how many capacity hours are left for new machines. So we increased our time to deliver a virtual machine (VM).

Gardner: What has this better IT visibility in operations and remediation brought to you in technical and in business terms?

Rudolph: The process is very easy, because we've seen that we reduced the time until we can deliver our root cause for our known problem by nearly 50 percent. We reduced the time for doing that, and this is also the best case for our customers -- that we can deliver faster solution for a system problem.

The second thing we've seen is that we can see earlier information about how the system is feeling. Through vCenter Operations and through the health status in the vC Ops we can see how our end-users feel. We can detect some problems before they occur, and that’s the best use case we can ever have.

When we detect problems faster and can resolve them faster, they have faster usage of the product.

Gardner: How about looking toward the future? We talked a little bit about your use of improved operations, but will this become important when you move to more cloud, software-as-a-service (SaaS), and/or mobile types of activities. How important is this proactive ability in management as you innovate?

Rudolph: It's very important for us. We currently have the vCenter orchestration platform implemented, and we're starting to deliver to the end-user a service portal. Where they can request more-and-more VMs. When we didn’t have the products to monitor this system and we come to great trouble. How can we else go further, maybe to a hybrid cloud environment, if we can’t manage our private cloud like now with the vCenter Orchestrator and also with the vC Ops.

Gardner: Taking a step back and reviewing how things have gone, do you have any recommendations or advice for other companies that might be pursuing higher levels of virtualization and perhaps looking for similar reduction in meantime to solution for problems?

Two recommendations

Rudolph: I see two recommendations. Not many people know how powerful vCenter Orchestration is. This is one powerful tool as an automatic way for deployment, for maintaining, and also to do some other basic tasks in your virtual infrastructure. This is one important step for us to go to a higher virtualization ratio, because it can be delivered faster to our end-users.

The second thing is really to take a look at vCenter Operations and definitely to the new version that’s coming up. This really helps us to understand how my infrastructure is working. When I don’t know that, I may have problem with one of my disks and I/O and this reflects back to one VM especially. You have to know that, otherwise you don’t have recognition from the end-user that virtualization is really working and that you can bring mission-critical systems to the virtual infrastructure.

Listen to the podcast. Find it on iTunes/iPod. Read a full transcript or download a copy. Sponsor: VMware.

You may also be interested in:

• SAP Runs VMware to Provision Private Clouds that Support Complex and Critical Training Applications
• Case Study: How SEGA Europe Uses VMware to Standardize Cloud Environment for Globally Distributed Game Development
• Germany's Largest Travel Agency Starts a Virtual Journey to Get Branch Office IT Under Control
• Virtualized Desktops Spur Use of 'Bring You Own Device' in Schools, Allowing Always-On Access to Education Resources
  
• From VMworld, Cosmetics Giant Revlon Harnesses the Power of Private Cloud to Produce Impressive Savings and Cost Avoidance
• From VMworld, NYSE Euronext on Hybrid Cloud Vision and Strategy Behind the Capital Markets Community Platform Vertical Cloud

Overlapping criminal and state threats pose growing cyber security threat to global Internet commerce, says Open Group speaker

Listen to the podcast. Find it on iTunes/iPod. Read a full transcript or download a copy. Sponsor: The Open Group.

This special BriefingsDirect thought leadership interview comes in conjunction with The Open Group Conference this January in San Francisco.

The conference will focus on how IT and enterprise architecture support enterprise transformation. Speakers in conference events will also explore the latest in service oriented architecture (SOA), cloud computing, and security.

We’re here now with one of the main speakers, Joseph Menn, Cyber Security Correspondent for the Financial Times and author of Fatal System Error: The Hunt for the New Crime Lords Who are Bringing Down the Internet.

Joe has covered security since 1999 for both the Financial Times and then before that, for the Los Angeles Times. Fatal System Error is his third book, he also wrote All the Rave: The Rise and Fall of Shawn Fanning's Napster.

As a lead-in to his Open Group presentation, entitled "What You're Up Against: Mobsters, Nation-States, and Blurry Lines," Joe explores the current cyber-crime landscape, the underground cyber-gang movement, and the motive behind governments collaborating with organized crime in cyber space. The interview is moderated by Dana Gardner, Principal Analyst at Interarbor Solutions. [Disclosure: The Open Group is a sponsor of BriefingsDirect podcasts.]

Here are some excerpts:

Gardner: Have we entered a new period where just balancing risks and costs isn't a sufficient bulwark against burgeoning cyber crime?

Menn: Maybe you can make your enterprise a little trickier to get into than the other guy’s enterprise, but crime pays very, very well, and in the big picture, their ecosystem is better than ours. They do capitalism better than we do. They specialize to a great extent. They reinvest in R&D.

On our end, on the good guys’ side, it's hard if you're a chief information security officer (CISO) or a chief security officer (CSO) to convince the top brass to pay more. You don’t really know what's working and what isn't. You don’t know if you've really been had by something that we call advanced persistent threat (APT). Even the top security minds in the country can't be sure whether they’ve been had or not. So it's hard to know what to spend on.

More efficient

The other side doesn’t have that problem. They’re getting more efficient in the same way that they used to lead technical innovation. They're leading economic innovation. The freemium model is best evidenced by crimeware kits like ZeuS, where you can get versions that are pretty effective and will help you steal a bunch of money for free. Then if you like that, you have the add-on to pay extra for -- the latest and greatest that are sure to get through the antivirus systems.

Gardner: When you say "they," who you are really talking about?

Menn: They, the bad guys? It's largely Eastern European organized crime. In some countries, they can be caught. In other countries they can't be caught, and there really isn't any point in trying.

It's a geopolitical issue, which is something that is not widely understood, because in general, officials don’t talk about it. Working on my book, and in reporting for the newspapers, I've met really good cyber investigators for the Secret Service and the FBI, but I’ve yet to meet one that thinks he's going to get promoted for calling a press conference and announcing that they can’t catch anyone.

So the State Department, meanwhile, keeps hoping that the other side is going to turn a new leaf, but they’ve been hoping that for 10 or more years, and it hasn’t happened. So it's incumbent upon the rest of us to call a spade a spade here.

What's really going on is that Russian intelligence and, depending on who is in office at a given time, Ukrainian authorities, are knowingly protecting some of the worst and most effective cyber criminals on the planet.

Gardner: And what would be their motivation?

The same resources, human and technical, that are used to rob us blind are also being used in what is fairly called cyber war.

Menn: As a starting point, the level of garden-variety corruption over there is absolutely mind-blowing. More than 50 percent of Russian citizens responding to the survey say that they had paid a bribe to somebody in the past 12 months. But it's gone well beyond that.

The same resources, human and technical, that are used to rob us blind are also being used in what is fairly called cyber war. The same criminal networks that are after our bank accounts were, for example, used in denial-of-service (DOS) attacks on Georgia and Estonian websites belonging to government, major media, and Estonia banks.

It's the same guy, and it's a "look-the-other-way" thing. You can do whatever crime you want, and when we call upon you to serve Mother Russia, you will do so. And that has accelerated. Just in the past couple of weeks, with the disputed elections in Russia, you've seen mass DOS attacks against opposition websites, mainstream media websites, and live journals. It's a pretty handy tool to have at your disposal. I provide all the evidence that would be needed to convince the reasonable people in my book.

Gardner: In your book you use the terms "bringing down the Internet." Is this all really a threat to the integrity of the Internet?

Menn: Well integrity is the key word there. No, I don’t think anybody is about to stop us all from the privilege of watching skateboarding dogs on YouTube. What I mean by that is the higher trust in the Internet in the way it's come to be used, not the way it was designed, but the way it is used now for online banking, ecommerce, and for increasingly storing corporate -- and heaven help us, government secrets -- in the cloud. That is in very, very great trouble.

Not a prayer

I don’t think that now you can even trust transactions not to be monitored and pilfered. The latest, greatest versions of ZeuS gets past multi-factor authentication and are not detected by any antivirus that’s out there. So consumers don’t have a prayer, in the words of Art Coviello, CEO of RSA, and corporations aren’t doing much better.

So the way the Internet is being used now is in very, very grave trouble and not reliable. That’s what I mean by it. If they turned all the botnets in the world on a given target, that target is gone. For multiple root servers and DNS, they could do some serious damage. I don’t know if they could stop the whole thing, but you're right, they don’t want to kill the golden goose. I don’t see a motivation for that.

Gardner: If we look at organized crime in historical context, we found that there is a lot of innovation over the decades. Is that playing out on the Internet as well?

Menn: Sure. The mob does well in any place where there is a market for something, and there isn’t an effective regulatory framework that sustains it -- prohibition back in the day, prostitution, gambling, and that sort of thing.

The mob does well in any place where there is a market for something, and there isn’t an effective regulatory framework that sustains it.

... The Russian and Ukrainian gangs went to extortion as an early model, and ironically, some of the first websites that they extorted with the threat were the offshore gambling firms. They were cash rich, they had pretty weak infrastructure, and they were wary about going to the FBI. They started by attacking those sites in 2003-04 and then they moved on to more garden-variety companies. Some of them paid off and some said, "This is going to look little awkward in our SEC filings" and they didn’t pay off.

Once the cyber gang got big enough, sooner or later, they also wanted the protection of traditional organized crime, because those people had better connections inside the intelligence agencies and the police force and could get them protection. That's the way it worked. It was sort of an organic alliance, rather than "Let’s develop this promising area."

... That is what happens. Initially it was garden-variety payoffs and protection. Then, around 2007, with the attack on Estonia, these guys started proving their worth to the Kremlin, and others saw that with the attacks that ran through their system.

This has continued to evolve very rapidly. Now the DOS attacks are routinely used as the tool for political repression all around the world --Vietnam, Iran and everywhere you’ll see critics that are silenced from DOS attacks. In most cases, it's not the spy agencies or whoever themselves, but it's their contract agents. They just go to their friends in the similar gangs and say, "Hey do this." What's interesting is that they are both in this gray area now, both Russia and China, which we haven't talked about as much.

In China, hacking really started out as an expression of patriotism. Some of the biggest attacks, Code Red being one of them, were against targets in countries that were perceived to have slighted China or had run into some sort of territorial flap with China, and, lo and behold, they got hacked.

In the past several years, with this sort of patriotic hacking, the anti-defense establishment hacking in the West that we are reading a lot about finally, those same guys have gone off and decided to enrich themselves as well. There were actually disputes in some of the major Chinese hacking groups. Some people said it was unethical to just go after money, and some of these early groups split over that.

In Russia, it went the other way. It started out with just a bunch of greedy criminals, and then they said, "Hey -- we can do even better and be protected. You have better protection if you do some hacking for the motherland." In China, it's the other way. They started out hacking for the motherland, and then added, "Hey -- we can get rich while serving our country."

It is much, much worse than anybody realizes. The US counterintelligence a few weeks ago finally put out a report saying that Russia and China are deliberately stealing our intellectual property.

So they're both sort of in the same place, and unfortunately it makes it pretty close to impossible for law enforcement in [the U.S.] to do anything about it, because it gets into political protection. What you really need is White House-level dealing with this stuff. If President Obama is going to talk to his opposite numbers about Chinese currency, Russian support of something we don’t like, or oil policy, this has got to be right up there too -- or nothing is going to happen at all.

Gardner: What about the pure capitalism side, stealing intellectual property (IP) and taking over products in markets with the aid of these nefarious means? How big a deal is this now for enterprises and commercial organizations?

Menn: It is much, much worse than anybody realizes. The U.S. counterintelligence a few weeks ago finally put out a report saying that Russia and China are deliberately stealing our IP, the IP of our companies. That's an open secret. It's been happening for years. You're right. The man in the street doesn’t realize this, because companies aren’t used to fessing up. Therefore, there is little outrage and little pressure for retaliation or diplomatic engagement on these issues.

I'm cautiously optimistic that that is going to change a little bit. This year the Securities and Exchange Commission (SEC) gave very detailed guidance about when you have to disclose when you’ve been hacked. If there is a material impact to your company, you have to disclose it here and there, even if it's unknown.

Register for The Open Group Conference
Jan. 30 - Feb. 3 in San Francisco.

Gardner: So the old adage of shining light on this probably is in the best interest of everyone. Is the message then keeping this quiet isn’t necessarily the right way to go?

Menn: Not only is it not the right way to go, but it's safer to come out of the woods and fess up now. The stigma is almost gone. If you really blow the PR like Sony, then you're going to suffer some, but I haven’t heard a lot of people say, "Boy, Google is run by a bunch of stupid idiots. They got hacked by the Chinese."

It's the definition of an asymmetrical fight here. There is no company that's going to stand up against the might of the Chinese military, and nobody is going to fault them for getting nailed. Where we should fault them is for covering it up.

Not only is it not the right way to go, but it's safer to come out of the woods and fess up now. The stigma is almost gone.

I think you should give the American people some credit. They realize that you're not the bad guy, if you get nailed. As I said, nobody thinks that Google has a bunch of stupid engineers. It is somewhere between extremely difficult to impossible to ward off against "zero-days" and the dedicated teams working on social engineering, because the TCP/IP is fundamentally broken and it ain't your fault.

... [These threats] are an existential threat not only to your company, but to our country and to our way of life. It is that bad. One of the problems is that in the U.S., executives tend to think a quarter or two ahead. If your source code gets stolen, your blueprints get taken, nobody might know that for a few years, and heck, by then you're retired.

With the new SEC guidelines and some national plans in the U.K. and in the U.S., that’s not going to cut it anymore. Executives will be held accountable. This is some pretty drastic stuff. The things that you should be thinking about, if you’re in an IT-based business, include figuring out the absolutely critical crown jewel one, two, or three percent of your stuff, and keeping it off network machines.

Short-term price

Gardner: So we have to think differently, don’t we?

Menn: Basically, regular companies have to start thinking like banks, and banks have to start thinking like intelligence agencies. Everybody has to level up here.

Gardner: What do the intelligence agencies have to start thinking about?

Menn: The discussions that are going on now obviously include greatly increased monitoring, pushing responsibility for seeing suspicious stuff down to private enterprise, and obviously greater information sharing between private enterprise, and government officials.

But, there's some pretty outlandish stuff that’s getting kicked around, including looking the other way if you, as a company, sniff something out in another country and decide to take retaliatory action on your own.

But, there's some pretty outlandish stuff that’s getting kicked around, including looking the other way if you, as a company, sniff something out in another country and decide to take retaliatory action on your own. There’s some pretty sea-change stuff that’s going on.

Gardner: So that would be playing offense as well as defense?

Menn: In the Defense Authorization Act that just passed, for the first time, Congress officially blesses offensive cyber-warfare, which is something we’ve already been doing, just quietly.

We’re entering some pretty new areas here, and one of the things that’s going on is that the cyber warfare stuff, which is happening, is basically run by intelligence folks, rather by a bunch of lawyers worrying about collateral damage and the like, and there's almost no oversight because intelligence agencies in general get low oversight.

Gardner: Just quickly looking to the future, we have some major trends. We have an increased movement toward mobility, cloud, big data, social. How do these big shifts in IT impact this cyber security issue?

Menn: Well, there are some that are clearly dangerous, and there are some things that are a mixed bag. Certainly, the inroads of social networking into the workplace are bad from a security point of view. Perhaps worse is the consumerization of IT, the bring-your-own-device trend, which isn't going to go away. That’s bad, although there are obviously mitigating things you can do.

The cloud itself is a mixed bag. Certainly, in theory, it could be made more secure than what you have on premise. If you’re turning it over to the very best of the very best, they can do a lot more things than you can in terms of protecting it, particularly if you’re a smaller business.

If you look to the large-scale banks and people with health records and that sort of thing that really have to be ultra-secure, they're not going to do this yet, because the procedures are not really set up to their specs yet. That may likely come in the future. But, cloud security, in my opinion, is not there yet. So that’s a mixed blessing.

Radical steps

You need to think strategically about this, and that includes some pretty radical steps. There are those who say there are two types of companies out there -- those that have been hacked and those that don’t know that they’ve been hacked.

Everybody needs to take a look at this stuff beyond their immediate corporate needs and think about where we’re heading as a society. And to the extent that people are already expert in the stuff or can become expert in this stuff, they need to share that knowledge, and that will often mean, saying "Yes, we got hacked" publicly, but it also means educating those around them about the severity of the threat.

One of the reasons I wrote my book, and spent years doing it, is not because I felt that I could tell every senior executive what they needed to do. I wanted to educate a broader audience, because there are some pretty smart people, even in Washington, who have known about this for years and have been unable to do anything about it. We haven't really passed anything that's substantial in terms of legislation.

As a matter of political philosophy, I feel that if enough people on the street realize what's going on, then quite often leaders will get in front of them and at least attempt to do the right thing. Senior executives should be thinking about educating their customers, their peers, the general public, and Washington to make sure that the stuff that passes isn't as bad as it might otherwise be.

Listen to the podcast. Find it on iTunes/iPod. Read a full transcript or download a copy. Sponsor: The Open Group.

Register for The Open Group Conference
Jan. 30 - Feb. 3 in San Francisco.

You may also be interested in:

• Exploring Business-IT Alignment: A 20-Year Struggle Culminating in the Role and Impact of Business Architecture
• Enterprise Architects Increasingly Leverage Advanced TOGAF 9 for Innovation, Market Response, and Governance Benefits
• Open Group Cloud Panel Forecasts Cloud s Spurring Useful Transition Phase for Enterprise Architecture
• The Open Group's Cloud Work Group Advances Understanding of Cloud-Use Benefits for Enterprises
• Exploring the Role and Impact of the Open Trusted Technology Forum to Ensure Secure IT Products in Global Supply Chains