Thursday, February 13, 2014

HP Access Catalog smooths the way for streamlined deployment of mobile apps

HP today announced HP Access Catalog, a SaaS-delivered mobile app and content store that allows corporations to quickly and securely deliver resources across mobile and desktop devices to their employees anywhere.

IT organizations are facing pressure to deliver a marketplace experience to employees who expect access to content and apps from their device of choice. But non-business controlled exchanges and app stores lack enterprise security and control. Companies must also protect their apps from access by outsiders.

So the new catalog from HP, which can be branded as the business's own store, offers organizations a secure, private “app store” for employees to browse, search and download mobile applications and digital content onto their devices, including mobile and tablets, as well as desktops. The catalog supports Android and iOS platforms, which make up close to 94 percent of the mobile-device market share in the third quarter of 2013.

Earlier this week HP launched the HP Vertica Marketplace, a hub for developers, partners and customers to create and share extensions, enhancements, and solutions that integrate with the HP Vertica Analytics Platform. Both the Vertica Marketplace and HP Access Catalog are powered by technology developed by Palm, which HP acquired in 2010.

Delivered via native mobile clients and a web interface, the HP Access Catalog is a pure software-as-a-service (SaaS) offering that helps organizations reduce the cost and complexity of managing applications on company-issued and bring-your-own-device (BYOD) mobile devices, said Tim Rochte, Director of Product Management at HP Software Web Services. [Disclosure: HP is a sponsor of BriefingsDirect podcasts.]

Streamlined deployment

Through the catalog’s native identity management system or seamless integration with enterprise identity systems, IT organizations ensure that users can find and download the right applications for their role, he said. Those organizations have 100% control over their content and apps.

In addition, the catalog allows IT organizations to drive updates to users to ensure they have the most current applications and data, increasing their mobile productivity and effectiveness without compromising security. Via a CDN, the delivery speed and global reach of the apps and content -- even large video objects and streams to remote branches -- is assured, something a home-grown app store may not be able to do, said Rochte.
As organizations embrace mobility, they need a simple, secure and reliable mechanism to manage the delivery of apps to their employees.

The Access Catalog uses HTML5 and single-sign-on authentication and authorization capabilities with SAML 2.0 integration. It coexists with "public" stores like iTunes and Google Play.

Hosted in HP’s PCI-compliant data center, the access catalog also is offered as an integrated component of the HP Anywhere enterprise mobility platform, enabling customers to manage all their mobile apps

While the HP Access Catalog is currently used for free content, an e-commerce element that allows selling and/or charge backs is in the offing, said Rochte. As applications developers go mobile-first, the store may become a primary way to distribute, track and manage all corporate applications. Or at least it will help manage the expected huge growth in mobile apps in businesses.

You could even say the Access Catalog marketplace model is the new intranet, for those of you that recall intranets.

HP Access Catalog will be available worldwide from HP and its channel partners in March. Pricing will be based on a simple per-user per month or annual subscription. The means the more content and apps per employee, the better the cost ratio -- and productivity.

Additional information is available at go.pronq.com/HP-Access-Catalog.

You may also be interested in:


Monday, February 10, 2014

HP adds new value to Vertica data analytics platform with community marketplace

HP today launched the HP Vertica Marketplace, a hub for developers, partners and customers to create and share extensions, enhancements, and solutions that integrate with the HP Vertica Analytics Platform.

These add-ons and solutions include connectors and third-party extensions, business intelligence (BI)  tools, exact transform load (ETL) and data transformation products, connectors and tools for HP HAVEn big-data analytics platform, as well as industry and other original equipment manufacturer (OEM) solutions.

In addition, the HP Vertica Marketplace includes the latest solutions from HP Vertica’s innovations incubation program, allowing users to create cutting-edge big-data applications. [HP is a sponsor of BriefingsDirect podcasts.]

“Our rapidly growing community of customers, partners and developers are building vertical and horizontal solutions on, and creating new add-on capabilities to, Vertica every day,” said Colin Mahony, vice president and general manager, Vertica, HP. “The HP Marketplace provides a place where our community can share and market their capabilities to help other organizations and developers fuel further innovation.”

New capabilities

With the HP Vertica Marketplace, developers and companies can:
  • Gain value from 100 percent of information -- spanning structured, semi-structured and unstructured data -- through connectors and extensions that store, manage and analyze big data. This includes integration with the Hadoop Distributed Filesystem (HDFS) and the HP Autonomy IDOL platform.
  • Get business insights from big data with flexible plug-ins and extensions to integrate and visualize users’ data, including BI and data-visualization tools and products.
  • Capitalize on shared intelligence by engaging developers via a social interface that allows users to pose questions, interact with subject matter experts, and review previous discussions, as all questions are cataloged and searchable.
    HP Vertica Marketplace members also will gain access to the latest innovations from HP Vertica through its incubation program.
HP Vertica Marketplace members also will gain access to the latest innovations from HP Vertica through its incubation program. These new technologies and solutions will be available for developers to evaluate and provide feedback, helping guide future development.

The current marketplace is geared toward free and open community sharing of extensions, connectors and tools, but I think this could easily blossom into a commerce hub for analytics apps and/or data enhancements. We'll have to keep an eye out for that. I also think some sort of vertical industry segmentation of analytics capabilities is in the offing. That would allow for ecosystem-defined solutions to emerge, either as open contributions or for-pay offerings. In any event, it's now quite a powerful destination for developers to showcase their big data analytics endeavors.


New innovations in the market include:
  • HP Vertica Distributed R, which helps data scientists overcome the scalability and performance limitations of R programming language and tackle problems not previously solvable by accelerating the analysis of large data sets by running R computations on multiple nodes.
  • HP Vertica Pulse, which helps organizations leverage an in-database sentiment analysis tool that scores short data posts, including social data, such as Twitter feeds or product reviews, to gauge the most popular topics of interest, analyze how sentiment changes over time, and identify advocates and detractors.
  • HP Vertica Place, which stores and analyzes geospatial data in real time, including locations, networks, and regions. This analytics pack provides Open Geospatial Consortium (OGC) standards–based functionality and integrates with third-party applications.
The HP Vertica Analytics Platform is a key component of the HP HAVEn big-data analytics platform, which enables HP customers and partners to create next-generation applications and solutions that accelerate the monetization of big data. HP HAVEn combines proven technologies including HP Autonomy IDOL, HP Vertica Analytics Platform, HP ArcSight Enterprise Security Manager and HP ArcSight Logger, as well as key industry offerings such as Hadoop.

HP Vertica Community Marketplace is currently availble at www.vertica.com/marketplace and can be accessed through the "Community" tab on www.vertica.com.

You may also be interested in:

Tuesday, February 4, 2014

Network virtualization eases developer and operations snafus in the mobile and cloud era

Listen to the podcast. Find it on iTunes. Read a full transcript or download a copy.

As developers are pressured to produce mobile and distributed cloud apps ever faster and with more network unknowns, the older methods of software quality control can lack sufficient predictability.

And as Agile development means faster iterations and a constant stream of updates, newer means of automated testing of the apps in near-production realism prove increasingly valuable.

Fortunately, a tag-team of service and network virtualization for testing has emerged just as the mobile and cloud era requires unprecedented focus on DevOps benefits and rapid quality assurance.

BriefingsDirect had an opportunity to learn first-hand how Shunra Software and HP have joined forces to extend the capabilities of service virtualization for testing at the recent HP Discover 2013 Conference in Barcelona.

Learn here how Shunra Software uses service virtualization to help its developer users improve the distribution, creation, and lifecycle of software applications from Todd DeCapua, Vice President of Channel Operations and Services at Shunra Software, based in Philadelphia. The discussion is moderated by me, Dana Gardner, Principal Analyst at Interarbor Solutions.

Here are some excerpts:
Gardner: There are a lot of trends affecting software developers. They have mobile on their minds. They have time constraints issues. They have to be faster, better, and cheaper along the apps lifecycle way. What among the trends is most important for developers?

DeCapua
DeCapua: One of the biggest ones -- especially around innovation and thinking about results, specifically business results -- is Agile. Agile development is something that, fortunately, we've had an opportunity to work with quite a bit. Our capabilities are all structured around not only what you talked about with cloud and mobile, but we look at things like the speed, the quality, and ultimately the value to the customers.

We’re really focusing on these business results, which sometimes get lost, but I try to always go back to them. We need to focus on what's important to the business, what's important to the customer, and then maybe what's important to IT. How does all that circle around to value?

Gardner: With mobile we have many more networks, and people are grasping at how to attain quality before actually getting into production. How does service virtualization come to bear on that?

Distributed devices

DeCapua: As you look at almost every organization today, something is distributed. Their customers might be on mobile devices out in the real world, and so are distributed. They might be working remotely from home. They might have a distribution center or a truck that has a mobile device on it.

There are all these different pieces. You’re right. Network is a significant part that unfortunately many organizations have failed to notice and failed to consider, as they do any type of testing.

Network virtualization gives you that capability. Where service virtualization comes into play is looking at things like speed and quality. What if the services are not available? Service virtualization allows you to then make them available to your developers.

In the early stage, where Shunra has been able to really play a huge difference in these organizations is by bringing network virtualization in with service virtualization. We’re able to recreate their production environments with 100 percent scale -- all prior to production.

When we think about the value to the business, now you’re able to deliver the product working. So, it is about the speed to market, quality of product, and ultimately value to your customer and to your business.

Gardner: And another constituency that we should keep in mind are those all-important operators. They’re also dealing with a lot of moving parts these days -- transformation, modernization, and picking and choosing different ways to host their data centers. How do they fit into this and how does service virtualization cut across that continuum to improve the lives of operators?
Service virtualization and network virtualization can benefit them is by being able to recreate these scenarios.

DeCapua: You’re right, because as the delivery has sped up through things like Agile, it's your operations team that is sitting there and ultimately has to be the owners of these applications. Service virtualization and network virtualization can benefit them by being able to recreate these in-production scenarios.

Unfortunately, there are still some reactive actions required in production today, so you’re going to have a production incident. But, you can now understand the network in production, capture those conditions, and recreate that in the test environment. You can also do the same for the services.

We now have the ability to quickly and easily recreate a production incident in a prior-to-production environment. The operations team can be part of the team that's fixing it, because again, the ultimate question from CIOs is, “How can you make sure this never happens again?”

We now have the way to quickly and confidently recreate incidents and fix it the first time, not having to change code in production, on the fly. That is one of the scariest moments in any of the times when I've been at the customer site or when I was an employee and had to watch that happen.

Agile iterations

Gardner: As you mentioned earlier, with Agile we’re seeing many more iterations on applications as they need to be rapidly improved or changed. How does service and network virtualization aid in being able to produce many more iterations of an application, but still maintain that high quality?

DeCapua: One of our customers actually told us that -- prior to leveraging network virtualization with service virtualization -- he was doing 80 percent of his testing in-production, simply because he knew the shortcomings, and he needed to test it, but he had no way of re-creating it. Now, let's think about Agile. Let's think about how we shift and get the proven enterprise tools in the developer’s hands sooner, more often, so that we can drive quality early in the process.

That's where these two components play a critical role. As you look at it more specifically and go just a hair deeper, how in integrated environments can you provide continuous development and continuous deployment? And with all that automated testing that you’re already doing, how you can incorporate performance into that? Or, as I call it, how do you “build performance in” from the beginning?

As a business person, a developer, a business analyst, or a Scrum Master, how is it that you’re building performance into your user scenarios today? How is it that you’re setting them up for understanding how that feature or function is going to perform? Let's think about it as we’re creating, not once we get two or three sprints into use and we have our hardening sprint, where we’re going to run our performance scenario. Let's do it early, and let's do it often.
Get the proven enterprise tools in the developer’s hands sooner, more often, so that we can drive quality early in the process.

Gardner: If we’re really lucky, we can control the world and the environment that we live in, but more often than not these days, we’re dealing with third-party application programming interfaces (APIs). We’re dealing with outside web services. We have organizational boundaries that are being crossed, but things are happening across that boundary that we can't control.

So, is there a benefit here, too, when we’re dealing with composite applications, where elements of that mixed service character are not available for your insight, but that you need to be able to anticipate and then react quickly should a change occur?

DeCapua: I can't agree with you more. It’s funny, I am kind of laughing here, Dana, because this morning I was riding the metro in Barcelona and before I got to the stop here, I looked down to my phone, because I was expecting a critical email to come in. Lo and behold, my phone pops up a message and says, “We’re sorry, service is unavailable.”

I could clearly see that I had one out of five bars on the Orange network, and I was on the EDGE network. So, it was about a 2.5G connection. I should still have been able to get data, but my phone simply popped up and said, “Sorry, cannot retrieve email because of a poor data connection.”

I started thinking about it some more, and as I was engaging with other folks today at the show, I asked them why is it that the developer of the application found it necessary to alert me three times in a row that it couldn’t get my email because of a poor data connection? Why didn’t it just not wait 30 seconds, 60 seconds, 90 seconds until it did, and then have it reach out and query it again and pull the data down?

Changing conditions

This is just one very simple example that I had this morning. And you’re right, there are constantly changing conditions in the world. Bandwidth, latency, packet loss and jitter are those conditions that we’re all exposed to every day. If you’re in a BMW driving down the road at 100 miles per hour, that car is now a mobile phone or a mobile device on wheels, constantly in communication. Or if you’re riding the metro or the tube and you have your mobile device on your hands, there are constantly changing conditions.

Network virtualization and service virtualization give you the ability to recreate those scenarios so that you can build that type of resiliency into your applications and, ultimately, the customers have the experience that you want them to have.

Gardner: Todd, tell us about so-called application-performance engineering solutions?

DeCapua: So, application performance engineering (APE) is something that was created within the industry over a number of years. It's meant to be a methodology and an approach. Shunra plays a role in that.

A lot of people had thought about it as testing. Then people thought about it as performance testing. At the next level, many of us in the industry have defined it is application engineering. It’s a lot more than just that, because you need to dive behind the application and understand the in’s and the out’s. How does everything tie together?
Understanding APE will help you to reduce those types of production incidents.

You’d mentioned some of the composite applications and the complexities there -- and I’m including the endpoints or the devices or mobile devices connecting through it. Now, you introduce cloud into the equation, and it gets 10 times worse.

Thinking about APE, it's more of an art and a skill. There is a science behind it. However, having that APE background knowledge and experience gives you the ability to go into these composite apps, go into these cloud deployments, and leverage the right tools and the right process to be able to quickly understand and optimize the solutions.

Gardner: Why aren’t the older scripting and test-bed approaches to quality control good enough? Why can't we keep doing what we've been doing?

DeCapua: In the United States recently, October 1 of 2013, there was a large healthcare system being rolled out across the country. Unfortunately, they used the old testing methodologies and have had some significant challenges. HP and Shunra were both engaged on October 2 to assist.

Understanding APE will help you to reduce those types of production incidents. All due to inaccurate results in the test environment, using the current methodologies, about 50 percent of our customers come to us in a crisis mode. They say, “We just had this issue, I know that you told us this is going to happen, but we really need your help now.”

They’re also thinking about how to shift and how to build performance in all these components -- just have it built in, have it be automatic, and get the results that are accurate.

Coming together

Gardner: Of course HP has service virtualization, you have network virtualization. How are they coming together? Explain the relationship and how Shunra and HP work together?

DeCapua: To many people's surprise, this relationship is more than a decade old. Shunra’s network-virtualization capability has, for a long time, been built in to HP LoadRunner, also is now being built into HP Performance Center.

There are other capabilities that we have that are built into their Unified Functional Testing (UFT) products. In addition, within service virtualization, we’re now building that product into there. It’s one that, when you think about anything that has some sort of distribution or network involved, network virtualization needs to come into play.

Some people have a hard time initially understanding the service virtualization need, but a very simple example I often use is an organization like a bank. They’ll have a credit check as you’re applying for a loan. That credit check is not going to be a service that the bank creates. They’re going to outsource it to one of the many credit-check services. There is a network involved there.

In your test environment, you need to recreate that and take that into consideration as a part of your end-to-end testing, whether it's functional, performance, or load. It doesn’t matter.
In your test environment, you need to recreate that and take that into consideration as a part of your end-to-end testing, whether it's functional, performance, or load.

As we think about Shunra, network virtualization and the very tight partnership that we've had with HP for service virtualization, as well as their ability to virtualize the users, it's been an OEM relationship. Our R and D teams sit together as they’re doing the development so that this is a seamless product for the HP customer to be able to get the benefit and value for their business and for their customers.

Gardner: Let's talk a little bit about what you get when you do this right. It seems to me the obvious point is getting to the problem sooner, before you’re in production, extending across network variables, across other composite application-type variables. But, I’m going to guess that there are some other benefits that we haven't yet hit on.

So, when you've set up you're testing, when you have virtualization as your tool, what happens in terms of paybacks?

DeCapua: There are many benefits there, which we have already covered. There are dozens more that we could get into. One that I would highlight, being able to pull all the different pieces that we've been talking about, are shorter release times.

TechValidate did a survey in February of 2013. The findings were very compelling in that they found a global bank was able to speed up their deployment or application delivery by 30 to 40 percent. What does that mean for that organization as compared to their competitor? If you can get to market 30 to 40 percent faster, it means millions or billions of dollars over time. Talk about numbers of customers or brands, it's a significant play there.

Rapid deployment

There are other things like rapid deployment. As we think about Agile and mobile, it's all about how fast we get this feature function out, leveraging service virtualization in a greater way, and reducing associated costs.

In the example that I shared, the customer was able to virtualize the users, virtualize the network, and virtualize the services. Prior to that, he would never have been able to justify the cost of rebuilding a production environment for test. Through user virtualization, network virtualization, and service virtualization, he was able to get to 100 percent at a fraction of the cost.

Time and time again we mention automation. This is a key piece of how you can test early, test often, ultimately driving these accurate results and getting to the automated optimization recommendations.

Gardner: What comes next in terms of software productivity? What should organizations be thinking in terms of vision?

Slow down

DeCapua: I see Agile, mobile, and cloud. There are some significant risks out in the marketplace today. As organizations look to leverage these capabilities to benefit their business and the customers, maybe they need to just slow down for a moment and not create this huge strategy, but go after “How can I increase my revenue stream by 20 percent in the next 90 days?” Another one that I've had great success with is, “What is that highest visibility, highest risk project that you have in your organization today?”

As I look at The Wall Street Journal, and I read the headlines everyday, it's scary. But, what's coming in the future? We can all look into our crystal balls and say that this is what it is. Why not focus on one or two small things of what we have now, and think about how we’re mitigating our risk of  looking at larger organizations that are making commitments to migrate critical applications into the cloud?

You’re biting off a fairly significant risk, which that there isn’t a lot there to catch you when you do it wrong, and, quite frankly, nearly everybody is doing it wrong. What if we start small and find a way to leverage some of these new capabilities? We can actually do it right, and then start to realize some of the benefits from cloud, mobile, and other channels that your organization is looking to.

Gardner: The role of software keeps increasing in many organizations. It's becoming the business itself and, as a fundamental part of the business, requires lots of tender love and care.
The more that we can think about that and tune ourselves and make ourselves lean and focused on delivering better quality products, we’re going to be in the winning circle more often.

DeCapua: You got it. The only other bit that I would add on to that is looking at the World Quality Report that was presented this morning by HP, Capgemini, and Sogeti, they highlighted that there is an increased spend from the IT budget, and a rather significant increase in spend from last year in testing.

It’s exactly what you’re saying. Organizations didn’t enter the market thinking of themselves as a software house. But time and time again, we’re seeing how people who treat what they do as a software house ultimately is improving not only life for their internal customers, but also their external customers.

So I think you’re right. The more that we can think about that and tune ourselves and make ourselves lean and focused on delivering better quality software products, we’re going to be in the winning circle more often.
Listen to the podcast. Find it on iTunes. Read a full transcript or download a copy.
Sponsor: HP.

You may also be interested in:

Wednesday, January 29, 2014

Healthcare among most opportunistic use cases for boundaryless information flow improvement

Listen to the podcast. Find it on iTunes. Read a full transcript or download a copy. Sponsor: The Open Group.

Healthcare, like no other sector of the economy, exemplifies the challenges and the opportunity for improving how the various participants in a complex ecosystem interact.

The Open Group, at its next North American conference on Feb. 3, has made improved information flow across so-called boundaryless organizations the theme of its gathering of IT leaders, enterprise architects, and standards developers and implementers.

And so the next BriefingsDirect discussion explores what it takes to bring rigorous interactions, process efficiency, and governance to data and workflows that must extend across many healthcare participants with speed and dependability.

Learn now how improved cross-organization collaboration plays a huge part in helping to make healthcare more responsive, effective, safe, and cost-efficient. And also become acquainted with what The Open Group’s new Healthcare Industry Forum is doing to improve the situation.

The panel of experts consists of Larry Schmidt, the Chief Technologist at HP for the America’s Health and Life Sciences Industries, as well as the Chairman of The Open Group Healthcare Industry Forum, and Eric Stephens, an Oracle Enterprise Architect. The moderator is Dana Gardner, Principal Analyst at Interarbor Solutions. [Disclosure: HP is a sponsor of BriefingsDirect podcasts. The views of the panelists are theirs alone and not necessarily those of their employers.]

Here are some excerpts:
Gardner: Why is healthcare such a tough nut to crack when it comes to this information flow? Is there something unique about healthcare that we don't necessarily find in other vertical industries?

Schmidt: We’ve progressed in healthcare from a delivery model that was more based on acute care -- that is, I get sick, I go to the doctor -- to more of a managed care-type capability with the healthcare delivery, where a doctor at times is watching and trying to coach you. Now, we’ve gotten to where the individual is in charge of their own healthcare.

A lot of fragmentation

With that, the ecosystem around healthcare has not had the opportunity to focus the overall interactions based on the individual. So we see an awful lot of fragmentation occurring. There are many great standards across the powers that exist within the ecosystem, but if you take the individual and place that individual in the center of this universe, the whole information model changes.

Then, of course, there are other things, such as technology advances, personal biometric devices, and things like that that come into play and allow us to be much more effective with information that can be captured for healthcare. As a result, it’s the change with the focus on the individual that is allowing us the opportunity to redefine how information should flow across the healthcare ecosystem.

The scenario of the individual being more in charge of their healthcare -- care of their health would be a better way to think of this -- is a way to see both improvements in the information flow  as well as making improvements in the overall cost of healthcare going forward.
Schmidt
Because the ecosystem had pretty much been focused around the doctor's visit, or the doctor’s work with an individual, as opposed to the individual’s work with the doctor, we see tremendous opportunity in making advancements in the communications models that can occur across healthcare.

Gardner: Larry, is this specific to the United States or North America, is this global in nature, or is it very much a mixed bag, market to market as to how the challenges have mounted?

Schmidt: I think in any country, across the world, the individual being the focus of the ecosystem goes across the boundaries of countries. Of course, The Open Group is responsible and is a worldwide standards body. As a result of that, it's a great match for us to be able to focus the healthcare ecosystem to the individual and use the capabilities of The Open Group to be able to make advances in the communication models across all countries around healthcare.

Gardner: Eric, thinking about this from a technological point of view, as an enterprise architect, we’re now dealing with this hub and spoke with the patient at the middle. A lot of this does have to do with information, data, and workflow, but we’ve dealt with these things before in many instances in the enterprise and in IT.

Is there anything particular about the technology that is difficult for healthcare, or is this really more a function of the healthcare verticals and the technology is really ready to step up to the plate?

Information transparency

Stephens: Well, Dana, the technology is there and it is ready to step up to the plate. I’ll start with transparency of the information. Let’s pick a favorite poster child, Amazon. In terms of the detail that's available on my account. I can look at past orders. I can look up and see the cost of services, I can track activity that's taking place, both from a purchase and a return standpoint. That level of visibility that you’re alluding to exists. The technology is there, and it’s a matter of applying it.

Stephens
As to why it's not being applied in a rapid fashion in the healthcare industry, we could surmise a number of reasons. One of them is potentially around the cacophony of standards that exist and the lack of a “Rosetta Stone” that links those standards together to maximum interoperability.

The other challenge that exists is simply the focus in healthcare around the healthcare technology that’s being used, the surgical instruments, the diagnostic tools, and such. There is focus and great innovation there, but when it comes to the plumbing of IT, oftentimes that will suffer.

Gardner: So we have some hurdles on a number of fronts, but not necessarily the technology itself. This is a perfect case study for this concept of the boundaryless information flow, which is really the main theme of The Open Group Conference coming up on February 3. [Register for the event here.]

Back to you, Larry, on this boundaryless issue. There are standards in place in other industries that help foster a supply-chain ecosystem or a community of partners that work together.

Is that what The Open Group is seeking? Are they going to take what they’ve done in other industries for standardization and apply it to healthcare, or do you perhaps need to start from scratch? Is this such a unique challenge that you can't simply retrofit other standardization activities? How do you approach something like healthcare from a standards perspective?
I think it's a great term to reflect the vast number of stakeholders that would exist across the healthcare ecosystem.

Schmidt: The first thing we have to do is gain an appreciation for the stakeholders that interact. We’re using the term “ecosystem” here. I think it's a great term to reflect the vast number of stakeholders that would exist across the healthcare ecosystem. Anywhere from the patient, to the doctor, to payment organization for paying claims, the life sciences organizations, for pharmaceuticals, and things like that, there are so many places that stakeholders can interact seamlessly.

So it’s being able to use The Open Group’s assets to first understand what the ecosystem can be, and then secondly, use The Open Group’s capabilities around things like security, TOGAF from an architecture methodology, enablement and so on. Those assets are things that we can leverage to allow us to be able to use the tools of The Open Group to make advances within the healthcare industry.

It’s an amazing challenge, but you have to take it one step at a time, and the first step is going to be that definition of the ecosystem.

Gardner: I suppose there’s no better place to go for teasing out what the issues are and what the right prioritization should be than to go to the actual participants. The Open Group did just that last summer in Philadelphia at their earlier North American conference. They had some 60 individuals representing primary stakeholders in healthcare in the same room and they conducted some surveys.

Larry, maybe you can provide us an overview of what they found and how that’s been a guide to how to proceed?

Participant survey

Schmidt: What we wanted to do was present the concept of boundaryless information flow across the healthcare ecosystem. So we surveyed the participants that were part of the conference itself. One of the questions we asked was about the healthcare quality of data, as well as the efficiency and the effectiveness of data. Specifically, the polling questions, were designed to gauge the state of healthcare data quality and effective information flow.

We understood that 86 percent of those participants felt very uncomfortable with the quality of healthcare information flows, and 91 percent of the participants felt very uncomfortable with the efficiency of healthcare information flows.

In the discussion in Philadelphia, we talked about why information isn’t flowing much more easily and freely within this ecosystem. We discovered that a lot of the standards that currently exist within the ecosystem are very much tower-oriented. That is, they only handle a portion of the ecosystem, and the interoperability across those standards is an area that needs to be focused on.

But we do think that, because the individual should be placed into the center of the ecosystem, there's new ground that will come into play. Our Philadelphia participants actually confirmed that, as we were working through our workshop. That was one of the big, big findings that we had in the Philadelphia conference.
We understood that 86 percent of those participants felt very uncomfortable with the quality of healthcare information flows.

Gardner: Just so our audience understands, the resulting work that’s been going on for months now will culminate with the Healthcare Industry Forum being officially announced and open for business,, beginning with the San Francisco Conference. [Register for the event here.]

Tell us a little about how the mission statement for the Healthcare Industry Forum was influenced by your survey. Is there other information, perhaps a white paper or other collateral out there, that people can look to, to either learn more about this or maybe even take part in it?

Schmidt: We presented first a vision statement around boundaryless information flow. I’ll go ahead and just offer that to the team here. Boundaryless information flow of healthcare data is enabled throughout a complete healthcare ecosystem to standardization of both vocabulary and messaging that is understood by all participants within the system. This results in higher quality outcomes, streamlined business processes, reduction of fraud, and innovation enablement.

When we presented that in the conference, there was big consensus among the participants around that statement and buy in to the idea that we want that as our vision for a Healthcare Forum to actually occur.

Since then, of course, we’ve published this white paper that is the findings of the Philadelphia Conference. We’re working towards the production of a treatise, which is really the study of the problem domain that we believe we can be successful in. We also can make a major impact around this individual communication flow, enabling individuals to be in charge of more of their healthcare.

Our mission will be to provide the means to enable boundaryless information flow across the ecosystem. What we’re trying to do is make sure that we work in concert with other standards bodies to recognize the great work that’s happening around this tower concept that we believe is a boundary within the ecosystem.

Additional standards

Hopefully, we’ll get to a point where we’re able to collaborate, both with those standards bodies, as well as work within our own means to come up with additional standards that allows us to make this communication flow seamless or boundaryless.

Gardner: Eric Stephens, back to you with the enterprise architect questions. Of course, it’s important to solve the Tower of Babel issues around taxonomy, definitions, and vocabulary, but I suppose there is also a methodology issue.

Frameworks have worked quite well in enterprise architecture and in other verticals and in the IT organizations and enterprises. Is there something from your vantage point as an enterprise architect that needs to be included in this vision, perhaps looking to the next steps after you’ve gotten some of the taxonomy and definitions worked out?

Stephens: Dana, in terms of working through the taxonomies and such, as an enterprise architect, I view it as part of a larger activity around going through a process, like the TOGAF methodology, it’s architecture development methodology.
In the healthcare landscape, and in other industries, there are a lot of players coming to the table and need to interact.

By doing so, using a tailored version of that, we’ll get to that taxonomy definition and the alignment of standards and such. But there's also the addressing alignment and business processes and other application components that comes into play. That’s going to drive us towards improving the viscosity of the information, that's moving both within an enterprise and outside of the enterprise.

In the healthcare landscape, and in other industries, there are a lot of players coming to the table and need to interact, especially if you are talking about a complex episode of care. You may have two, three, or four different organizations in play. You have labs, the doctors, specialized centers, and such, and all that requires information flow.

Coming back to the methodology, I think it’s bringing to bear an architecture methodology like provided in TOGAF. It’s going to aid individuals in getting a broad picture, and also a detailed picture, of what needs to be done in order to achieve this goal of boundaryless information flow.

Drive standardization

One of the things that we can do in the Forum is start to drive standardization, so that we have the data and devices working together easily, and it provides the necessary medical professionals the information they need, so they can make more timely decisions. It’s giving the right information, to the right decision maker, at the right time. That, in turn, drives better health outcomes, and it's going to, we hope, drive down the overall cost profile of healthcare, specifically here in the United States.

Gardner: Getting back to the conference, I understand that the Healthcare Industry Forum is going to be announced. There is going to be a charter, a steering committee program, definitions, and treatise in the works. So there will be quite a bit kicking off. I would like to hear from you two, Larry and Eric, what you will specifically be presenting at the conference in San Francisco in just a matter of a week or two. Larry, what’s on the agenda for your presentations at the conference? [Register for the event here.]

Schmidt: Actually, Eric and I are doing a joint presentation and we’re going to talk about some of the challenges that we think we can see is ahead of us as a result of trying to enable our vision around boundaryless information flow, specifically around healthcare.
As an enterprise architect, I look at things in terms of the business, the application, information, technology, and architecture.

The culture of being able to produce standards in an industry like this is going to be a major challenge to us. There is a lot of individualization that occurs across this industry. So having people come together and recognize that there are going to be different views, different points of views, and coming into more of a consensus on how information should flow, specifically in healthcare. Although I think any of the forums go through this cultural change.

We’re going to talk about that at the beginning in the conference as a part of how we’re planning to address those challenges as part of the Industry Forum itself.  Then, other meetings will allow us to continue with some of the work that we have been doing around a treatise and other actions that will help us get started down the path of understating the ecosystem and so on.

Those are the things that we’ll be addressing at this specific conference.

Stephens: As an enterprise architect, I look at things in terms of the business, the application, information, technology, and architecture. When we talk about boundaryless information flow, my remarks and contributions are focused around the information architecture and specifically around an ecosystem of an information architecture at a generic level, but also the need and importance of integration. I will perhaps touch a little bit on the standards to integrate that with Larry’s thoughts.

Soliciting opinions

Schmidt: Dana, I just wanted to add the other work that we’ll be doing there at the conference. We’ve invited some of the healthcare organizations in that area of the country, San Francisco and so on, to come in on Tuesday. We plan to present the findings of the paper and the work that we did in the Philadelphia Conference, and get opinions in refining both the observations, as well as some of the direction that we plan to take with the Healthcare Forum.

Obviously we’ve shared here some of the thoughts of where we believe we’re moving with the Healthcare Forum, but as the Forum continues to form, some of the direction of it will morph based on the participants, and based on some of the things that we see happening with the industry.

So, it’s a really exciting time and I’m actually very much looking forward to presenting the findings of the Philadelphia Conference, getting, as I said, the next set of feedback, and starting the discussion as to how we can make change going toward that vision of boundaryless information flow.
We’re actually able to see a better profile of what the individual is doing throughout their life and throughout their days.

Gardner: I should also point out that it’s not too late for our listeners and readers to participate themselves in this conference. If you’re in the San Francisco area, you’re able to get there and partake, but there are also going to be online activities. There will be some of the presentations delivered online and there will be Twitter feeds.

So if you can't make it to San Francisco on February 3, be aware that The Open Group Conference will be available in several different ways online. Then, there will be materials available after the fact to access on-demand. Of course, if you’re interested in taking more activity under your wing with the Forum itself, there will be information on The Open Group website as to how to get involved.

Before we sign off, I want to get a sense of what the stakes are here. It seems to me that if you do this well and if you do this correctly, you get alignment across these different participants -- the patient being at the hub of the wheel of the ecosystem. There’s a tremendous opportunity here for improvement, not only in patient care and outcomes, but costs, efficiency, and process innovation.

So first to you Larry. If we do this right, what can we expect?

Schmidt: There are several things to expect. Number one, I believe that the overall health of the population will improve, because individuals are more knowledgeable about their individualized healthcare and doctors have the necessary information, based on observations in place, as opposed to observations or, again, through discussion and/or interview of the patient.

We’re actually able to see a better profile of what the individual is doing throughout their life and throughout their days. That can provide doctors the opportunity to make better diagnosis. Better diagnosis, with better information, as Eric said earlier, the right information, at the right time, to the right person, gives the whole ecosystem the opportunity to respond more efficiently and effectively, both at the individual level and in the population. That plays well with any healthcare system around the world. So it’s very exciting times here.

Metrics of success

Gardner: Eric, what’s your perspective on some of the paybacks or metrics of success, when some of the fruits of the standardization begin to impact the overall healthcare system?

Stephens: At the risk of oversimplifying and repeating some of things that Larry said, it comes down to cost and outcomes as the two main things. That’s what’s in my mind right now. I look at these very scary graphs about the cost of healthcare in the United States, and it's hovering in the 17-18 percent of GDP. If I recall correctly, that’s at least five full percentage points larger than other economically developed countries in the world.

The trend on individual premiums and such continues to tick upward. Anything we can do to drive that cost down is going to be very beneficial, and this goes right back to patient-centricity. It goes right back to their pocketbook.

And the outcomes are important as well. There are a myriad of diseases and such that we’re dealing with in this country. More information and more education is going to help drive a healthier population, which in turn drives down the cost. The expenditures that are being spent are around the innovation. You leave room for innovation and you leave room for new advances in medical technology and such to treat diseases going. So again, it’s back to cost and outcomes.
Anything we can do to drive that cost down is going to be very beneficial, and this goes right back to patient centricity.

Gardner: Very good. I’m afraid we will have to leave it there. We’ve been talking with a panel of experts on how the healthcare industry can benefit from improved and methodological information flow. And we have seen how the healthcare industry itself is seeking large-scale transformation and how improved cross-organizational interactions and collaborations seem to be intrinsic to be able to move forward and capitalize and make that transformation possible.

And lastly, we have learned that The Open Group’s new Healthcare Industry Forum is doing a lot now and is getting into its full speed to improve the situation.

This special BriefingsDirect discussion comes to you in conjunction with The Open Group Conference on February 3 in San Francisco. It’s not too late to register at The Open Group website and you can also follow the proceedings during and after the conference online and via Twitter.

So a big thank you to our panel, Larry Schmidt, the Chief Technologist at HP for the America’s Health and Life Sciences Industries, as well as the Chairman of The new Open Group Healthcare Industry Forum, and Eric Stephens, an Oracle Enterprise Architect. We appreciate your time Eric.

I'm Dana Gardner, Principal Analyst at Interarbor Solutions, your host and moderator for this look at the healthcare ecosystem process. Thanks for listening, and come back next time for more BriefingsDirect podcast discussions.
Listen to the podcast. Find it on iTunes. Read a full transcript or download a copy. Sponsor: The Open Group. Register for the event here.

Transcript of a BriefingsDirect podcast on how The Open Group is addressing the information needs and challenges in the healthcare ecosystem. Copyright The Open Group and Interarbor Solutions, LLC, 2005-2014. All rights reserved.

You may also be interested in:

Thursday, January 23, 2014

Siemens Brazil blazes a best practices path to deliver work flow applications on mobile devices

Listen to the podcast. Find it on iTunes. Read a full transcript or download a copy.

As enterprise IT departments scramble to meet demand for more mobile applications, many are charting entirely new terrain. Lessons from applications development and support from PCs and notebooks don't necessarily provide a guide for the mobile tier.

Indeed, mobile apps are very different in what end users expect from them. So how to learn new best practices and simultaneously meet the demand for rapid mobile apps development?

Siemens Brazil in São Paulo has learned several valuable lessons from its mobile app development experiments and subsequent full roll-out of high-demand work flow apps for business managers.

BriefingsDirect had an opportunity to learn first-hand how Siemens Brazil has succeeded at its initial mobile apps at the recent HP Discover 2013 Conference in Barcelona when we interviewed Alexandre Padeti, IT Consultant and Applications Integration Technician with Siemens Brazil. The discussion is moderated by me, Dana Gardner, Principal Analyst at Interarbor Solutions.

Here are some excerpts:
Gardner: Tell us about what Siemens Brazil. Then let's learn about your transition to mobile applications.

Padeti: Siemens Brazil is a public utilities engineering company in Brazil, responsible for 50 percent of the energy transmission in Brazil. With the mobility scenario within Siemens Brazil, we’re just starting right now to implement them in our field applications.

Padeti
Gardner: What types of applications have you targeted first for moving out to the mobile tier?
Padeti: The main applications that we are working with at the moment is Workflow Approval, which integrates with our back-end SAP ERP system. We’re trying to give the managers mobility, the option to make their approval on an ongoing daily basis in a different way.

Real-time basis

Gardner: So it's more important to have workflow approved and managed on a real-time basis, wherever these individuals are and whatever device they happen to be using?

Padeti: Yes. These are the main points of the solution. We’re trying to give this especially to our managers, who are used to being in meetings or moving from one place to another. They gain the ability to make this kind of approval on the go.

Gardner: Why didn’t you just make these applications internally, customize them, host them, and deliver them? What was missing from your being able to do this all yourselves?

Padeti: In the beginning, we were looking for a tool that gave us the freedom to develop for any device. That's the main reason that we chose HP Anywhere. We have the freedom now to choose -- or give the freedom to the users to choose -- the device.

https://hpln.hp.com/group/hp-anywhereGardner: Tell us a little bit about that process of adoption. You've had a proof of concept (POC) phase?

Padeti: Initially we had a POC with HP Anywhere together with HP Brazil and a local partner. From the beginning, it was well-suited. So we decided to go with HP Anywhere in production, and now we’re running a project that will cover nearly 200 users by the end of January.

Gardner: Do you think this will lead to more applications and more mobile users? Does this seem to be a larger undertaking with movement toward even more mobility?
We’re quite sure that 90 percent of the devices will be running on Android and a small percentage on iOS.

Padeti: Yes, that's for sure. This will become bigger in Siemens Brazil, because it's a change of the mindset of the users. They will begin to change the way they’re thinking about requesting solutions from the IT department. In the future, I believe that we’ll have a lot of requirements to develop more such mobile applications.

The standard for Siemens Brazil is based on Android. So we’re quite sure that 90 percent of the devices will be running on Android and a small percentage on iOS.
Gardner: As you've gone through this process, are there any lessons learned that you could share for other organizations? What lessons have you learned, or what advice could you offer them?

Small processes

Padeti: The first one would be to think first about smaller processes. At Siemens Brazil, we’re starting with a not-so-big process. We’re using a not-so-complex one to start. This is a good thing to engage the users and allow them to be comfortable, and furnish proof of use on the solution.

The next one would be to talk a lot with the users, because in our case we have requirements that the user could not think of before. We're learning constantly about what is possible with mobility.
When you give to them the freedom with the mobility, new ideas will come up.

I really advise you to talk with the users and know what they want, because most of the times they don’t come up with an idea until they use mobile, because they’re only thinking initially of desktop or notebook PCs. So when you give to them freedom with mobility, new ideas come up.
Listen to the podcast. Find it on iTunes. Read a full transcript or download a copy. Sponsor: HP.

You may also be interested in:

Wednesday, January 8, 2014

Nimble Storage leverages big data and cloud to produce data performance optimization on the fly

Listen to the podcast. Find it on iTunes. Read a full transcript or download a copy. 

If, as the adage goes, you should fight fire with fire then perhaps its equally justified to fight big data optimization requirements with -- big data.

It turns out that high-performing, cost-effective big-data processing helps to make the best use of dynamic storage resources by taking in all the relevant storage activities data, analyzing it and then making the best real-time choices for dynamic hybrid storage optimization.

In other words, big data can be exploited to better manage complex data and storage. The concept, while tricky at first, is powerful and, I believe, a harbinger of what we're going to see more of, which is to bring high intelligence to bear on many more services, products and machines.

To explore how such big data analysis makes good on data storage efficiency, BriefingsDirect recently sat down with optimized hybrid storage provider Nimble Storage to hear their story on the use of HP Vertica as their data analysis platform of choice. Yes, it's the same Nimble that last month had a highly successful IPO. The expert is Larry Lancaster, Chief Data Scientist at Nimble Storage Inc. in San Jose, California. The discussion is moderated by me, Dana Gardner, Principal Analyst at Interarbor Solutions.

Here are some excerpts:
Gardner: How do you use big data to support your hybrid storage optimization value?

Lancaster: At a high level, Nimble Storage recognized early, near the inception of the product, that if we were able to collect enough operational data about how our products are performing in the field, get it back home and analyze it, we'd be able to dramatically reduce support costs. Also, we can create a feedback loop that allows engineering to improve the product very quickly, according to the demands that are being placed on the product in the field.

Lancaster
Looking at it from that perspective, to get it right, you need to do it from the inception of the product. If you take a look at how much data we get back for every array we sell in the field, we could be receiving anywhere from 10,000 to 100,000 data points per minute from each array. Then, we bring those back home, we put them into a database, and we run a lot of intensive analytics on those data.

Once you're doing that, you realize that as soon as you do something, you have this data you're starting to leverage. You're making support recommendations and so on, but then you realize you could do a lot more with it. We can do dynamic cache sizing. We can figure out how much cache a customer needs based on an analysis of their real workloads.

We found that big data is really paying off for us. We want to continue to increase how much it's paying off for us, but to do that we need to be able to do bigger queries faster. We have a team of data scientists and we don't want them sitting here twiddling their thumbs. That’s what brought us to Vertica at Nimble.

Using big data

Gardner: It's an interesting juxtaposition that you're using big data in order to better manage data and storage. What better use of it? And what sort of efficiencies are we talking about here, when you are able to get that data in that massive scale and do these analytics and then go back out into the field and adjust? What does that get for you?

Lancaster: We have a very tight feedback loop. In one release we put out, we may make some changes in the way certain things happen on the back end, for example, the way NVRAM is drained. There are some very particular details around that, and we can observe very quickly how that performs under different workloads. We can make tweaks and do a lot of tuning.

Without the kind of data we have, we might have to have multiple cases being opened on performance in the field and escalations, looking at cores, and then simulating things in the lab.

It's a very labor-intensive, slow process with very little data to base the decision on. When you bring home operational data from all your products in the field, you're now talking about being able to figure out in near real-time the distribution of workloads in the field and how people access their storage. I think we have a better understanding of the way storage works in the real world than any other storage vendor, simply because we have the data.

Gardner: So it's an interesting combination of a product lifecycle approach to getting data -- but also combining a service with a product in such a way that you're adjusting in real time.

Lancaster: That’s right. We do a lot of neat things. We do capacity forecasting. We do a lot of predictive analytics to try to figure out when the storage administrator is going to need to purchase something, rather than having them just stumble into the fact that they need to provision for equipment because they've run out of space.
That’s the kind of efficiency we gain that you can see, and the InfoSight service delivers that to our customers.

A lot of things that should have been done in storage from the very beginning that sound straightforward were simply never done. We're the first company to take a comprehensive approach to it. We open and close 80 percent of our cases automatically, 90 percent of them are automatically opened.

We have a suite of tools that run on this operational data, so we don't have to call people up and say, "Please gather this data for us. Please send us these log posts. Please send us these statistics." Now, we take a case that could have taken two or three days and we turn it into something that can be done in an hour.

That’s the kind of efficiency we gain that you can see, and the InfoSight service delivers that to our customers.

Gardner: Larry, just to be clear, you're supporting both flash and traditional disk storage, but you're able to exploit the hybrid relationship between them because of this data and analysis. Tell us a little bit about how the hybrid storage works.

Challenge for hard drives

Lancaster: At a high level, you have hard drives, which are inexpensive, but they're slow for random I/O. For sequential I/O, they are all right, but for random I/O performance, they're slow. It takes time to move the platter and the head. You're looking at 5 to 10 milliseconds seek time for random read.

That's been the challenge for hard drives. Flash drives have come out and they can dramatically improve on that. Now, you're talking about microsecond-order latencies, rather than milliseconds.

But the challenge there is that they're expensive. You could go buy all flash or you could go buy all hard drives and you can live with those downsides of each. Or, you can take the best of both worlds.

Then, there's a challenge. How do I keep the data that I need to access randomly in flash, but keep the rest of the data that I don't care so much about in a frequent random-read performance, keep that on the hard drives only, and in that way, optimize my use of flash. That's the way you can save money, but it's difficult to do that.

It comes down to having some understanding of the workloads that the customer is running and being able to anticipate the best algorithms and parameters for those algorithms to make sure that the right data is in flash.
It would be hard to be the best hybrid storage solution without the kind of analytics that we're doing.

We've built up an enormous dataset covering thousands of system-years of real-world usage to tell us exactly which approaches to caching are going to deliver the most benefit. It would be hard to be the best hybrid storage solution without the kind of analytics that we're doing.

Gardner: Then, to extrapolate a little bit higher, or maybe wider, for how this benefits an organization, the analysis that you're gathering also pertains to the data lifecycle, things like disaster recovery (DR), business continuity, backups, scheduling, and so forth. Tell us how the data gathering analytics has been applied to that larger data lifecycle equation.

Lancaster: You're absolutely right. One of the things that we do is make sure that we audit all of the storage that our customers have deployed to understand how much of it is protected with local snapshots, how much of it is replicated for disaster recovery,  and how much incremental space is required to increase retention time and so on.

We have very efficient snapshots, but at the end of the day, if you're making changes, snapshots still do take some amount of space. So, learning exactly what is that overhead, and how can we help you achieve your disaster recovery goals.

We have a good understanding of that in the field. We go to customers with proactive service recommendations about what they could and should do. But we also take into account the fact that they may be doing DR when we forecast how much capacity they are going to need.

Larger lifecycle

It is part of a larger lifecycle that we address, but at the end of the day, for my team it's still all about analytics. It's about looking to the data as the source of truth and as the source of recommendation.

We can tell you roughly how much space you're going to need to do disaster recovery on a given type of application, because we can look in our field and see the distribution of the extra space that would take and what kind of bandwidth you're going to need. We have all that information at our fingertips.

When you start to work this way, you realize that you can do things you couldn't do before. And the things you could do before, you can do orders of magnitude better. So we're a great case of actually applying data science to the product lifecycle, but also to front-line revenue and cost enhancement.

Gardner: How can you actually get that analysis in the speed, at the scale, and at the cost that you require?
I have to tell you, I fell in love with Vertica because of the performance benefits that it provided.

Lancaster: To give you a brief history of my awareness of HP Vertica and my involvement around the product, I don’t remember the exact year, but it may have been eight years ago roughly. At some point, there was an announcement that Mike Stonebraker was involved in a group that was going to productize the C-Store Database, which was sort of an academic experiment at UC Berkeley, to understand the benefits and capabilities of real column store.

[Learn more about column store architectures and how they benefit data speed and management for Infinity Insurance.]

I was immediately interested and contacted them. I was working at another storage company at the time. I had a 20 terabyte (TB) data warehouse, which at the time was one of the largest Oracle on Linux data warehouses in the world.

They didn't want to touch that opportunity just yet, because they were just starting out in alpha mode. I hooked up with them again a few years later, when I was CTO at a company called Glassbeam, where we developed what's substantially an extract, transform, and load (ETL) platform.

By then, they were well along the road. They had a great product and it was solid. So we tried it out, and I have to tell you, I fell in love with Vertica because of the performance benefits that it provided.

When you start thinking about collecting as many different data points as we like to collect, you have to recognize that you’re going to end up with a couple choices on a row store. Either you're going to have very narrow tables and a lot of them or else you're going to be wasting a lot of I/O overhead, retrieving entire rows where you just need a couple fields.

Greater efficiency

That was what piqued my interest at first. But as I began to use it more and more at Glassbeam, I realized that the performance benefits you could gain by using HP Vertica properly were another order of magnitude beyond what you would expect just with the column-store efficiency.

That's because of certain features that Vertica allows, such as something called pre-join projections. We can drill into that sort of stuff more if you like, but, at a high-level, it lets you maintain the normalized logical integrity of your schema, while having under the hood, an optimized denormalized query performance physically on disk.

Now you might ask you can be efficient if you have a denormalized structure on disk. It's because Vertica allows you to do some very efficient types of encoding on your data. So all of the low cardinality columns that would have been wasting space in a row store end up taking almost no space at all.

What you find, at least it's been my impression, is that Vertica is the data warehouse that you would have wanted to have built 10 or 20 years ago, but nobody had done it yet.
Vertica is the data warehouse that you would have wanted to have built 10 or 20 years ago, but nobody had done it yet.

Nowadays, when I'm evaluating other big data platforms, I always have to look at it from the perspective of it's great, we can get some parallelism here, and there are certain operations that we can do that might be difficult on other platforms, but I always have to compare it to Vertica. Frankly, I always find that Vertica comes out on top in terms of features, performance, and usability.

Gardner: When you arrived there at Nimble Storage, what were they using, and where are you now on your journey into a transition to Vertica?

Lancaster: I built the environment here from the ground up. When I got here, there were roughly 30 people. It's a very small company. We started with Postgres. We started with something free. We didn’t want to have a large budget dedicated to the backing infrastructure just yet. We weren’t ready to monetize it yet.

So, we started on Postgres and we've scaled up now to the point where we have about 100 TBs on Postgres. We get decent performance out of the database for the things that we absolutely need to do, which are micro-batch updates and transactional activity. We get that performance because the database lives on Nimble Storage.

I don't know what the largest unsharded Postgres instance is in the world, but I feel like I have one of them. It's a challenge to manage and leverage. Now, we've gotten to the point where we're really enjoying doing larger queries. We really want to understand the entire installed base of how we want to do analyses that extend across the entire base.

Rich information

We want to understand the lifecycle of a volume. We want to understand how it grows, how it lives, what its performance characteristics are, and then how gradually it falls into senescence when people stop using it. It turns out there is a lot of really rich information that we now have access to to understand storage lifecycles in a way I don't think was possible before.

But to do that, we need to take our infrastructure to the next level. So we've been doing that and we've loaded a large number of our sensor data that’s the numerical data I have talked about into Vertica, started to compare the queries, and then started to use Vertica more and more for all the analysis we're doing.

Internally, we're using Vertica, just because of the performance benefits. I can give you an example. We had a particular query, a particularly large query. It was to look at certain aspects of latency over a month across the entire installed base to understand a little bit about the distribution, depending on different factors, and so on.
I'm really excited. We're getting exactly what we wanted and better.

We ran that query in Postgres, and depending on how busy the server was, it took  anywhere from 12 to 24 hours to run. On Vertica, to run the same query on the same data takes anywhere from three to seven seconds.

I anticipated that because we were aware upfront of the benefits we'd be getting. I've seen it before. We knew how to structure our projections to get that kind of performance. We knew what kind of infrastructure we'd need under it. I'm really excited. We're getting exactly what we wanted and better.

This is only a three node cluster. Look at the performance we're getting. On the smaller queries, we're getting sub-second latencies. On the big ones, we're getting sub-10 second latencies. It's absolutely amazing. It's game changing.

People can sit at their desktops now, manipulate data, come up with new ideas and iterate without having to run a batch and go home. It's a dramatic productivity increase. Data scientists tend to be fairly impatient. They're highly paid people, and you don’t want them sitting at their desk waiting to get an answer out of the database. It's not the best use of their time.

Gardner: Larry, is there another aspect to the HP Vertica value when it comes to the cloud model for deployment? It seems to me that if Nimble Storage continues to grow rapidly and scales that, bringing all that data back to a central single point might be problematic. Having it distributed or in different cloud deployment models might make sense. Is there something about the way Vertica works within a cloud services deployment that is of interest to you as well?

No worries

Lancaster: There's the ease of adding nodes without downtime, the fact that you can create a K-safe cluster. If my cluster is 16 nodes wide now, and I want two nodes redundancy, it's very similar to RAID. You can specify that, and the database will take care of that for you. You don’t have to worry about the database going down and losing data as a result of the node failure every time or two.

I love the fact that you don’t have to pay extra for that. If I want to put more cores or  nodes on it or I want to put more redundancy into my design, I can do that without paying more for it. Wow! That’s kind of revolutionary in itself.

It's great to see a database company incented to give you great performance. They're incented to help you work better with more nodes and more cores. They don't have to worry about people not being able to pay the additional license fees to deploy more resources. In that sense, it's great.

We have our own private cloud -- that’s how I like to think of it -- at an offsite colocation facility. We do DR through Nimble Storage. At the same time, we have a K-safe cluster. We had a hardware glitch on one of the nodes last week, and the other two nodes stayed up, served data, and everything was fine.
If you do your job right as a cloud provider, people just want more and more and more.

Those kinds of features are critical, and that ability to be flexible and expand is critical for someone who is trying to build a large cloud infrastructure, because you're never going to know in advance exactly how much you're going to need.

If you do your job right as a cloud provider, people just want more and more and more. You want to get them hooked and you want to get them enjoying the experience. Vertica lets you do that.
Listen to the podcast. Find it on iTunes. Read a full transcript or download a copy. Sponsor: HP.

You may also be interested in: