The Promises, Payoff, and Products of Hybrid Clouds

The cloud promised to provide us all flexibility. The opportunity to access infinite resources as and when we need them and pay accordingly. We would no longer have to spend time installing, configuring and maintaining servers; we were promised more time to "just code."

Instead, we got increasing vendor lock-in and a handful of cloud players so large that if a data center experiences problems, significant sections of the internet go offline. Naturally, we created more tools and practices to cope with the problem we created for ourselves, and dear readers, welcome to hybrid clouds.

I am of course being slightly facetious. In reality, hybrid clouds are a method for building flexibility and redundancy into a cloud infrastructure. The past decade has taught us that relying on one provider is a bad idea, and we should use a mixture of public and private platforms and switch between them as required for operational or financial reasons.

There are several reasons you might want to consider a hybrid cloud instead of throwing all your egg-shaped services into one cloud-shaped basket.

For regulatory or architectural reasons, an application may contain data that you need to store in particular regions or on servers that you have more control over.

Some cloud providers provide better value for certain services than others, or you might want to take advantage of the best deals with specific providers.

While increasingly unlikely as most software vendors rush to the cloud, you may have legacy or custom services that only run on particular private machines or third-party providers. This includes services that you intend to migrate eventually but haven't yet.

It may surprise you to hear, but most cloud providers are supportive of hybrid clouds, especially those that connect their services to legacy and on-premises systems. After all, they are removing barriers for potential customers. Here's a couple of factors to consider in your hybrid-cloud strategy.

While in theory developer standards are widely adopted, you can potentially experience library or protocol inconsistencies between providers, so do your research and testing before a major rollout.

As I hope you are already doing, naturally you need to encrypt all communications between services and make sure that public endpoints are secured.

Again, although cloud services, CDNs, and transmission mechanisms are continually improving, the more hops you introduce, the more the opportunity for lag, latency, and 'moving parts' that you need to debug in case of a problem.

Now for everyone's favorite discussion, let's talk about the tools available to help you create, manage, and tweak your hybrid cloud setup. I've tried to break them into categories, but there is some crossover.

AWS has an entire suite of tools to help their services form a part of your hybrid cloud, including:

• AWS Storage Gateway: for using on-premises storage as part of AWS storage devices

• Amazon VPC: for creating a VPN between AWS and other parts of your network, plus the capability to manage IP address ranges if you need to use specific values

• AWS Direct Connect: similar to VPC but for creating direct connections

• AWS OpsWorks: for those of you following "infrastructure as code" practices, AWS's offering can also manage on-premises servers

Azure has a similar offering in the form of Azure Stack. What features it offers and how you use them is a little unclear and lurks behind a sign-up form. Other tools offered by Azure that relate to hybrid clouds are:

• Logic Apps: for pulling data from on-premises applications into public cloud applications

• Service Bus: for inter-cloud messaging

• StorSimple: for consolidated storage

You can integrate many of the smaller hosting players with a hybrid cloud, using any of the commercial tools below that support your provider(s), or a roll-your-own option if you put in the work. There are companies like Joyent who focus their business on helping you integrate them with other larger players, which is a smart move, and open-source their tools.

One of the many all-in-one solutions, CoreStack coins another buzzword to add into the mix: cloud governance. Aimed more at operations and business people than developers, the service focuses on defining how your services fit together based on consumption and cost and don't provide a tremendous amount of detail before an appointment with sales.

There are also a handful of companies such as ParkMyCloud and Replex that focus entirely on the money-saving aspect, helping you save as much money as possible by shifting application components around as efficiently as possible.

Cloud Controller pull in a lot of enterprise-friendly service providers such as Oracle, Citrix, and Red Hat. With another new buzzword is Nutanix and their "hyperconverged infrastructure technology." They both have an impressive client list and support a lot of enterprise-friendly software components, but again it's hard to know how their platform works.

Finally, of course, Cisco has their own solution in the shape of CloudCenter that has a few extra useful features such as budget plans, centralized security, and supports over 20 providers.

There are plenty of choices in the open-source realm too that you can install and manage yourself, or find preinstalled on public and private clouds. While many developers will use more complex (and thus scalable options) for managing Docker containers across multiple hosts, for simple setups, Docker machine and Swarm could be enough for your needs.

A small project called Kubernetes has hybrid cloud functionality (or as they call it "cluster federation") in the form of kubefed. It's a little complex to set up, but read this Google blog post for some ideas. If you are interested in Kubernetes but don't want install and manage it yourself, then look no further than my roundup of Kubernetes managed hosting options.

Somewhat overshadowed by Kubernetes these days but still a powerful option is Apache Mesos, which uses interesting paradigms to treat your distributed computing resources as one collective whole.

In a similar vein is OpenStack, which puts hybrid clouds front and center, and I would hazard a guess that some of the commercial vendors use it behind the scenes as well.

Lesser known but with equal vintage and aims is Apache CloudStack.

An older but well-established option is OpenNebula, it's not immediately apparent that it's open source (thought it declares loudly that it is). After some digging, I finally found the codebase.

Finally, take a look at OneOps from Walmart, yes, the retail chain. At least you know it's production-tested.

Tools for monitoring hybrid clouds are also plentiful. What you choose mostly depends on your setup and what you want to monitor. Common tools such as DataDog, New Relic, Prometheus, and the Elastic stack should suit your needs and are widely available.

In reality, the hybrid cloud is what the cloud should have been in the first place: a flexible suite of services that do what we ask, when we ask, and charge us accordingly.

Granted, a hybrid cloud requires more initial steps than we might have all hoped, but few people also want to run and maintain their own servers anymore, so it's a happy compromise.