The Relationship Between Virtualization, Big Data and Cloud Computing

Virtualization is about taking a single large compute resource and making it act like many smaller resources.

Big Data is about taking many smaller compute (and storage) resources and making them act like one big resource.

Cloud computing is about easily changing my compute and storage resources as needed.

Big Data can leverage cloud computing to scale the size of the "one big" resource as needed.

Cloud Computing, Virtualization, IaaS, PaaS and SaaS - Part 2

In part one of this post, I looked in to the question of why cloud computing is important. In this post, I will look into the question of what cloud computing is.

NIST's definition of cloud computing gives a useful model for deciding what is and is not a cloud deployment:
 "cloud computing is a model for enabling ubiquitous, convenient, on-demand network access to a shared pool of configurable computing resources (e.g., networks, servers, storage, applications and services) that can be rapidly provisioned and released with minimal management effort or service provider interaction."

But to understand what "the cloud" is really made up of, it helps to look at how cloud computing is packaged and sold.

• Infrastructure as a Service (IaaS)
• Platform as a Service (PaaS)
• Software as a Service (SaaS)

Infrastructure as a Service (Iaas)
In this cloud offering, the seller provisions virtualized compute, network and storage upon which the buyer installs his own operating system and software. This is most easily thought of as buying bare computers and storage onto which your own IT dept will put everything required for your specific uses.
A few major vendors who offer these products are: Amazon Web Services, RackSpace and AT&T Synaptic Hosting.

Virtualization software is key to making IaaS feasible. Without virtualization, physical hardware would need to be provisioned for each customer. This would make it prohibitively expensive to quickly provision, re-provision and scale resources to meet customer needs.
The major virtualization s/w vendors include VMWare (ESX), Microsoft (HyperV) and an open source virtualization solution Xen.

More on virtualization in another post.

Platform as a Service (Paas)
In the platform level cloud offering, the seller provisions some combination of operating system, middleware and runtime packages which enable the buyer to develop and run applications of their choosing. This can be thought of as buying access to services which enable programs to easily access the resources they need to run.
Vendors provide a variety of offerings in this space. A few major offerings are below:

• Google App Engine provides a number of runtime environments for web application developers.
• Amazon Simple Storage Service (S3) provides file based storage to any application which needs it while Amazon Elastic Block Store (EBS) provides block level storage for applications the need direct storage access (like DBs).
• Microsoft Azure provides access to a few runtime envrionments,  SQL DB services for applications which need DB services and the virtual network fabric required to link together multiple server's services.

Software as a Service (Saas)
In this level of cloud offering, the seller provides the buyer with access to end applications while hiding the infrastructure, middleware and runtime components. Pretty much every time you hear about a "web application" being offered by a company, you are seeing a SaaS product.
There are probably thousands of SaaS offerings available so covering any significant fraction of the space here is futile. But here are a few examples of SaaS offerings:

• Web email - from google, Yahoo, etc
• Office 365 and Google Apps - On line productivity apps for creating, accessing and collaborating with others on a variety of document types (word processing, presentations, spreadsheets, picture editing, etc).
• Sustainable Supply Chain (SSC) - Supply chain survey management for corporate social responsibility reporting from CSRware.
• Facebook and Google + - public social networking tools for keeping tabs on your friends around the world.
• Socialcast, Clearvale , Jive and Spigit -  Enterprise social networking tools for keeping abreast of news and status, smoothing workflow, fostering collaboration and spurring idea generation in a corporate context.

One additional consideration is which user base a cloud deployment is intended to serve. This leads to the ideas of: Private, Public and Hybrid Clouds

• Private cloud = services and (often hardware) are strictly for a single company's use. Frequently this implies that the company consuming the services will deploy the cloud services on hardware behind their corporate firewall.
• Public cloud = services are available for all users anywhere. Though users may need to pay for services... This is the context in which most people experience the cloud because of the heavy reliance start-ups offering SaaS products have on public cloud services.
• Hybrid cloud = multiple cloud systems connected in such a way as to allow programs and data to be easily moved between private to public clouds.

Cloud Computing, Virtualization, IaaS, PaaS and SaaS - Part 1

What is cloud computing and why is it important?

On the question of importance, cloud computing addresses several business problems.

Perhaps the most mundane problem that cloud computing addresses is one of cost for getting applications up and keeping them running.

• Availability 
• Because of the underlying technology used in cloud computing (Virtualization - more on this next post.) it becomes relatively easy to implement failover and high availability for servers. This makes the customer experience better (fewer interruptions) and allows more routes to addressing business critical applications (lowers barrier to entry on some enterprise applications).
• Application deployment & maintenance 
• Depending on the level of the cloud you engage at (Infrastructure, Platform or Software - more on this next post), the level of IT expertise required to use the applications / services you want can be greatly reduced. Software patching, upgrades, hardware provisioning and maintenance is left to the cloud providers who have this expertise on staff.

These "mundane" aspects of cloud computing enable significantly less mundane capabilities. The most immediate of which are the new business models that become feasible.

• Software start ups galore
• Because compute power can be purchased on-demand and in relatively small units, capital costs for starting a software company are significantly reduced. This enables more players to try more things in search of the next big thing.
• F2P (Free to Play), Freemiums and ad-driven businesses
• Because the marginal cost of adding users to a centralized s/w application can be nearly zero when deployed in a dynamically scalable infrastructure such as the cloud provides, companies can explore business models that "give away" software, or more precisely: give away access to software, while closely tracking and tailoring the user experience more easily. This leads to businesses that thrive on micro transactions within the software or on ad revenue instead of from sales of the software itself.
• On-line special events
• Relatively low cost and rapid provisioning of compute, network and storage resources allows for companies to generate increased community engagement or media attention by using this burst capacity to host periodic special events. For a small fraction of the cost of buying and setting up the h/w and software required to gain this capacity, the same buzz generating potential can be realized.
• Mobile
• By using the relatively limited compute power of networked mobile devices to drive the user interface and pushing the computationally intensive tasks to servers in the cloud, the capabilities of mobile devices become nearly unlimited. This creates a new class of applications (e.g. voice recognition, navigation, etc) for entrepreneurs to explore and market, driving sales of mobile computing hardware (e.g. smart phones and tablets) or of the applications (as apps or as services) themselves.
• Distributed work force, virtual desktops and collaboration
• Because public cloud resources can be accessed from any internet connected device and data storage is easily centralized, the cloud makes it more feasible than ever to provision specialized applications to anyone, anywhere, to keep company data centralized and provide tools by which physically separated groups can easily interact and exchange information in real time in order to improve productivity.

As Corporate Social Responsibility (CSR) and sustainability concerns increase, the "green" aspects of cloud computing also become more important.

• Economies of scale in "green" data centers may mean that renting capacity via "the cloud" is greener than you can afford on your own.  
• Making an efficient data center requires high up front capital and expertise costs. Considerations like the ones below are more affordable by large businesses or by dedicated cloud hosting companies where the risk associated with the capital costs are more acceptable for the long term return.
• efficient server h/w
• low energy and passive cooling strategies
• occupancy based lighting, energy efficient lighting fixtures and daylighting strategies
• green power purchase agreements
• on-site generation via renewables (e.g. co-gen, bio-fuel based fuel cells, PV, Solar thermal, etc.)

Next time some details about the the "what" part of the question.