We are entering into an interesting time in the industry witnessed by the release of VMware View. View Manager, View Composer, Thin App, and Offline Desktop are well positioned to accelerate the adoption of virtual desktops in datacenters and to remote access devices.
Today I’d like to share some thoughts specific to reducing storage costs with VDI or View. A major component in VMware View Composer is Linked Clones, which provides the ability to deploy hundreds of virtual desktops from a single desktop image. In addition to storage savings, customers will be able to update deployed desktops deployed with View Composer by patching the master desktop image.
Recently I have received a number of inquiries as to how View Composer and Linked Clones integrate with NetApp’s storage saving Dedupe and zero cost provisioning FlexClone technologies. I’d like to take this time to discuss this topic and how VMware and NetApp are engineering the best of breed virtual desktop solutions.
I will caution you in advance, this post is a bit lengthy because we have much to share and discuss.
Let’s Focus on View Composer
As I want to discuss storage savings and integration we will focus primarily on View Composer. I can only imagine the length of this post if I were to cover all of the components in the view suite.
Virtual Desktop Types 101
When one considers virtual desktops there are for all practical reasons two types: permanent and non-permanent. With VMware View these two types are available in three flavors*: individual, persistent, and non-persistent.
A permanent desktop is much like one’s workstation complete with all of the personalization and personal data stored locally. VMware refers to this design is referred to as an individual desktop.
Virtual Desktop Types 102
A non-permanent desktop is one where the OS is served from a temporary image (such as a Linked Clone), and the user data is stored elsewhere in a separate shared storage pool.
If the user data is stored on the network and served via Microsoft roaming profiles then VMware terms this design a non-persistent desktop.
New to VDI is the concept of storing the user data on a second virtual disk (aptly titled the ‘user data disk’). VMware refers to this design as a persistent desktop.
Note in both non-permanent designs the user data is separate from the OS data. It is this separation that enables the OS to be ‘modular’ and allows the ability to dynamically update desktops.
Virtual Desktop Types Addendum
View Manager can manage and deploy all three types of the VMware virtual desktop types; however, View Composer’s Linked Clones are only available with the non-persistent and persistent (or non-permanent) desktop types.
Customers who require individual (or permanent) virtual desktops will still be able to use the wealth of technology available in View Manager minus the storage savings provided Linked Clones.
Zero Cost Clone Copies
For both VMware and NetApp, the ability to provide zero cost copies of data has its roots in our respective snapshot implementations.
Linked Clones leverages the VMSnap process to make clones from a ‘master desktop image.’ Each clone consists of VMSnap redo log(s) for each clone that is provisioned. As redo logs grow over time, routine maintenance is required to keep storage utilization at a minimum. Not to worry, View Composer address these tasks.
Enhanced Virtual Desktop Management
With non-permanent desktop types, VMware View provides the means where a user can log off of their desktop and when they log back on their desktop will be ‘patched’ or ‘updated.’ NetApp customers who marvel at some of the things we do with WAFL can sometimes over-think this component. The elegance of View Composer is that there is no ‘block level magic’ going on here.
Within VDI in general, users are connected to their desktops via a connection broker. VMware View Manager performs this function and as the OS disks are separate from the user data it has the ability to connect the user to a different (or updated) desktop and the user is unaware of the change as their user data and settings won’t appear to be changed. This is pretty cool to see in action (hint – call your VMware account manager now).
As an example:
Deploy 100 non-permanent desktops (Linked Clones).
Update the master image VM.
Deploy 100 new non-permanent desktops (Linked Clones).
Move the users from old version to new via a View Manager policy.
Delete original 100 desktops.
NetApp with VMware View
VMware linked clones are valuable as they are tightly integrated into View Manager & View Composer. Linked clones can reduce initial storage capacity by 90%. However, the storage for the serving the desktop OS and application set is only a portion of the data involved in a VDI solution.
A non-permanent desktop type requires user data to either reside in network attached home directories, in user data disks (which reside in a VMware datastore), or possibly in both (for those who may want full data access with offline VDI). In most cases user data is equal to or greater than the amount of data required for the operating system and applications. NetApp dedupe can reduce the data redundancy in the user home and profile directories.
For customers that require individual or the permanent desktop type they too can receive storage savings for OS and applications via NetApp dedupe. In fact, NetApp provides a Rapid Cloning Utility which can mass provision hundreds or thousands of individual mode desktops and post deployment allows these desktops to be imported in View Manager for ongoing life cycle management.
BTW – The RCU is free to NetApp customers.
The Power of Joint Engineering
So customers can deploy VMware View and NetApp efficiency technologies together and gain tremendous benefits from both. In this case, it is not an either/or proposition. You can have your cake and eat it too.
Non-permanent desktop types leverage Linked Clones to reduce OS and application storage requirements while NetApp Dedupe reduces the large component of the data that is user home and user profile data.
NetApp reduces their storage requirements for permanent desktop types, and even provides a mass deployment tool, which the VMs deployed with, can be imported into View for management.
The vStorage Effect
But wait, this solution gets better… At VMworld 2008, VMware announced the vStorage initiative, which is a common management framework for storage integration.
From the vStorage press release: vStorage APIs for array integration will enable customers to leverage array-based capabilities, such as snapshots, provisioning, replication and restore, directly with individual virtual machines in conjunction with the clustering and pooling capabilities of VMware Infrastructure
In the keynote presentations that covered vStorage, NetApp announced many of our engineering efforts one of which is to integrate FlexClone (file, LUN, and volume level cloning) into the VMware Linked Clone API.
I would like to point out here that virtual desktop or virtual machines are files, and NetApp is the only storage vendor providing zero cost file level cloning along with LUN and volume cloning.
This means that with vStorage integration, customers will be able to upgrade VMware View & NetApp Data ONTAP and by doing so will receive linked clones that are enhanced by FlexClone.
The benefit of VMs provisioned via the Linked Clone API and FlexClone is that customers will receive all of the operational benefits of the View Manager tools while being able to scale their VDI solution farther than previously available.
Dichotomies Everywhere
Providing dense VDI solutions is an interesting challenge. While one can drive down storage capacity requirements with technologies such as FlexClone, dedupe, and Link Clone, the storage performance required for each desktop still needs to be considered.
We all agree that each individual desktop (not the type but a single desktop) has low IO or performance demands; however, VDI solutions can aggregate 100s or 1000s of desktops. Storage reduction technologies that remove physical disk drives can exacerbate this scenario by removing the means to provide storage performance.
VDI introduces the issue of IO ‘storms’. As an example, consider the impact of the 2 hour window every morning when all of the users log on (aka a logon storm). Maybe I’ll blog more about boot, logon, HA failover, and antivirus IO storms in the near future.
The solution to this challenge appears to be in array system cache, gobs and gobs of array cache.
Most are unaware that NetApp storage arrays provide dedupe aware cache along with dedupe aware storage. Advanced storage architectures such as this extend dense storage deduplication ratios throughout the array. With VDI, disk utilization increases to 250:1 as well as array cache utilization (note these ratios are specific to VDI cloned images).
Dedupe aware cache is available today with the Performance Accelerator Module and will be available in main systems memory with the release of 7.3.1.
Addressing all the needs of a Virtual Infrastructure
Linked clones will be available for virtual servers in the future; however as you can see, they are not designed for permanent virtual machines. NetApp dedupe enables customers to virtualize more systems than traditional storage vendors.
Wrapping This Post Up
√ VMware and NetApp are long time and deeply integrated partners
√ NetApp is one of the storage partners working with VMware to develop the open standards for vStorage
√ VMware View Composer and Linked Clones save storage capacity for non-permanent desktops
√ NetApp dedupe saves storage capacity for the user data of non-permanent desktops (NTAP unique)
√ NetApp dedupe and FlexClone saves storage capacity for permanent desktops (NTAP unique)
√ In the future vStorage will make linked clones call NetApp FlexClone for greater solution scalability and no redo log maintenance
√ NetApp Dedupe and FlexClone reduces the storage required for virtual servers (NTAP unique)
√ NetApp dedupe aware array cache extends the performance of arrays to greater than their physical limits (NTAP unique)
Virtualization changes everything…
* or ‘flavours’ as I am reminded by my friends and family in Ontario. See you on Facebook
Welcome Back to the blogosphere Vaughn! I enjoyed the View overview.
Hi,
As you said, article is lengthy but its really very informative… Liked Joint Engineering point of VMWare and NetApp.
Himangi
Times Of Virtualization
Thanks for the excellent post – the only thing I’d add is that while RCU is indeed free it requires FlexClone (which is most definitely not free unfortunately).
@Andrew – great point. FlexClone is required for zero cost desktops, and while it isn’t free, neither is other storage savings technologies such as VMware’s Linked Clones.
We’re doing as much as we can to provide value with FlexClone by allowing customers to use it for more than virtual desktops. Applying FlexClone (via the RCU) to virtual servers and adding NetApp deduplication (which is free) can reduce your production storage footprint by 60-80%.
Deduped footprints make for dedupe replicas for DR, and dedupe footprints provide built in WAN acceleration for the replication of the data set.
Thanks for the FlexClone response — this has recently gotten easier in the channel with the bundle approach to software as well.
For this…
“In the keynote presentations that covered vStorage, NetApp announced many of our engineering efforts one of which is to integrate FlexClone (file, LUN, and volume level cloning) into the VMware Linked Clone API.”
Is there any timeline or detailed feature announcements available by any chance?
Re: [NetApp – The Virtual Storage Guy] Andrew Miller submitted a comment to A New Era for Virtual Desktops
@Andrew NetApp provides FlexClone integration for the cloning of both datastores and VMs with the Rapid Cloning Utility (or RCU) vCenter plug-in. This allows for instantaneous, pre-deduplicated cloning of a VM or 1,000s of VMs (for VDI) in VMFS NFS datastores. The cloning offloads the IO from the storage network and ESX/ESXi hosts.
VMware does have VAAI, the vStorage API for Array Integration, which will be released in phases beginning with ESX/ESXi 4.1 in the first half of this year. With this release all supported storage arrays will be able to offload the cloning process (for VMFS) from the ESX/ESXi host to the array. This API is referred to as Copy Offload. One should note, this process operates will vary based on array capabilities. NetApp will still provide instant, pre-deduplicated VM clones for both VMFS and NFS. Copy Offload will also allow for the offload of Storage VMotion operations.
Also included in the first release of the APIs is Write Same and Atomic Test Set. Write Same will allow for the formatting of EagerZeroed Thick VMDKs to be handled by arrays. ATS will replace the current SCSI locking mechanism in VMFS allowing for larger ESX/ESXi clusters (beyond the 8 node max that is a common practice today).
I should add that, storage controller software may have to be upgraded in order to enable these features along with the hypervisor upgrade to 4.1.
Which is to integrate FlexClone (file, LUN, and volume level cloning) into the VMware Linked Clone API.”
As wide as industrial sector is growing day by day so the use of computer is also becoming a basic need of these sectors. Very good article and very appreciated too.