Data reduction discussion - playing with SAN

[Dieselboy]

I've been working on setting up our new Nimble SAN over the past couple of weeks and I'm slowly moving our VM disks over. I think I've found something pretty interesting but to be honest this might be well known and just new to me.
If I build a VM, say complete vanilla Windows 2012 with gui and install all the latest updates, antivirus etc just the basic stuff every server must have before you go and install the application specific stuff. I can then go over to our nimble SAN and take a snapshot, clone this snapshot into a new iSCSI volume then mount this new volume into a new VM on RHEV. What I now have is 2 vanilla servers only consuming the space of one. The 2nd VM only stores the new changes. I can keep the original server as a base and each time I need a new VM just clone the original again. I could build a tree and fork the original server into two, one with a gui and one without a gui. Each time I'm only consuming the space of the additional changes. I was aware of how snapshots worked already but I think the penny has dropped here (unless I've misunderstood). Although with Windows there's that whole sysprep / computer ID thing that causes weird things to happen when you clone machines (unless, fingers crossed, M$ have resolved that somehow).

The biggest benefit here is our Red Hat VMs. We have 90 VMs and most of those are RHEL 6 base installation / no gui. Or base installation with webmin

Thinking about later on after updates have been applied the forked VMs would have to install those as per any other VM. The original VM can be updated and another snapshot taken for cloning more base VMs with the latest security updates. Likewise down the tree, if I need a RHEL6 with webserver / tomcat I can keep a base snapshot of that config for cloning.

Found this by accident. I am doing some tasks on a windows 7 vm, from the console. The VM doesn't have any network access at all. I wanted to multitask and test the snapshotting out so I took a snapshot and cloned it into a new volume. I then mounted this new LUN into a new VM and started multitasking. I noticed that the new VM has only used 500MB of it's 50GB LUN. I then needed another clone so done the same again; clone the previous snapshot into a new LUN, mount the LUN into a new VM. I now have 3 VMs running for the price of 1.0001

ps I've tried deleting the original snapshot / volume and it wont let me, saying that this is the base of a clone. - which is nice.

[Dieselboy]

To work around the Microsoft SID issue, run sysprep. I found this web guide on how to do that: https://connectedenterprise.wordpress.com/2012/08/03/correct-a-duplicate-sid-in-a-virtual-machine-clone/
I use sysprep for our laptop builds. From that I found that you cannot sysprep more than once, so with that you must keep the base build for cloning, don't clone a clone.

[Otanx]

I'm not a storage guy, but my understanding is the data dedup stuff on the new SAN gear should work without you having to clone the disks. It should detect the duplicate data, and just store one copy. You can still use normal build procedures, and get this benefit. For our RHEL stuff we just have a kickstart, and then add it to the correct channels in Satellite.

-Otanx

[Dieselboy]

Hi Otanx, you're correct or at least I believe you are. Data is "compressed" in that the disk usage as per the VM is higher than what's really in use on the SAN side. "Deduplication" at the moment is only available for all-flash arrays with Nimble. The dedupe "enable" checkbox is there but it's in a "coming soon" phase. I wonder when / if this gets fully implemented by Nimble if the cloning will then be  redundant and will offer the same space savings as the clone method and this is why I don't see that kind of space saving with regular disk / LUNs.


But since my OP yesterday I've been testing this out, building systems etc. I have one "alpha" build, which is windows 2012 server, with latest updates. I cloned the vanilla VM, run sysprep afterwards and then began customisations form that point on as a new server. The "alpha" server has everything that every server must have except server specific customisations, like configuring applications.

With this I can see that for 1 x alpha VM and 4 x working clones, data has gone down from 4x15GB (approx 60GB) - to 10GB + 4GB = 14GB. Each clone is consuming round 800MB to 2GB of new data.

I've set up a pool of Windows 7 VMs as well some time ago that need re-building / cleaning. They are each consuming 35GB to 50GB of data. If I rebuild these and implement the above method I can reduce this footprint from just over 200GB to 39GB.
I see another benefit to this that we were not doing already, as VMs are built from a single VM clone, that is now a build image template. Time taken has been seriously reduced and is going to be the same configuration each time.

I'm not sure why I find this so exciting, it's like magic

[wintermute000]

Hi Otanx, you're correct or at least I believe you are. Data is "compressed" in that the disk usage as per the VM is higher than what's really in use on the SAN side. "Deduplication" at the moment is only available for all-flash arrays with Nimble. The dedupe "enable" checkbox is there but it's in a "coming soon" phase. I wonder when / if this gets fully implemented by Nimble if the cloning will then be  redundant and will offer the same space savings as the clone method and this is why I don't see that kind of space saving with regular disk / LUNs.


But since my OP yesterday I've been testing this out, building systems etc. I have one "alpha" build, which is windows 2012 server, with latest updates. I cloned the vanilla VM, run sysprep afterwards and then began customisations form that point on as a new server. The "alpha" server has everything that every server must have except server specific customisations, like configuring applications.

With this I can see that for 1 x alpha VM and 4 x working clones, data has gone down from 4x15GB (approx 60GB) - to 10GB + 4GB = 14GB. Each clone is consuming round 800MB to 2GB of new data.

I've set up a pool of Windows 7 VMs as well some time ago that need re-building / cleaning. They are each consuming 35GB to 50GB of data. If I rebuild these and implement the above method I can reduce this footprint from just over 200GB to 39GB.
I see another benefit to this that we were not doing already, as VMs are built from a single VM clone, that is now a build image template. Time taken has been seriously reduced and is going to be the same configuration each time.

I'm not sure why I find this so exciting, it's like magic

Not to go OT but shouldn't having multiple client OS-es be a classic use case for VDI which would leverage some kind of clone/template system in the background?