Category Archives: Virtualization

Troubleshooting Tip – The resource ‘104’ is in use

View Administrator experienced a strange issue not long before a scheduled upgrade to Horizon 6. Each time Composer attempted to spin up a new desktop, the task would error out in vCenter with the error The resource ‘104’ is in use. One would think that this issue would not have much impact. Unfortunately, this issue wreaked havoc on a linked clone pool. Over a thousand errors were reported in View Administrator overnight when this issue arose, as the desktop pool was not set to Stop provisioning on error. Upon refreshing the ports on the vSwitch, the error went away.

A quick search turned up a VMware KB article discussing that the number 104 refers to a specific port on the virtual distributed switch. Looking at the ports on the virtual distributed switch, it was observed that an older version of a template that was once servicing a full clone pool was occupying the port in question. There seemed to be some sort of confusion on the side of vCenter. Errors like this had been solved in the past by rebooting vCenter. The next outage window available for a vCenter reboot was allocated for the upgrade window. Since the first step in the upgrade was to reboot vCenter, this seemed like a good time. After the reboot of vCenter, and a successful upgrade the issue persisted.

It was time to look at the template. Since the template was an older version of the standard desktop that was kept around for fallback purposes, it was determined that a good first step was to convert the template to a virtual machine, and disconnect it from the vDS. Upon right-clicking on the template, it was discovered that the option to Convert to Virtual Machine was grayed out. A quick search returned the VMware KB article discussing possible permissions issues. Since permissions were not the case, the article recommended removing the VM from inventory. Removing the template from inventory fixed the problem, and it has yet to return. Rather than re-add the VM to inventory, the VM was deleted, as it was no longer in use.  

Advertisements

VMFS Datastore Sizing and SCSI Reservations

Introduction

This post discusses datastore sizing concerns in the production design of many virtual infrastructures. During VMware View Health Checks, an issue may be identified regarding VMFS datastore sizes. The priority 1 recommendation suggested that datastore sizes be reduced, accessed by fewer hosts, allocating no more than 64VMs per LUN/VMFS datastore. The implementation of the smaller datastore has become pain point support teams. This document explains the rationale behind that decision, as well as a path going forward.

Background

In previous versions of vSphere, VMFS used SCSI reservations on storage devices that did not support hardware acceleration. In addition, previous versions of Data ONTAP, SVC, or other storage array did not support hardware acceleration. When SCSI reservations happen, they lock an entire storage device during any operation that requires metadata protection, up until the time the operation completes. Since this lock is exclusive, excessive SCSI reservations can cause performance degradation on any host accessing the same VMFS volume.

SCSI Reservations

SCSI reservations can be caused by certain activities performed on disk. Many of these activities are listed in the section below.

Things that cause SCSI reservations:

  • Creating, resignaturing, or expanding a VMFS datastore
  • Powering on a virtual machine
  • Creating or deleting a file
  • Creating a template
  • Deploying a virtual machine from template
  • Creating a new virtual machine
  • Migrating a virtual machine with vMotion
  • Growing a file, such as a thin provisioned virtual disk

A VMware View environment is ripe for SCSI reservation issues, given the following:

  • View desktops experience far more power cycles than a server
  • View desktops and their associated disks are created and deleted on a regular basis
  • View desktops are thin provisioned, and grow daily
  • View desktops are typically smaller than servers, and have more virtual disk files, resulting in more VMs and VMDKs per datastore than a server infrastructure
  • View linked clones are based on a snapshot of the parent, and typically generate a significant amount of SCSI reservations

A few approaches to reduce SCSI reservation conflicts are listed below:

  • Limit the number of operations on different hosts that require SCSI reservations at the same time
  • Thick provision disks
  • Limit the number of hosts that can access a specific LUN
  • Reduce the number of snapshots
  • Reduce the number of virtual machines per LUN

Many of the approaches listed to reduce SCSI reservations introduce additional cost and administrative overhead, some are simply not possible. For example, using thick provisioned disks greatly increases the amount of SAN storage required by the virtual infrastructure. Dealing with the design considerations for addressing SCSI reservation issues is not a trivial task and requires design updates as new product improvements are released.

The New Approach

Introduction of Atomic Test and Set (ATS) in vSphere 5.0, using VAAI helps resolve SCSI-2 reservation issues. ATS is available on storage devices that support hardware acceleration. ATS is sometimes referred to as hardware assisted locking. Unlike SCSI reservations, ATS supports discrete locking per disk sector. In these cases where hardware acceleration is available on the storage array VMFS uses ATS rather than SCSI reservations. In order to ensure all ATS capabilities are available, the VMFS datastore must be at version 5. Also note that VMFS datastores that were upgraded from VMFS-3 to VMFS-5 may revert back to using SCSI reservations to handle locking.

Storage devices that support hardware acceleration perform the following tasks more efficiently:

  • Migration of virtual machines with Storage vMotion
  • Deploy virtual machines from templates
  • Cloning virtual machines and templates
  • VMFS clustered locking and metadata operations for virtual machine files
  • Writes to thin provisioned and thick virtual disks
  • Creating fault-tolerant virtual machines
  • Creating and cloning thick disks on NFS datastores

Solution

The current storage design can be updated to reflect new features introduced as a part of the Hypervisor. The implementation of the recommendation from a View health check is causing pain in that the growth of View persistent disks has begun to cause space issues on VMFS datastores. Space issues are compounded by the inability to leverage storage vMotion for Linked clones, as a rebalance operation is the only supported method for balancing out datastore usage. Storage vMotion is supported on full clones, so putting datastores in datastore clusters and enabling SDRS would helpful. Given that, in many environments, full clones and linked clones reside on the same datastores. This option is not viable without some logical changes to the infrastructure. Finally, some may be consumed unnecessarily for old replicas.

All this being said, the solution must be two-fold:

  1. The solution must continue to ensure that SCSI reservation issues do not occur (or are minimized to a great extent)
  2. The solution must address space concerned expressed by View support teams.

Ensure SCSI reservation issues remain a non-issue

  • Confirm Hardware Acceleration is supported on the storage array
  • Ensure that all primitives are available on each host
  • Provision new storage volumes as new VMFS-5 datastores.

Address datastores running out of space

  • Separate datastores for full clones and linked clones
  • Configure datastore clusters and SDRS for full clone datastores
  • Regularly rebalance linked clones across linked clone datastores
  • Clean up old replicas
  • (Optional) Increase size of datastores, and rely on the fact that ATS will handle disk sector locks, rather than having SCSI-2 reservation issues.

Implementation Approach

To implement the above solution, the approach to be used will likely follow, at a high level, the steps listed below:

Dedicate datastores either as a full clones or for linked clones

  1. Migrate full clones to their appropriate datastores
  2. Migrate linked clones to their appropriate datastores by performing a rebalance operation
  3. Combine full clone datastores into a datastore cluster
    1. Enable SDRS
  4. Resize LUNs, increasing them to more palatable size
  5. Rebalance linked clones
  6. Enjoy a balanced, automated storage infrastructure

     

These steps require testing in a non-production environment, as the approach may vary depending on the environment.

Conclusion

This document addressed the rationale for leveraging smaller VMFS datastores, as well as the implications of SCSI reservations. Additionally, recommendations were provided, as a path forward, to alleviate the pains resulting in smaller LUNs.

References

VMFS Locking Mechanisms – p157 for SCSI Reservations; p215 for hardware assistance

http://pubs.vmware.com/vsphere-51/topic/com.vmware.ICbase/PDF/vsphere-esxi-vcenter-server-511-storage-guide.pdf

vSphere 5.1 Excessive SCSI Reservations

http://pubs.vmware.com/vsphere-51/index.jsp?topic=%2Fcom.vmware.vsphere.troubleshooting.doc%2FGUID-EAAFC7EE-4325-43BA-905D-329DF142E66E.html

Discussion Regarding Reservations and the new ATS

http://blogs.vmware.com/kb/2012/05/determining-root-cause-for-a-scsi-reservation-conflict-issue.html

How to detect datastores upgraded to VMFS-5

http://kb.vmware.com/selfservice/microsites/search.do?language=en_US&cmd=displayKC&externalId=2003813

Differences between Upgraded and newly created VMFS-5 datastores:

http://blogs.vmware.com/vsphere/2011/07/new-vsphere-50-storage-features-part-1-vmfs-5.html

Atomic Test and Set

http://blogs.vmware.com/vsphere/tag/atomic-test-and-set

 

 

 

Configuring Shares in Resource Pools

As virtual infrastructures grow in size, resource allocation is an important design consideration, as it ensures the organization is getting the most bang for its buck. Proper cluster and resource pool design help ensure both resources and licenses are allocated in the most optimal manner. VMware offers many ways to carve up resources, by clusters based on physical hosts and by resource pools based on assigned values.

Clusters

Some environments divide clusters into multiple tiers, similar to what one would do with storage. Higher tiers would have newer hardware, with faster processors and more RAM. The sizing would be based on a predetermined vCPU to physical core ratio and overall memory consumption, possibly allocating a host for maintenance of failover purposes, the +1 in N+1. Other environments align their clusters by operating system, such as Windows and Linux. This is a cost isolation/reduction model, providing isolation of workloads for host-based operating system licensing, which are available for both Windows and Red Hat Linux. Additionally, this allows the environment to achieve better memory utilization, by taking advantage of transparent page sharing.

Resource Pools

Regardless of the preferred model for creating clusters, resource pools can be created to further divide resources. One approach to further divvy up resources is to create resource pools with a set amount of resources, which is a practical approach if a company has an accounting structure that allows them to charge back for a specific amount of resources allocated.

Another way to take advantage of both cluster designs is to leverage the operating system cluster model, with resource pools inside of the cluster. High, Normal, Low are typical names, based on the default VMware values for shares. This is a good model, if it is maintained. That being said, the default values associated with the resource pools are useful only as a starting point, but the values must be modified depending on what is being placed in each resource pool.

A High share value should be twice that of the Normal share value, and the normal share value should be twice that of the Low share value. This ensures that the VM’s in the High resource pools have twice the priority when accessing resources than the VMs in the Normal resource pool, and the VMs in the Normal resource pool have twice the priority to resources as the VMs in the Low resource pool.

The model is not so much based on the number of virtual machines, but rather the number of virtual cpu’s and RAM assigned inside of the resource pool.

The method for properly calculating the share values assigned to the High, Normal, and Low resource pools are as follows.

  1. Assign an arbitrary number to each share, keeping in mind that the High share value should be twice that of the Normal share value, and the normal share value should be twice that of the Low share value.
    1. Example: High=20, Normal=10, Low=5
  2. Calculate the number of vCPU’s assigned to all of the VM’s in the High resource pool, and multiple by the value assigned in the previous step
    1. Example: 197 times 20 = 3940
  3. Calculate the amount of RAM assigned to all of the VMs in the High resource pool, and multiply that by the value assigned in step 1.
    1. Example: 304072MB times 20 = 6081440
    2. Note: Whether RAM is measured in GB or MB is meaningless, as long as all calculations are done using the same
  4. Repeat Steps 2 and 3 for the Normal and Low resource pools
  5. The results of these calculations will be what will be assigned to each share value in the resource pool.

Here is an example of what the share values would look like in an environment with 322 VMs

Headings

VMs

Value

Total RAM

Total vCPUs

RAM Shares

CPU Shares

Percent RAM of Cluster

Percent CPU of Cluster

Normal

183

10

1331880

522

13318800

5220

62%

50%

Low

107

5

449172

265

2245860

1325

10%

13%

High

32

20

304072

197

6081440

3940

28%

38%

21646100

10485

 

Notice that the value associated with the Normal resource pool is higher than that of the high resource pool. This is because the RAM assigned to the all of the VM’s in the Normal resource pool is 4 times that of the High resource pool. Notice, though, that the share value is only twice as much. Remember, these shares are not in place to say 25% of the resources go to this pool, they guarantee that a value of 10 is assigned to each VM in the resource pool. This ensures that the VM’s in the High resource pools have twice the priority when accessing resources than the VMs in the Normal resource pool, and the VMs in the Normal resource pool have twice the priority to resources as the VMs in the Low resource pool.

One thing to note about this model is that it takes maintenance. There are 3 events that require a re-calculation

  1. Adding a VM to the resource pool
  2. Removing a VM from the resource pool
  3. Changing the CPU or RAM resources assigned to a VM in the resource pool

These are pretty common tasks, so it is recommended to revisit the values on a monthly basis. Exporting lists from vCenter, and performing the calculations with Excel take no more than an hour.

Regardless of the chosen design, it is always beneficial to periodically review the environment to ensure it continues to abide by the design set forth. Many times one will find that the original design becomes outdated. Never be afraid to update the model if it makes sense.

Leveraging Flash in a Virtual Environment

Introduction

Flash is an emerging trend in the storage landscape, as the traditional monolithic array can no longer offer the performance demands of modern workloads. Flash is available three ways:

  • High-cost all flash arrays
  • Hybrid arrays used in replacement of existing storage
  • Easy to add host-based flash

Focusing on the virtual infrastructure, host-based flash seems to be the most cost effective approach. Leveraging host-based flash adds much needed performance, while continuing to leverage the remaining storage capacity of the existing SAN-based storage devices currently deployed.

Host-based Flash

Host-based flash architecture views storage differently, broken into two high level tiers: Data in motion, and data at rest. Data at rest is static or lightly used data, the capacity side of data, which continues to reside on SAN-based storage. Data in motion is the active data sets, residing in the cache locally on the host, while also keeping a copy on the SAN for long term storage and backup purposes. This architecture effectively decouples performance from capacity. The primary benefits of this architecture are increased performance, and extending the useful life of SAN storage devices.

Host-based flash accelerates the data in motion. This done by to keep reads and writes (where applicable) local to the hypervisor host, on high speed, low latency solid state devices.

Components

There are two components to a host based caching infrastructure, those being hardware and software. The host-based hardware provides the cache capacity and performance, while the software provides the mechanism to determine which pieces of data are cached and facilitates compatibility with existing features like vMotion, HA, and DRS. Below are the various hardware and software options available:

Hardware

The hardware component can consist of multiple possibilities:

  • SAS/SATA attached solid state drives
  • PCIe-based solid state storage
  • System RAM

Software

The software component. There are 3 primary types:

  • Static Read Cache
  • Dynamic Read Cache
  • Dynamic Read/Write Cache

Hardware

In order to implement a host-based flash solution, flash hardware must be installed in the server. This section details the hardware options available to be implemented in such a solution.

Tier 1 – SATA/SAS attached SSD Flash

Most software products on the market today support SSD-based flash devices attached via SATA and SAS. Both rack-optimized and blade servers have slots for the disk controllers and drives. The cost for these devices is relatively low, compared to other options. These factors lower the barrier of entry, allowing for flash devices to be easily be added to an existing infrastructure.

Performance on these devices pales in comparison to PCIe devices. This performance hit comes into play with the need to cross two controllers to reach the flash device. The first controller is the SATA/SAS disk controller. The second controller is the controller built into the SSD, converting SATA/SAS to flash.

It is worth noting that SATA/SAS based SSD have a limited write life, and RAID levels can be introduced to ensure flash availability. That being said, the additional protection provided by RAID will increase the load on the SATA/SAS controller, introducing latency, as well as additional cost. Also, RAID 5 and 6 result in a write penalty for the parity calculation and writing that is undesirable in a flash infrastructure. Software designed to provide a flash virtualization platform provide cache redundancy across hosts to ensure a cache is always available, even in the event of losing flash on a single host.

Tier 2 – PCIe-based Flash

All software products on the market today support PCIe based flash devices. Rack optimized servers usually have more PCIe slots than are needed, although in smaller form factor systems, all available slots may be consumed by redundant 10GB Ethernet and 8GB HBA adapters. Additionally, some server vendors partner with flash vendors to provide specially designed PCIe flash devices that will fit into a blade server. HP offers IO Accelerators for BladeSystem c-Class. Since PCIe system run at the speed of the PCIe bus, and has direct access to the CPU without having to cross multiple controller, they offer staggering performance over SSD, which is the name of the game in this architecture.

What is also staggering about PCIe-based flash is the price tag, costing anywhere from 3x to 10x that of SSD. That being said, the devices are also limited by the capacity of the PCIe bus, which means if there are a lot of devices on your PCIe bus, there may be some level of contention. Adding redundancy to PCIe flash devices requires additional devices as well as software to support mirroring the device. Again, this is where flash virtualization platform software comes into play.

Tier 3 – RAM

Some software products on the market support allocating a section of RAM on the host to provide the cache storage. All servers have RAM, although it is a precious and expensive resource. RAM is the fastest and highest performing method of caching data on a host, and also the most expensive. Redundancy is not a major concern, as there are usually in upwards of 16 DIMM slots in a server, with all in use.

Hardware Price and Performance Comparison

The table below details the various options, highlighting both price and performance. One thing to note about the HP vs Micron SSD options is that HP’s 3 year warranty ensures the drives will be placed upon failure for 3 years. The Micron drives are priced such that they can be mirrored or be kept on hand in the event of failure. The remaining information is rather straight-forward, reflecting what the sections above discuss.

Vendor Size

Class

List

Price

B/W

Random Read

Random Writes

Sequential Reads

Sequential Writes

Price/GB

HP 32GB

RAM

$1,399

varies

100k IOPS

unavail

5500 MiB/s

5500 MiB/s

$43

HP 16GB

RAM

$425

varies

100k IOPS

unavail

5500 MiB/s

5500 MiB/s

$27

Micron 175GB

Full PCIe

$3,200

16Gbps

unavail

unavail

unavail

unavail

$18

Micron 350GB

Full PCIe

$6,500

16Gbps

unavail

unavail

unavail

unavail

$19

HP 365GB

in-blade PCIe

$8,500

16Gbps

71k IOPS

32.5K IOPS

860 MiB/s

560 MiB/s

$23

HP 100GB

SSD

$670

6Gbps

63k IOPS

19.2k IOPS

480 MiB/s

185 MiB/s

$6.60

HP 200GB

SSD

$1,400

6Gbps

63k IOPS

32k IOPS

480 MiB/s

350 MiB/s

$7

HP 400GB

SSD

$2,660

6Gbps

63k IOPS

35k IOPS

480 MiB/s

450 MiB/s

$6.60

Micron 120GB

SSD

$192

6Gbps

63k IOPS

23k IOPS

425 MiB/s

200 MiB/s

$1.60

Micron 240GB

SSD

$324

6Gbps

63k IOPS

33k IOPS

425 MiB/s

330 MiB/s

$1.40

Micron 480GB

SSD

$579

6Gbps

63k IOPS

35k IOPS

425 MiB/s

375 MiB/s

$1.20

 

Software

The available options coinciding with the software component of the host-based flash infrastructure come in multiple flavors. The sections below detail the various software options available.

Tier 1 – Static Read Cache

The basic cache is what some refer to as a static read cache. This means that a portion of a locally attached flash device is statically assigned to a specific VM. There is only one product on the market worth mentioning in this realm, detailed below:

Vmware – vFlash Read Cache

VMware’s vFlash Read Cache is included free with vSphere Enterprise Edition, assuming the virtual infrastructure has been upgraded to vSphere 5.5 and the VM is running hardware version 10. It provides an effective method for caching commonly read blocks to the flash device. It fully supports vMotion, including pre-warming of the cache on the destination host prior to the vMotion completing. This avoids the need to pre-populate the flash. It is VMware certified, and supports both block-based and network-attached storage. Upgrading and patching of the hypervisor software is fully supported by VMware. The product is managed from within vCenter, although performance monitoring must be done via ESXTOP.

There are many complexities in configuring the static read cache, and including choosing the correct block size. Choosing the wrong block size can result in worse performance for the VM’s leveraging cache. The largest complexity is the fact that the cache is static. The means that an administrator must plan the flash usage, on a per-VM basis, in advance. This means that the administrator must know the usage pattern of each VM in advance. When dealing with virtual infrastructures that include hundreds of virtual machines, this task is nearly impossible. Also, if additional flash is needed for newly deployed VM’s, and there is no more available flash, additional hardware will need to be procured to provide for the newly deployed VM. This product also lacks write buffering, to accelerate writes. It is not fully clustered, so any time a host goes down, the cache data is lost.

Tier 2 – Dynamic Read Cache

The next level cache is what some refer to as a dynamic read cache, or write back caches. This means that a local flash device is assigned as a pool of flash resources. The dynamic read cache is smart enough to identify usage patterns of VM’s assigned to the cache, growing and shrinking the per-VM cache usage as it sees fit. All products listed support vMotion, DRS, and HA. There are three products on the market worth mentioning in this realm, detailed below:

Proximal Data – AutoCache

Flash indexes in RAM, algorithm to keep index size small. Other products store indexes on the flash device, resulting in an added latency penalty for lookups. Uses Most Recently Used, Most Frequently Used, and proprietary caching. Active real-time feedback to determine which algorithms are performing best at any given time, resulting in more resources to the most affective algorithm that adapts to changes and prolongs life of flash.

List Price of $999 for 0-500GB per host; $1999 for 500GB-1TB; >1TB for $2999 – with flash prices going down over time, customers starting out on the low end will eventually upgrade to the $2999 price point. As with most software vendors, 20% recurring maintenance cost is assumed, although not confirmed.

Fusion-io – ioTurbine

Fusion-io’s ioTurbine product seems to be one of the more well-known dynamic read caching products on the market. This is likely due to the fact that Fusion-io has been selling various PCIe-based flash devices for quite some time. They could be considered a one-stop shop for all of your flash needs, as they also have SAN based flash arrays as caches for storage, operating specific cache software, for both server and workstations.

Little stands out regarding io-Turbine on the feature side, as it seems to match the feature sets of the other two players in this class of software solutions. The caching software can be configured for specific VMDK files, specific VMFS datastores, and specific virtual machines. Caching is done using the Most Recently Used algorithm, which caches only the most recently used data, regardless of the frequency of use.

Much stands out from an architectural standpoint. Limited hardware support, as Fusion-io PCI flash devices are the only supported hardware. Since Fusion-io only makes the expensive PCIe devices, and no SSD, the entry price is very high. The software requires a host driver, and a dedicated virtual machine for management. Additionally, a driver can be installed in the guest for additional file level control, but this approach is not scalable for environment with more than a couple dozen systems on the cache. Documentation is unclear as to whether the host driver operates in user mode or kernel mode.

List Price $3,900 per host, regardless of the size of flash devices in the host. As with most software vendors, 20% recurring maintenance cost is assumed, although not confirmed.

SANDisk – FlashSoft

FlashSoft, which was acquired by SANdisk not long ago, also offers caching software for both Windows, Linux, and for virtual environments. This software uses the Most Frequently Used algorithm, meaning the hottest blocks used by the VMDK’s are cached. This software is configured on a per-VMDK basis only. Supporting both PCI and SSD, the software comes with the capability to track, monitor and predict end of life for SSD devices. That being said, HP provides the same type of tools for its SSD devices.

List Price $3,900 per host, regardless of the size of the flash devices in the host. As with most software vendors, 20% recurring maintenance cost is assumed, although not confirmed.

Tier 3 – Dynamic Read/Write Cache

The top tier technology for host-based flash is what some would call a dynamic read/write cache, although it is really much more than that. This means that the local flash devices across multiple hosts are assigned as a pool

PernixData – Flash Virtualization Platform (FVP)

PernixData provides the only dynamic host-based caching software that can cache both reads and writes. Few software providers have the resources available to develop such software, as the CTO of the firm helped develop the VMFS file system used by Vmware ESX. This inside knowledge has allowed the software to be the most tightly integrated with the vSphere hypervisor and underlying stack. The FVP software is fully clustered, using “Flash Cluster Technology” to allow any host to remotely access the flash devices on any other host. This ensures that data written to cache is fault tolerant in the event of hardware failure. The software can be configured to cache specific virtual machines or specific datastores. Based on customer stories using other caching software, PernixData’s FVP continues to operate problem free when host hypervisor upgrades occur. This is important given that upgrades are release on an annual basis. The caching software will also continue to provide virtual machines access to cache even if a SSD fails. This is done by accessing the fault tolerant copy of the cache from another host. The same occurs in an HA event, if host hardware fails. PernixData is also the only software that can leverage system RAM to act as a cache for storage data.

The only drawback with this software is price. With a list price of $7,500 per host, and a 20 % annual maintenance fee, the entry point can be quite salty. Discounted pricing makes the product less than $3,000 per host, making it less than Fusion-io.

Product Capability Comparison

The table below compares the three tiers of caching software available for host-based flash.

Column1

Vmware vFRC

Fusion-io ioTurbine

Proximal Data AutoCache

PernixData FVP v2.0

Write-Through Caching (Reads)

X

X

X

X

Write-Back Caching (Writes)

NO

NO

NO

X

Dynamically Assigned

NO

X

X

X

Caches to PCI Flash

X

X

X

X

Caches to SAS/SATA SSD

X

NO

X

X

Caches to RAM

NO

NO

NO

X

Pre-Assignment required

X

NO

NO

NO

In-depth knowledge of workloads required

X

NO

NO

NO

Clustered

NO

NO

NO

X

Supports vMotion

X

X

X

X

Supports vMotion Maintaining Cache

NO

X

X

X

Continued acceleration upon failure of SSD

NO

NO

NO

X

Supports DRS

X

X

X

X

Supports HA (cache info is lost)

NO

NO

NO

X

Vmware Certified

X

NO

NO

X

Block Storage

X

X

X

X

NFS

?

X

X

X

Outage Required to add VM

X

X

X

NO

vSphere 5.0 or better required

NO

X

X

X

vSphere 5.5 H/W v10 required

X

X

N/A

N/A

vCenter Plug-in for Configuration

X

NO

X

X

vCenter Plug-in for Performance Mgmt

NO

NO

X

X

Seamless Hypervisor Upgrade

X

NO

NO

X

Licensing

n/a

Per host

Per host

Per host

1st Year pricing per host

Free with vSphere 5.5 EE

$3,900

$999 (<500GB)

$7,500

Recurring cost per host

Free with vSphere 5.5 EE

$780.0

$200

$1,567

 

Product Price Comparison

The table below compares the five caching software products available for host-based flash.

Vendor

Product

License Model

License Cost

Maintenance Cost

Notes

Vmware

vFlash Read Cache

vSphere Feature

n/a

n/a

Assuming vSphere 5.5 EE

Proximal Data

Auto Cache

0-500GB

$999

$200

List Price

Proximal Data

Auto Cache

500GB-1TB

$1,999

$400

List Price

Proximal Data

Auto Cache

>1TB

$2,999

$600

List Price

Fusion-io

ioTurbine

per host

$3,900

$780

List Price

SANDisk

Flashsoft

per host

$3,900

$780

List Price

PernixData

Flash Virtualization Platform 2.0

per host

$7,500

$1,570

List Price

Conclusion

This document provided a comprehensive look at the options, both hardware and software, as well as pricing estimates for the components required to deploy a host-based flash cluster in a virtual infrastructure. PernixData seems offer the best overall capabilities, given the fact that it is fully clustered, the most tightly integrated with vSphere, and provides caching of writes. Solid state drives are the best point of entry for an initial foray into the realm of host-based flash given cost and ease of deployment.

With any new technology, proof of concept is the best place to start. PernixData is willing to provide evaluation software and loaner SSDs to facilitate testing in a proof of concept environment. This proof of concept should be executed on a fraction of production hosts, focusing on problematic virtual machines, to see how the solution performs in the real world.

Pending a successful POC, and full deployment to production an organization can expect to see reduced CPU utilization on the existing SAN storage as well as a lower demand for IOPS on their SAN-attached storage arrays. With virtual machines depending more on flash disk for reads, their performance will improve while also freeing up resources on the SAN-attached storage arrays helping to improve performance for other SAN-based workloads.