VMware

How to setup ESXi with Fusion IO

Today I installed an ESXi host with an Fusion I/O Accelerator card and I would like to share my experience with you.

Background
This server will be used by VMware Horizon to run VM’s on it.

Hardware
HP ProLiant DL380p Gen8
HP 1.6TB HH/HL Value Endurance (VE) PCIe Workload Accelerator

Documentation, firmware and drivers
http://h20565.www2.hpe.com/hpsc/swd/public/readIndex?sp4ts.oid=7169465&swLangOid=8&swEnvOid=4166

Installation of ESXi
The first step when installing ESXi on a new HP server is to run the HP Proliant Support Pack to update the server with new firmware versions (if applicable).

After updating the server I installed and configured ESXi. Now it was time to add a datastore, but when I tried to add a datastore there was nothing to add. I hoped to found the I/O card and create a datastore on it.

RTFM
Because I was unable to add a datastore on the I/O card I started looking for information and found documentation, firmware and drivers. The “USER GUIDE FOR VMWARE ESXI” was very helpful and explained what to do. This guide explained how to install drivers manually, but because I expect to install much more of these servers I hoped to automate the installation of the drivers.

VUM
I was hoping to add the drivers to VUM, create a baseline and apply the drivers to this (and in the future to others) host. I added the drivers below to the Patch Repository in VUM by using the Import Patches feature:
– libvsl-1.0.0-550-offline-bundle.4.1.2.428.zip
– scsi-iomemory-vsl4-55L-4.1.2.428-offline_bundle-2368400.zip

I used the above drivers because I use ESXi 5.5 U2. If you are using ESXi 5.x (not 5.5) the use these drivers:
– libvsl-1.0.0-550-offline-bundle.4.1.2.428.zip
– scsi-iomemory-vsl4-5X-4.1.2.428-offline_bundle-2366406.zip

vum-drivers

Now it is time to create a baseline. I called mine: HP I/O Driver. The baseline type is Host Extension and I added both drivers to this baseline.
baseline

baselinedrivers

This baseline can now be attached to hosts containing I/O cards and you can start remediating the host(s) to install these drivers.

Firmware
Now that the drivers are installed I was hoping (again) to add a datastore. Still no luck. So I went back to the user guide and noticed the following: With the command fio-status you can check the status of the I/O card. I did this and noticed: Attach status is “Status unknown: Driver is in MINIMAL MODE:”. The user guide explained that this could have something to do with the firmware version. So next step: Update firmware.

On datastore1 (ESXi is installed on a local SSD disk) I created a folder called Bundles and put the firmware file in that folder (ioaccelerator_4.1.2-20141212.fff).

Now I can update the firmware by running the command below from the console:
fio-update-iodrive -d /dev/fct0 /vmfs/volumes/datastore1/Bundles/ioaccelerator_4.1.2-20141212.fff

After 5 or 10 minutes the firmware was installed and I rebooted the server.

After the reboot I was finally able to configure a datastore on the I/O card.

Bandwidth and PCI slot
When running the fio-status command again I now see the I/O card to be online and Attached. I do get a warning message however about the PCI slot. I think this can be solved by moving the I/O card to a different (faster) PCI slot. Maybe I’ll update this post once this is done.
bandwidth

Other settings
The user guide also talks about: Power settings, how to format the I/O card correctly, how to address possible memory issues, Disable CPU Frequency Scaling and Limiting ACPI C-States. So I recommend you to read this guide carefully and consult you manufacturer for these kind of settings. The user guide is from SanDisk (I think HP rebranded it) and not from HP so before adjusting any setting contact you manufacturer. Maybe I’ll update this post once the settings become clear.

~~~UPDATE~~~

HP investigated this issue and gave the following recommendations:

  • Install the IO accelerator in the PCI Slot 5.
  • Adjust power settings in BIOS
    • HP Power Profile – Max Performance
    • HP Power Regulator – HP Static High Performance Mode
    • Advanced Power Management Options – Minimum Processor idle state – No “C” states

After changing the BIOS and moving the card to a different PCI slot the warning disappeared.

Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s