FreeNAS and ESX

In my neverending quest to run a decent ESX lab at home I’ve been messing around with some storage options on the cheap.  Currently I use my iMac to serve up NFS storage to my ESX and ESXi hosts at home.  This works fine but I wanted something a little, well…better!

FreeNAS has been around for a while as well as OpenFiler.  I downloaded and played around with both, and just found FreeNAS to be a little simpler to use.  The nice thing about both packages is 1. They’re FREE! and 2. They can run on standard PC hardware rather efficiently.  I had an AMD64 3200 mobo laying around and threw in 1GB worth of memory sticks I had into a beat up old case.  The neat thing about FreeNAS is that if you can boot from an USB stick there is an embedded version that will consume a quaint 32MB of space and conserves your SATA ports for data disks.

I started off with a 160GB drive and had some storage up in under 30 minutes.  But that wasn’t quite “geeky” enough for me so I picked up 2 x 750GB drives off NewEgg for $120.  Setting up the RAID1 mirroring was a little tricky though.

I’m not going to get into specifics on how to install FreeNAS, there are hundreds of other websites out there to explain that.  But here is how to get thru the RAID1 setup.

First when you log into the WebGUI (default login is admin/freenas) Go under Disks then Management.  Even though FreeNAS can see your disks you have to manually add them.

Disk ManagementClick on the little plus sign (+) and add your disk….for each disk you add, change the “Preformatted File System” to SoftwareRAID

picture-3

Repeat this for each physical disk you have in your machine.  Be sure to click the button (not shown) that says “Apply Changes”  You should have something that looks like this….

picture-13

The next step is to go to Disks, Software RAID.  This is where we configure our RAID setup.  This is where I got a little thrown off.  Click on the little plus sign (+) to create your RAID.  In my case, I have 2 drives so I’ll go with RAID1 so click on the RAID1 tab.

picture-8

picture-14

Give your RAID array a name then ctrl-click both drives.  IMPORTANT – be sure to click the option to Create and initialize RAID, otherwise you go nuts trying to figure out the next step!

Now go to Disk, Format.  This threw me off because I thought my drives were already formatted, however now you need to format the RAID array, not the disks that “make up” the array.  FreeNAS is built on FreeBSD and UFS is the default file system so use that.  Don’t worry, NTFS and Apple-Journaled work just fine with UFS and will do some VMFS later.  Simply pick your newly created RAID under the Disk drop down and give the volume a name.  You should see a ton of text generated while the disk is formatted.

picture-9
(LOL…in this example, the Volume Name I used was invalid…can’t have an underscore in there!)

Finally the last step is to create a mount point so you can actually start using the storage.  Click on Disk, Mount Point.

picture-15

Type should be Disk, the Disk should be your RAID array you created (properly formatted hopefully!)  You need to specify a Mount Point, you need to remember this for when you enable some services, like CIFS.  Again, be sure you click “Apply Changes” after you click Save.
So cool!  Now we have some storage to use.  Lets test it by creating a CIFS share.  Click on Services, CIFS/SMB.  There is a bunch of configuration on the opening page, but the most important thing is clicking the little Enable checkbox on the right.  You can change the name, authentication, etc as you wish.

picture-16

Next click on the Shares tab.  Give your Share a name, comment and you need to specify a Path.  Click the browse button and another browser window should open.  Here is where you specify the Mount Point you created earlier

picture-18

picture-17

Lets test it out to see if we can access the share.  From my XP machine, I’ll just browse My Network Places, or from Start, Run go to \IP of FreeNas

picture-19 While we’re here, lets create a new folder called “nfs” and an empty file called “iscsi

picture-20

So everything seems to be working just fine…Lets move on to VMWare.  First, lets try using NFS.  Creating an NFS share is similar to creating a CIFS share.  Click on Services, NFS.  Again, enable the service by clicking the same box on the right.

picture-21

Next Click on the Shares tab and click the little plus sign (+)

picture-22

Here we need to specify the path again…this is why I created that “nfs” folder earlier.  This way all my VM data doesn’t get lumped together with my CIFS share data.  You also need to specify a network that can access the NFS share.  In this case, I’m allowing any server on my network.  Realistically this would present a problem in the enterprise, but this is for a home lab.  I’m sure there are some settings in FreeNAS to lock this down further, I’m just scratching the surface here.

If we go to our ESX server, and add storage, we can map to our NFS share using the Add Storage Wizard in VI Client

picture-24

You can see the 675 GB of RAID1 goodness!

picture-25

What about iSCSI?  Sure we can do that too!  Click on Service, and iSCSI Target.  Again we need to enable the service first by clicking the little box on the right and applying our changes.  Next we need to click on the little plus (+) sign next to Extents and create a new one.

picture-26

picture-27

First the Extent needs a name, the default one will work just fine.  However we’re going to change the Type to File.  Remember that “iscsi” file we created before?  We’re going to point to that file.  The next thing is to give it an actual size.  In this example we’re going to make it 150GB.

The next thing we need to do is on the Services, iSCSI Target main menu we need to create a target.  This target initializes FreeNAS’s iSCSI target and tells it to use the Extent we just created.  You noticed after you add it, the “Name” will be the FreeNAS initiator name…

picture-28

picture-29

Click “Save and Restart” and we’re ready to head back to the ESX server!  I’m not going to get into detail on how to configure iSCSI on ESX.  But real quick, the first thing we need to do is add the target of our FreeNAS box in ESX.  Under your Configuration tab, click on Storage Adapters, then highlight your iSCSI Software Adapter (if you don’t see it, your missing a VMKernel port) and click Properties.

picture-32

Once you’ve enabled iSCSI if you haven’t already, click on the Dynamic Discovery Tab to add the IP address of the FreeNAS box.

picture-33

Now go back to your Configuration tab under Storage Adapters and click Rescan….

picture-34

Shazam!!!  Now you should see your iSCSI target listed under Details

picture-35

The final step for iSCSI is to go under the Configuration tab and click on Storage and run the Add Storage Wizard.  Except this time, unlike NFS, we’re going to select Disk/LUN and use our new iSCSI target

picture-36

Viola!  iSCSI Storage in ESX in my basement!

picture-37

I hope you found this info useful.  I was looking at the new IoMega StorCenter’s that are certified with VMware for storage, but the reviews were terrible.  For half the price of a StorCenter and with some regular hardware I had laying around I was able to get more storage (750GB vs. 500GB) and iSCSI.  I’m noticing that my screenshots aren’t coming up 100% perfectly, I’ll fix those but in the meantime if you get stuck, drop a post and I’ll try to help!

This entry was posted in Storage, VDI, VMware and tagged , , , . Bookmark the permalink.

17 Responses to FreeNAS and ESX

  1. Pingback: FreeNAS with ESX via NFS or iSCSI with a dash of RAID 1 « Network Attached Storage News

  2. Really cool. Thanks for sharing this. It’s exactly what I was looking to do.

  3. VMdoug says:

    Good post. Have you heard about Starwind? They have a free iSCSI SAN solution (runs on Windows) that supports up to 2TB of data. http://www.starwindsoftware.com/free

  4. tony says:

    @VMDoug – I follow you on Twitter 🙂 Yes, I have. Its similar to DataCore’s SANMelody which is also pretty cool (but costs $) I wanted to stick with something open-source and I really like the embedded feature of FreeNAS.

    Thanks for the reply!

  5. Josef says:

    Yes thanks alot for sharing, worked for me too!

    One question…
    In this example, you used a file for the ISCSI target in FreeNas which does work.
    If I try use a whole disk as the target, Vmware complains about not being able to get partition information.

    Has anyone been able to do it this way? Please let me know.
    Thanks.

    • tony says:

      @Josef – Thanks for the comment! The setup is pretty much the same except instead of picking a file for the extent, it’s the device. What format is the disk in FreeNAS? Unformatted or UFS?

      One other thing, and I haven’t played around with this confirm this configuration, but when you’re using a file, you can stripe across disks like RAID1…not sure if you can do it without a hardware based RAID controller

  6. Phase Dave says:

    Hey Tony,

    awesome post…I have two 120gb drives software raided in freenas. I got the nfs datastore created in esxi 4. I am having trouble getting the iscsi data store created. When I attempt to create the datastore in vmware to connect to the freenas iscsi target, I get the following error:

    “…Unable to create Filesystem…”

    I created an empty file using fsutil and specified in freenas to make it 50gb. The file is indeed 50gb now.

    any suggestions?

    Similar issue found here:

    http://communities.vmware.com/thread/218982

    • tony says:

      Do a chmod 777 to the iSCSI file you created….I had the same error and it seemed like ESX didn’t have write permission to create the VMFS file system.

  7. Saqib says:

    You Da man :))

  8. Ahmed says:

    Yes thanks alot for sharing, worked for me too!
    It helps alot..
    rocks

  9. Sean says:

    Thanks for this tony. Busy setting up my ESX server on an old Poweredge 2450 once again, already got the NAS share sorted thanks to your post 🙂

  10. Ryan says:

    Hi !

    Great Post !

    Will be setting up the following in the comming week :

    2x VM HOSTS ( ESXI 4)
    – OS to run off 1 GB memory card per Host
    – Hardware : 2x HP Proliant ML 110 G5 servers

    1x Vcenter server

    1x storage box ( FreeNAS )
    – OS to run off 128mb memory card
    – 2 x 250GB sataII drives ( Taken from the proliant servers [ ESXi Hosts ] )

    Ill start building the Freenas box tonight . Hopefully all goes well 🙂

  11. Steve says:

    I had FreeNas working for a few days. then I rebooted everything and now this is the error

    Create NAS datastore192.168.1.2
    Error during the configurati-on of the host: NFS Error: Unable to Mount filesystem:
    Unable to
    connect to
    NFS server

  12. Jeff Campitelli says:

    Hi Tony,
    I just setup up 2-500GB Raid 1 set and initializing is taking forever. It has been over 4 hrs. Should I expect a long init time.

    Thanks,
    Jeff Campitelli

  13. John says:

    Thanks for the howto. Although this sheds some light as to why I’m having difficulties with my setup, I’m slightly confused by the “container” approach illustrated above in creating a samba (CIFS) share and then placing an NFS folder and iscsi target file inside of it. Does it mean that samba users would have automatic access to the other share types within or will NFS and iscsi inherit to the filesize limitations of samba? Which types of share work best at the container level? The docs online don’t go into much detail about mixing share types from a single disk.

    Thanks,
    John

  14. rick says:

    One Question –

    Is it possible to create a mount point smaller than the drive size?

    In my case, I have a 5TB storage device ready to mount to ESX using iSCSI and I like to create mount point smaller than 2TB.

    Thanks if you know the answer.

    Rick

  15. tony says:

    @rick Sorry for the delay!

    Yeah, when you create the extent for iSCSI, the size you specify is the size of the mount point ESX sees. Is that what you mean?

Leave a Reply

Your email address will not be published. Required fields are marked *