cyborgworkshop.org

By default, when you enable iscsi sharing within zfs, the share is created and bound to all available ethernet interfaces. This isn’t necessarily a bad thing, but if for some reason you can reach your iscsi share via two paths, you run the chance of sending iscsi traffic over a non optimized path and really messing with your performance. Fortunately, a way exists to bind iscsi to specific interfaces using interface groups.

First, we need to create the interface group. This is assuming that the IP 192.168.1.1 is the IP address that is assigned to the interface (or in the case of bound channels, multiple interfaces) that you want a specific share to use.

iscsitadm create tpgt
iscsitadm modify tpgt -i 192.168.1.1 1

A quick

iscsitadm list tpgt -v 1

Will let you know if this worked.

Now that your interface group is created, all you have to do is bind it to a specific share.

iscsitadm modify target -p 1 zpool/iscsiTarget

Done! This leaves open some interesting opportunities for using the same iscsi SAN to service connections on different networks in a relatively secure manner. Have fun!

How many of you saw this one coming? In my haste, I accidentally did a zpool destroy -r on the wrong pool this morning and offlined my SAN. Not good. I walked around the entire rest of the day bumming over all of the stuff I’d lost since my last backup to tape because now all of my snapshots were gone. I spent my lunch hour browsing some docs over on sun admin and came across what ended up being my salvation. How to undestroy a destroyed pool. To save you the time, you need to know two commands

zpool import -D

That will show you what pools you have that are still around, but have been deleted.

zpool import -Df tank

That command does the leg work and brings the good old tank pool back online. Hopefully you didn’t lose any drives during this process, but in theory you should be able to recover a degraded pool. Best bet though, don’t try to clean up the SAN at 5:30am before you go to work.

The other night, I moved from a ZFS pool on 3 IDE drives to a pool on 3 SATA drives on the same box. The volumes that I moved though were exported via iscsi to a vmware server, so they had no file system that I could just run a cp on. Instead, I used the versatile ZFS send and receive commands. The source pool is called tank and the target is vault

zfs send tank/Linux | zfs receive vault/Linux

seriously. That’s it. The new volume is created and we’re good to go. Notice that “|”. The pipe character gets any creative thinking geek a little excited because it means we’re working in the realm of STDIN and STDOUT. So let’s say for a minute that my pools were on different machines. I could perform the same function but use ssh to transport the data between servers like so.

zfs send tank/Linux | ssh new-san “zfs receive vault/Linux”

Solaris had seen better days with the release of Solaris 9.  No ground breaking innovations had occurred, the sparc architecture had started to lose it’s place as the data center chip of choice and linux was really kicking it in the teeth with it’s ease of access by the younger sysadmins. An x86 version existed, but it was really just a hobby OS  and no data center in its right mind would deploy it as production.  Things looked bleak, and then came Solaris 10. Solaris 10, and the cool threads/niagra CPUs, helped to put the shine back on Sun. Zones and containers helped to virtualize server hardware, giving a bit more return on investment, but what really did it for the geeks was ZFS.   ZFS is coined “the last word in file systems” and I gotta say, I believe it.  It combines LVM, RAID a journaled atomic file system and manages to increase performance all at the same time. Add to the equation that VMWare recently released ESXi (the bare metal hypervisor that they had been charging 3500 per node for) and you have a really sweet SAN backed virtualization solution in the making.

First things first, install open solaris and immediatly patch it.  You can find instructions on how to do that Here but the condensed version is

pfexec pkg refresh
pfexec pkg image-update
pfexec mount -F zfs rpool/ROOT/opensolaris-2 /mnt
pfexec /mnt/boot/solaris/bin/update_grub -R /mnt

Depending on your internet connection this may take an hour or a few.  The reason for the upgrade is that the shipping version of Open Solaris (2008.5) has a bug with the serial number generation that prevents VMWare from using volumes exported via iscsi.   Once you’ve upgraded solaris, we need to create our pool. We’re going to assume three drives, c0t0d0, c0t0d1 and c0t0d2 and we’re going to put them into a raidz (better look this up, think raid 5 but better)

zpool create tank raidz c0t0d0 c0t0d1 c0t0d2

And you can check your handy work by running

zpool status -v tank

So, we now have a zfs pool called tank that is made of 3 drives we’re going to create a 100 gig volume that we’ll use in the SAN.

zfs create -V 100g tank/iscsi-vol

We now have a 100 gig volume in /tank called iscsi-vol. Next step is to share that bugger out via iscsi

zfs set shareiscsi=on tank/iscsi-vol

and we’re done. you can verify with

iscsitadm list target -v

Now that we have the volume shared out, we need to get access to it with vmware. I’m assuming here that you have a single ESXi 3.5 Update 2 node to play with, so this is assuming a virtual center client to a single ESXi node. This is a pretty simple operation.  In the vmware console, click on configuration and go to networking.  add a vmkernel and then click properties and enable iscsi for that adapter.  Back to the main configuration tab,  click on storage adapters and select properties for the iscsi software adapter.  You’ll need to enable the device and then click on and close the window.  Open that property window again and go to dynamic discovery. Here you’ll add the IP of the Solaris box and then click ok.

Right click on the iscsi adapter and select rescan, this may take a minute.   When it’s done go into storage and click add storage. Looky what shows up in your vmfs storage pools, our new 100 gig volume.

It couldn’t get much easier then this. Assuming that your SAN is on 192.168.168.168 and that your lun doesn’t need CHAP authentication, give these commands a run.

iscsiadm add discovery-address 192.168.168.168:3260
iscsiadm list target
devfsadm -i iscsi

Now you can run format and access that disk,
enjoy!