cyborgworkshop.org

Updated for ESXi 4
If you have a linux box and a 1 gig thumb drive, here is how to install VMWare ESXi onto that drive in 2 commands.    Download the ESXi ISO and then mount it in linux like so

mount -t iso9660 -o loop VMware-VMvisor-Installer-4.0.0-171294.x86_64.iso /mnt/esx

Now we’re going to write the installer file to the thumbdrive. I’m assuming the thumbdrive is /dev/sdb

tar xvzf /mnt/esx/image.tgz usr/lib/vmware/installer/VMware-VMvisor-big-171294-x86_64.dd.bz2 -O | bzip2 -d -c | dd of=/dev/sdb

This will take a little bit, but once it’s done you’re good to go. pop that drive into a USB boot capable board, configure your storage and away you go!

I enabled ssh on my ESXi server to help me troubleshoot what appears to be a hardware problem.  Hardware problems can be a real pain to troubleshoot and since this one involved the USB storage that was encapsulating my ESXi install, sometimes rebooting when it was broken didn’t really work. It would just hang and I would have to power cycle the server.  Turns out that was an issue with ESXi trying to be responsible and syncing the filesystems  before it goes into reboot and, when the Hypervisor1 and Hypervisor2 fileystems aren’t available, this turns into a dead lock.  Not to fear! simply ssh into the server and, without using tab completion, type this entire command into the console and hit enter.

reboot -f -n

That tells ESXi to reboot without doing a sync/flush to disk and without going through init. Essentially pulling the power cord. It’s drastic and it can cause no shortage of issues on your VMs filesystems, but when you are remote and can’t reach the cord this is a good option to have!

ESXi 3.5 only allows 4 virtual NICs to be assigned to a virtual machine. For many uses, that’s just fine, but my latest overly complex home network requires for 5 wireless network, 3 wired networks, 2 storage networks and a public hotspot type network. That’s no less then 11 network interfaces. Fortunately, I have the hardware to handle 802.1q vlans. Unfortunately, vmware doesn’t seem to like passing 802.1q headers through the vmnics and into the outside world. I spent a fair amount of time with a sniffer trying to figure this situation out and came across a white paper that saved my bacon. When you create a virtual interface in the VI Client, you have to set it to use vlan 4095 if you plan on using it as a pass-thru trunk interface. If you don’t, none shall pass. How frustrating is that?
Hey VMWare, consider this a feature request. How about a flipping check box labeled “Pass-Thru Interface” instead of a white paper buried on under the interwebs?

update The latest ESXi configuration guide mentions this on page 30 http://www.vmware.com/pdf/vi3_35/esx_3i_e/r35/vi3_35_25_3i_server_config.pdf

In a never ending goal to have enterprise level stuff in my humble home, I have put this short guide together on how to assemble the cheapest VMWare ESXi server possible.  There are a few guidelines to keep in mind.

First, ESXi will run using SATA drives in AHCI mode connected directly to a motherboard that uses an Intel ICH7 or ICH9 chipset, but NOT an ICH8 or ICH10.

VMWare will use as much RAM as you can give it. RAM is more important then CPU.

Having said that, a multicore CPU is really helpful. ecspecially if you plan on using the software iSCSI initiator.

Use an Intel Pro 100 or Pro 1000 NIC. NIC support is pretty bare in ESXi, so just hop on newegg and spend 20 bucks on one.

We have two options, the first and cheapest is to use a motherboard based on the ICH7 or ICH9 chipset. ESXi will support SATA drives in AHCI mode connected to this chipset,so an ICH7/9 board, a few gig of RAM, a SATA drive and an Intel NIC will have you on your feet.  If you have this kind of setup, all you’re going to need to do is boot off of the ESXi installer CD and you’ll be done a short time later.

The second option is a bit more involved, but will work on ICH8 and ICH10 motherboards. I personally run an ICH10 based board with a quad core and 8 gig of RAM using this method.  This method also requires iSCSI storage to be available. I use Solaris and ZFS for my SAN. So, if you have an ICH8 or ICH10 motherboard, you need to create a USB bootable thumbdrive and install ESXi onto that. You can follow these simple instructions for linux or you can google for instructions for windows. Once you have your boot drive created, you will need to give your host an IP and use the management console to setup iscsi storage.  Have fun!

I really only have 1 lun that is important to me on my home san. It stores a VM running ubuntu that I remote into using NoMachine on a fairly constant basis from all over the place. When I upgraded the SAN and did a zfs send and receive from one SAN to another, I neglected to think about the iscsi IDs getting changed between SANs thus screwing up my ESXi machine. I could reimport the LUNs easy enough, but when you import a LUN into an ESX node, the first thing it does is format it! Not good. So after digging around, I found a way to tell ESXi to simply scan the SAN for available LUNs and to allow access to VMFS that might be on them.
In your VI client, do the following.

• Select the Configuration tab
• Select Advanced Settings
• Select LVM in the left pane
• Set LVM.EnableResignature to 1 and hit OK
• Go back to Configuration
• Click Storage Adapters
• Right click on your iscsi adapter and select rescan. This will take a minute or two.
• In storage, you’ll now see a bunch of LUNs labeled snapshot-whatever_you_called_the_LUN
• Rename the snapshots to just be whatever_you_called_the_LUN and go along your merry way

Crisis averted!