cyborgworkshop.org

ESXi is one of my favorite products on the market and is really excellent for home use. It’s missing out on a few enterprise features like SNMP (I’ll show you how to get around that in a later post) but is otherwise very complete. I had previously been using a separate storage server using iscsi for my vmfs, but the dependency of the two boxes kept me from shutting down either server as needed. So, I converted everything back to local SATA storage (another post) and went on a hunt for how to backup my VMDKs. My iSCSI SAN had dual power supplies, 2 hot standby drives and I could do snapshot based backups whenever I wanted. My ESXi server has two SATA drives, one power supply and no easy way to backup. Enter GhettoVCB. GhettoVCB is a brilliant little shell script that runs on the ESXi server and performs backups of your vms using snapshots that can be stored on local, attached or network storage.  Getting GhettoVCB up and running isn’t difficult at all, you just need to make sure you have a few things in place first.

• A place to store the backups (NFS, iSCSI, Local storage and potentially DAS)
• Decide how many backups you want to have available
• Decide How you want to store the files (thin, thick, etc) This consideration will impact how many backups you have available because of the different space requirements.
• And lastly when you want your backups to run.

I’ve decided to take thin copies of my vmdks at midnight every night and store them on an NFS share with 5 backups available before the older ones get replaced.  I’m not going to try to replicate the excellent instructions that already exist for GhettoVCB,  but here is how I did it.

1. Create an NFS mount on my NFS server with the following options

/mnt/raid5/VMWare       10.0.0.0/24(rw,async,all_squash,anonuid=99,anongid=99)

2. Mount the NFS share in vSphere and call it Backups
3. enable ssh on your ESXi server
4. scp ghettovcb.tar.gz to your ESXi server
5. untar ghettovcb.tar.gz to /usr/ghettovcb
6. edit ghettoVCB.sh and change VM_BACKUP_VOLUME to be VM_BACKUP_VOLUME=/vmfs/volumes/Backups
7. edit DISK_BACKUP_FORMAT to be DISK_BACKUP_FORMAT=thin
8. edit VM_BACKUP_ROTATION_COUNT to VM_BACKUP_ROTATION_COUNT=5
9. edit EMAIL_LOG to EMAIL_LOG=0  unless you want to setup logs to be emailed to you.
10. edit cron by running this command

echo "0 6 * * * /usr/ghettoVCB/ghettoVCB.sh -a > /var/log/ghettoVCB-backup-$(date +\%s).log" >> /var/spool/cron/crontabs/root
kill $(cat /var/run/crond.pid)
busybox crond

11. My cron is set for 6am because my clock is set to UTC, to 6am is midnight for me
12. Wait until midnight for the cron to run or run it by hand yourself.

Logs will be stored in /var/log/  .  I’m backing up 8 VMs in roughly an hour using about 60GB of space (thin provisioned) and have tested restores successfully. Good luck and let me know if you have any issues!

When you are working with VMs, the hierarchy of needs tends to be Memory, Storage, Network, CPU.  You can never have too much RAM when you are talking about a VM server.  Since you usually consume all of your memory first, it’s nice to have a little one liner that lets you know how much memory you have allocated to VMs, and using the VirtualBox command line, this does exactly that. You’ll note that this isn’t my best work, but it’s quick and I’m not worried about all of the forking because we’re only talking about a dozen or so lines at most. Feel free to add tweaks in the comments if you can pretty it up!

VBoxManage list -l runningvms |grep Memory | awk {‘print $3′} | awk -F”MB” {‘print $1′} | echo `sed ‘s/$/+/’` | sed ‘s/+$//g’ | bc

Or this one is a little fancier and takes the total memory installed in the system, subtracts the VM memory in use +512 for the host and gives you the total physical RAM left for VM usage

VM=`VBoxManage list -l runningvms |grep Memory | awk {‘print $3′} | awk -F”MB” {‘print $1′} | echo \`sed ‘s/$/+/’\` | sed ‘s/+$//g’ | bc`; MEM=`grep MemTotal /proc/meminfo | awk {‘print $2′} | sed ‘s/$/ \/ 1024/g’|bc` >/dev/null; echo “$MEM – $VM – 512″ | bc -l

I have a box at home that gets lots of information, but is more of an appliance then most of my machines. I would like it to be able to send events that get displayed to my MythTV frontends, but installing mythtvosd isn’t an option. Fortunately, in this case, MythTV uses a non encrypted XML over UDP scheme to get messages sent to the various frontend nodes. This means I can use netcat in UDP mode to just shoot an xml file out over my network and get the data I want on my TV screen without having to shoehorn mythtvosd onto the appliance.
I’ve created a file called message.xml that has the message that I would like to display on my TV in the format of my choosing. I prefer the scroller, but you can use the CID or alert XML schemas if you so choose. See the main.c source code file for mythtvosd in the contrib directory for the schemas.

<?xml version=”1.0″?>
<mythnotify version=”1″><container name=”news_scroller”><textarea name=”text_scroll”><value>Testing</value></textarea></container></mythnotify>

Then I just use netcat to send that message via UDP to my main frontend. I could broadcast this out to all of my frontend nodes, but prefer to do these messages in a targeted manner.

cat message.xml | nc -uv -q1 serenity 6948

The end result is that I can now get messages on my TV for everything from RSS updates to stock reports to firewall alerts. Anything that can be sent as a text string can be displayed!

Whenever WHIA encounters an error, I have it send me a twitter message letting me know.  Using the twitter API and curl, this is a simple operation.

curl –basic –user “Twitter_Username:Twitter_Password” –data status=”Twitter Message Here” http://twitter.com/statuses/update.xml

replace Twitter_Username with your username and Twitter_Password with your password, keeping the colon “:” as the separator.

I have this perl script called Get_Temperature.pl that has been giving me fits. It simply reads a serial port that gets updated every second and does some regex on those values. The script was slowly consuming more and more CPU over time and creating a huge backlog of data. By the time it had run for an hour, it was at 60% CPU. To help troubleshoot, I wanted to output the amount of CPU that the script was using every time it processed a string, so I used this.

ps -o pcpu,pid,user,args -C Get_Temperature.pl

the -C flag allows me to specify the process name instead of a PID and the output looks like this

%CPU PID USER COMMAND
0.0 1075 jason /usr/bin/perl ./Get_Temperature.pl

Quick, low impact and works.