Now that I have the LAN almost 100% virtual, I thought I would put a little post together on my ESXi 4.1 whitebox. Most of you won’t care, but ESXi whitebox info is kinda hard to come by, so maybe someone on google will find this useful someday. First, the VM Load. I’m currently running 7 VMs.
Boomer ubuntu 10.04 utility box that lives in my DMZ
Hybrid pfSense firewall with 4 physical NICs and 1 virtual attached
GLaDOS OpenFiler NAS with 3 2TB drives setup for raw sata access
Vault ubuntu 10.10 server with 3 1TB drives setup for raw sata access for backups of GLaDOS
Serenity XP VM that I use when I’m oncall
Sleven Windows 7 that I use for nefarious deeds
DC1 Windows 2008 Server setup as a DC to experiment with.
The ESXi server itself (Jane, Ender fan anyone?) is a quad core Xeon X3220 with 16 gigs of DDR3 RAM on a gigabyte motherboard that has an ICH10 chipset allowing for local SATA storage. I’ve had to add in an additional SIL3114 SATA 1 PCI controller to support the additional drives that are attached to the vault. SATA 1 drive performance hasn’t really been an issue since backups are mostly occurring over a WAN link anyhow. I was given an Intel quad port gigabit PCIEx4 adapter, so that’s my primary NIC and then I have a few old Intel dual 10/100 cards installed just in case. The server itself boots off of a USB thumbdrive and everything is stored in a 4U case with 7 Kingwin trayless racks holding the drives in place. Here are the actual parts I’m using
GIGABYTE GA-P43T-ES3G motherboard
Intel X3220 Quad core Xeon CPU
GSkill RipJaws DDR3 1333 4GB DIMMS x 4 (16GB)
Seagate 2TB SATA drive x3
Seagate 1TB SATA Drive x3
Vantec 6 port SATA II 150 PCI SATA card
Intel EXPI9404PTG2L20 quad port ethernet adapter (freebie)
Intel Dual port 10/100 ethernet adapter (junk parts store)
Kingwin SATA hotswap rack x7
generic 4U 7 5.25 bay rack mount enclosure
The whole system was put together piece by piece over time, but if I had to buy it all again, I would anticipate around $900 if you had no parts at all. My CPU load is almost always below 25% and RAM is around 50%. The amazing part though is that this setup has replaced at least 7 physical machines and very likely more around 10 when you figure in the “I want to try out solaris again” whims. I literally have a 7 foot rack in my basement that has a single server in it. Looks kinda sad really. I have only 1 upgrade planned for the year and that is to replace the USB boot drive with an SSD that I have, but otherwise this server is setup to carry my needs for a good long time.
Hehe, I’m one of those guys who was looking for info on ESXi on Google. I’ve also read your guide on raw sata access which is just what I am trying to achieve myself. I myself am an ESXi starter so it’s great to hear how powerfull it is as you’ve replaced so many machines with virtual ones. Thanks
Glad it was helpful. It’s been a champ so far, up to 7 raw drives across three vms without a hitch.
Thanks for posting your setup. I found it cause I was looking to see if a EXPI9404PTG2L20 was NIC I could use for ESXi before I bought it. So your efforts didn’t go to waste and are much appreciated.
Hi man! I´m just curious about power consumption and costs. I´m from Brazil and here we have a very expensive electricity power costs. To give you a exemple if I let powered on a computer to act as a server with a 600W power supply, I need to expend about $50 dollars/month to the power bill increase. Just one computer 24/7. As a read in USA is very common computers powered on 24/7 in houses. How much cost this per month. Just curious.
Thaks a lot your attention.
As it happens, I’ve redone this box a little bit. Here are the new specs
i5 2500k
32 GB of RAM
an Intel 4 port GB network adapter
2 Intel single GB network adapters
2 120GB OCZ SSDS
3 1.5TB harddrives
a 600 Watt 80+ rated power supply
an external RAID 1 array connected via eSATA
an external RAID 1 array connected via USB
on the support side we have
a cable modem
2 Asus wireless routers
an Ooma Telo VOIP box
an 8 port switch
so all of that running at once is consuming 178W according to my UPS. If you take off the external stuff and just worry about the server, it’s 118W. I actually just tested this out yesterday as I was moving equipment around and got to power things up one at a time.
If you wanted to build something that is very low power, I would suggest
an i3 2100T (1/3 of the electrical power of my i5 and still plenty of CPU)
Western Digital Green drives. they are a little slower then regular drives, but they are much more power efficient.
32 GB of RAM, but get the slowest you can get. The higher the clock rate the more power you will burn and with an i3, it’s not worth it.
Intel GB nics are very low power already as long as you are getting the single interface NICs and not the dual or quads.
SSDs are also very low power, but high cost.
I would estimate that you could get all of your power consumption down to around 60 watts if you buy a nice power supply (80+ efficiency rating) and don’t add too many drives. Best of luck to you!