Virtualization Journey » ESXi

Death by Command Line – NOT

vittorioviarengo — Wed, 29 Apr 2009 02:00:28 +0000

http://viarengo.com has been running smoothly on VMware for more than 10 days now.

CPU utilization is around 10% which is common in many IT applications out there.

CPU Usage Post Virtualization

I am running two virtual machines on the server now:

My web site
A file server.

Unfortunately I don’t really have the need to utilize the available CPU cycles for other applications. My use case is really about encapsulating a legacy application to be able to run it in the future in its current configuration on any x86 platform. Eventually, I will want to run it in the cloud (more on this later).

A Pleasant Surprise

One pleasant surprise in using VMware Converter, ESXi and the VI Client is their ease of use. The reason I am surprised is that way too often server software is plagued by the “tyranny of the geeks” syndrome where installing, configuring and managing it is done through “Death by Command Line”.

Don’t get me wrong. If you read Sriram Krishnan’s post on the tyranny of the Geeks, I am NOT against being accurate and strict at the level where VMware software works. Managing hardware resources is not like the Web where the KISS principle rules and sloppiness is not only acceptable but often useful as described in this great post by Adam Bosworth.

Also, I do get the value of having command line and scripting interfaces for automated management and the ability to integrate with existing management tools.

What I am talking about is the fact that ease of use and management tools are often an afterthought in server-side software. I remember when I worked at Object Design, we had some really amazing engineers (mostly from MIT). They built an amazingly powerful product (Object Store) that stored ANY C++ structure on disk transparently to the C++ developer. Very powerful and elegant.

The problem was that once you stored your C++ objects into the database, the only way to get at the data was through the C++ application that put it there in the first place. No browsing, query, reporting, or visual administration tools. This was good for me as it allowed me and my partners to build a startup to fill these gaps but it was bad for early customers trying to build mainstream applications on ObjectStore.

Born to Suffer

The early adopters of any given technology don’t care as much about ease of use. As I sometimes joke about it with colleagues and friends, the early adopters are ‘born to suffer‘. They kind of like it. They feel empowered by making the product work, cracking the code, finding the undocumented command line argument that does the trick.

The rest of us just get frustrated. Note that this does not apply to most SaaS applications as the lack of usability prevents them from taking off in the first place.

While it is natural for a new technology to be slighlty hard to use and lack some of the more mainstream tools, in my experience it requires a different type of product management and engineering culture to take a product to the next level of usability. But this is not the subject of this post.

The Simple Baker Usability Test

This post is about how refreshing it was to be able to install, configure, run and administer my VMware instance ALL through the UI of the provided management tools (other than the small workaround to make ESXi work on an unsupported disk controller). No complex command line commands, switches, messing with configuration files, etc.

Although my use case is relatively straightforward, I touched many parts of the VMware stack foundation and tools.

PTV (Physical to Virtual) Conversion – VMware Stand Alone Converter
ESXi installation – VMware ESXi
ESXi Configuration – VMware Virtual Infrastructure Client
VM Deployment – VMware Stand Alone Converter
VM Creation – VMware Virtual Infrastructure Client
Additional Storage Installation – VMware Virtual Infrastructure Client
VM Performance Monitoring – VMware Virtual Infrastructure Client

It will be intersting to go learn from VMware customers what their experience is in more complex IT scenarios.

No “tyranny of the geeks” so far.

Viarengo.com on ESXi

vittorioviarengo — Fri, 17 Apr 2009 17:52:59 +0000

My virtualization journey continues.

Quick recap: I created a Virtual Disk File (luckily 3 days before the hard drive of the server died) containing my web application plus all related data. This will turn to be a management challenge later on given the size of the VM (120GB).

My alternatives now are to deploy this Virtual Machine file on

Options 1 – VMware Server (vistualization server hosted on Windows) or
Options 2 – VMware ESXi, the bare metal hypervisor

I decided to pursue Option 2 as it offers more opportunity to learn (and to go to Fry’s and buy some hardware new hardware components, and I can tell the wife it is work related. Yeah!!!).

Later I will explore Options 3 - deploy the virtual machine to a VMware-enabled hosting provider.

The goal today: to run my web application as a virtual machine on VMware ESXi.

Hardware Upgrade

I am not going to buy a new server. I am using a HP desktop that I have had for 6 years (an interesting compatibility challenge for ESXi).

Hard Drive: First of all I had to fix my hardware. My hard drive failed, so I went to Fry’s and bought a 500 Gbyte EIDE drive.
I also found some 512 Mb or Ram laying around in my hardware drawer, so I brought the total memory of my server to 2GB
I ended up with HP Pentium 4 280Mhz, 500GB EIDE drive, 2Gb memory

Software

Next I had to install ESXi and deploy the VM file that I created with VMware Converter earlier on (before the hard drive failure).

Here are the steps I followed:

Download VMware ESXi, the free hypervisor from VMware. It is a ISO CD image
Create a boot CD with ESXi on it
Install ESXi on the target machine

Here I ran into my first trouble. The ESXi installer gave the following error:

“Unable to find a supported device to write the VMware ESX Server 3i 3.5.0 image to“.

EIDE Support Problem

It turns out that ESXi does not support my EIDE disk controller (I don’t think it supports any really, see Hardware Compatibility List here).

After some research I found this great workaround for EIDE controller and I managed to install ESXi just fine.

The workaround required me to use VI 20 years later… I forgot the feeling of using an editor that works like it has a glass in front of the text and you have to press the Ins key to actually edit it.

Oh well… :wq

ESXi installed just fine after this. Note that at this point I am pretty much on my own as I am running a hardware configuration that is not officially supported by VMware. The price to pay for re-using my old piece of hardware.

Reboot
Press F2 to customize the server
Configure the network.

I am planning on assigning a static IP address to this server (your usual 192.168.1.2) and later put it in the DMZ within my home NAT. This probably does not matter as what it really matters for the DMZ is the IP of the VM. I will figure this out later once the VM is up and running.

Network Card Compatibility problem

Here I ran into my second problem. The changes I made to the network configuration did not take. When trying to run the network management diagnostic, I would get an error.

After inspecting the logs, I realized that the network service did not start.

Back to google-ing the user groups for answers.

It turns out that the network card on my motherboard was not supported. After trying few other cards from my hardware drawer without success, I made another trip to Fry’s and bought an Intel-based network card as suggested by the people on the user groups.

After installing the new network card and rebooted, everything worked fine and I could configure my ESXi network parameters just fine.

BTW, here is a useful trick to access the Linux control for ESXi newbies like myself.

Managing ESXi

ESXi is super lightweight. It does not embed high level management tools. So, here is what you do next:

Access http://192.168.1.2 from another desktop
The web page at this address has links to download location for VMware management tools

ESXi Managemnt Web Page

Download Vmware Infrastructure Client
Connect to the ESXi server

After connecting I could see the server and storage.

Uploading a Virtual Machine file

Now all I ad to do was to upload the file containing my web site image that I create earlier with VMware Standalone Converter.

After poking around the VI Client interface to find the “Upload VM File” functionality, I realized that the way to do it was using the Converter itself.

Run VMware Converter
Load the load VM wizard
Select my new ESXi server as target
Finish

VMware Converter Running

Oh man, do i regret having created a VM file that includes all my pictures…. 100Gb work of them. This makes the deployment process kind of slow. It took around 9 hours to deploy it.

Once this is up and running I need to figure out how to separate my storage from my Virtual Machine and shrink the VM file size significantly.

Success!!!!

At the end of the conversion/deployment process, I launched the console within the VMware Infrastructure Client and:

Log in the running virtual machine
Change the network configuration from DHCP to 192.168.1.2 so that the VM would be put in the DMZ by my DSL router

Virtual Machine Console

Tada!!!!! Success!

http://Viarengo.com is back online just like it was before my hard drive failure

It is fun to (pretend to) be a Geek

Man, how fun was this??? I got to mess around with hardware (and go to Fry’s twice), use VI after 20 years, browse all these techies forums, learn a ton and most importantly recover smoothly from a hard drive failure!!

Summary

If you factor out the two hardware compatibility problems I encountered due to the need to reuse old and unsupported hardware, the process was *very* smooth overall.

Turn a physical server into a VM file using VMware Converter
Install ESXi on the target server
Install VMWare Infrastructure Client to manage and configure VMs on the server
Deploy the VM file onto ESXi using VMware Converter

In my case the server to be virtualized was also the server that I use to run ESXi. If this was not the case, I could have combined step 1 and 4.

I am now officially virtualized.

What’s next:

Separate storage from my virtual machine and shrink the size of it.
Learn how to back up my VM regularly and without service interruptions
create another virtual machine from scratch on the same server and see how the process works and what is the impact on the existing viarengo.com VM
learn what kind of management and monitoring tools are included in the VMware Infrastructure client by looking at the running instances on my server.
create a second ESXi server and start playing with Vmotion and other VMWare Infrastructure capabilities
see how I can make my web site highly available using virtualization capabilities.

ESXi or VMWare Server?

vittorioviarengo — Wed, 15 Apr 2009 23:05:25 +0000

Now that I have my Virtual Machine file with 10 years of my life on it (as well as the last semi-production code I have written) I need to deploy it an run it on a virtual server.

I have two alternatives:

Run VMware Server on top of my new 4 core desktop running Vista 64 or
Rebuild my existing Pentium 4 based server and install VMware ESXi, the OS-independent VMware hypervisor

Option 1 would allow me to get rid of one physical server and make my household a little greener. VMware ESXi is supposed to have a lower performance hit than VMware Server but given the load on my web server (yes, I do have a big extended family back in Italy but I still get only an average 20 hits a day on my web site…) and th epower of my new desktop processor (Intel i7), I should be just fine.

Option 2 would be a better learning experience as it is a more production-like configuration, I would have to install ESXi from scratch and manage it remotely using VMware management tool. Moreover, later on I could play with vMotion and why not, use VMware technology to make my website highly available (I have a pretty aggressive SLA with “the family” back in Italy )

I have a third option actually. I could run my virtual machine in the cloud using one of VMware’s partners service. This would be a great option actually as it would get me out of the business of procuring, installing and running my own hardware but at this stage I want to eat the dog food directly to learn.

Hardware != Control

Interestingly, as I write this I just realized that I am dealing with the issue of control. Owning your own hardware gives you the illusion of control. The reason I say illusion is that it is more like hardware is controlling you and your budget in multiple ways (see my previous post on hard drive failure). What you really want is control over provisioning of applications and resource allocation. In my case capacity is not really an issue (20 hits a day…) but provisioning in case of a failure is. In real world IT scenarios, the issue of (perceived) control, provisioning, resource planning and balancing are big issues that virtualization can actually address effectively and elegantly. But first things first. Let me first explore the basics of virtualization technology before I drill down into budgeting, organizational and political issues.

I guess I will pursue both option 1 and 2 to learn the most. Then decide which one I will go with.

I will definitely explore options 3 later on as I move from virtualization to cloud computing (and after I cook my way through convincing Ben (who manages our partner program) to get me a discount from one of our hosting partners