I mostly talked about the Virtualization Journey as a necessary condition to build a private cloud.

Based on what I leanred from customers, I positioned the journey as the mechanism to provide existing applications with the core characteristic of cloud computing, that is

• Abstraction
• Pooling
• Elasticity
• Management Automation

As I discussed in multiple posts in the past, from a business perspective the journey starts with consolidation and savings, it evolves around business continuity and quality of services and it lands with business agility. From a cloud readiness perspective, it starts with abstration and pooling, then moves to elasticity and management automation.

Here is the video, the panel starts around minute 9

Vittorio

Let’s now double-click on the Business Production phase.

During the IT Production phase customers tend to virtualize Test and Dev servers as well as assets that IT owns, infrastructure services such as file, print, web, domain controller and so on. Customer stay in this loop for a while until one of these three main event happen

1. One, they deliver so much value in the form of cost savings, that somebody more senior in the organization notices and they provides them with the air cover to scale the virtualization effort
2. Two, they run out of things to virtualize and so they have to go and find something else to do. This typically happens around 25-35% virtualization level (unless they are an ISV)
3. Third, they get hit by a big major trigger. For example they buy a new company and they need to integrate the assets, or they are about to have to build a new data center and that pushes them to the next phase.

The next phase is what we call business production and we call it business production because this is when a customer virtualizes their first mission critical application or database. Customers in this stage are in a different league because they need to learn a  whole set of new tricks, they need to learn how to deal with application owners.

This transition is not always easy and the reason why this is a chasm for most customers is that all of a sudden, IT organizations go from virtualizing what they own to virtualizing business applications that are owned by somebody else.

That somebody else is typically an application owner or line of business IT and those guys sit at the top of a pile of concerns of which saving money on servers is not the top priority.

They are concerned with risks, performance, ISV support, uptime, business continuity… and so on and so forth.

We find that as customers transition through this phase successfully, these concerns will go away over time and they get into what we call, “resistance is futile” phase your application will be virtualized and we are not even going to tell you. There are a lot of customers that do this, but it’s a transition. They need to get comfortable, learn how to use features like HA, FT, SRM, and so on to address the line of business owner concerns. Once the information about initial successes gets around, the push back from application owners and database administrators goes away. Sponsorship from the CIO always helps.

The main business drivers for this phase are

• Business continuity
• Better quality of service for business application and databases
• OPEX savings

Capex savings are still a factor but it is like customers at this point take them for granted and move their attention to building a better environment for business applications and databases to run on.

In the next few posts I am going to cover what happens in this phase, what type of applications get virtualized, what are the best practices around it and so on. Stay tuned

Vittorio

• The CIO sits in the same room with the rest of the team (typically less that 10 people overall) in most decisions, so when they get comfortable with the technology they immediately get top level air cover
• The business is not as involved in the technical decisions, so there is less FUD and push back from application owners. When IT is comfortable with the virtualization technology, then they go ahead and quickly virtualize everything that comes their way including business applications and databases
• Their IT scale is smaller (typically 100-500 servers)
  
• It is easier for them to change their processes, thus removing friction and shortening the path to full value realization
  
• There don’t have barriers between network, storage, network and security teams which is one of the biggest obstacle to virtualization
• Because they reach critical mass quickly, they benefit by automating management using features like DRS early in the journey
• For the same reason they can have a robust DR solution across most of the assets very early in their journey

We hear many of them saying “we virtualized the whole environment in 12 months, and we have not bought a physical server since“

Obstacles

One of the biggest obstacles we heard from SMBs  is need to buy pre-tested configurations of server, storage, network and virtualization software. They don’t have the staff and the time to go through lengthy evaluation and testing and because they are not big companies, they don’t get the same level of attention from the vendors.

Quality of Life

interestingly and refreshingly, the  ‘better quality of life’ theme came up very often in out SMBs interviews.

One example if the ability to do disruption-free hardware maintenance thanks to vMotion: “We don’t have to come in during weekends to do hardware maintenance and upgrade anymore, we vMotion the virtual machine to a different server and we do this during working hours without business disruption“.

Another good one is the use od DRS to automatically manage the virtual infrastructure based on quality of service policies: “We used to obsess about whether all my server lights were green, now DRS does that for us“

Vittorio

Virtualization has been recognised as a very effective way to increase server utilization therefore cutting on hardware and maintenance costs. But this is just a part of a bigger story. As we learned from our customers, server consolidation for cost efficiency is just the first step of their virtualization journey which then evolves along two main additional phases that we call Business Production and ITaaS. At each step of the journey, IT addresses different issues that are at the core of most board room agendas:

• Cost control
• Business continuity
• Business agiilty

We are all familiar with the cost saving benefits of virtualization, let’s talk about business  continuity and business agility (see also this post on the topic)

Business Production

Customers move into this stage when they first virtualize a mission critical application or database. When we talk to customers who are in this phase, we hear that the predominant reason why they do this is to achieve better business continuity and quality of service for their business applications. These customers have a higher level of sophistication compared to the ones that are still in the consolidation phase. They still enjoy cost saving from consolidating servers but that is not the driving business proposition enymore. They are now addressing another big board room level concern: business continuity.

ITaaS

This is the more advanced stage of virtualization adoption journey. It is the point of no return where customers have virtualized more than 50% of their systems and virtualization becomes their default computing platform. These customers are clearly on the path to the private cloud. Most of them don’t call it that. They don’t yet refer to this stage as their private cloud but when we break down what we mean by private cloud as in the following diagram

then they nod their head and generally agree that this is what they are doing through their virtualization journey.

Indeed if we agree on the above definition of cloud computing, then you can see how customers are already building the foundation for their cloud even in the early stages by creating the abstraction and pooling layer through virtualization. Then they tackle the service layer and quality of service (Control) in the Business Production phase and finally move into Zero-Touch Infrastructure as they achieve higher level of provisioning, self-service, management and scaling automation in the ITaaS phase.

Much more on topic later.

The increased level of automation achieved in this later stage gives our customers a much more agile IT infrastructure which helps them address two additional concerns of the board room: time to market for business services and applications and the ability to respond faster to changes in business requirements.

Off that Agenda then…

So, based on what we learned from our most advanced customers, there is a strong correlation between their level of adoption of virtualization technology and private cloud computing and their ability to be on the board room agenda for the right reasons. Stay tuned as I get permission to publish the details of some of these customers stories in the near future.

Vittorio

VMware has a huge number of deployed customers that have virtualized all kinds of workloads (a term we use to describe what runs within a virtual machine). As it often happens, customers are using our technology in ways that we may not have anticipated.

Also, some customers have been more aggressive than others about their use of virtualization technology and have virtualized most of their infrastructure, while others have virtualized 1000′s of servers but mostly within the realm of a given type of workload (say web servers).

The Journey

The main question we are trying to answer is: what is the typical virtualization journey?

Is there a typical journey in the first place? Is there one by industry? Company size? Workload type?

In my first month here at VMware, I talked to many smart people (primarily people close to the customers such as in professional service, system engineers, sales etc) and I got some good answers to the above questions. What I want to do now is to go directly to the source and find out from our customers and from their perspective how they achieved high level of virtualization and why.

A Different type of Listening

If you read my previous post on listening to customers, it was inspired by years of working in either a product management or product development capacity. In that context, I was typically meeting customers to validate a new product idea or gather input for a new release of an existing product. This time is different.

I don’t have an idea to validate or requirements to gather. I don’t really have a preconceived notion of what I am going to find out. I am just going out there, listen and learn what our customers have done with VMware technology.

This is not going to be a quantitative type of research. It is going to be qualitative. We want to find out

• When – the project started and finished
• Why – the project got started
• What – was virtualized
• What – products were used
• How – it was done
• Who – drove the project
• Who – helped
• Who – sponsored it
• What – were the technical and business results
• What – processes and organizations changed as a result
• How/When/Why – the project influenced the next one

This last point is critical. We really want to understand the relationship between multiple virtualization projects and how  they come together in an overall journey (when they do).

We selected a number of customers in different geographies and in different stages of virtualization adoption. Cant’ wait to meet them.

Let the learning begin!!

Vittorio

http://viarengo.com has been running smoothly on VMware for more than 10 days now.

CPU utilization is around 10%  which is common in many IT applications out there.

CPU Usage Post Virtualization

I am running two virtual machines on the server now:

• My web site
• A file server.

Unfortunately I don’t really have the need to utilize the available CPU cycles for other applications. My use case is really about encapsulating a legacy application  to be able to run it in the future in its current configuration on any x86 platform. Eventually, I will want to run it in the cloud (more on this later).

A Pleasant Surprise

One pleasant surprise in using VMware Converter, ESXi and the VI Client is their ease of use. The reason I am surprised is that way too often server software is plagued by the “tyranny of the geeks” syndrome where installing, configuring and managing it is done through “Death by Command Line”.

Don’t get me wrong.  If you read Sriram Krishnan’s post on the tyranny of the Geeks, I am NOT against being accurate and strict at the level where VMware software works. Managing hardware resources is not like the Web where the KISS principle rules  and sloppiness is not only acceptable but often useful as described in this great post by Adam Bosworth.

Also, I do get the value of having command line and scripting interfaces for automated management and the ability to integrate with existing management tools.

What I am talking about is the fact that ease of use and management tools are often an afterthought in server-side software. I remember when I worked at Object Design, we had some really amazing engineers (mostly from MIT). They built an amazingly powerful product (Object Store) that stored ANY C++ structure on disk transparently to the C++ developer. Very powerful and elegant.

The problem was that once you stored your C++ objects into the  database, the only way to get at the data was through the C++ application that put it there in the first place. No browsing, query,  reporting, or visual administration tools. This was good for me as it allowed me and my partners to build a startup to fill these gaps but it was bad for early customers trying to build mainstream applications on ObjectStore.

Born to Suffer

The early adopters of any given technology don’t care as much about ease of use. As I sometimes joke about it with colleagues and friends, the early adopters are ‘born to suffer‘. They kind of like it. They feel empowered by making the product work, cracking the code, finding the undocumented command line argument that does the trick.

The rest of us just get frustrated.  Note that this does not apply to most SaaS applications as the lack of usability prevents them from taking off in the first place.

While it is natural for a new technology to be slighlty  hard to use and lack some of the more mainstream tools,  in my experience it requires a different type of product management and engineering culture to take a product to the next level of usability.  But this is not the subject of this post.

The Simple Baker Usability Test

This post is about how refreshing it was to be able to install, configure, run and administer my VMware instance ALL through the UI of the provided management tools (other than the small workaround to make ESXi work on an unsupported disk controller). No complex command line commands, switches, messing with configuration files, etc.

Although my use case is relatively straightforward, I touched many parts of the VMware stack foundation and tools.

• PTV (Physical to Virtual) Conversion – VMware Stand Alone Converter
• ESXi installation – VMware ESXi
• ESXi Configuration – VMware Virtual Infrastructure Client
• VM Deployment – VMware Stand Alone Converter
• VM Creation  – VMware Virtual Infrastructure Client
• Additional Storage Installation – VMware Virtual Infrastructure Client
• VM Performance Monitoring – VMware Virtual Infrastructure Client

It will be intersting to go learn from VMware customers what their experience is in more complex IT scenarios.

No “tyranny of the geeks” so far.

Hardware Upgrade

I am not going to buy a new server. I am using a HP desktop that I have had for 6 years (an interesting compatibility challenge for ESXi).

• Hard Drive: First of all I had to fix my hardware. My hard drive failed, so I went to Fry’s and bought a 500 Gbyte EIDE drive.
• I also found some 512 Mb or Ram laying around in my hardware drawer, so I brought the total memory of my server to 2GB
• I ended up with HP Pentium 4 280Mhz, 500GB EIDE drive, 2Gb memory

Software

Next I had to install ESXi and deploy the VM file that I created with VMware Converter earlier on (before the hard drive failure).

Here are the steps I followed:

• Download VMware ESXi, the free hypervisor from VMware. It is a ISO CD image
• Create a boot CD with ESXi on it
• Install ESXi on the target machine

Here I ran into my first trouble. The ESXi installer gave the following error:

Managing ESXi

• Access http://192.168.1.2 from another desktop
• The web page at this address has links to download location for VMware management tools

ESXi Managemnt Web Page

• Download  Vmware Infrastructure Client
• Connect to the ESXi server

Uploading a Virtual Machine file

VMware Converter Running

Success!!!!

• Log in the running virtual machine
• Change the network configuration from DHCP to 192.168.1.2 so that the VM would be put in the DMZ by my DSL router

Virtual Machine Console

Tada!!!!! Success!

http://Viarengo.com is back online just like it was before my hard drive failure

It is fun to (pretend to) be a Geek

Summary

If you factor out the two hardware compatibility problems I encountered due to the need to reuse old and unsupported hardware, the process was *very* smooth overall.

1. Turn a physical server into a VM file using VMware Converter
2. Install ESXi on the target server
3. Install VMWare Infrastructure Client to manage and configure VMs on the server
4. Deploy the VM file onto ESXi using VMware Converter

In my case the server to be virtualized was also the server that I use to run ESXi. If this was not the case, I could have combined step 1 and 4.

I am now officially virtualized.

What’s next:

• Separate storage from my virtual machine and shrink the size of it.
• Learn how to back up my VM regularly and without service interruptions
• create another virtual machine from scratch on the same server and see how the process works and what is the impact on the existing viarengo.com VM
• learn what kind of management and monitoring tools are included in the VMware Infrastructure client by looking at the running instances on my server.
• create a second ESXi server and start playing with Vmotion and other VMWare Infrastructure capabilities
• see how I can make my web site highly available using virtualization capabilities.

I have two alternatives:

1. Run VMware Server on top of my new 4 core desktop running Vista 64 or
2. Rebuild my existing Pentium 4 based server and install VMware ESXi, the OS-independent VMware hypervisor

Option 1 would allow me to get rid of one physical server and make my household a little greener. VMware ESXi is supposed to have a lower performance hit than VMware Server but given the load on my web server (yes, I do have a big extended family back in Italy but I still get only an average 20 hits a day on my web site…) and th epower of my new desktop processor (Intel i7), I should be just fine.

Option 2 would be a better learning experience as it is a more production-like configuration, I would have to install ESXi from scratch and manage it remotely using VMware management tool. Moreover, later on I could play with vMotion and why not, use VMware technology to make my website highly available (I have a pretty aggressive SLA with “the family” back in Italy )

I have a third option actually. I could run my virtual machine in the cloud using one of VMware’s partners service. This would be a great option actually as it would get me out of the business of procuring, installing and running my own hardware but at this stage I want to eat the dog food directly to learn.

Hardware != Control

Interestingly, as I write this I just realized that I am dealing with the issue of control. Owning your own hardware gives you the illusion of control. The reason I say illusion is that it is more like hardware is controlling you and your budget in multiple ways (see my previous post on hard drive failure). What you really want is control over provisioning of applications and resource allocation. In my case capacity is not really an issue (20 hits a day…) but provisioning in case of a failure is. In real world IT scenarios, the issue of (perceived) control, provisioning, resource planning and balancing are big issues that virtualization can actually address effectively and elegantly. But first things first. Let me first explore the basics of virtualization technology before I drill down into budgeting, organizational and political issues.

I guess I will pursue both option 1 and 2 to learn the most. Then decide which one I will go with.

I will definitely explore options 3 later on as I move from virtualization to cloud computing (and after I cook my way through convincing Ben (who manages our partner program) to get me a discount from one of our hosting partners

First, let’s talk about what runs on the server.

Hardware:

• Intel-Pentium 4 based HP desktop, 2.8Mhz with 1.5 G of RAM and a couple of ~100G ADE Hard Drive.

Software:

• Operating System
  • Windows XP Professional
• Web Application
  • Back-end: XML Database
  • Front-end: IIS + XSL
• Storage
  • Gigabytes of pictures and videos are store on the ADE Hard Drive (for which I have a backup on an staging external drive)

Software Stack

I wrote the application as part of a previous “eat the dog food” effort in 1999. Back then, sharing pictures online was not mainstream so I build my own web app to catalog and publish digital pictures a videos. A flickr wanna-be.

The application has served me (and my family in Italy) well for 10 year but the XML database back-end is a legacy product now and if I had to rebuild the server from scratch due to a hardware failure or upgrade, it would take me a good day (assuming all the legacy software would still be compatible with whatever new hardware I used).

This is a very good use case for virtualization of legacy applications. You virtualize an instance that works and then run it as a virtual machine on any VMware Server (operating system hosted) or ESXi (hypervisor-based, bare metal virtual server).

Physical to Virtual Conversion

First, I downloaded and installed the VMware vCenter Converter Standalone on my server and started the conversion process keeping pretty much all the default parameters.

The conversion progress seemed to get stuck at 4-5%, so I restarted it a couple of times only to realize that it was actually taking longer than I thought (more on this later). I let it run overnight and after around 20 hours I had my Virtual machine file all ready to go.

I then realized that the VM file was more than 100 Gig. In retrospcet I should have separated the multimedia storage from the server image so that the resulting VM file would be smaller and more manageable.

A timely Hardware Failure

Now, believe it or not, when I went back to rebuild the virtual machine without including storage, the unexpected actually happened. I had a hard disk failure and the server was not starting anymore. The usual attempts to restore the operating system failed.

Google Analytics for Viarengo.com

Man, am I glad I created a Virtual Machine of my server… Luck or good “eat the dog food” karma???

In the next post, I will describe the process of running my website as a Virtual Machine.

I have to decide whether I am going to run it on VMware ESXi or VMWare Server.

More later.

Virtual-Earl

VMware also has a new management team that is committed to keep the company growing an innovating. Like most people, I think that the team I work with is the most important factor in a  job selection. I am glad and excited to work with Tod Nielsen and Rick Jackson again. I am also looking forward to working with the rest of the team here at VMware. A company does not become this successful without awesome people.

In my first two weeks here I worked to get up to speed on the core technology, the product line and how customers use it in their IT departments. The feeling that I keep experiencing as I learn more about VMware technology is of incredulity. This virtualization stuff really sounds like magic, but it actually works. Customers *love* the technology and the percentage of VMware customers that use our products in mission critical production environments is staggering.

It reminds me of my experience at BEA, where WebLogic developers were devoted fans of the product, technology and the team behind them.

The reason I decided to blog about this new experience comes from a conversation I just had at lunch with Pratap Subrahmanyam, a VMware veteran who knows more about virtualization than most people.  I was explaining to him that to get up to speed I like to eat the dog food and use the product first hand. I have a web site that I built in 1999 and it runs on a legacy platform that is in maintenance mode today. This is a very good (although relatively basic) use case for virtualization. I could virtualize the whole environment and then run it as a virtual machine on VMware Server (or ESX, more later) for many years in the future without worrying about hardware failures, hardware and software upgrades, and so on. Moreover, I just learned that the carbon footprint of a medium server is equivalent to driving an SUV, therefore it would be nice to consolidate my web server and my multimedia desktop into one and take an SUV off the road.

Anyway, as I was telling Pratap what I am trying to do, he suggested I would blog about it for other people’s benefit. So here it goes.

I am also refreshing my knowledge of operating systems by taking this online course courtesy of Berkeley University 20 or so years after taking a similar course with Joy Marino at the University of Genova, Italy. Oh my, has it been that long??  I just feel like I need to go deeper to better understand and appreciate virtualization technology, be credible with customers and the engineering team at VMware.

Let the virtualization journey begin!