I’ve been rediscovering the great esxtop tool lately, and I have to say that I’m really impressed with the tremendous amount of information you can get out of this little thing!

But today I will be talking about the (esxtop modes) which I believe not many of us know much about. With a quick search also over the Planet V12N blogosphere I couldn’t find any posts talking about this topic, so there you have it.

We all know about the traditional way of running the esxtop tool through the esx host service console (or resxtop through RCLI). This is in fact called “interactive mode” where you get to see the statistics or information live in front of you and interact with it using deferent sets of keys (m = memory, d = disk and so forth). But this is actually only one of three modes that esxtop can run into. Here are the other two:

• Replay Mode
  

In this mode, you can record and play the esxtop statistics for a specific period of time, and use also the interval of your choice. But before going into details, you may be wondering why would you want to do that? Well, it could be for support purposes, for example if VMware wants to have a snapshot of your performance statistics they may ask you to run this and send them the output. For me, this came in handy when one of my VMs was having a very strange behavior during an early morning hour. This was happening on daily basis and since I was not in favor of staying awake to see what’s the deal with that, I set a small cron job to run/record the esxtop stats for this specific period of time. The next day I played the stats and I was very grateful for the amount of information I got for troubleshooting the problem.

Now, let’s see how we can do this:

First, you will need to be “root” in order to issue the record command as follows:

vm-support -S -i 5 -d 120

We can see here that the interval is 5 seconds, and the duration for recording the statistics is 120 seconds.

The esxtop will start recording the stats and then compress the output into a .tgz file.

We can issue a command to see the .tgz file as shown below. Make sure you are in the right partition when issuing this command and that you have a reasonable disk space if you will run this command for long time.

Next, we need to uncompress the file as follows:
tar -xzf esx*.tgz

Finally, you need to issue the replay command: (thanks to @vRobM for bringing my attention to this missing command)
esxtop -R vm-support*

And there you have it, the statistics will run as if you are sitting at the point of time the stats were recorded in:

One thing to note here though, the time that will show on the top line, will always reflect the current time you are running in, not the time of the recording, so don’t be confused by that.

• Batch mode
  

In the batch mode, you can dump your esxtop stats into a .csv file for later use. You can utilize either Microsoft Excel, or Microsoft perfmon tool to view these data at any time.

Here is the command to achieve this:
esxtop -b -a > output_file.csv

This command will dump “all” the fields from the esxtop to the file. But what if you want specific columns only? Well, that’s the real beauty of the batch mode. You can always choose what information you are interested in, here is how:

• Run the esxtop command in interactive mode.
• In each of the panels, select the columns you want.
• Save this configuration to a file (by default ~/.esxtop4rc) using the W interactive command.

Now you can use the batch command and it will dump only the comlums that you have selected.

What’s next?
Use Excel or perfmon to analyze your data, I personally prefer to use my Vista’s perfmon for this unless you have a specific requirement for using Excel.

• Go to Start -> Run -> type perfmon.
• Go to Performance Monitor section.
• Click of the “View Log Data” in the tool bar at the right as shown in the screen shot below.

   

After that, you need to use the file you’ve dumped the esxtop data to:

You can choose here which columns/counters you want to graph and analyze

And here you see your selected columns graphed in perfmon

And you are done!

Today I received an email from the HyTrust Appliance (HTA) team giving me a great tip about the Community Edition that I blogged about earlier this month. This information will be actually announced officially next Monday, so you get to know about it earlier.

As most of you know, the HTA Community Edition protects up to 3 ESX hosts. However, the configuration assessment capability was left intentionally unrestricted. This means you have the ability to assess the configuration of your entire ESX environment for as many hosts as you want!

Here’s how to configure Community Edition for assessment of your ESX hosts:

1. Log-in to the HTA UI with your web browser, or navigate to the “HyTrust” tab in the Virtual Infrastructure Client (if you have the vCenter Plugin installed.)
2. Select “Hosts” from the “Hosts” menu.
3. Press the “Add” button to add a host.
4. Enter the credentials and details of the ESX host you want to add. Make sure that the “Protected” option is unselected. If you forget, don’t worry! The CE license of the HTA will refuse to add more than 3 protected hosts.
5. Once you’re done adding hosts to the HTA, you can select the check boxes next to the hosts you want to audit, and press “Assess”.
6. You’re done!

Thanks to the HTA team for allowing me to blog about this great trick before the official announcement.

I just ran into a problem now while I was patching one of my ESX 3.5 servers. I got this message from vCenter:

Insufficient disk space on datastore ‘service console’

I SSH’ed into my ESX host and ran the df –ah command which came back with the following:

Apparently I had some big files sitting somewhere on the root partition, and I remembered at once that I was doing some throughput testing from my ESX to the FC SAN and iSCSI targets, so I knew where to go:

And here they are, two iso files consuming 3GB of my precious service console datastore:

Alternatively, you can run the find command with the following syntax to return the file size of your choice, sort them in human readable format and then write them in a text file (here it’s larger than 300MB):

find / -type f -size +300000k -exec ls -lh {} \; 2> /dev/null | awk ‘{ print $NF “: ” $5 }’ | sort -nrk 2,2 > HyperViZor-FileSize.txt

ALWAYS Keep your Service Console FS clean!

Al Pacino – (Scent of a Woman): “It’s a wonderful day for singing a song…it’s a wonderful day for mooooooving a long…HOOOWAAA”

Downloads:
Download Page: https://www.vmware.com/download/
Free 60 days Trial: https://www.vmware.com/tryvmware/index.php?p=vsphere&lp=1

Documentations:
http://www.vmware.com/support/pubs/vs_pubs.html

Third-Party Downloads:
Cisco Nexsus 1000V Trial: http://www.cisco.com/go/1000vdownload

Video Tutorials from VMware:
1 - ESX Installation and Configuration
2 – ESXi Installation and Configuration
3 – VMware vCenter Server
4 – VMware vSphere Client
5 – Networking configuration
6 – Storage configuration (iSCSI)
7 – Create and manage virtual machines
8 – VMware Host Profiles
9 – VMware Storage VMotion
10 – VMware vCenter Server Linked Mode
11 – VMware vNetwork Distributed Switch (vDS)

To be updated with the latest and greatest links…Stay Tuned!

When you find a great blog post that you know from the first glance that it’s going to be big, you really find yourself dragged to refer to it even if you won’t add much.

Eric Gray from VMware, has finally done it! He showed us on his great blog, VCritical.com, how we can run ESX 4.0 as a virtual machine inside a physical ESX host. That’s right, not a VMware Workstation, but a real physical ESX server.

Any regular reader of my blog or a follower on Twitter knows that I’ve been trying to do that since I had my hands on ESX 4.0 starting from the early beta to the RC, but I had no luck doing that. As per Eric, it seems that the new GA build allows you to do this with a little bit configuration tweaks. Eric also mentioned that internally VMware has been running ESX as a VM inside ESX servers with a special build to achieve that.

Which leads me to the question I actually get from people when they know what I want to do: Why would someone need to do something crazy like this?

There are many reasons actually, but here is my personal favorite:
- I want to be able to create a fairly big and complicated ESX labs, and in the same time, I have a fairly huge CPU and memory resources on my existing physical ESX clusters.
- Running ESX on VMware workstation is great, but you are always limited to the amount of memory you’ve got on your PC/Laptop. 4GB or even 8GB won’t really allow you to do much.
- In the Workstation scenario you can’t keep your VMs running all the time, you will always have to shut them down to free up your resources.
- Having your ESX servers as virtual machines gives you extreme flexibility, starting from creating as much NICs/Networks as you want, all the way to creating your own libraries for deferent configurations. Imagine now doing that with VMware Lab Manager and the flexibility you can get for building/sharing these libraries with your peers.

Of course everything has its limitation, and one of the things that I can think of right away, is the fact that you won’t run some features like FT within a virtualized ESX server. I haven’t tried it obviously, but form the restricted CPU requirements for running FT, I don’t think it would work in this scenario.

However, I am as much excited about the GA release as many out there now, where I’ll finally be able to virtualize ESX 4.0. It’s worth mentioning also that, as far as I know, doing the same for other hypervisors is not possible, like Hyper-V or XenServer. So, thanks to Eric for bringing this to our attention, and to VMware for their great flexibility and open mind to allow people to do something great like this.