XenGatherer Update

Today the XenGatherer tool got a bit more functionality, an update about this progress will be given first. This will be followed by a description of the issues that still exist. Next week we'll have a look at the overhead that running this tool causes (in particular on a CPU usage level).

The things that were added to/modified on the XenGatherer tool today:

• First of all some small issues that were introduced while coding at home (without being able to run everything) were fixed (e.g. disk columns were not well formatted).
• The overview of the CPU usage was made more readable (see snapshot under the blogpost).
• Dom0 stats were added to the disk section of the resource usage overview. In this overview a MB/s read and write column was added (iostat provided a -m option to facilitate this).
• An option to provide a file containing the vm names to monitor was added, namely "--vmfile". If this optional argument is not provided the application will now parse the 'xm list' output and use all the vms that are currently running.
• After copying the 'vmlinux' kernel file to the node (and rebooting the VMs) the oprofile functionality to monitor the occurrence of cache miss and hit events worked fine. But since the generated output was rather extensive (eg for one domain there was a counter of samples for the kernel, xen, modules,...), we now take the sum of these counters (for each domain) and present these values in the overview. Note that there is still an option that can be used when the full output is preferred: "--nosumcache".

Here the topics that need further attention are listed:

• First of all dom0 stats should still be added to the cache hits/misses section.
• The CPU stats cause problems when the domain id crosses the 32 boundary, the XenBaked code should be optimized such that no problems arise.
• Also, we should have a look whether the LLC_REF events do indicate a cache hit or whether the cache miss events should be subtracted from this number to get the number of cache hits.
• And last but not least, the source code/design should be reviewed and comments should be written.

Xen Monitoring Tool (cache)

The monitoring tool now gathers information about the cache hits and misses as well, using OProfile. Through program arguments it is now possible to choose what will be monitored (e.g. --nodisk triggers the disk information to no longer be included). Also the output is formatted now, such that the columns are alligned and everything is more readable. An example of this output can be downloaded here.

OProfile => XenGatherer

I extended the XenGatherer prototype such that when it is started opcontrol starts gathering information about cache misses (event LLC_MISSES):

• opcontrol --reset
• opcontrol --start-daemon --event=LLC_MISSES:10000 --xen=... --vmlinux=...
• opcontrol --start

When the XenGatherer tool is stopped, opcontrol is stopped as well:

• opcontrol --stop
• opcontrol --shutdown

And when you choose to make a report (throught the XenGatherer CLI) the retrieved information is written to file. The information gathered by OProfile is requested by the following command:

• opreport event:LLC_MISSES

An updated version of the Python source files can be found here.

Patching the Linux Kernel

During this week I managed to get the newer kernel version to work with oprofile: I had to make changes to a couple of source file to get it to compile. Then I tried to make a patch file by using the diff command that checks the differences that exist between nonpatched and patched kernel files. But applying this patch didn't work. So, now I googled how to create a patch file again and found some things here that I probably did wrong:

• When creating the patch, the new kernel was compiled, which explains the fact that I genereated a 200 MB patch when using the root folders of the kernels.


• "Do not ever create a patch with uneven directories" does the site state, well I guess this is what I did.

Next week I'll first try to get this patch working, afterwards I'll start writing a Python application that already gathers one of the parameters (learning the Python language in the mean time).