Going to have to try eclim


MacVim icon, glossy style
Image via Wikipedia

For my MSc project I made heavy use of the Eclipse IDE to write various Groovy programs that took an XML input and output an SVG (of course SVG is XML also, but I hope you understand).

Groovy was a great choice as, while not as fast as C, for instance, it was easy to write something that could hack XML and SVG – all I had to worry about was the algorithm as much of the infrastructure for handling the file formats was to hand.

And Eclipse made perfect sense as the IDE as it had good Groovy support.

But my problem was I am a VIM user most of the time and so there was more than one time when I had to go back and clean up the :w mess I had left behind.

Now, it seems, there may be a solution to hand – eclim – which allows me to use VIM in Eclipse and vice versa. I will try it in the next few days and see how I get on.

Fixing the problem with XSLT 2.0


Wot, no namespaces?
Image by psd via Flickr

I asked about my XSLT issues on the Gnome XSLT mailing list and got this reply, which neatly summarises where I was going wrong… (though the being ‘shouted at’ part tells you a lot about the dysfunctional aspects of a lot of online software development, but I am used to that by now)

The name of the element is a complex object with a
namespace and a local name.

With <svg width=”1000px” height=”800px” version=”1.1″
xmlns=”http://www.w3.org/2000/svg“>, the element’s name is
{http://www.w3.org/2000/svg}svg.  With <svg width=”1000px”
height=”800px” version=”1.1″>, the element’s name is {}svg.  These are
NOT THE SAME, and XSLT that works to transform one will not transform
the other.  You may as well ask why changing <svg> to <gvs> does not work.

There is a neat answer to this, specifying xpath-default-namespace="http://www.w3.org/2000/svg" in the xsl tag at the start of the stylesheet (I have to lose the non-standard indent attribute too) – giving this:

<?xml version="1.0"?>
<xsl:stylesheet version="2.0"
xmlns:xsl="http://www.w3.org/1999/XSL/Transform"
xpath-default-namespace="http://www.w3.org/2000/svg">
<xsl:param name="colour">yellow</xsl:param>
<xsl:template match="/">
	<xsl:apply-templates select="svg"/>
</xsl:template>
<xsl:template match="svg">
	<xsl:copy>
		<xsl:for-each select="@*">
			<xsl:copy/>
		</xsl:for-each>
		<xsl:text>
</xsl:text>
		<xsl:apply-templates select="rect"/>
		<xsl:apply-templates select="line"/>
		<xsl:apply-templates select="text"/>
		<xsl:apply-templates select="circle"/>
	</xsl:copy>
</xsl:template>
<xsl:template match="line">
	<xsl:copy>
		<xsl:for-each select="@*">
			<xsl:copy/>
		</xsl:for-each>
	</xsl:copy>
<xsl:text>
</xsl:text>
</xsl:template>
<xsl:template match="rect">
	<xsl:copy>
		<xsl:for-each select="@*">
			<xsl:copy/>
		</xsl:for-each>
	</xsl:copy>
<xsl:text>
</xsl:text>
</xsl:template>
<xsl:template match="text">
	<xsl:copy>
		<xsl:for-each select="@*|node()">
			<xsl:copy/>
		</xsl:for-each>
	</xsl:copy>
<xsl:text>
</xsl:text>
</xsl:template>
<xsl:template match="circle">
	<xsl:if test="@stroke=$colour">
		<xsl:copy>
			<xsl:for-each select="@*">
				<xsl:copy/>
			</xsl:for-each>
		</xsl:copy>
		<xsl:text>
</xsl:text>
	</xsl:if>
</xsl:template>
</xsl:stylesheet>

This is only supported in XSLT 2.0, an accepted standard, but more or less orphaned by the big software houses and, in particular, the browser builders. But help is at hand in the form of the Saxonb XSL processor, so a command line like this:

saxonb-xslt -xsl:pickcolour.xsl -s:memmap.svg -o:test.svg colour=green does the job.

XSL problem solved (sort of)


I half remembered doing something like this before … and it works: though I don’t know if this is a bug in xsltproc or what…

To get the stylesheet to work I have to delete the namespace declaration in the svg tag in the graph – ie change

<svg width="1000px" height="800px" version="1.1" xmlns="http://www.w3.org/2000/svg">

to this:

<svg width="1000px" height="800px" version="1.1">

and then run xsltproc with --novalid specified.

Will enquire further on the appropriate mailing list.

XSL problem


W3c's SVG logo
Image via Wikipedia

A while back I wrote some XSL to manipulate the SVG files my Groovy programs were outputting.

Now either I did something fancy back then that I have forgotten or else some piece of code has broken in the last couple of months (or maybe become less tolerant of my bad XML?) – because it doesn’t work now.

The XSL reads:

<?xml version="1.0"?>
<xsl:stylesheet version="1.0" indent="yes"
xmlns:xsl="http://www.w3.org/1999/XSL/Transform">
<xsl:param name="colour">yellow</xsl:param>
<xsl:template match="/">
	<xsl:apply-templates select="svg"/>
</xsl:template>
<xsl:template match="svg">
	<xsl:copy>
		<xsl:for-each select="@*">
			<xsl:copy/>
		</xsl:for-each>
		<xsl:text>
</xsl:text>
		<xsl:apply-templates select="rect"/>
		<xsl:apply-templates select="line"/>
		<xsl:apply-templates select="text"/>
		<xsl:apply-templates select="circle"/>
	</xsl:copy>
</xsl:template>
<xsl:template match="line">
	<xsl:copy>
		<xsl:for-each select="@*">
			<xsl:copy/>
		</xsl:for-each>
	</xsl:copy>
<xsl:text>
</xsl:text>
</xsl:template>
<xsl:template match="rect">
	<xsl:copy>
		<xsl:for-each select="@*">
			<xsl:copy/>
		</xsl:for-each>
	</xsl:copy>
<xsl:text>
</xsl:text>
</xsl:template>
<xsl:template match="text">
	<xsl:copy>
		<xsl:for-each select="@*|node()">
			<xsl:copy/>
		</xsl:for-each>
	</xsl:copy>
<xsl:text>
</xsl:text>
</xsl:template>
<xsl:template match="circle">
	<xsl:if test="@stroke=$colour">
		<xsl:copy>
			<xsl:for-each select="@*">
				<xsl:copy/>
			</xsl:for-each>
		</xsl:copy>
		<xsl:text>
</xsl:text>
	</xsl:if>
</xsl:template>
</xsl:stylesheet>

While the SVG (rather a small fraction of it) reads:

<?xml version="1.0" encoding="UTF-8" standalone="no"?>
<!DOCTYPE svg PUBLIC "-//W3C//DTD SVG 1.1//EN" "http://www.w3.org/Graphics/SVG/1.1/DTD/svg11.dtd">
<svg width="1000px" height="800px" version="1.1" xmlns="http://www.w3.org/2000/svg">
<rect x='0' y='0' width='1000' height='800' fill='white' />
<text x='100' y='800' style='font-family: Helvetica; font-size: 10; fill: black'>Page size: 4096: 0 to 2% memory</text>
<line x1='90' y1='705' x2='905' y2='705' stroke='black' stroke-width='10' />
<line x1='95' y1='705' x2='95' y2='100' stroke='black' stroke-width='10' />
<line x1='100' y1='715' x2='100' y2='100' stroke='lightgrey' stroke-width='1' />
<text x='95' y='720' style='font-family: Helvetica; font-size:10; fill: maroon'>0</text>
<line x1='80' y1='100' x2='900' y2='100' stroke='lightgrey' stroke-width='1' />
<text x='30' y='105' style='font-family: Helvetica; font-size:10; fill: maroon'>7bb0</text>
<line x1='300' y1='715' x2='300' y2='100' stroke='lightgrey' stroke-width='1' />
<text x='295' y='720' style='font-family: Helvetica; font-size:10; fill: maroon'>75341312</text>
<line x1='80' y1='250' x2='900' y2='250' stroke='lightgrey' stroke-width='1' />
<text x='30' y='255' style='font-family: Helvetica; font-size:10; fill: maroon'>6cc4</text>
<line x1='500' y1='715' x2='500' y2='100' stroke='lightgrey' stroke-width='1' />
<text x='495' y='720' style='font-family: Helvetica; font-size:10; fill: maroon'>150682624</text>
<line x1='80' y1='400' x2='900' y2='400' stroke='lightgrey' stroke-width='1' />
<text x='30' y='405' style='font-family: Helvetica; font-size:10; fill: maroon'>5dd8</text>
<line x1='700' y1='715' x2='700' y2='100' stroke='lightgrey' stroke-width='1' />
<text x='695' y='720' style='font-family: Helvetica; font-size:10; fill: maroon'>226023936</text>
<line x1='80' y1='550' x2='900' y2='550' stroke='lightgrey' stroke-width='1' />
<text x='30' y='555' style='font-family: Helvetica; font-size:10; fill: maroon'>4eec</text>
<line x1='900' y1='715' x2='900' y2='100' stroke='lightgrey' stroke-width='1' />
<text x='895' y='720' style='font-family: Helvetica; font-size:10; fill: maroon'>301365249</text>
<line x1='80' y1='700' x2='900' y2='700' stroke='lightgrey' stroke-width='1' />
<text x='30' y='705' style='font-family: Helvetica; font-size:10; fill: maroon'>4000</text>
<text x='25' y='300' transform='rotate(90, 25, 300)' style='font-family: Helvetica; font-size:10; fill:red'>PAGES</text>
<text x='100' y='750.0' style='font-family:Helvetica; font-size:10; fill:red'>INSTRUCTIONS (376706 per pixel)</text>
<text x='800' y='90' style='font-family:Helvetica; font-size:10; fill: black'>html</text>
<rect x='910' y='120' width='5' height='5' fill='red' stroke='black' stroke-width='1' />
<text x='920' y='125' style='font-family:Helvetica; font-size:10; fill:black'>Instructions</text>
<rect x='910' y='150' width='5' height='5' fill='green' stroke='black' stroke-width='1' />
<text x='920' y='155' style='font-family:Helvetica; font-size:10; fill:black'>Modify</text>
<rect x='910' y='180' width='5' height='5' fill='blue' stroke='black' stroke-width='1' />
<text x='920' y='185' style='font-family:Helvetica; font-size:10; fill:black'>Load</text>
<rect x='910' y='210' width='5' height='5' fill='yellow' stroke='black' stroke-width='1' />
<text x='920' y='215' style='font-family:Helvetica; font-size:10; fill:black'>Store</text>
<circle cx='100' cy='700' r='1' fill='none' stroke='red' stroke-width='1' />
<circle cx='100' cy='699' r='1' fill='none' stroke='red' stroke-width='1' />
<circle cx='101' cy='699' r='1' fill='none' stroke='red' stroke-width='1' />
<circle cx='102' cy='699' r='1' fill='none' stroke='red' stroke-width='1' />
<circle cx='103' cy='699' r='1' fill='none' stroke='red' stroke-width='1' />
<circle cx='104' cy='699' r='1' fill='none' stroke='red' stroke-width='1' />
<circle cx='105' cy='699' r='1' fill='none' stroke='red' stroke-width='1' />
<circle cx='106' cy='699' r='1' fill='none' stroke='red' stroke-width='1' />
<circle cx='107' cy='699' r='1' fill='none' stroke='red' stroke-width='1' />
<circle cx='108' cy='699' r='1' fill='none' stroke='red' stroke-width='1' />
<circle cx='109' cy='699' r='1' fill='none' stroke='red' stroke-width='1' />
<circle cx='110' cy='699' r='1' fill='none' stroke='red' stroke-width='1' />
<circle cx='111' cy='699' r='1' fill='none' stroke='red' stroke-width='1' />
<circle cx='112' cy='699' r='1' fill='none' stroke='red' stroke-width='1' />
<circle cx='113' cy='699' r='1' fill='none' stroke='red' stroke-width='1' />
<circle cx='113' cy='700' r='1' fill='none' stroke='red' stroke-width='1' />
<circle cx='113' cy='698' r='1' fill='none' stroke='red' stroke-width='1' />
<circle cx='113' cy='697' r='1' fill='none' stroke='red' stroke-width='1' />
<circle cx='113' cy='676' r='1' fill='none' stroke='red' stroke-width='1' />
<circle cx='113' cy='683' r='1' fill='none' stroke='red' stroke-width='1' />
<circle cx='113' cy='678' r='1' fill='none' stroke='red' stroke-width='1' />
<circle cx='113' cy='674' r='1' fill='none' stroke='red' stroke-width='1' />
<circle cx='113' cy='675' r='1' fill='none' stroke='red' stroke-width='1' />
<circle cx='113' cy='679' r='1' fill='none' stroke='red' stroke-width='1' />
<circle cx='113' cy='680' r='1' fill='none' stroke='red' stroke-width='1' />
<circle cx='113' cy='682' r='1' fill='none' stroke='red' stroke-width='1' />
<circle cx='113' cy='685' r='1' fill='none' stroke='red' stroke-width='1' />
<circle cx='113' cy='684' r='1' fill='none' stroke='red' stroke-width='1' />
<circle cx='113' cy='687' r='1' fill='none' stroke='red' stroke-width='1' />
<circle cx='113' cy='686' r='1' fill='none' stroke='red' stroke-width='1' />
<circle cx='113' cy='694' r='1' fill='none' stroke='red' stroke-width='1' />
<circle cx='113' cy='689' r='1' fill='none' stroke='red' stroke-width='1' />
<circle cx='113' cy='690' r='1' fill='none' stroke='red' stroke-width='1' />
<circle cx='113' cy='692' r='1' fill='none' stroke='red' stroke-width='1' />
<circle cx='113' cy='693' r='1' fill='none' stroke='red' stroke-width='1' />
<circle cx='113' cy='677' r='1' fill='none' stroke='red' stroke-width='1' />
<circle cx='113' cy='681' r='1' fill='none' stroke='red' stroke-width='1' />
<circle cx='114' cy='699' r='1' fill='none' stroke='red' stroke-width='1' />
<circle cx='114' cy='700' r='1' fill='none' stroke='red' stroke-width='1' />
<circle cx='114' cy='698' r='1' fill='none' stroke='red' stroke-width='1' />
<circle cx='114' cy='697' r='1' fill='none' stroke='red' stroke-width='1' />
<circle cx='114' cy='679' r='1' fill='none' stroke='red' stroke-width='1' />
<circle cx='114' cy='674' r='1' fill='none' stroke='red' stroke-width='1' />
<circle cx='114' cy='660' r='1' fill='none' stroke='red' stroke-width='1' />
<circle cx='114' cy='659' r='1' fill='none' stroke='red' stroke-width='1' />
<circle cx='114' cy='689' r='1' fill='none' stroke='red' stroke-width='1' />
<circle cx='114' cy='694' r='1' fill='none' stroke='red' stroke-width='1' />
<circle cx='114' cy='680' r='1' fill='none' stroke='red' stroke-width='1' />
<circle cx='114' cy='683' r='1' fill='none' stroke='red' stroke-width='1' />
<circle cx='114' cy='681' r='1' fill='none' stroke='red' stroke-width='1' />
<circle cx='114' cy='682' r='1' fill='none' stroke='red' stroke-width='1' />
<circle cx='114' cy='676' r='1' fill='none' stroke='red' stroke-width='1' />
<circle cx='115' cy='698' r='1' fill='none' stroke='red' stroke-width='1' />
<circle cx='115' cy='697' r='1' fill='none' stroke='red' stroke-width='1' />
<circle cx='115' cy='699' r='1' fill='none' stroke='red' stroke-width='1' />
<circle cx='115' cy='700' r='1' fill='none' stroke='red' stroke-width='1' />
<circle cx='106' cy='687' r='1' fill='none' stroke='yellow' stroke-width='1' />
<circle cx='107' cy='697' r='1' fill='none' stroke='yellow' stroke-width='1' />
<circle cx='107' cy='687' r='1' fill='none' stroke='yellow' stroke-width='1' />
<circle cx='108' cy='697' r='1' fill='none' stroke='yellow' stroke-width='1' />
<circle cx='108' cy='687' r='1' fill='none' stroke='yellow' stroke-width='1' />
<circle cx='109' cy='697' r='1' fill='none' stroke='yellow' stroke-width='1' />
<circle cx='109' cy='687' r='1' fill='none' stroke='yellow' stroke-width='1' />
<circle cx='110' cy='697' r='1' fill='none' stroke='yellow' stroke-width='1' />
<circle cx='110' cy='687' r='1' fill='none' stroke='yellow' stroke-width='1' />
<circle cx='111' cy='697' r='1' fill='none' stroke='yellow' stroke-width='1' />
<circle cx='111' cy='687' r='1' fill='none' stroke='yellow' stroke-width='1' />
<circle cx='112' cy='697' r='1' fill='none' stroke='yellow' stroke-width='1' />
<circle cx='112' cy='687' r='1' fill='none' stroke='yellow' stroke-width='1' />
<circle cx='112' cy='671' r='1' fill='none' stroke='yellow' stroke-width='1' />
<circle cx='112' cy='698' r='1' fill='none' stroke='yellow' stroke-width='1' />
<circle cx='113' cy='698' r='1' fill='none' stroke='yellow' stroke-width='1' />
<circle cx='113' cy='697' r='1' fill='none' stroke='yellow' stroke-width='1' />
<circle cx='113' cy='671' r='1' fill='none' stroke='yellow' stroke-width='1' />
<circle cx='113' cy='687' r='1' fill='none' stroke='yellow' stroke-width='1' />
<circle cx='113' cy='659' r='1' fill='none' stroke='yellow' stroke-width='1' />
<circle cx='113' cy='658' r='1' fill='none' stroke='yellow' stroke-width='1' />
<circle cx='114' cy='671' r='1' fill='none' stroke='yellow' stroke-width='1' />
<circle cx='114' cy='659' r='1' fill='none' stroke='yellow' stroke-width='1' />
<circle cx='114' cy='658' r='1' fill='none' stroke='yellow' stroke-width='1' />
<circle cx='114' cy='698' r='1' fill='none' stroke='yellow' stroke-width='1' />
<circle cx='114' cy='697' r='1' fill='none' stroke='yellow' stroke-width='1' />
<circle cx='894' cy='658' r='1' fill='none' stroke='green' stroke-width='1' />
<circle cx='894' cy='698' r='1' fill='none' stroke='green' stroke-width='1' />
<circle cx='895' cy='657' r='1' fill='none' stroke='green' stroke-width='1' />
<circle cx='895' cy='671' r='1' fill='none' stroke='green' stroke-width='1' />
<circle cx='895' cy='653' r='1' fill='none' stroke='green' stroke-width='1' />
<circle cx='895' cy='658' r='1' fill='none' stroke='green' stroke-width='1' />
<circle cx='896' cy='657' r='1' fill='none' stroke='green' stroke-width='1' />
<circle cx='896' cy='671' r='1' fill='none' stroke='green' stroke-width='1' />
<circle cx='897' cy='671' r='1' fill='none' stroke='green' stroke-width='1' />
<circle cx='897' cy='657' r='1' fill='none' stroke='green' stroke-width='1' />
<circle cx='897' cy='653' r='1' fill='none' stroke='green' stroke-width='1' />
<circle cx='897' cy='656' r='1' fill='none' stroke='green' stroke-width='1' />
<circle cx='898' cy='657' r='1' fill='none' stroke='green' stroke-width='1' />
<circle cx='898' cy='671' r='1' fill='none' stroke='green' stroke-width='1' />
<circle cx='899' cy='657' r='1' fill='none' stroke='green' stroke-width='1' />
<circle cx='899' cy='671' r='1' fill='none' stroke='green' stroke-width='1' />
<circle cx='899' cy='653' r='1' fill='none' stroke='green' stroke-width='1' />
<circle cx='899' cy='656' r='1' fill='none' stroke='green' stroke-width='1' />
<circle cx='900' cy='657' r='1' fill='none' stroke='green' stroke-width='1' />
</svg>

But using xsltproc generates an empty output…

Using XSLT to manipulate an SVG file


I am generating a lot of the graphics for my project using Scalable Vector Graphics (SVG) – an XML format.

The advantages of SVG are obvious – it is human readable, it preserves some of the data in the output (eg in the relative placing of the dots on a graph), Groovy has good support for it and, in theory at least XML Stylesheet Transformations (XSLT) means that it can be manipulated outside of a formal programming environment using another bit of XML (an XSL stylesheet) and freely available XSLT tools – eg xsltproc on Linux.

But XSLT is one of those Cinderellas of the computing world – widely relied on but not generally understood or widely discussed. As a result I have struggled over the last 24 hours to work out how to do what ought to be a simple thing: removing a set of dots of one colour from an SVG. I still have not got to the bottom of all the issues – specifically xltproc seems to have an issue with XML namespace declarations and/or DTD declarations – but I have fixed it for all practical purposes, so thought I should document it for future users…

Before I describe the problem more fully as well as the solution, I have to recommend this book – XSLT: Mastering XML Transformations. For Beginners and Advanced Users – which gave me the pointers to a solution I just could not find online.

So here is the graph (in PNG format because WordPress does not support SVG) I want to manipulate.

Firefox memory mapThis is memory map – via Valgrind – of Firefox (7) opening and closing (I created a page that, once Firefox was configured, would close the browser.)

The different types of memory accesses – for instructions (red), loads (blue), stores (yellow) and modifies (green) are all on superimposed and which colour you see depends on the order they are written if they access the same page.

So the answer, with an SVG, is obvious, just look at the colour you want to see.

Ought to be easy, right? Well, I suppose when you know how to do it, it is easy. But it’s not completely simple!

Each dot on the graph is drawn with an XML element like this:

<circle cx='102' cy='699' r='1' fill='none' stroke='red' stroke-width='1' />

The trick is to create a template for all the elements but inside that template only copy out the elements that match the correct stroke attribute. (Attributes are not children of node elements either, so you have to copy them out explicitly.)

Here’s a stylesheet that does it:

<?xml version="1.0"?>
<xsl:stylesheet version="1.0" indent="yes"
xmlns:xsl="http://www.w3.org/1999/XSL/Transform">
<xsl:param name="colour">yellow</xsl:param>
<xsl:template match="/">
	<xsl:apply-templates select="svg"/>
</xsl:template>
<xsl:template match="svg">
	<xsl:copy>
		<xsl:for-each select="@*">
			<xsl:copy/>
		</xsl:for-each>
		<xsl:text>
</xsl:text>
		<xsl:apply-templates select="rect"/>
		<xsl:apply-templates select="line"/>
		<xsl:apply-templates select="text"/>
		<xsl:apply-templates select="circle"/>
	</xsl:copy>
</xsl:template>
<xsl:template match="line">
	<xsl:copy>
		<xsl:for-each select="@*">
			<xsl:copy/>
		</xsl:for-each>
	</xsl:copy>
<xsl:text>
</xsl:text>
</xsl:template>
<xsl:template match="rect">
	<xsl:copy>
		<xsl:for-each select="@*">
			<xsl:copy/>
		</xsl:for-each>
	</xsl:copy>
<xsl:text>
</xsl:text>
</xsl:template>
<xsl:template match="text">
	<xsl:copy>
		<xsl:for-each select="@*|node()">
			<xsl:copy/>
		</xsl:for-each>
	</xsl:copy>
<xsl:text>
</xsl:text>
</xsl:template>
<xsl:template match="circle">
	<xsl:if test="@stroke=$colour">
		<xsl:copy>
			<xsl:for-each select="@*">
				<xsl:copy/>
			</xsl:for-each>
		</xsl:copy>
		<xsl:text>
</xsl:text>
	</xsl:if>
</xsl:template>
</xsl:stylesheet>

(This code be shortened to just copy-of the line, rect and text elements.)

And here’s an example:

xsltproc  -o inst.svg --stringparam colour "red" pickcolour.xsl master.svg

Instructions executed by Firefox 7

Help! Computing power or better algorithm required


Peter Denning
Peter Denning, Image via Wikipedia

I have a serious problem with my MSc project.

For the first part of this I am seeking to demonstrate/investigate the underlying assumptions about locality, phases of locality and so on that underlie the working set method of VM management.

The graphs on other blogs here are some of the output of the test cases I have been using and they seem to work well.

In addition, though, I want to show what Peter Denning refers to as a “lifetime curve” for a process – essentially showing how increasing the size of the resident set (or in my case increasing the time for which pages remain resident) changes the time between page faults.

This should not (and early results of mine show) it isn’t linear (though it is monotonic). But to calculate this seems, under the algorithm I am using, to require vast amounts of computing power.

My algorithm (in Groovy) is essentially this:

1. Set time (if we are plotting 600 points then first time would be 1/600th of the process lifetime) for page expiry

2. Reading the lackeyml file and using a groovy/java map, store the page number and the reference time -if the page wasn’t referred to in the map, increase the fault count

3. Look through the map to see if there are any pages which are older than the page expiry times, and evict them from the map

4. Move to the next record in the lackeyml file and return to 2

5. Get to the end of the file and count the total faults.

The problem with all this is (2) above – typically it is being called millions of times and iterating through a map in this way is extremely slow. I think on a dual AMD64 box here with the 48 million instruction count for xterm (pretty much a ‘toy’ application in this context) – this will take 20 days to run, even with a decent bit of parallelization in the code.

Hence I need a better algorithm or massive (and affordable) computing power. Anyone have a cluster they could lend me for a few hours?

The price of syntactical sugar?


A selection of programming language textbooks ...
Image via Wikipedia

The programming language Groovy contains lots of “syntactical sugar” designed to make the programmer’s task easier and the code more expressive and so, presumably, easier to maintain.

But my experience seems to suggest it also comes with a heavy performance price. I have updated some code I am working on to create XML files and now check some values against a range – of the form max .. min and performance has fallen through several floors: a script that was running in 20 minutes has now been cranking away for over 145 minutes.

Introducing the lackeyml format


Example of a XML file
Image via Wikipedia

The Valgrind simulation program has a lot of options and for many of them it supplies an XML output option, though not so for the basic “lackey” option that allow one to look at basic memory reads and writes.

I need some XML output from that – it’s simply easier to manipulate and interpret the output (eg to transform into a graph) when you can use a known and well supported format. So I wrote some code to do that and defined a new format – lackeyml.

You can get the Groovy code here – and use it like this:

First run Valgrind and output the raw text to a file (using xterm here as the targeted application):

valgrind --tool=lackey --trace-mem=yes --log-file=xtlackey.txt xterm

Then run the groovy script against that file:

groovy lackey_xml.groovy xtlackey.txt

If, and when, I get the next stage right – producing an SVG from the output, I’ll post that on the blog also.

Eighteen million lines of XML!


Valgrind
Image via Wikipedia

I have just finished a Groovy (the language) helper application to process the output of Valgrind‘s “lackey” package into an XML file for further analysis. More proof of the power of Groovy (a lanuage I am coming to like) – but also a sign that maybe I should get a more powerful computer – the XML file that came from the loading of xterm ran to over 18 million lines.

XPath tool


An illustration of XSL transformation and form...
Image via Wikipedia

This is a helpful XPath tool if you are looking to sharpen your skills.

You can test your XPath expressions online, and even load your own file to check against.