hexmon BASIC

How to export from LyX to LaTeX


All of this information is out there, but it took me more than a few hours to successfully manage this on my Mac, so here are the steps, using LyX and TeXShop.

 

  1. Go to File->Export->LyX Archive
    This generates a zipped tarball of your .lyx file and, crucially, all the other elements (such as graphics and bibliography) in a hierarchy of files.
  2. Unroll the tar ball (i.e. tar -xvzf yourarchive.tar.gz) in a suitable place.
    You now have all the files you need in a standalone hierarchy (best to place your tarball in its own directory before you do this as the subdirectories you create could go under your root directory etc)
  3. Convert the LyX file to raw LaTeX.
    At this point all you have a LyX archive and if you really need LaTeX you need to convert the LyX file. On a Mac the easiest way to do this is to point LyX.app at the .lyx file in your archive and open it, then run another export – this time to LaTeX (pdflatex). This will create a .tex file in your archive. (I messed this stage up in the original posting – thanks, as always to Paul Rubin for putting me on the straight and narrow here.)

  4. Make sure your bibliography is in the right place.
    Your bibliography needs to be in the same directory. So if your .tex file is in the bizarro directory, your bib file must be too. (I said before it should have the same name as your .tex file but that is not necessary.) 
  5. Open your .tex file in TeXShop.
    You may need to do some editing, for instance if your .tex file refers to your bibliography  under some path, so for the example above you need to make your bibliography reference look like this:
    \bibliography{bizarrothings}

  6. Run the following sequence of commands in TeXShop:
    Typeset LaTeX
    Typeset BibTeX
    Typeset LaTeX
    Typeset LaTeX

    Hopefully you now have compiled a PDF file with all the correct references, which means if you package up your archive directory it will now contain the correct .tex file (it will also contain the PDF and .lyx file unless you delete them first).

Hope this is helpful – if I have missed anything, let me know in the comments.

 

Advertisements
Real time devices

A very first draft…


Our proposal here is to meet this need to use a memory space much greater than the locally available low-latency storage head on. The problems faced by programmers of today’s many- and multi-core devices are, in a sense, far from new: in the 1960s an earlier generation of programmers, experimenting with virtual memory and time-sharing computers, and using programs generated by the first generation of high-level languages were struck down by the phenomenon of thrashing, where computing time was lost to excessive memory management [23]. The solutions we propose here draw on the lessons learned at that time, but significantly adapt the methods to account for the realities of using multicore devices with very limited local storage.

We propose to enable virtual memory [24] for such systems, using a novel form of paging which, instead of transferring whole pages across the memory hierarchy, concentrates on a minimal transfer of partial pages [50]. Here hard faults still impose a significant penalty of lost cycles but the cost of paging in and out of memory is amortised across multiple memory reads and writes. This avoids unnecessary transfer and limits congestion in the memory interconnect, which we find to be a significant determinant of program performance.

One of the lessons of the 1960s was that maintaining a program’s working set in low latency memory was the most effective way of limiting paging and thrashing [22]. Much research at that time and into the 1970s and later concentrated on effective ways to maintain the working set of pages in fast memory and innovation continued into the 1990s (and beyond), for instance with the 2Q algorithm in Linux [38]. Here, though, we establish that better performance in embedded and real-time manycore systems is unlikely to come from a better page replacement algorithm but from a reorientation towards better management of the internals of the pages.

General computing page replacement algorithms are generally robust [49] and with the decreasing cost of memory, much recent research has concentrated on using large page sizes, not least to minimise TLB misses [9], we return to earlier research findings which emphasise the efficiency of small pages on small systems and we show that combining this with a FIFO page replacement policy may, perhaps counter-intuitively, deliver more time predictable or even faster performance than attempts to use some version of a least-recently used policy, though we also show that choice of page replacement algorithm is likely to be strongly related to the variation in working set size over a program’s lifecycle.

Concerns about power have always been important for embedded devices, but changing characteristics of hardware mean new issues such as “dark silicon” [28] will be important as we move deeper into the coming manycore era. We demonstrate here that constraints in power may be, to some extent, mitigated by a loosening of bottlenecks elsewhere in a realtime system, particularly in the memory interconnect: in a system where power is no constraint then congestion here dominates performance, where power concerns limit processor activities they may also lessen congestion.

Further we demonstrate results that some frequency scaling in manycore systems may not limit performance if it contributes to lessening congestion in the memory interconnect: results that suggest that the dark silicon issue may not be as big a barrier to performance in real-world applications as might be expected.

Getting a job


I have, essentially, two sets of skills and experience.

One is as a political campaigner and communicator. I did well out of that for a while and more than that, did some things I am proud of and feel really privileged to have had a chance to be part of.

But it’s fair to say that road seems to have hit a dead end.  If you want to run a serious, progressive, campaign then I am certainly still interested, but I am not sure there is much of that out there today.

So then there are the other skills – ones that I am told are in high demand.

Namely as a software designer/writer/developer.

I can do this and I am much better these days than I used to be: unlike, say, running I am still getting faster and sharper. C/C++/Embedded/Perl/Linux/Groovy/DSLs/R/Deep Learning – I can tick all those boxes.

But where to begin? The reputation of IT recruitment agencies is pretty grim, though I have no direct experience. I have registered with one, but I am being sent invitations to be a senior C++ engineer in Berlin on a salary of €150,000 per annum which even I think is probably a bit over-ambitious for someone with no commercial experience.

(NB: If you want to see what I have done have a look at https://github.com/mcmenaminadrian).

Giving up on the convolutional network?


For almost three months now I have been trying to build and train a convolutional network that will recognise chess puzzles: but I don’t feel I am any closer to succeeding with it than I was at the start of September and so I wonder if I should just give up.

The network itself is built, and as far as I can see, works except for the fact that I just cannot get it to converge on the training set.

The (learning) code is here: https://github.com/mcmenaminadrian/ChessNet/tree/learning

The training set is here: https://github.com/mcmenaminadrian/ChessNet/tree/learning/images

There are 25 possible classes of outcome – from an empty white square to a black square hosting a black king, and the network outputs a value between -1 (no match) and 1 (perfect match).

There are 25 convolutional fibres each with seven layers, going from a 100 x 100 input layer to a final filter (feature map) of 88 x 88 which are then fully connected to 25 output neurons (there is no pooling layer): as you can see that means there are 88 x 88 x 25 x 25  + 25 (4.84 million, plus 25 for bias) connections at the final, fully connected, layer (or alternatively each output neuron has 193601 input connections).

Perhaps the issue is that the scale of the fully connected layer dwarfs the output and influence of the feature maps? I don’t know, but what I do know that, as training goes along, the output neurons generally begin in a low (i.e., close to -1) state and then edge towards a high state, but as they do they are suddenly overwhelmed and everything returns to an even lower state than before.

Envisaging this as a three dimensional surface, we creep up a steep hillside and then fall down an even deeper hole just as we appear to be getting towards a summit: the problem seems to be that training doesn’t really teaching the network to differentiate between any of the training images, it just pushes the network towards a high value. Then, suddenly images which should be reported as low are reported as high and the error values flood the network on back propagation.

To explain further: in the training set our image X will will always be relatively infrequently seen so most results should be low and are low, with small error values (deltas as they are usually called) – so small that they are generally ignored. The deltas for X are then large and they feed into the network, dragging our response towards high. Eventually we cross a threshold and all the results – for good and bad images – are reported as high and so there lots of big deltas which overwhelm the small number of correct positives. At least that is what I think is happening.

Of course what really should happen is that the network learns to discriminate between the ‘good’ and ‘bad’ images, but that just seems as far away as ever.

Any tips, beyond giving up, gratefully received.

Conv-nets are hard


Months ago I started work on a convolutional neural network to recognise chess puzzles. This evening after mucking about with the learning phase for weeks I thought I had scored a breakthrough – that magic moment when, during learning, a tracked value suddenly flips from the wrong result to the right one.

Brilliant, I thought – this is about to actually work, and I started tracking another value. Only to come back to the original and see that it had all gone wrong again.

False minima abound in training – which is essentially about getting the right coefficients for a large set of non-linear equations each with many thousands of parameters. Or maybe it wasn’t a false minimum at all – but the real one, but it’s just operating over an extremely small range of parameter values.

Will I ever find it again? And if I do can I find it for the other 25 classification results too?

(As an aside: I made the code parallel to speed it up, but it’s a classic example of Amdahl’s law – even on a machine with many more processors than the 26 threads I need and with no shortage of memory, the speed-up is between 3 and 4 even with the most heavy-duty calculations run in parallel.)

More and more spam reviews on Amazon


Earlier this month I highlighted how a book that claims to be about using Python to build convolutional neural networks and yet, say readers, contains not a single line of Python, was garnering rave reviews on Amazon.

The trend hasn’t stopped and it is pretty clear to me that these are, in fact, spam.

Plainly Amazon’s review system is broken.

 

Can you get a useful result with a random convolution filter?


In a number of places I’ve seen it remarked that a random convolution filter makes for a reasonably efficient edge detector for images, so I thought I’d test this.

The answer, perhaps surprisingly, seems to be yes.

With 25 input filters in an untrained convolutional neural net (where kernel values were pseudo-randomly distributed between -0.5 and 0.5), all but three of the first level filters returned something that suggested edge detection (though given the original image was a collection of edges this is not much of a claim.) Some of the second or even third level filters also showed patterns, but most delivered something like universal blackness.

Admittedly this is a small sample size, with just one test image.

Here is the original (100 x 100) image:

orig

Here are some of the useful or at least interesting (98 x 98) filtered images:

I cannot really think of a useful application of this finding, but it does interest me none the less.

Strange reviews on Amazon


Messing about with convolutional neural networks (CNNs) continues to take up some of my time (in case my supervisor reads this – I also have a simulation of a many core system running on the university’s computer atm).

I started my research here with a much cited, but really well-out-0f-date book – Practical Neural Network Recipes in C++. What’s nice about that book is that it is orientated towards getting things done, though the C++ is really from the “C with classes” era.

Another book – Guide to Convolutional Neural Networks: A Practical Application to Traffic-Sign Detection and Classification – which I can access through the University of York’s library, helped fill in some of the theoretical and other gaps and also showed me why I needed to move away from the perceptron model promoted in the earlier book and move towards a CNN. But like many of Springer’s books it is poorly edited and not fully and properly translated to use online.

So I’m still on the look out for the perfect match – a book with practical coding examples that clearly explains the theory and, bluntly, is written in good English with all the maths actually reproduced in the online format (as I am just not going to be able to afford a printed copy.)

In particular I want a clear explanation of how to do back propagation in a CNN – as it’s plain that the general method outlined in “Practical Neural Network Recipes” doesn’t work beyond a fully connected layer, while the explanation in “Guide to…” is impenetrable and, actually, rather odd (as it seems to imply that we use a fixed weight for every neuron in a filter as opposed to using fixed weights across each filter – if I have explained that properly).

So, I’ve just had another look and came across this book … “Convolutional Neural Networks in Python: Introduction to Convolutional Neural Networks

This book has managed (at time of writing to have collected two, one star, reviews on the Amazon UK website):

Amazon reviews

I have no idea how fair those reviews are, but this passage from the preview version available on the website doesn’t suggest the author is yet rivalling Alan Turing:

But, here’s the odd thing. It would appear a number of “purchasers” of this book through Amazon.com are very enthusiastic about it and all felt the need to say so this very day (9 August):

odd reviews

Even more oddly, the reviews all read like spam comments I get on this blog. But I have no evidence to suggest these are anything other than genuine, if odd, comments on the book…

Free software to chop up your JPEGs


As a public service announcement – if you need some software (on Linux but may well compile and run on other systems if they support Qt) to chop a big JPEG up into smaller files, I have written this (in my case to support building a test set for a neural network).

It’s at https://github.com/mcmenaminadrian/TestSetCreator and it’s GPL licenced (v 3).

Even more about neural networks


Because I essentially got it wrong in the last post … turns out that a fully connected network is, generally, not a great idea for image processing and that partial connections – through “convolution layers” are likely to be more efficient.

And my practical experience backs this up: my first NN did, in effect, have two convolution layers (or filters), although somewhat eccentrically designed as 100 x 1 and 1 x 100 filters. And this network performs better than the single hidden layer fully connected alternative. That may just be because it takes an age to train the fully connected network and converge of the error levels towards a low number is just taking for ever (a convolution layers has many fewer connections and so can be trained much faster).