A hash algorithm, for computer scientists, is a way of turning one long string (some words, a number etc) into a shorter “hash code“.
Hashing is used in multiple ways – for instance to check that a file you have downloaded matches the one on the server (you hash the downloaded file data and check it matches the advertised hash on the server). This is much quicker than comparing the numbers byte by byte and, if a good hashing algorithm is chosen, the chances of the “collision” (in other words the false matching of two hashes) are low, especially for the common types of error, so you can be pretty sure a matched hash means the download is a good one.
It seems that the UK’s Department for Work and Pensions “Universal Credit” project relies extensively on hashing to check that their data on people’s incomes are correct. And the failure to match hashes is very high – suggesting some sort of fundamental failure in the system.
I have written about the enormous risk that the Universal Credit project represents – at a time of dwindling resource budgets the government is seeking to deliver an IT project that has a direct bearing on the lives of millions of the most financially vulnerable people to a very tight deadline and using a development method – “agile” – that has been sold as the answer to all the problems of government IT despite the fact most software development textbooks will tell you this is not what you should be using “agile” for.
It might still come off – certainly the department are refusing to admit that there is any possibility of failure – but I think the evidence that it is not ready is beginning to amass.
At least, though, the latest story, even if it is not encouraging in terms of the project’s overall chances of success, does show that the DWP are at least carrying out part of the “agile” function – testing the software. Previously there was next to no sign of it. But “agile” is also supposed to mean being open about progress, getting “stakeholder buy-in” (yes, I know it’s a horrible phrase, but it communicates what this is about) and getting lots of user feedback as the development process goes on. Where is any of that? The less we see of it, the more it looks like the department have got something to hide.
And, one final comment. Ruth Owen from HMRC appears to quote a much lower hash match failure rate – 5% instead of 25% – but 5% in the real world would mean literally millions of failures every year. A 5% failure rate would be as broken as a 25% failure rate.
Related articles
- Universal Credit on-track – and thanks to agile, says discharged IT boss (computerweekly.com)
- Fog of confusion still surrounds “biggest agile project ever” (cartesianproduct.wordpress.com)
- xxHash : new version (fastcompression.blogspot.com)
- How to explain Hash DoS to your parents by using cats (anchor.com.au)
cartesian product ... stuff about computing mostly 