Programming

It recently occurred to me that a certain website that I check regularly could be given an unofficial RSS feed by writing a script to pull in data from its SOAP interface and spit it out as a feed. Not many websites have SOAP interfaces, but this one does, and while I’m not sure they created it to allow third-party RSS feeds in particular, I’m sure they don’t mind if I just use it for my own secret consumption. That means I won’t be disclosing which site it is that my script is accessing, although the general principle about how to write a script like this should be more useful anyway. Of course I used PHP for the script, because that’s what webhosts support, so if you want a version written in a nice language like Groovy I’m afraid you’ll have to port this script to that language.

Like the mythical Cassandra, I occasionally find myself aware of an impending disaster but unable to prevent it. The particular disaster I am talking about is writing verbose, ugly, unclear code in PHP that can be written succinctly, beautifully, and readably in Groovy (or at least one out of those three). This would not feel so ironic, were it not for the fact that I never have quite the fluency with Groovy to actually be able to write the alternative code, and thus I am forced to go with the PHP solution that I know to be suboptimal. A recent case, in particular, has highlighted this fact to me, as I had the task of turning a hash (actually a list of hashes) into a CSV file (actually a string in CSV format), while filtering the hash based on a whitelist of keys. Although this turned out to be implementable in 7 lines of PHP, I was sure it must be a one-liner in Groovy.

One interesting thing about software is that because it is written in a language that a computer can understand, it is also possible, in theory, to get the computer to tell you things about that software. There are various classes of utilities from static analysers to code style checkers, but I have recently been considering how possible it is to check for uses of specific deprecated functions and classes in a PHP project. The actual checker I was thinking of creating would use regular expressions and be written in PHP itself, but because the process would be a static analysis one, which I hoped didn’t have to resort to tokenising the PHP source code it was examining, this checker could in fact be written in any language just as (if not more) easily. As is often the case in my blog posts about PHP recently, it turns out that various “features” of PHP make things more difficult (regardless of the language that the checker is written in), although trying to solve a problem the wrong way doesn’t help either.

By a strange coincidence, I’ve recently bumped into another PHP gotcha, well, actually three of them depending on how you’re counting, but I’m sure that these must be the last three unexpected things about PHP and I won’t soon have to write another blog post detailing something else unintuitive that PHP does. The reason these could be seen as one gotcha is that they all involve PHP’s support for floating point numbers, so one workaround would just be to not use that datatype at all in your programs. As I will show though, it is harder than you might think to avoid them, so perhaps the best advice is to avoid using numbers at all. To be on the safe side though, maybe you should just use a different programming language.

A project at work recently reached an interesting crossroads where it had to be decided what the spelling convention should be for some code which the company’s partners would have to interact with. The use of English as a common language had been uncontroversial up to that point, but there had crept into the software words which had different spellings between American English (en_US) and British English (en_GB). As a passionate European, it made sense to me to choose the version of English spoken by the English, but the more I researched it, the more I realised that the correct international standard really was en_US. Perhaps the deciding factor for me was that the coding standard for the European Space Agency requires the use of en_US spelling, which I will mention below in my depressingly convincing argument for that same linguistic standard.

While my previous blog post covered what may be an actual bug in PHP, there are still a couple of gotchas which might be worth mentioning. Firstly there is the inconsistent way it deals with single and double quotes around strings, and then there are the undesirable consequences of how it juggles variable types, for which I have an example. Of course, all languages have their problems, with the obvious exception of Groovy, the superiority of which I will demonstrate with a one-liner, possibly accompanied by an excuse to explain why that simple one-liner was so hard to write.

The title of this post is even less helpful (and less fair) than the title of the blog post which inspired it, so it should probably be viewed as parody. However, I do sadly have a file on my computer called hate.php containing a series of lines of valid PHP source code which produce unexpected and even perplexing results. Admittedly there are only 6 lines, and the behaviour of half of them is as documented, but I hold PHP to a higher standard than I would a “messy” language like Perl, so I thought this deserved a blog post.

A friend of mine was apparently inspired by my solution to the problem of finding the first 10 digit prime number in the digits of e, and he told me about a seemingly similar problem he faced while trying to manipulate a text file he had created in a custom format. While some techniques from solving the e problem may be applicable, his data format allowed for arbitrary length strings, rather than having this 10 digit limitation, which made the problem suddenly much harder. I will detail the exact problem below, as well as listing some of the methods we used to tackle it.

If you have used SSH at all, you are likely to have bumped into a warning message when accessing a new host for the first time. This is because unlike with HTTPS, there is no central list of which public keys belong to which hosts (or rather, which entities should be trusted to make those connections). For example, if you access www.google.com over HTTPS in a browser, then your browser does the equivalent of downloading a public key from that host, and a certificate saying that this really is Google’s public key. This certificate itself has to be checked cryptographically, but it will have been created by one of the few central trusted points for handing out certificates, and your browser will be pre-loaded with a list of these trusted points.

As SSH clients don’t tend to have pre-loaded lists like this, you are left to make the judgement yourself about whether a given SSH public (host) key belongs to a given host. One way to do this is by comparing the key’s fingerprint (which is presented in the warning message) with another copy of that fingerprint retrieved over a different medium, preferably a trusted one, such as in person. To give someone the information needed to check these fingerprint warnings, you might think of sending them a known_hosts file, which contains a list of hosts and cryptographic data about their public keys, but unfortunately this cryptographic data isn’t stored in the same format as fingerprints. Where there’s a will, there’s a script, though, and so I detail below what can be done.

I do a lot of programming, and I like writing one-liners to help me with things, so it’s perhaps not surprising that I’ve ended up writing a one-liner to help me with my programming. I should point out that the initial motivation to write this one-liner was not that I made any mistakes in the code I wrote, rather it was someone else’s code I was looking at which needed correction, but it would be hubris to assume I’m never going to make any mistakes myself, so I’m sure this script will be useful for my own code. Of course, nowadays editors will at least spell check the comments in your code for you, but it is also good to make sure your variable names don’t contain misspelled words, as that makes it harder for people (who know the correct spelling) to collaborate with you. This one-liner is rather crude and does produce a lot of noise in the output, but it is also interesting from a technical point of view, so I will discuss below how I came up with it and how it works.

Every now and then I find my mind gets caught on some seemingly trivial observation, and I end up following a chain of thought tangential to the one I was originally on, until I arrive at somewhere quite unexpected. Whereas people in former times may have been unable to travel too far down these intellectual rabbit holes, we now live in a world where Google and Wikipedia have made us seemingly omniscient, and hypertext in particular allows us to jump from one idea to the next, wherever our curiosity takes us. The secondary limit, I suppose, would be the ability to process all of this information that we amass while browsing the Web. As a programmer, though, there are certain options for information processing which are open to me but would not be readily available to non-programmers, and even if what I do with the processed information isn’t particularly ground-breaking, it can at least be the subject of a new blog post. As the title of this post suggests, my most recent such endeavour involved looking at Linux application names, and dictionary words, and below I explain what I found and how I found it.

Signing people’s keys is one problem, but when people sign your key that requires its own procedure with its own set of potential issues. You will likely receive several emails, one from each person with whom you exchanged keyslips, and these emails will each include an attachment or message body that contains encrypted data for you to decrypt. The plaintext you get from decrypting should be a copy of your public key that has been signed by someone’s private key. It is this signature that you’re looking for, and GPG can extract it and add it to your key. Once you have collected the signatures, you can upload your key to a keyserver where the information about who has signed your key can be publicly stored. That’s the principle, but the steps to do all this can be quite cumbersome, so I used some console one-liners to speed some of the steps up, and I include them below with an explanation of what they do.

Cryptography is hard to do right, and even if it is implemented correctly, the user is often required to perform some complicated operations to make use of it. One area of cryptography which involves a great deal of activity from the user is the signing of public keys used in public key cryptography. Not only does it usually require users actually meet each other and do some sort of identity verification, it then requires quite an involved process at the computer involving retrieving keys, checking signatures, and sending emails. To automate this as much as possible, I have come up with a little script which helps me perform some GPG and email operations quickly and simply, without, I hope, reducing my security.