Taking back the Web

Of course, the blog is already free of spam, but only at the expense of putting up barriers which prevent legitimate contributions from being made. It was always my intention with this blog to allow people to comment on it and leave trackbacks, but until now I had been stopped by this false dichotomy. To be precise, I had allowed comments, but only from people who had accounts (which meant me and my sister) and I had allowed trackbacks, but they had to be manually approved by me. The manual approval process was not, originally, too onerous, and has produced a positive effect. Recenly, however, the amount of spam had become considerable (currently I have 158 comments awaiting moderation) and I sought answers. My search was also made more pressing by a conversation with someone I had encouraged to use WordPress, who told me that they were having to turn off these community features because of the spam problem.

The solution

So it was these two factors that made me want to change the system I was using, and the principle I mentioned earlier that made me think the new system should be more open. Fortunately, and perhaps because things have moved on since the first time I looked, answers were available for both the comment spam and trackback spam problems, in the form of two WordPress plugins. With the elegance of a concise theorem, the strength of the solutions stood out in a way that didn’t need words. I could see that they both exploited the inherent advantages of the media they were trying to protect and the inherent disadvantages of the attack vector. This is in contrast with traditional solutions such as blacklisting (as in virus checkers) and statistical modelling (as in email spam filters) which propose a never-ending war against the behaviour they are trying to stop. In particular, the comment spam solution involved asking the human comment writer (who must be there for the comment to be valid) a simple but (in theory) AI-complete question before accepting the message. The trackback spam solution, on the other hand, required that the blog leaving the trackback contain a link to my blog (which must exist if the trackback is valid).

As for where I found them, the first answer is “The WordPress Codex”, and the second answer is “On the developer’s site”. The trackback spam prevention plugin was the easiest to install, requiring just 4 steps (one of which was optional): download, decompress, activate, (configure). Only the comment spam prevention plugin took particular effort, a fact recognised by the author who explained the benefits that came with this cost, in comparison with another similar plugin, saying

Any ways: if you are afraid of modifying the source code of your theme, I recommend that you go for the Did You Pass Math? plugin.

For a start, the installation process of this plugin required extra steps, consisting as it did of the first three of the previous plugin, and the fourth which was no longer optional, then (and most significantly) a fifth and final step of

Add code to the comments.php of your theme.

An example was given, which one would expect, leaving only a few small problems, such as where to find comments.php (see list below), where to paste the code (although near the bottom is usually best, with the other <input> tags) and making any customisations necessary to fool bots.

Upgrading

That should be all that is needed to get the two plugins working, but it does present a small on-going concern that now each upgrade to WordPress requires careful hand-crafting of the comments.php file again. Over time, these several small changes can amount to a lot of work, and last time I did such an upgrade I realised that my notes are not up-to-date for making the process as simple as possible. As such I feel that now is a useful time to give myself a list of settings to be re-applied when the next version of WordPress comes out:

• Follow the instructions in the Codex, including the steps about turning off, copying across and turning on plugins
• Copy across the custom style sheet or it will default to a similar, but less complete theme (the colours will be noticeably different)
• Change /wp-content/themes/$themeName/sidebar.php to include the Atom link, and any other changes
• Change /wp-content/themes/$themeName/header.php to improve the semantics of the blog title text
• Change /wp-content/themes/$themeName/comments.php to add the anti-spam functionality discussed in this post
• Change /wp-includes/template-functions-general.php to set the date format of the sidebar (unless a recent bug fix provides the functionality)
• Change /wp-includes/links.php to give the sidebar section headings a common style

That’s quite a few changes. Also in anticipation of a future upgrade, I must point out that the creation of a tool that maximised the simplicity of making these changes, possibly under certain requirements that make more obvious options unfeasible, would be something worthy of my time.

Testing

Back to the issue of stopping spam (and letting through legitimate comments) you can see below that legitimate trackbacks are still allowed through, not being rejected as false positives, and that the ability to leave comments is open to unregistered users. In this regard, the plugins are a success, certainly not making the pages of my blog any worse, and even causing some improvement, by increasing the democratising opportunities for people to comment on what I write (although I reserve the right to screen any comments to maintain the signal to noise ratio — if someone wants to write thousands of words condemning me, they are free to write their own blog doing so).

The fact that a few good comments get through doesn’t immediately prove the success of the system. It is too early to say if the heurisitic of the trackback spam prevention plugin can really catch all the types of pages that a spammer can set up (although it would be 100% successful against the ones I have checked out so far). Perhaps more interesting is the comment spam prevention plugin, which is effectively a CAPTCHA-based defense, and that leads to several questions. The most common type of formal CAPTCHA is the “Type the letters in this image” test. I specifically did not choose this type because of the implications for certain categories of legitimate visitor, including the blind, users of text-only browsers, and users who may pay by the byte. Some text-only CAPTCHAs, on the other hand, have the problem that they are not neutral with respect to culture and language. With a blog written predominantly in English, it is not unreasonable to expect a comprehension of English from those who wish to comment, but I am fully aware that people who don’t speak English may still find my blog post while searching for error messages or code snippets, and want to leave a useful comment to assist speakers of their own language. My principle about linguistic accessibility is that the requirement to pass the CAPTCHA can still be met if the instructions are translated by a machine into another human language. For reference, using Google to translate the question at the bottom of the page into simplified Chinese and back into English produces this result:

Please enter the sum of these two figures :

One must also consider the chances of false negatives — of the system not detecting spam as being spam. When a friend exhibited their PEAR text-CAPTCHA package to me, I glibly commented that solving their maths question didn’t even require writing a compiled program or a parser or at all; one could simply extract the string from the first digit to the last and ask Google, then take the number after the equals sign. I hope I have managed to avoid the irony of relying on just the technology that I scourned, by using a form slightly less Googleable, but perhaps no harder.

Accepting the limited nature of the advantage I currently have against spammers, though, and, as mentioned, trying to avoid an arms race, in particular by making the challenge AI-complete, I should at least give a hint at the future advancements I’m prepared to make (if and when the spam problem makes this necessary). Of course, not publishing a solution for scrutiny is a sure sign of reliance on security through obscurity and ignorance of Schneier’s Law, so here goes. I believe that by coming up with a simple mathematical problem that is suitably unique to my blog, with parameters which can be programmatically and randomly changed by the plugin, an attacker would, in the general case, have to write software which can understand a sufficiently large subset of natural language as to be AI-complete. (When that happens, of course, I will re-examine a lot more than just access to my blog). In particular, by putting in numbers that aren’t used in the calculation (“Add the square numbers from this list”) and using alternatives to the numbers themselves (“number of sunrises in a day”, “number of players in a duet”, “number of sides on a triangle”) it is prohibitively expensive for a spammer to compete.

Conclusion

Given my premature confidence with the comment spam prevention plugin and cautious optimism about the trackback spam prevention plugin, I feel a need to try to express some recent thoughts on a certain sort of victory. This post already mentions the threat posed to Freedom on the Web, and the threat is real, but the “war” of which this battle is a part, the “War on Information” is one in which the victory of information is almost directly derivable from the laws of nature, at least assuming the world does not throw away the progress of the last 100 years. That is to say, we face a downhill battle if we want to expand the potential of the Internet, and we need only fight to limit the damage caused by our adversaries in their death throes. If society can’t win the “War on Computer Viruses” (I apologise to any Orwellians that I’m not picking vague enough concepts for these wars to be against) it certainly can’t stop people exercising their legally enshrined Fair Use rights (even by changing the law).

To show the connection between, but not the equivalence of, viruses, copyright infringement and Fair Use, consider this thought experiment: A virus is released (or commands are sent across a botnet) that result in millions of technologically-inexperienced computer users finding a “My Hollywood Movies” folder appear on their computer. It will have been made visible after filling with 10 hours (at a quality dependent on their harddrive space) of films downloaded by the virus over an anonymised, censorship resistant peer-to-peer file sharing network. Moreover, pretty HTML instructions will have been saved in the directory for how to use the newly installed peer-to-peer client. No anti-virus company would risk the class-action lawsuit from users who found that their legitimate peer-to-peer software had been disabled by it, and any news generated by the outbreak of this virus would just attract more people to the network. One attack from the media industries would be to poison the networks with fake files that matched whatever pattern the peer-to-peer software had been instructed to look for, but this could be countered by using a distributed Web of Trust and digital signing of uploads using anonymous keys. Now, I’m not naive enough to think that media industries would not sue people for the actions of a virus under some extremely dubious interpretation of “contributory infringement” or “negligence”, but it would at least require those industries to find a way around the protections of the peer-to-peer software and would prompt a debate about whether a country can afford to criminalise millions of citizens for merely owning a computer connected to the Internet.

This was just a thought experiment, but there have been so many times when the overwhelming power of information, exercised through the skill of hackers (and sadly crackers too) has defeated the effort of an adversary that is seemingly orders of magnitude more powerful. Whether it is releasing a new method for running Linux on the Xbox within two weeks of new security measures being imposed, or releasing a new method to format-shift iTunes songs within one day of Apple imposing a new restriction system, or the resumption of service by The Pirate Bay only three days after a potentially illegal raid on their (and others’) servers. So evocative and iconic are these examples of triumph far beyond mere survival, that they seem to need a term of their own to make them stand out from the everyday individual struggles we make for democracy, culture and the rule of law. The word I propose, then, is “smiting” from “smite”:

smite (v.tr.)

To deliver a powerful blow to suddenly and sharply

There are religious overtones to this word which I would want to avoid, but I think most definitions separate the word from any such relation. I merely intend to consider smitings in all their forms, and be impressed at the suddeness and power with which people can respond to curtailments of their freedom.

When will the “War on War” end?