Category Archives: IT

IT related posts, technical stuff

Safely running bulk operations on Redis with lua scripts

This article was also posted on the Gumtree devteam blog

If there was one golden rule when working with redis in production, it would be

“Don’t use KEYS”

The reason for this is that it blocks the redis event loop until it completes, i.e. while it’s busy scanning its entire keyspace, it can’t serve any other clients.

Recently, we had a situation where code was storing keys in redis without setting an expiry time, with the result that our keyspace started to grow:
Continue reading

13″ Retina Macbook Pro (late 2013) – Buyer Review

As with all my reviews, this is a totally subjective personal view and not an in-depth technical analysis. For more mainstream reviews, check out Engadet, Pocket Lint, Expert Reviews, and Casey Johnston’s Air vs Pro comparison on Arstechnica if you’re also considering an Air.

Retina vs Dell

My old faithful Dell E4300 has done its dash. Actually it still works; it runs Ubuntu well, it has an SSD and 4GB of ram which makes it pretty nippy for web browsing and lightweight tasks, but what sealed its fate was my work laptop – a 15″ Retina Macbook Pro. After getting used to that gorgeous 2880×1800 screen, I found I just couldn’t go back to the Dell any more with its 1280×800 TN LCD (ugh), horrible touchpad and 2009-era performance.

Continue reading

It’s nice to be right some times

Five short years ago I wrote an article about my desire for a Nokia N900. I was extremely enthusiastic about the device, which I saw as the future of computing and a sign of things to come. I also said:

Personally I think Linux usage overtaking Windows on personal computing devices is inevitable, and this is how it’s going to happen (although the capabilities of the N900 will have to move down to a much lower price point first). We’ll see if I’m right in 5-10 years time.

It’s now 4 years and 4 months later. I was right about Linux overtaking windows on personal computing devices, but I was wrong about how, and it happened far more quickly than I could have imagined.

Continue reading

Which laptop to buy, 2014 edition

In what could only fall firmly into the first-world problems category, I’m currently suffering a dilemma as to what laptop I should buy. My requirements are common – a good balance of power, performance and portability. I’ve decided the specification I should go for is:

  • Intel Core i5 (4th generation, Haswell)
  • 8Gb ram
  • 256Gb SSD
  • 13″ display, resolution at least 1920×1080

I think these specs make for the best price / performance balance on most of the laptops I’ve priced up.

Continue reading

Fixing Puppet 3.2 symlinks on OSX Mavericks

Received the following error when running puppet after upgrading to Mavericks:

/System/Library/Frameworks/Ruby.framework/Versions/2.0/usr/lib/ruby/2.0.0/rubygems/core_ext/kernel_require.rb:45:in `require': cannot load such file -- puppet/util/command_line (LoadError)
	from /System/Library/Frameworks/Ruby.framework/Versions/2.0/usr/lib/ruby/2.0.0/rubygems/core_ext/kernel_require.rb:45:in `require'
	from /usr/bin/puppet:3:in `<main>'

Solution is to symlink the packages to the new ruby 2.0.0 directory:

#!/bin/bash

sudo ln -s /usr/lib/ruby/site_ruby/1.8/puppet /System/Library/Frameworks/Ruby.framework/Versions/2.0/usr/lib/ruby/site_ruby/2.0.0/puppet
sudo ln -s /usr/lib/ruby/site_ruby/1.8/puppet.rb /System/Library/Frameworks/Ruby.framework/Versions/2.0/usr/lib/ruby/site_ruby/2.0.0/puppet.rb
sudo ln -s /usr/lib/ruby/site_ruby/1.8/semver.rb /System/Library/Frameworks/Ruby.framework/Versions/2.0/usr/lib/ruby/site_ruby/2.0.0/semver.rb
sudo ln -s /usr/lib/ruby/site_ruby/1.8/facter /System/Library/Frameworks/Ruby.framework/Versions/2.0/usr/lib/ruby/site_ruby/2.0.0/facter
sudo ln -s /usr/lib/ruby/site_ruby/1.8/facter.rb /System/Library/Frameworks/Ruby.framework/Versions/2.0/usr/lib/ruby/site_ruby/2.0.0/facter.rb
sudo ln -s /usr/lib/ruby/site_ruby/1.8/hiera /System/Library/Frameworks/Ruby.framework/Versions/2.0/usr/lib/ruby/site_ruby/2.0.0/hiera
sudo ln -s /usr/lib/ruby/site_ruby/1.8/hiera.rb /System/Library/Frameworks/Ruby.framework/Versions/2.0/usr/lib/ruby/site_ruby/2.0.0/hiera.rb

Should be fixed in the next major version.

Reference: https://projects.puppetlabs.com/issues/18205

Changing boot order in Ubuntu 13.04 (or Debian) – the easy way

I wanted nice, concise instructions on changing the boot order in Ubuntu 13.04, which uses Grub 2. Being a newbie focused OS however, Googling “ubuntu boot order” results in SEO blogs with lots of fluff, and then the actual instructions start out with “install package from ppa”…

What the hell, I just want to change the boot order!

Continue reading

Calling mysqldump in Python

Python is a fantastic tool to know, and despite being a beginner I find myself using it more and more for everyday tasks. Bash is great for knocking together quick scripts, but when you want to something a little more complex such as interfacing with APIs or other systems over a network, you really need a more fully-featured programming language.

The topic of this post, however, is the kind of task that bash is perfect for. Thanks to mysqldump, a database backup script can be written in a few lines and dump/restores are easily automated. So why on earth would we do this in Python?
Continue reading

How not to program Bash

Came across this gem today:

COMP=$1

if [ -e $COMP ]; then
    echo "Please supply a competition_id"
    exit
fi

On first read it looks backwards – “if exist $COMP then exit”, and the programmer clearly wanted to do the exact opposite. However it actually works as desired. Mostly.

If no argument is supplied, $COMP will be a blank string, i.e. “”. Thus [ -e $COMP ] will parse to [ -e ], which returns 0 (effectively true). It’s basically like saying “does nothing exist”.

This is because the -e argument tests whether a file exists. So if an argument is supplied, [ -e $COMP ] will return 1 (effectively false), which does what the programmer intended UNLESS the argument supplied is a valid file path.

In this case a number was expected, so sure it’s unlikely to fail in this way, but it’s still an incredibly bad way to test if an argument is set. Not to mention confusing to read!

The correct way to test this by the way would be to use -z, which simply tests if a string’s length is zero:

COMP=$1

if [ -z "$COMP" ]; then
    echo "Please supply a competition id"
    exit
fi

Or better still, use getopts. For more info run ‘man test’ from a bash terminal.

Xen Server “The SR failed to complete the operation”

Had this problem when trying to start a newly created VM and install from certain ISO files. Some ISO images would work and others would not.

Removing the ISO image allowed the VM to start but obviously there was no OS image to install from.

The error message is generic and not very helpful, but if you do encounter it check that the ISO is readable by the Xen server. In our case it was on an NFS share but the permissions were read-only to all but the owner…. so a simple `chmod a+r *.iso` fixed it!