Category Archives: Linux

Building and Packaging a Python command-line tool for Debian

python-logo-notext-svgPython packaging has a chequered past.

Distutils was and still is the original tool included with the standard library. But then setuptools was created to overcome the limitations of distutils, gained wide adoption, subsequently stagnated, and a fork called distribute was created to address some of the issues. Distutils2 was an attempt to take the best of previous tools to support Python 3, but it failed. Then distribute grew to support Python 3, was merged back in to setuptools, and everything else became moot!

Unfortunately, it’s hard to find reliable information on python packaging, because many articles you might find in a Duckduckgo search were created before setuptools was reinvigorated. Many reflect practices that are sub-optimal today, and I would disregard anything written before the distribute merge, which happened in March 2013. Continue reading

Ubuntu Home Server 14.04 – A DIY NAS

It’s been more than 4 years since I wrote about home servers, but my Ubuntu Home Server article was, for a while, the most popular post on this blog. Since moving to the UK though, I’ve taken a more appliance-based approach to my home network. For the last few years I’ve been using a Boxee Box for media playback, and a 4-bay Netgear ReadyNAS duo NV2+ for storage, mainly to keep the bulk of my possessions to a minimum.

The appliance approach does have advantages. It is power efficient, easy to setup, and very low maintenance. But after getting an internet connection with decent upload speed, I wanted to run CrashPlan on the NAS without having to have another PC running. I managed to get it running by following directions I found here.

There’s just one problem:

3.3 months to upload 350GB is a little too long

3.3 months to upload 350GB is a little too long

Performance is abysmal, and I’ve only selected the most important data – my photos. I’m limited not by my internet connection, but by the NAS’s anaemic CPU and lack of ram (just 256Mb). Furthermore, it’s always had very slow read and write speeds – generally around 2Mb/sec, and loading a large directory via its Samba shares can take a while.

So I started to look for a replacement. My requirements:

  • Minimum 2GB ram
  • Strong CPU, preferably x86
  • 4+ drive bays
  • Linux based OS
  • Root access to said OS

The best pre-built option I could find which meets those requirements is the Thecus N5550, but at £383 it is a long way from cheap. And it barely meets the specs; an Atom CPU is strong for a NAS but not by modern x86 standards.

While the customised software shipped with a NAS does offer some conveniences, it also gets in the way of using newer Linux features such as BTFS RAID 5/6 (which is currently not considered stable but should be within the next 12 months). You’re also reliant on the vendor for distribution upgrades, and the priority is going to be shiny features which consumers will appreciate, not keeping the foundation OS up to date. The ReadyNAS NV2+ is currently running Debian Squeeze, and will be until the day support ends.

At this point I realised that a pre-made NAS with the level of power and flexibility I wanted doesn’t exist at a realistic price point. And with the end of Boxee support its days as a useful device are numbered, so a HTPC could be on the cards as well. It’s time to build my own server again.

Continue reading

13″ Retina Macbook Pro (late 2013) – Buyer Review

As with all my reviews, this is a totally subjective personal view and not an in-depth technical analysis. For more mainstream reviews, check out Engadet, Pocket Lint, Expert Reviews, and Casey Johnston’s Air vs Pro comparison on Arstechnica if you’re also considering an Air.

Retina vs Dell

My old faithful Dell E4300 has done its dash. Actually it still works; it runs Ubuntu well, it has an SSD and 4GB of ram which makes it pretty nippy for web browsing and lightweight tasks, but what sealed its fate was my work laptop – a 15″ Retina Macbook Pro. After getting used to that gorgeous 2880×1800 screen, I found I just couldn’t go back to the Dell any more with its 1280×800 TN LCD (ugh), horrible touchpad and 2009-era performance.

Continue reading

It’s nice to be right some times

Five short years ago I wrote an article about my desire for a Nokia N900. I was extremely enthusiastic about the device, which I saw as the future of computing and a sign of things to come. I also said:

Personally I think Linux usage overtaking Windows on personal computing devices is inevitable, and this is how it’s going to happen (although the capabilities of the N900 will have to move down to a much lower price point first). We’ll see if I’m right in 5-10 years time.

It’s now 4 years and 4 months later. I was right about Linux overtaking windows on personal computing devices, but I was wrong about how, and it happened far more quickly than I could have imagined.

Continue reading

Pausing Spotify and playing a random video in Python – A party trick for Halloween

For a Halloween party last weekend I wrote a python script to pause Spotify, play a random video and start music playback again. The videos were basic ogg files I cobbled together which showed a scary image and evil laughs or screaming with OpenShot. I can’t really share them, as I don’t have rights to the media, but it’s pretty simple to recreate them yourself.

The code for this script is on Github, and I’ve reproduced the latest snapshot below. Feel free to fork and improve if you want to scare your guests, or add support for other OS’s. Presently it only supports Linux because I used dbus to perform the play/pause actions.

#!/usr/bin/python

'''
This is a Halloween party script which pauses Spotify and plays a video
at random intervals.
'''

import random
import subprocess
from subprocess import call
from time import sleep
import os
import datetime

start_time = datetime.time(21, 0, 0)
stop_time = datetime.time(23, 0, 0)

video_dir = '/home/alex/Videos/scream/'
videos = { 'scream1_nofade.ogg': 30,
'happy.ogg': 1,
'evil_laugh.ogg': 5,
}

def time_in_range(start, end, x):
"""Return true if x is in the range [start, end]"""
if start <= end:
print("start<end")
return start <= x <= end
else:
print("end<start")
return start <= x or x <= end

def weighted_choice(weights):
total = sum(weights[video] for video in weights)
r = random.uniform(0, total)
upto = 0
print("total: %s\nrandom: %s" % (total, r))

for video in weights:
w = weights[video]
if upto + w > r:
return video
upto += w
assert False, "shouldn't get here"

def spotifyPause():
command = "dbus-send --print-reply --dest=org.mpris.MediaPlayer2.spotify /org/mpris/MediaPlayer2 org.mpris.MediaPlayer2.Player.Pause"
print("pausing spotify")
os.system(command)

def spotifyPlay():
print("playing spotify")
command = "dbus-send --print-reply --dest=org.mpris.MediaPlayer2.spotify /org/mpris/MediaPlayer2 org.mpris.MediaPlayer2.Player.PlayPause"
os.system(command)

def play_video(video_file):
print("Playing %s" % video_file)
#call(['/usr/bin/mplayer', '-fs', video_file], stdout=None, stderr=None)
#result = subprocess.Popen(['/usr/bin/mplayer', '-really-quiet', '-fs', video_file])
result = subprocess.check_call(['/usr/bin/mplayer', '-really-quiet', '-fs', video_file], stdout=None, stderr=None)
return result

def playBuzz(buzzfile):
print("Buzz...")
result = subprocess.check_call(['/usr/bin/mplayer', '-really-quiet', '-ss', '18', buzzfile], stdout=None, stderr=None)
return result

def infiniteLoop():
while 1:
current_time = datetime.datetime.now().time()
#if current_time > stop_time or current_time < midday:

choice = weighted_choice(videos)

random_time = random.randrange(1200,2400)
random_time = 3

video_file = video_dir + choice
print("Chose video %s after %s seconds" % (video_file, random_time))
sleep(random_time)

# Whether to play buzz
buzz = False
if random.randrange(0,100) > 90:
buzz = True

# Continue if outside time range
if not time_in_range(start_time, stop_time, current_time):
print("Not playing video, outside time range")
continue

# Do it
spotifyPause()
if buzz:
playBuzz('/home/alex/Videos/scream/audio/buzz.mp3')
play_video(video_file)
spotifyPlay()

if __name__ == "__main__":
infiniteLoop()

Changing boot order in Ubuntu 13.04 (or Debian) – the easy way

I wanted nice, concise instructions on changing the boot order in Ubuntu 13.04, which uses Grub 2. Being a newbie focused OS however, Googling “ubuntu boot order” results in SEO blogs with lots of fluff, and then the actual instructions start out with “install package from ppa”…

What the hell, I just want to change the boot order!

Continue reading

Calling mysqldump in Python

Python is a fantastic tool to know, and despite being a beginner I find myself using it more and more for everyday tasks. Bash is great for knocking together quick scripts, but when you want to something a little more complex such as interfacing with APIs or other systems over a network, you really need a more fully-featured programming language.

The topic of this post, however, is the kind of task that bash is perfect for. Thanks to mysqldump, a database backup script can be written in a few lines and dump/restores are easily automated. So why on earth would we do this in Python?
Continue reading

How not to program Bash

Came across this gem today:

COMP=$1

if [ -e $COMP ]; then
    echo "Please supply a competition_id"
    exit
fi

On first read it looks backwards – “if exist $COMP then exit”, and the programmer clearly wanted to do the exact opposite. However it actually works as desired. Mostly.

If no argument is supplied, $COMP will be a blank string, i.e. “”. Thus [ -e $COMP ] will parse to [ -e ], which returns 0 (effectively true). It’s basically like saying “does nothing exist”.

This is because the -e argument tests whether a file exists. So if an argument is supplied, [ -e $COMP ] will return 1 (effectively false), which does what the programmer intended UNLESS the argument supplied is a valid file path.

In this case a number was expected, so sure it’s unlikely to fail in this way, but it’s still an incredibly bad way to test if an argument is set. Not to mention confusing to read!

The correct way to test this by the way would be to use -z, which simply tests if a string’s length is zero:

COMP=$1

if [ -z "$COMP" ]; then
    echo "Please supply a competition id"
    exit
fi

Or better still, use getopts. For more info run ‘man test’ from a bash terminal.

Creating samba share in Nautilus: ‘net usershare’ returned error 255

I was having this problem on Ubuntu 12.04 (precise), but most of the Google results pointed to a bug in Hardy. However there are other causes of this problem.

In my case it was a previously-created share with a different user ID – Nautilus couldn’t create the share because there was already a share file with the same name owned by a different user.

The directory is /var/lib/samba/usershares. You should already have write access assuming you’re a member of the sambashare group (which the gui should handle for you), so all that remains to be done is remove the offending share with the same name as the one you’re trying to create.

alex@al4:~$ cd /var/lib/samba/usershares/
alex@al4:/var/lib/samba/usershares$ ls -lah
total 16K
drwxrwx--T 2 root       sambashare 4.0K Jul 25 12:33 .
drwxr-xr-x 5 root       root       4.0K May  1 10:40 ..
-rw-r--r-- 1 2046297271 2046296576  142 Oct 25  2011 music
-rw-r--r-- 1 2046297271 2046296576  128 Feb  7 17:13 videos
alex@al4:/var/lib/samba/usershares$ sudo rm music
[sudo] password for alex:
alex@al4:/var/lib/samba/usershares$ sudo rm videos
alex@al4:/var/lib/samba/usershares$

After doing the above, Nautilus was able to recreate the shares without trouble.

Photo workflow script

After trying several photo management tools I’ve concluded that the best way to manage a photo library is the simplest – copy the files yourself and name the directory.

I also often shoot in RAW+JPG mode on my camera and like to keep the files separate to make viewing saner, so under each event directory I will have a RAW and JPG directory as well.

After downloading, the first step is always to delete the photos that didn’t come out. With both raw and jpeg files to delete this process is a bit of a pain, especially when they’re in separate directories, so I wrote a script that deletes the raw file when the jpg is missing. That way I can simply browse through the JPG directory with Eye of Gnome, delete the photos I don’t want, and the script handles cleanup of the RAW directory.

https://github.com/al4/scripts/blob/master/bash/rawdel.sh

The code is below but you should use the github link above in case it gets improved at some point in the future.

#!/bin/bash

# rawdel.sh, a photo workflow script to delete raw files when the jpg has been removed
# By Alex Forbes
#
# I frequently shoot in RAW+JPG mode and when downloading from the camera I like to separate 
# the raw and jpg files into separate directories. I then go through the jpg directory and 
# delete the rejects. It is a pain to have to manually delete the corresponding raw files as 
# well, so I wrote a script to do it for me.
#
# It simply removes RAW files from a directory when the corresponding JPG file has been removed.

# Set these
rawextn="CR2"	# raw file extension (Canon is CR2)
rawdir="./RAW"	# directory where raw files reside
jpgdir="./JPG"	# directory where jpg files reside
				# rawdir and jpgdir can be the same

# Working variables, leave as-is
list=""			# list of files that have been deleted
rawlist=""		# the list of raw files that we will delete
filecount=""	# number of files we will delete

# Operate on each raw file
for f in $(ls -1 $rawdir/*.$rawextn); do 
	# Corresponding JPG file is:
	jpgfile=$(basename $f | sed "s/\.$rawextn$/.JPG/")

	# If this JPG file doesn't exist
	if [ ! -f $jpgdir/$jpgfile ]; then
		# Add to our list of files that have been deleted
		list=$(echo -e "${jpgfile} ${list}")
	fi
done

# Convert jpg filenames back to corresponding raw filenames
rawlist=$(echo ${list} | sed 's/\.JPG$/.CR2/g')
filecount=$(echo -e ${rawlist}| awk 'END{print NF}')

if [ $filecount == 0 ]; then
	echo "No files to delete"
	exit 0
fi

echo -e "About to remove $filecount files:\n${rawlist}"
read -p "Continue? [Y/N] " prompt
 
if [[ $prompt = "Y" || $prompt = "y" ]]; then
	# Delete all files in the list
	for f in ${rawlist}; do
		rm -v $rawdir/$f
	done
	exit 0
else
	echo -e "\nAborted."
	exit 1
fi