Category Archives: Articles

Ubuntu Home Server 14.04 – A DIY NAS

5 Replies

It’s been more than 4 years since I wrote about home servers, but my Ubuntu Home Server article was, for a while, the most popular post on this blog. Since moving to the UK though, I’ve taken a more appliance-based approach to my home network. For the last few years I’ve been using a Boxee Box for media playback, and a 4-bay Netgear ReadyNAS duo NV2+ for storage, mainly to keep the bulk of my possessions to a minimum.

The appliance approach does have advantages. It is power efficient, easy to setup, and very low maintenance. But after getting an internet connection with decent upload speed, I wanted to run CrashPlan on the NAS without having to have another PC running. I managed to get it running by following directions I found here.

There’s just one problem:

3.3 months to upload 350GB is a little too long

3.3 months to upload 350GB is a little too long

Performance is abysmal, and I’ve only selected the most important data – my photos. I’m limited not by my internet connection, but by the NAS’s anaemic CPU and lack of ram (just 256Mb). Furthermore, it’s always had very slow read and write speeds – generally around 2Mb/sec, and loading a large directory via its Samba shares can take a while.

So I started to look for a replacement. My requirements:

  • Minimum 2GB ram
  • Strong CPU, preferably x86
  • 4+ drive bays
  • Linux based OS
  • Root access to said OS

The best pre-built option I could find which meets those requirements is the Thecus N5550, but at £383 it is a long way from cheap. And it barely meets the specs; an Atom CPU is strong for a NAS but not by modern x86 standards.

While the customised software shipped with a NAS does offer some conveniences, it also gets in the way of using newer Linux features such as BTFS RAID 5/6 (which is currently not considered stable but should be within the next 12 months). You’re also reliant on the vendor for distribution upgrades, and the priority is going to be shiny features which consumers will appreciate, not keeping the foundation OS up to date. The ReadyNAS NV2+ is currently running Debian Squeeze, and will be until the day support ends.

At this point I realised that a pre-made NAS with the level of power and flexibility I wanted doesn’t exist at a realistic price point. And with the end of Boxee support its days as a useful device are numbered, so a HTPC could be on the cards as well. It’s time to build my own server again.

Continue reading →

13″ Retina Macbook Pro (late 2013) – Buyer Review

1 Reply

As with all my reviews, this is a totally subjective personal view and not an in-depth technical analysis. For more mainstream reviews, check out Engadet, Pocket Lint, Expert Reviews, and Casey Johnston’s Air vs Pro comparison on Arstechnica if you’re also considering an Air.

Retina vs Dell

My old faithful Dell E4300 has done its dash. Actually it still works; it runs Ubuntu well, it has an SSD and 4GB of ram which makes it pretty nippy for web browsing and lightweight tasks, but what sealed its fate was my work laptop – a 15″ Retina Macbook Pro. After getting used to that gorgeous 2880×1800 screen, I found I just couldn’t go back to the Dell any more with its 1280×800 TN LCD (ugh), horrible touchpad and 2009-era performance.

Continue reading →

Pausing Spotify and playing a random video in Python – A party trick for Halloween

2 Replies

For a Halloween party last weekend I wrote a python script to pause Spotify, play a random video and start music playback again. The videos were basic ogg files I cobbled together which showed a scary image and evil laughs or screaming with OpenShot. I can’t really share them, as I don’t have rights to the media, but it’s pretty simple to recreate them yourself.

The code for this script is on Github, and I’ve reproduced the latest snapshot below. Feel free to fork and improve if you want to scare your guests, or add support for other OS’s. Presently it only supports Linux because I used dbus to perform the play/pause actions.

[python]
#!/usr/bin/python

”’
This is a Halloween party script which pauses Spotify and plays a video
at random intervals.
”’

import random
import subprocess
from subprocess import call
from time import sleep
import os
import datetime

start_time = datetime.time(21, 0, 0)
stop_time = datetime.time(23, 0, 0)

video_dir = ‘/home/alex/Videos/scream/’
videos = { ‘scream1_nofade.ogg’: 30,
‘happy.ogg’: 1,
‘evil_laugh.ogg’: 5,
}

def time_in_range(start, end, x):
“””Return true if x is in the range [start, end]”””
if start <= end:
print(“start<end”)
return start <= x <= end
else:
print(“end<start”)
return start <= x or x <= end

def weighted_choice(weights):
total = sum(weights for video in weights)
r = random.uniform(0, total)
upto = 0
print(“total: %s\nrandom: %s” % (total, r))

for video in weights:
w = weights
if upto + w > r:
return video
upto += w
assert False, “shouldn’t get here”

def spotifyPause():
command = “dbus-send –print-reply –dest=org.mpris.MediaPlayer2.spotify /org/mpris/MediaPlayer2 org.mpris.MediaPlayer2.Player.Pause”
print(“pausing spotify”)
os.system(command)

def spotifyPlay():
print(“playing spotify”)
command = “dbus-send –print-reply –dest=org.mpris.MediaPlayer2.spotify /org/mpris/MediaPlayer2 org.mpris.MediaPlayer2.Player.PlayPause”
os.system(command)

def play_video(video_file):
print(“Playing %s” % video_file)
#call([‘/usr/bin/mplayer’, ‘-fs’, video_file], stdout=None, stderr=None)
#result = subprocess.Popen([‘/usr/bin/mplayer’, ‘-really-quiet’, ‘-fs’, video_file])
result = subprocess.check_call([‘/usr/bin/mplayer’, ‘-really-quiet’, ‘-fs’, video_file], stdout=None, stderr=None)
return result

def playBuzz(buzzfile):
print(“Buzz…”)
result = subprocess.check_call([‘/usr/bin/mplayer’, ‘-really-quiet’, ‘-ss’, ’18’, buzzfile], stdout=None, stderr=None)
return result

def infiniteLoop():
while 1:
current_time = datetime.datetime.now().time()
#if current_time > stop_time or current_time < midday:

choice = weighted_choice(videos)

random_time = random.randrange(1200,2400)
random_time = 3

video_file = video_dir + choice
print(“Chose video %s after %s seconds” % (video_file, random_time))
sleep(random_time)

# Whether to play buzz
buzz = False
if random.randrange(0,100) > 90:
buzz = True

# Continue if outside time range
if not time_in_range(start_time, stop_time, current_time):
print(“Not playing video, outside time range”)
continue

# Do it
spotifyPause()
if buzz:
playBuzz(‘/home/alex/Videos/scream/audio/buzz.mp3’)
play_video(video_file)
spotifyPlay()

if __name__ == “__main__”:
infiniteLoop()
[/python]

5 tips on travelling with a wide-angle lens

6 Replies

Three years ago, on a bit of a whim, I bought a Canon EF-S 10-22mm lens. The action was probably symptomatic of Gear Acquisition Syndrome, but I’ve fortunately managed to keep it under control since – I’m still using the same set of lenses I bought around that time!
Canon EF-S 10-22mm f/3.5-5.6 USM

It was a pricey item, and to this day I don’t really know what possessed me to spend such a sum on a piece of glass that I didn’t really know how to use. What’s more, I wasn’t sure I’d use it regularly. But all the pictures in this article were taken with it, and these days it hardly leaves the camera.

Others have asked me about wide angle photography, and I’ve even loaned my 10-22 out a couple of times, so I thought it was about time I put down some words about traveling with a wide angle lens.

Continue reading →

Buying a PC Power Supply in the UK, January 2012

4 Replies

In the process of building my first desktop PC in about 4 years, I’ve come to realise that I no longer know anything about PC hardware. At one stage I considered myself, perhaps incorrectly, to be fairly knowledgeable about the subject of PC power supplies, even writing a PSU guide for a forum I used to frequent. Most of that guide is now irrelevant but looking back it wasn’t terrible advice for a newbie looking to buy a PSU at that time.
Continue reading →

Upgrading my E4300

3 Replies

Two years is quite a long time for me to keep a computer, but this Dell has lasted surprisingly well. It lags a bit in the 3D graphics department, but CPU wise it’s still perfectly acceptable and 4gb of ram is still a decent amount. And the 13″ form factor is perfect for my needs.

One of the major advantages of business models is the long product life cycle, which means an ample supply of spare parts and accessories. In the case of the Dell E series the docking stations and some of the other accessories are compatible with all other E series laptops. The greater amount of resources that go into the design and the higher build quality are also apparent – once you’ve had a good business laptop it’s hard to go back to disposable consumer machines!

Recently I’ve had a few gripes with it however:

  • The lack of bluetooth has become an inconvenience as the E4300 is chronically short of USB ports. Two really isn’t enough, and I want to get a bluetooth mouse to free up one of them.
  • I moved to the UK last year and the USA keyboard layout lacks the keys for € and £ symbols.
  • The fan started to buzz loudly now and again about six months ago. It was relatively infrequent and only annoyed me perhaps once a week, but it was a sure sign that the bearings were on the way out.

When the fan finally outright stalled I was able to get it to move again after a rest, but I decided it was time to kill 3 pigs with 1 bird and give the old girl a new lease of life.

I picked up the following parts on ebay: Continue reading →

Getting the most out of Terminator

21 Replies

Terminator in action

Terminator is a must-have tool for Linux administrators. It’s a terminal emulator that supports multiple terminals via tabs, but also by dividing up its window with horizontal and vertical splits.

The user documentation is a bit sparse, in fact what you see in the man page is what you get. In this post I’ll take you through the features that I think make Terminator the best terminal emulator around.
Continue reading →

Setting up a secure Ubuntu LAMP server

47 Replies

Disclaimer: This article is provided for your information only, and simply following this guide will not make your server “secure”. As the server administrator you are ultimately responsible for its security!

Intro

Having recently been through the process of setting up a few Ubuntu LAMP (Linux, Apache, MySQL, PHP) servers lately I thought I’d make an article out of my notes and provide a starters guide to setting up the LAMP stack on Ubuntu.

It goes without saying that the only truly secure computer is one with no network connection, no ports or input devices and is locked in a bank vault, but such a machine is not terribly useful. Regretfully, compromises must be made to allow functionality! Besides presuming insecurity, there are a lot of things you can do to make your server more secure and keep out the vast majority of would-be hackers running port scans, meta-exploit scripts and dictionary attacks.
Continue reading →

Recovering a RAID5 mdadm array with two failed devices

34 Replies

Update 1/11/2019

If you’ve reached this article by Googling how to recover a RAID array, I suggest you don’t use this guide. The Linux RAID wiki has much more correct, complete, and authoritative information. In retrospect I was very lucky not to destroy the data, but this article is an interesting account of how I did it.

Update

Before reading this article you should know that it is now quite old and there is a better method – ‘mdadm –assemble –force’ (it may have been there all along). This will try to assemble the array by marking previously failed drives as good. From the man page:

If mdadm cannot find enough working devices to start the array, but can find some devices that are recorded as having failed, then it will mark those devices as working so that the array can be started.

I would however strongly suggest that you first disconnect the drive that failed first. If you need to discover which device failed first, or assemble doesn’t work and you need to manually recreate the array, then read on.

I found myself in an interesting situation with my parents home server today (Ubuntu 10.04). Hardware wise it’s not the best setup – two of the drives are in an external enclose connected with eSATA cables. I did encourage Dad to buy a proper enclosure, but was unsuccessful. This is a demonstration of why eSATA is a very bad idea for RAID devices.

What happened was that one of the cables had been bumped, disconnecting one of the drives. Thus the array was running in a degraded state for over a month – not good. Anyway I noticed this when logging in one day to fix something else. The device wasn’t visible so I told Dad to check the cable, but unfortunately when he went to secure the cable, he must have somehow disconnected the another one. This caused a second drive to fail so the array immediately stopped.

Despite having no hardware failure, the situation is similar to someone replacing the wrong drive in a raid array. Recovering it was an interesting experience, so here I’ve documented the process.
Continue reading →

Configuring the backup system

Leave a reply

This article is part of a series about setting up a home server. See this article for further details.

Surprisingly, this is one of the easiest bits. If you don’t mind sticking with the options presented by the GUI, Back In Time makes backups so simple it’s almost criminal not to use it. The use of the GUI itself is fairly straightforward so I’m not going to go step by step and instead go for the important bits.

Just make sure you use the root shortcut (Back In Time – root) to prevent any permissions problems.

I’ve used NTFS for the backup volume because it supports hard links and is readable by Windows machines if something goes wrong. A native Linux file system would be preferable for many, but whatever you do don’t use FAT32 (FAT32 doesn’t support hard links, so every snapshot would consume 100% of its size whether the file was changed since the last backup or not).

Creating the Job

This is all done in the settings menu, which isn’t labelled but represented by the classic screwdriver and spanner icon – intuitive enough.

Under General, make sure you’re saving snapshots to your backup volume. Set the schedule to whatever you like, but I prefer to handle the schedule manually as it doesn’t give enough options. For a desktop machine the “daily” option would make sense, but as this machine will be on 24/7 I want it to run at a set time each day, not whenever it feels like it. So we will setup a cron job manually later.

Under the Include tab add your data folder (/media/data). Under exclude I removed all the preset options as I want everything on the data volume backed up. Everything that is except the lost+found folder, so I would suggest clicking Add folder and adding “/media/data/lost+found”.

The auto-remove options are up to you. I set the free space threshold to 1Gb, checked the smart-remove box, and chose not to remove named snapshots as they all seem fairly logical. The expert options don’t really need tweaking unless you want to do different schedules for different folders.

Click OK to save and you can now take a backup.

Altering the schedule

As I explained above we want to make sure the backup runs at a set time, which the gui for Back In Time doesn’t allow for, so fire up a terminal and enter the command: ‘sudo crontab -e’

The crontab is like task scheduler on Windows, but arguably a lot more powerful and flexible. The ‘-e’ option just tells crontab to edit the existing crontab instead of overwriting.

The screenshot below shows my crontab.

The @daily line is the line that the Back In Time gui added. I’m not so concerned about ‘niceness’ at 4am (nice values on Linux serve the same purpose as task priority on Windows), so I left that out. The final line is:
0 4 * * * /usr/bin/backintime --backup-job >/dev/null 2>&1

For an explanation of the crontab, see this crontab quick reference. Basically all you need to know though, is that the first number is the minute and the second is the hour. So if for example you would rather it ran at 1.30am instead of 4am, change the first number to 30 and the second to 1 so it reads:
30 1 * * * /usr/bin/backintime --backup-job >/dev/null 2>&1

Later on we will modify this to also email the result.

Important Caveat

I just discovered that the Back In Time gui blitzes any lines in the the crontab that contain the string “backintime” whenever you click OK from the preferences window. This is a rather annoying problem, as I can easily see this happening.

I recommend making sure the gui schedule is set to every day rather than disabled, which means that if someone does fiddle at least the backup will still happen once a day. The solution to this is to call a wrapper script which does not contain “backintime” in its name… I’ll update this once I’ve written and tested it.

Next part – Monitoring and email configuration