Author Archives: Alex

N900 PR 1.2 finally released

It’s about time.

I know it’s really sad to eagerly await a firmware update, but since this one fixes a lot of bugs, improves stability and adds Skype video calling, I think any eagerness is warranted.

The update was released over-the-air in the UK today and the worldwide release will follow tomorrow. But rather than wait I elected to do a clean flash.

First impressions? I’ve hardly had a chance to play with it, but it looks good so far. I don’t think it’ll be enough to stop me wishing I’d bought a Nexus One though.

Links

Configuring the backup system

This article is part of a series about setting up a home server. See this article for further details.

Surprisingly, this is one of the easiest bits. If you don’t mind sticking with the options presented by the GUI, Back In Time makes backups so simple it’s almost criminal not to use it. The use of the GUI itself is fairly straightforward so I’m not going to go step by step and instead go for the important bits.

Just make sure you use the root shortcut (Back In Time – root) to prevent any permissions problems.

I’ve used NTFS for the backup volume because it supports hard links and is readable by Windows machines if something goes wrong. A native Linux file system would be preferable for many, but whatever you do don’t use FAT32 (FAT32 doesn’t support hard links, so every snapshot would consume 100% of its size whether the file was changed since the last backup or not).

Creating the Job

This is all done in the settings menu, which isn’t labelled but represented by the classic screwdriver and spanner icon – intuitive enough.

Under General, make sure you’re saving snapshots to your backup volume. Set the schedule to whatever you like, but I prefer to handle the schedule manually as it doesn’t give enough options. For a desktop machine the “daily” option would make sense, but as this machine will be on 24/7 I want it to run at a set time each day, not whenever it feels like it. So we will setup a cron job manually later.

Under the Include tab add your data folder (/media/data). Under exclude I removed all the preset options as I want everything on the data volume backed up. Everything that is except the lost+found folder, so I would suggest clicking Add folder and adding “/media/data/lost+found”.

The auto-remove options are up to you. I set the free space threshold to 1Gb, checked the smart-remove box, and chose not to remove named snapshots as they all seem fairly logical. The expert options don’t really need tweaking unless you want to do different schedules for different folders.

Click OK to save and you can now take a backup.

Altering the schedule

As I explained above we want to make sure the backup runs at a set time, which the gui for Back In Time doesn’t allow for, so fire up a terminal and enter the command: ‘sudo crontab -e’

The crontab is like task scheduler on Windows, but arguably a lot more powerful and flexible. The ‘-e’ option just tells crontab to edit the existing crontab instead of overwriting.

The screenshot below shows my crontab.

The @daily line is the line that the Back In Time gui added. I’m not so concerned about ‘niceness’ at 4am (nice values on Linux serve the same purpose as task priority on Windows), so I left that out. The final line is:
0 4 * * * /usr/bin/backintime --backup-job >/dev/null 2>&1

For an explanation of the crontab, see this crontab quick reference. Basically all you need to know though, is that the first number is the minute and the second is the hour. So if for example you would rather it ran at 1.30am instead of 4am, change the first number to 30 and the second to 1 so it reads:
30 1 * * * /usr/bin/backintime --backup-job >/dev/null 2>&1

Later on we will modify this to also email the result.

Important Caveat

I just discovered that the Back In Time gui blitzes any lines in the the crontab that contain the string “backintime” whenever you click OK from the preferences window. This is a rather annoying problem, as I can easily see this happening.

I recommend making sure the gui schedule is set to every day rather than disabled, which means that if someone does fiddle at least the backup will still happen once a day. The solution to this is to call a wrapper script which does not contain “backintime” in its name… I’ll update this once I’ve written and tested it.

Next part – Monitoring and email configuration

Creating user accounts and setting up the file shares

This article is part of a series about setting up a home server. See this article for further details.

In this section we will create accounts for each user that will access the server, create a folder for each user, make sure the permissions are sane, and configure the samba shares.

For home environments a single user account that everyone uses can be good enough. However I like to have some semblance of security to raise barriers for viruses (who knows what’s going to be connecting to the network), so I setup guest to be read-only and assign write permissions only to authenticated users.

But before we proceed…

One Caveat when doing remote administration in a NeatX session

For some reason NeatX breaks policykit, which means any buttons in control panel applets that require root privileges will simply fail to work.

The way around this is to run the applets with gksu. The most convenient way to do this in my opinion is to create a desktop shortcut.

Go to System > Administration, right click on the users and groups icon in the menu, and “Add this launcher to desktop”. Next, right click on the resulting desktop icon, click properties and in the “Command” field, prepend gksu so that it reads “gksu users-admin”.

Double-clicking on the icon should then prompt you for your password, and all the buttons will work. Hopefully in the future this won’t be necessary!

I also created desktop shortcuts for Disk Utility and Back In Time (root).

Adding the users

Creating users is simple enough, but afterwards we need to add the users to two groups – “sambashare” and “users” (you should be able to figure this out). After doing this, go to Advanced Settings > select the Advanced tab and change the main group to users.

The reason for changing the primary group is so that any files the user creates are also accessible to others in the users group – which will include anyone that we want to be able to access files on the server. If you want to keep files private it is best to leave the primary group as the user name. Old school Unix people tend to know this, but for Windows refugees the lower level Linux concepts such as user groups and file system permissions can seem a bit strange, as they work quite differently.

Folders and Permissions

The raid array in my home server is mounted at /media/data. I like mount points to be owned by root to avoid accidental tampering:
chown root.root /media/data
chmod 755 /media/data

The octal permissions are 755, which means read/write/execute for the owner, read/execute for the group and all other users. For newbies I must confess that the rwx permissions notation is easier to understand, but unfortunately I learned octal permissions and it’s become a habit!

Under the data folder I have a folder for each user. The owner of the folder is the user, and the group is users, as I want Mum and Dad to be able to see each others files:
mkdir /media/data/mum
chown -R mum.users /media/data/mum
chmod -R 775 /media/data/mum

Repeat this for each user (substitute the user for “mum” in the example above). Note the use of the -R switch which applies the command to all sub-folders and files.

If Mum wanted keep her files private, both the owner and group would be the user name, e.g.:
chown -R mum.mum /media/data/mum
And the permissions would be:
chmod -R 700 /media/data/mum
Remember to make sure the primary group is the user name as well.

Setting up sharing (Samba)

I’m not 100% sure I’ve done this the officially sanctioned way, especially since it involves the decidedly old-school method of editing smb.conf. However for anyone comfortable with the terminal I think it works perfectly well.

First open /etc/samba/smb.conf in your favourite text editor (my preference is vim). There’s no need to modify any of the configuration, so scroll down to the bottom where the shares are located. I always comment out the printer shares (print$ and printers), as sharing printers via samba is a fool’s errand in my opinion, just get a blimmin’ network printer.

My shares are setup as follows:
[Data]
comment = Raid5 array, backed up daily
path = /media/data
browseable = yes
read only = no
guest ok = yes

[Backup]
comment = Backup drive, read only, no guest
path = /media/backup
browseable = yes
read only = yes
guest ok = no

[Media]
comment = Media files for XBMC
path = /media/media
browseable = yes
read only = no
guest ok = yes

Some explanation is definitely needed. Firstly while [Data] allows guests and the share is not read-only, guests will not be able to write because of the file system permissions which only allow the owner and group to modify files. You may want to create a public folder with permissions 777, which would allow guests to copy files on to the server. Or you may want to set up a another share and change guest ok to “no” for the data share.

The backup drive is read-only because I don’t want anyone to modify files on the backup drive, and file system permissions are no protection due to it being NTFS (Linux doesn’t and really shouldn’t support NTFS permissions). It would be too easy to go back to a previous version of a file and accidentally save it, and I’m not sure how Back In Time would handle a backed up file being newer than the source. Altering a file in the backup would also change every linked copy, so basically writing to files on the backup volume is bad mmkay? It is shared only to make restoring previous versions convenient.

NFS?

I haven’t covered setting up NFS here, as Mum and Dad both run Windows machines. If you do decide to setup NFS it’s fairly straightforward, but to save yourself some pain make sure the user ID’s match on all machines – NFS matches uid and not the actual user name. Off the top of my head the packages to install are nfs-common and portmap, and the config file to modify is /etc/exports.

A final note on samba passwords

I have found that a login to the local machine is required in order for the samba password to be synchronised with the unix password. If after logging in you still can’t access samba shares with that account, use the command smbpasswd to set the password, e.g.:
sudo smbpasswd mum

If you need to restart samba you can do so with the command ‘service smbd restart’.

Next section – Configuring the backup system

Running EpNamer on Ubuntu

EpNamer is a useful tool for renaming TV episodes. It’s not open source, but it is written in .NET and the developer has released a version that works with mono.

To run it on Ubuntu, you need to download the Mac OSX version from epnamer.com, mount the image, copy the application to your home folder and install some mono dependencies.

This would be quite complicated via gui so I figure 6 lines at the command line is going to be easier for most:

wget http://www.epnamer.com/?dl=EpNamer1.4.4.dmg
mount -o loop -t hfsplus ./EpNamer1.4.4.dmg /mnt
mkdir ~/epnamer
cp -R /mnt/EpNamer.app/Contents/Resources/* ~/epnamer
sudo apt-get install mono-winforms*
mono ~/epnamer/EpNamer.exe

If anyone’s found a better tool for renaming media files on Linux I’d be keen to hear about it! Unfortunately I couldn’t get the current Windows release of EpNamer (1.4.5) working.

Configuring the raid array

This article is part of a series about setting up a home server. See this article for further details.

Most of this can actually be achieved with the GUI these days, RedHat’s disk utility has improved a lot since the version that was included with karmic. To load it go to Settings > Administration > Disk Utility.

Make sure all the disks to be used have no partitions on them, then go to Create > Raid array.
512KiB was the default stripe size and is what I used initially, but I later switched to a 128KiB stripe. The example below uses a 512KiB stripe.

Also, it is a good idea to reduce the size to 128mb or so below the max capacity. Drives of the same advertised capacity can vary slightly in actual size, and if you replace a disk you don’t want the rebuild to fail because the drive is a few megabytes too small.

After selecting the disk members and pressing create the array will be in a “degraded” state until all the disks are synchronised. The amount of time it takes depends on the size of the array, but I’d suggest letting it finish before proceeding, I received errors in disk utility if I tried to create a partition too soon.

Also note that I’ve used the GPT partitioning scheme. GPT is designed to replace the old MBR scheme which has some limitations and can be restrictive these days, so I elected to use GPT. MBR is a safer option if your array is less than 2TB. If you use new hard drives with 4096-byte sectors such as Western Digital “advanced format” drives, you should use GPT. The use of GPT means that fdisk cannot be used, because it doesn’t support it. In its place we use parted and gdisk (“aptitude install gdisk” if you don’t yet have it).

Creating a partition on the array

The only slightly tricky part is creating a partition that is aligned with the raid stripe. You can’t create a partition starting at sector 0 because that’s where the partition table lives, so disk utilities will always offset the start of the first partition. However in order to get the best performance you need to align the partitions with the stripe size.

After creating a partition with disk utility I was greeted with the following:

It appears that disk utility isn’t quite intelligent enough to create properly aligned partitions on its own just yet.

In my case the partition was offset by 17.4KiB, which doesn’t align with the stripe size of 512KiB:

I found the easiest way to get a properly aligned partition was to create a partition in disk utility with no file system, and note the offset given to you by the warning message that appears after the partition has been created. Then, simply delete the partition and use gdisk to create the partition with the original offset plus the figure given to you in the warning.

To get the existing offset, run “parted /dev/md0”, type “unit b” to switch the working units to bytes and type p to print a list of partitions on the volume.

In my example the original offset was 17408 bytes (34 sectors * 512 bytes/sector), and the partition was out of alignment by 506880 bytes. This means it should actually be at byte 524288, which also happens to match the stripe size of 512KiB.

gdisk works in sectors however, so we need to divide the result by 512, giving sector 1024 (524288/512=1024). So in this example you would run gdisk, type n to create a new partition, enter 1024 for the first sector, and accept the default for the last. For the current type I used code 0700, which is “Linux/Windows data”.

The second time around I used a stripe size of 128KiB (it seemed more “normal”), and with this stripe you would offset the start of the partition by 256 sectors (131072 bytes). I don’t believe the stripe size matters much – larger stripe sizes probably perform marginally better with larger files but I wanted something more general purpose as there will be a lot of small documents as well as large photos.

An example of creating a partition in gdisk follows. Note that I’m using /dev/null in this example to avoid destroying my laptop, you would probably want to use the actual blank raid array, which is likely to be /dev/md0. You can delete partitions with gdisk (with d), but for the newbies it’s probably easier to delete any previous attempts in the gui disk utility first.
sudo gdisk /dev/null
GPT fdisk (gdisk) version 0.5.1

Command (? for help): n
Partition number (1-128, default 1): 1
First sector (34-18446744073709551582, default = 34) or {+-}size{KMGT}: 256
Last sector (256-18446744073709551582, default = 18446744073709551582) or {+-}size{KMGT}:
Current type is 'Unused entry'
Hex code (L to show codes, 0 to enter raw code): 0700
Changed system type of partition to 'Linux/Windows data'

Command (? for help): w

Final checks completed. About to write GPT data. THIS WILL OVERWRITE EXISTING
MBR PARTITION!! THIS PROGRAM IS BETA QUALITY AT BEST. IF YOU LOSE ALL YOUR
DATA, YOU HAVE ONLY YOURSELF TO BLAME IF YOU ANSWER ‘Y’ BELOW!

Do you want to proceed, possibly destroying your data? (Y/N) Y
OK; writing new GPT table.
The operation has completed successfully

Creating the file system

Next create the file system on the partition:
mkfs.ext4 /dev/md0p1

I haven’t yet investigated tuning an ext4 partition for raid 5 arrays. There are probably some tweaks to be made here, please comment if you have a suggestion.

Autostarting and mounting the array

At this point you should have a working raid array, shown as running in disk utility and an ext4 partition on the disk. If you reboot you will notice that the array doesn’t start automatically, you have to go into the disk utility and start it manually each time.

Getting it to start by itself also requires falling back to the command line. Make the array is running before proceeding.

First we need to get the config line:
mdadm --detail --scan
You should get a line like the following:
ARRAY /dev/md0 level=raid5 num-devices=4 metadata=01.02 name=:Raid5 UUID=7198dc4b:0b61431d:99f71126:c2d41815
Paste this line into /etc/mdadm/mdadm.conf, below the line that says “definitions of existing MD arrays”. Reboot, load disk utility and you should see that your array has been started automatically.

Automounting the file system on the array simply involves putting a line in /etc/fstab, which unfortunately every technical Linux user still needs to know about (it seems to be the most prominent legacy hangover, but it’s still a pretty good system when you know how it works). My line is as follows:
UUID=47b4d934-c0c3-46c6-b9f7-09c1c7a94774 /media/data ext4 rw,nosuid,nodev 0 1
Note that the UUID here is the UUID of the partition and not the raid array. To get the UUID of your partition, use the command “sudo blkid” and note the UUID of the partition on your raid array, which should be /dev/md0.

Another useful command is “cat /proc/mdstat” which gives some info about active arrays.

You shouldn’t need to create the mount point in /media, in my experience this happens automatically.

Next part – Creating user accounts and setting up the file shares

OS and package installation

This article is part of a series about setting up a home server. See this article for further details.

I did a simple install of Ubuntu using the alternate media written to a USB stick. You could use the ordinary desktop CD, or Ubuntu server (but you will need to install a lot more packages on the server version). I did not configure the raid arrays on install. The reason for this is that I didn’t have all the disks ready, but I didn’t want to install OS data on them anyway.

Installing packages

For media playback, the medibuntu repository is essential. See the Medibuntu site for more info, but the commands I used (lifted straight off the Medibuntu site) are:
sudo wget --output-document=/etc/apt/sources.list.d/medibuntu.list http://www.medibuntu.org/sources.list.d/$(lsb_release -cs).list && sudo apt-get --quiet update && sudo apt-get --yes --quiet --allow-unauthenticated install medibuntu-keyring && sudo apt-get --quiet update

sudo apt-get --yes install app-install-data-medibuntu apport-hooks-medibuntu

Next I installed some additional software:

aptitude install openssh-server backintime-gnome gstreamer0.10-plugins-ugly gstreamer0.10-plugins-bad gstreamer0.10-ffmpeg ntp samba winbind libpam-smbpass apache2.2-bin lib-apache2-mod-dnssd mdadm

Some of these warrant explanation:

  • The gstreamer plugins packages are codecs, which may or may not be legal in your country depending on its stance on patents (like anyone really pays an attention to that). However you need to ensure you have codecs available for any media you wish to play back.
  • ntp is for time synchronisation, which isn’t strictly necessary in a home environment but I like to have an accurate source of time on any network.
  • samba, winbind, libpam-smbpass, apache2.2-bin, lib-apache2-mod-dnssd are all related to file sharing. Winbind allows the system to lookup other hosts with Netbios, which Windows uses on small networks without a local DNS server (like most homes). I don’t feel it is necessary to provide a DNS server in a home with non-technical users, and to me it is for a network appliance such as an ADSL router to handle anyway.
  • apache2.2-bin and lib-apache2-mod-dnssd are required for the “personal file sharing” control panel in Gnome to work. The developer has stated that the Apache won’t be required in the future (see this bug report for details. You may not need this functionality for your home server, but I thought it was nice to have in case it’s needed.
  • mdadm is for raid
  • gdisk is for partitioning GPT partition tables. If you prefer to stick with MBR partition tables you don’t need this

Installing NeatX

NeatX is a free implementation of NoMachine’s nx server, originally written by Google for an internal project. It seems to be the easiest and quickest way to get going with Windows RDP equivalent functionality, in fact it’s just a few lines:
add-apt-repository ppa:freenx-team/ppa
aptitude update
aptitude install neatx-server

Next simply download the client from nomachine.com and away you go. There are a few rough edges and I have encountered errors on reconnect, but it’s good enough for me. It is much more efficient than VNC, and the speed increase is more than enough for me to put up with the bugs.

Internal Errors on reconnect

If you encounter internal errors when connecting, delete all directories from /var/lib/neatx/sessions. For some reason it isn’t always cleaning up properly, even if you logoff.

Next part – Configuring the raid array

N900 – 3 months on

My last post on this blog was a review of the Nokia N900, and that was a whole quarter ago. The last 3 months have been hectic to say the least but I now have a lot more free time!

So how has the N900 turned out?

In short, OK.

My conclusion still stands – the N900 is not a suitable phone for most people, and it probably isn’t the best phone for me either. For developers of applications for Nokia’s QT platform it’s the reference device and thus an essential piece of kit. But the general public are better served by the existing Symbian range.

So what’s good and what’s bad?
Continue reading

It’s here!





Currently updating the firmware to the latest version, I’ll be writing a review once I’ve had a good play and am familiar with the device.

Initial impression: awesome.

Mozilla Prism – what’s the point?

Mozilla prism is a framework for packaging web apps as applications. It uses the gecko rendering engine and is basically Firefox without the user interface. My reaction upon first reading about it a couple of years ago was as per the title – this is just a web browser with a restricted interface, and thus offered no advantage over a simple desktop url shortcut.

But there actually is a good reason to use prism, and that is privacy. Prism uses seperate profiles for each application, thus if you are logged in to Gmail, you do not have to be logged in while using Firefox and have Google track all your searches in addition to indexing your email. You can use Facebook in prism and not have to worry about third party sites accessing your profile, like the much-maligned beacon “service” facilitated recently.

In short prism affords a lot of convenience for those of us that like to keep their web identities segregated, but if you’re really really paranoid about privacy then Gmail and Facebook are two sites you probably shouldn’t use.

On Ubuntu you can install Gmail for Prism with the following command:

  • aptitude install prism-google-mail

For Facebook:

  • aptitude install prism-facebook

Once the prism package is installed you can also convert any site into a prism app by going to Tools > “Convert Website to Application”.

Of Bioshock 2 and DRM

This is a game I’ve been anticipating and I’m sure I’m not the only one. I was all ready to pull the trigger on a pre-order but the DRM has given me pause.

First there’s the whole Games for Windows Live (GFWL) thing. It annoyed the hell out of me in Batman Arkham Asylum, and I’m not particularly thrilled about it here. Had I known what GFWL was when I bought Arkham Asylum I might have thought twice. I know Steam is a similar concept, but somehow it’s more tolerable, although if you buy Bioshock 2 through Steam you have to put up with both!

The game is also protected by Securom, and has a 15 activation limit enforced by GFWL. With Bioshock 1 this caused issues, and later on I believe they removed the limit (although back then the limit was enforced by Securom).

If you preorder the game it will cost $90 NZD boxed or $63 NZD on Steam. With the Steam version you effectively have to handle two (no wait, three) DRM systems – GFWL, Steam and SecuROM! It really doesn’t get much more ridiculous than that.

Boycott this is not, but I think I might vote with my feet here, even if I am in the minority. Or I’ll just wait until they remove the DRM and drop the price like they did for the original Bioshock.

My opinion of Steam and GFWL can be summed up thusly:

Steam serves the publisher while throwing the customer a bone, and while you give up some rights you do gain some conveniences. It is also cheaper.

GFWL seems to be designed to serve only the publisher, and is little more than an annoyance for gamers.

At least the pirates won’t have to deal with this crap.

More info at Arstechnica, the Steampowered Forums and the 2k Games Forums.