Category Archives: Ubuntu

Samba config for Apple Time Machine

I’ve been using samba’s vfs_fruit module to enable backing up my mac laptop to my ubuntu-based NAS. I’ve found the configuration fiddly, and it occasionally breaks with macOS upgrades.

Anyway, I thought I would document my settings in case it helps anyone else out there. The file system backing my Time Machine share is ZFS, and I am using Samba 4.15 and MacOS Sonoma 14.4.1.

Global config

In /etc/samba/smb.conf under the [global] section, I have the following (not complete config, just the relevant settings):

protocol=SMB3
vfs objects = acl_xattr fruit streams_xattr aio_pthread
fruit:aapl = yes
fruit:model = MacSamba
fruit:posix_rename = yes
fruit:metadata = stream
fruit:nfs_aces = no
recycle:keeptree = no
oplocks = yes
locking = yes

Some comments:

  • I believe SMB3 is required – Time Machine struggles with older protocols.
  • The order of vfs objects is important – aio_pthread must go last.
  • Without aio_pthread, my backups fail while scanning. I suspect Time Machine is heavily threaded and does a lot of requests in parallel – apparently too much for a single samba thread.
  • fruit:posix_rename = yes appears to be the default and can probably be omitted
  • fruit:metadata = stream was a copy-paste and not thought through by me – I’m unsure of the implications of this

These work for me as a general set of settings for mac clients – I don’t use Windows or Linux clients often, so I don’t know how well it works for them. It’s possible some of these options are not required, as they’ve accumulated over time.

Share Config

The share itself is configured like so:

[TimeMachine NAS]
path=/pool1/backup/timemachine
comment=Time Machine
valid users = alex
writable = yes
durable handles = yes
kernel oplocks = no
kernel share modes = no
posix locking = no
ea support = yes
browseable = yes
read only = no
inherit acls = yes
fruit:time machine = yes

According to the docs, “fruit:time machine = yes” sets durable handles, kernel oplocks, kernel share modes and posix locking – you can probably omit these.

/pool1/backup/timemachine is a ZFS volume with a quota, which was set with “zfs set quota=3TB pool1/backup“. The available space is reported correctly to the client (Finder), so I’d expect it to work fine for restricting disk usage, and for time machine to manage its snapshots.

See also

ZFS compression and encryption

Up until a recent overhaul, I was using btrfs in raid1 to manage the 4 drives I had in my NAS. However it’s been clear for a while that the momentum is behind zfs. It has more features, better stability, and generally inspires much more confidence when things go wrong. btrfs still has its place in managing single-device boot volumes, but for multiple physical devices, I would definitely recommend zfs over btrfs.

When I added a couple of new 16TB disks, I opted to create a new pool with a single mirror vdev. If I need to expand it in future, I’ll add another mirrored vdev to the pool.

Continue reading

Building and Packaging a Python command-line tool for Debian

python-logo-notext-svg

Python packaging has a chequered past.

Distutils was and still is the original tool included with the standard library. But then setuptools was created to overcome the limitations of distutils, gained wide adoption, subsequently stagnated, and a fork called distribute was created to address some of the issues. Distutils2 was an attempt to take the best of previous tools to support Python 3, but it failed. Then distribute grew to support Python 3, was merged back in to setuptools, and everything else became moot!

Unfortunately, it’s hard to find reliable information on python packaging, because many articles you might find in a Duckduckgo search were created before setuptools was reinvigorated. Many reflect practices that are sub-optimal today, and I would disregard anything written before the distribute merge, which happened in March 2013.

Continue reading

Ubuntu Home Server 14.04

I had grand intentions.

This home server article was to be a detailed masterpiece, a complete documentation of my home server setup.

It hasn’t turned out that way, and many pieces are missing. Turns out, that writing a detailed article on setting up a server is much harder than just doing it! So what you see here is what I finally managed to publish, 5 months after actually building it. I hope you find it useful, and I don’t rule out the possibility that I may update parts of it in future. Continue reading

Ubuntu 14.04 – No USB keyboard after upgrading kernel

After upgrading my Ubuntu 14.04 LTS install from linux kernel 3.13 to 3.16, USB input devices, particularly my keyboard, stopped working.

On rebooting to an older kernel, the keyboard worked again. The reason for this, is that the base kernel package doesn’t include the usbhid module, which is require for USB input devices.

The solution, is to install the linux-image-extra package for your kernel. In my case it was:

[shell]
sudo apt-get install linux-image-extra-3.16.0-28-generic
[/shell]

You can either do this via ssh, or boot to an older working kernel first.

Afterwards, you should be able to do [shell]modprobe usbhid[/shell], or simply reboot, and your usb input devices should function correctly.

Ubuntu Home Server 14.04 – A DIY NAS

It’s been more than 4 years since I wrote about home servers, but my Ubuntu Home Server article was, for a while, the most popular post on this blog. Since moving to the UK though, I’ve taken a more appliance-based approach to my home network. For the last few years I’ve been using a Boxee Box for media playback, and a 4-bay Netgear ReadyNAS duo NV2+ for storage, mainly to keep the bulk of my possessions to a minimum.

The appliance approach does have advantages. It is power efficient, easy to setup, and very low maintenance. But after getting an internet connection with decent upload speed, I wanted to run CrashPlan on the NAS without having to have another PC running. I managed to get it running by following directions I found here.

There’s just one problem:

3.3 months to upload 350GB is a little too long

3.3 months to upload 350GB is a little too long

Performance is abysmal, and I’ve only selected the most important data – my photos. I’m limited not by my internet connection, but by the NAS’s anaemic CPU and lack of ram (just 256Mb). Furthermore, it’s always had very slow read and write speeds – generally around 2Mb/sec, and loading a large directory via its Samba shares can take a while.

So I started to look for a replacement. My requirements:

  • Minimum 2GB ram
  • Strong CPU, preferably x86
  • 4+ drive bays
  • Linux based OS
  • Root access to said OS

The best pre-built option I could find which meets those requirements is the Thecus N5550, but at £383 it is a long way from cheap. And it barely meets the specs; an Atom CPU is strong for a NAS but not by modern x86 standards.

While the customised software shipped with a NAS does offer some conveniences, it also gets in the way of using newer Linux features such as BTFS RAID 5/6 (which is currently not considered stable but should be within the next 12 months). You’re also reliant on the vendor for distribution upgrades, and the priority is going to be shiny features which consumers will appreciate, not keeping the foundation OS up to date. The ReadyNAS NV2+ is currently running Debian Squeeze, and will be until the day support ends.

At this point I realised that a pre-made NAS with the level of power and flexibility I wanted doesn’t exist at a realistic price point. And with the end of Boxee support its days as a useful device are numbered, so a HTPC could be on the cards as well. It’s time to build my own server again.

Continue reading

It’s nice to be right some times

Five short years ago I wrote an article about my desire for a Nokia N900. I was extremely enthusiastic about the device, which I saw as the future of computing and a sign of things to come. I also said:

Personally I think Linux usage overtaking Windows on personal computing devices is inevitable, and this is how it’s going to happen (although the capabilities of the N900 will have to move down to a much lower price point first). We’ll see if I’m right in 5-10 years time.

It’s now 4 years and 4 months later. I was right about Linux overtaking windows on personal computing devices, but I was wrong about how, and it happened far more quickly than I could have imagined.

Continue reading

Pausing Spotify and playing a random video in Python – A party trick for Halloween

For a Halloween party last weekend I wrote a python script to pause Spotify, play a random video and start music playback again. The videos were basic ogg files I cobbled together which showed a scary image and evil laughs or screaming with OpenShot. I can’t really share them, as I don’t have rights to the media, but it’s pretty simple to recreate them yourself.

The code for this script is on Github, and I’ve reproduced the latest snapshot below. Feel free to fork and improve if you want to scare your guests, or add support for other OS’s. Presently it only supports Linux because I used dbus to perform the play/pause actions.

[python]
#!/usr/bin/python

”’
This is a Halloween party script which pauses Spotify and plays a video
at random intervals.
”’

import random
import subprocess
from subprocess import call
from time import sleep
import os
import datetime

start_time = datetime.time(21, 0, 0)
stop_time = datetime.time(23, 0, 0)

video_dir = ‘/home/alex/Videos/scream/’
videos = { ‘scream1_nofade.ogg’: 30,
‘happy.ogg’: 1,
‘evil_laugh.ogg’: 5,
}

def time_in_range(start, end, x):
“””Return true if x is in the range [start, end]”””
if start <= end:
print(“start<end”)
return start <= x <= end
else:
print(“end<start”)
return start <= x or x <= end

def weighted_choice(weights):
total = sum(weights for video in weights)
r = random.uniform(0, total)
upto = 0
print(“total: %s\nrandom: %s” % (total, r))

for video in weights:
w = weights
if upto + w > r:
return video
upto += w
assert False, “shouldn’t get here”

def spotifyPause():
command = “dbus-send –print-reply –dest=org.mpris.MediaPlayer2.spotify /org/mpris/MediaPlayer2 org.mpris.MediaPlayer2.Player.Pause”
print(“pausing spotify”)
os.system(command)

def spotifyPlay():
print(“playing spotify”)
command = “dbus-send –print-reply –dest=org.mpris.MediaPlayer2.spotify /org/mpris/MediaPlayer2 org.mpris.MediaPlayer2.Player.PlayPause”
os.system(command)

def play_video(video_file):
print(“Playing %s” % video_file)
#call([‘/usr/bin/mplayer’, ‘-fs’, video_file], stdout=None, stderr=None)
#result = subprocess.Popen([‘/usr/bin/mplayer’, ‘-really-quiet’, ‘-fs’, video_file])
result = subprocess.check_call([‘/usr/bin/mplayer’, ‘-really-quiet’, ‘-fs’, video_file], stdout=None, stderr=None)
return result

def playBuzz(buzzfile):
print(“Buzz…”)
result = subprocess.check_call([‘/usr/bin/mplayer’, ‘-really-quiet’, ‘-ss’, ’18’, buzzfile], stdout=None, stderr=None)
return result

def infiniteLoop():
while 1:
current_time = datetime.datetime.now().time()
#if current_time > stop_time or current_time < midday:

choice = weighted_choice(videos)

random_time = random.randrange(1200,2400)
random_time = 3

video_file = video_dir + choice
print(“Chose video %s after %s seconds” % (video_file, random_time))
sleep(random_time)

# Whether to play buzz
buzz = False
if random.randrange(0,100) > 90:
buzz = True

# Continue if outside time range
if not time_in_range(start_time, stop_time, current_time):
print(“Not playing video, outside time range”)
continue

# Do it
spotifyPause()
if buzz:
playBuzz(‘/home/alex/Videos/scream/audio/buzz.mp3’)
play_video(video_file)
spotifyPlay()

if __name__ == “__main__”:
infiniteLoop()
[/python]

Changing boot order in Ubuntu 13.04 (or Debian) – the easy way

I wanted nice, concise instructions on changing the boot order in Ubuntu 13.04, which uses Grub 2. Being a newbie focused OS however, Googling “ubuntu boot order” results in SEO blogs with lots of fluff, and then the actual instructions start out with “install package from ppa”…

What the hell, I just want to change the boot order!

Continue reading

Creating samba share in Nautilus: ‘net usershare’ returned error 255

I was having this problem on Ubuntu 12.04 (precise), but most of the Google results pointed to a bug in Hardy. However there are other causes of this problem.

In my case it was a previously-created share with a different user ID – Nautilus couldn’t create the share because there was already a share file with the same name owned by a different user.

The directory is /var/lib/samba/usershares. You should already have write access assuming you’re a member of the sambashare group (which the gui should handle for you), so all that remains to be done is remove the offending share with the same name as the one you’re trying to create.

alex@al4:~$ cd /var/lib/samba/usershares/
alex@al4:/var/lib/samba/usershares$ ls -lah
total 16K
drwxrwx--T 2 root       sambashare 4.0K Jul 25 12:33 .
drwxr-xr-x 5 root       root       4.0K May  1 10:40 ..
-rw-r--r-- 1 2046297271 2046296576  142 Oct 25  2011 music
-rw-r--r-- 1 2046297271 2046296576  128 Feb  7 17:13 videos
alex@al4:/var/lib/samba/usershares$ sudo rm music
[sudo] password for alex:
alex@al4:/var/lib/samba/usershares$ sudo rm videos
alex@al4:/var/lib/samba/usershares$

After doing the above, Nautilus was able to recreate the shares without trouble.