Author Archives: Alex Forbes

Setting up a secure Ubuntu LAMP server

Disclaimer: This article is provided for your information only, and simply following this guide will not make your server “secure”. As the server administrator you are ultimately responsible for its security!

Intro

Having recently been through the process of setting up a few Ubuntu LAMP (Linux, Apache, MySQL, PHP) servers lately I thought I’d make an article out of my notes and provide a starters guide to setting up the LAMP stack on Ubuntu.

It goes without saying that the only truly secure computer is one with no network connection, no ports or input devices and is locked in a bank vault, but such a machine is not terribly useful. Regretfully, compromises must be made to allow functionality! Besides presuming insecurity, there are a lot of things you can do to make your server more secure and keep out the vast majority of would-be hackers running port scans, meta-exploit scripts and dictionary attacks.
Continue reading

Changes

I made a small change recently – I’ve moved from WordPress.com hosting to a self-hosted server.

The main reason for this is education – I want to learn more about e-commerce and running my own site, which is a bit difficult when WordPress.com places adsense ads on your site (and not on your behalf!). This way I get to keep any adsense revenue, although to be honest if it covers even the small costs of running this site I will eat my hat, shorts and tshirt.

An unfortunate side effect of this is that WordPress.com accounts can no longer comment and you need to manually enter details each time, so I expect comments will drop. But for those that do I appreciate the feedback!
Continue reading

A quick look at Unity in Ubuntu 11.04

I usually jump on the latest Ubuntu release before it hits the final release stage, but this time it was with a bit more trepidation than usual. You see they’ve replaced the shell with a completely new one – Unity. And to say that not everyone likes it would be a minor understatement.

The good news is that Unity is undoubtedly a step forward in practical user interface terms. The classic Gnome panel is really showing its age, and doesn’t lend itself to wide screen formats due to having panels on both the top and bottom of the screen. SUSE addressed this by created a start menu like launcher, but it was always a bit clunky and barely an improvement over Windows XP.

So how is Unity different? It is perhaps more informative to talk about how it is similar…
Continue reading

Recovering a RAID5 mdadm array with two failed devices

Update 1/11/2019

If you’ve reached this article by Googling how to recover a RAID array, I suggest you don’t use this guide. The Linux RAID wiki has much more correct, complete, and authoritative information. In retrospect I was very lucky not to destroy the data, but this article is an interesting account of how I did it.

Update

Before reading this article you should know that it is now quite old and there is a better method – ‘mdadm –assemble –force’ (it may have been there all along). This will try to assemble the array by marking previously failed drives as good. From the man page:

If mdadm cannot find enough working devices to start the array, but can find some devices that are recorded as having failed, then it will mark those devices as working so that the array can be started.

I would however strongly suggest that you first disconnect the drive that failed first. If you need to discover which device failed first, or assemble doesn’t work and you need to manually recreate the array, then read on.

I found myself in an interesting situation with my parents home server today (Ubuntu 10.04). Hardware wise it’s not the best setup – two of the drives are in an external enclose connected with eSATA cables. I did encourage Dad to buy a proper enclosure, but was unsuccessful. This is a demonstration of why eSATA is a very bad idea for RAID devices.

What happened was that one of the cables had been bumped, disconnecting one of the drives. Thus the array was running in a degraded state for over a month – not good. Anyway I noticed this when logging in one day to fix something else. The device wasn’t visible so I told Dad to check the cable, but unfortunately when he went to secure the cable, he must have somehow disconnected the another one. This caused a second drive to fail so the array immediately stopped.

Despite having no hardware failure, the situation is similar to someone replacing the wrong drive in a raid array. Recovering it was an interesting experience, so here I’ve documented the process.
Continue reading

Quick analysis of a phishing attack

Twice in three days I have received emails purporting to be from ASB and ANZ Bank. Both are New Zealand banks, and the fact that I’ve received two of them clearly indicates that my email address is on a spam database somewhere and geographically tagged New Zealand. Easy enough – my .co.nz domain uses it as the registration address, and it has a New Zealand residential address on it.

I see these all the time, but the execution of this particular attack struck me as unusually slick however, hence the blog post.

The email

The ANZ email subject was “Please remove your Online Banking Limitation! Last warning!”, whereas the ASB email was titled “Online banking suspension warning!”. Both are clearly designed to panic the user into clicking the link and entering their banking details. The ANZ subject has a hint of ESOL and the grammar in the emails is sub-standard, I suspect the origin is a country where English is not the first language.
Continue reading

Splitting files with dd

We have an ESXi box hosted with Rackspace, it took a bit of pushing to get them to install ESXi it in the first place as they tried to get us to use their cloud offering. But this is a staging environment and we want something dedicated on hardware we control so we can get an idea of performance without other people’s workloads muddying the water.

Anyway, I’ve been having a bit of fun getting our server template uploaded to it, which is only 11GB compressed – not exactly large, but apparently large enough to be inconvenient.

In my experience the datastore upload tool in the vSphere client frequently fails on large files. In this case I was getting the “Failed to log into NFC server” error, which is probably due to a requisite port not being open. I didn’t like that tool anyway, move on.

The trusty-but-slow scp method was also failing however. Uploads would start but consistently stall at about the 1GB mark. Not sure if it’s a buffer or something getting filled in dropbear (which is designed to be a lightweight ssh server and really shouldn’t need to deal with files this large), but Googling didn’t turn up much.
Continue reading

An Ubuntu 10.04 Home Server

I’ve recently been setting up a home server for my parents using lucid. While it’s not quite a point and click setup process, the process is a lot more streamlined than it used to be.

They have an individual computer each running Windows 7 and, one laptop between them running XP. Mum is also a photographer and generates a large amount of data. Dad also generates a fair bit of data, less than Mum although he does do the occasional home video.

Backups are an ad-hoc affair. Mum has three hard disks in her computer which she manually copies files between and tries to ensure she has two copies of everything. Dad has a portable external drive which he backs up to infrequently. Between them, neither is confident that they’d get all their data back in the event of a disaster.

Dad also liked how my HTPC (running XBMC) worked, and decided one of those would be nice too. So I decided to setup a home server for them and solve all their computer problems. Well, almost.

I started writing this as a single article, but it got a bit long so I’ve decided to break it up into a series. This first post is an overview, the links to the other posts are at the bottom of this article.

I’m assuming a fairly good degree of technical knowledge here, but if there are any gaps you feel I should add please feel free to leave a comment. I am aiming this at a reader who is familiar with Linux and Ubuntu, has installed software with apt-get or Synaptic, is comfortable with using the command line, and understands the implications of using raid5.

Overview

This home server will perform the following tasks:

  • Play music and video via the TV
  • Present a file share to the network, with individual folders for Mum and Dad
  • Backup the contents of their folders nightly to an external hard drive
  • Provide a GUI-based remote administration interface
  • Monitor backups and the raid array, sending emails to both Mum and Dad if something is amiss

Software that needs to be configured to perform these tasks:

  • MDADM for RAID
  • Xbox Media Center (XBMC) for media playback
  • Samba for file sharing
  • Back in Time for backup
  • NeatX for remote administration

The main boot device in this case will be an IDE compact flash card. I did this partly because it makes recovery easier (just write an image to a flash card rather than a whole hard drive), but mainly because it frees up a SATA port!

The hardware components for this particular HTPC are:

  • Gigabyte M85M-US2H motherboard
  • AMD Athlon II 250
  • 2gb DDR2 ram
  • 4x640gb Western Digital 6400AAKS hard drives
  • 1x1TB Western Digital Green
  • 1x2TB Western Digital Green (in external esata case)
  • 4 Raidon/Stardom hotswap drive bays
  • IDE Compact Flash adaptor and 8gb 133x CF card

A note on raid

The 4x640gb drives are configured in a raid 5 array. Personally, this is about as large an array as I would trust Raid5 to, the future is redundancy at the file system layer, as ZFS and Btrfs are capable of. ZFS can’t be used in the Linux kernel and Btrfs isn’t even close to production-ready yet, so for now I believe Raid is still the most sensible option. But if you’re reading this in 2012, you should probably be using Btrfs instead.

Storage

The 1TB hdd is just a single disk for media to be played back on the TV. Anything here is considered replaceable (think of it like the internal HDD in a MySky or TiVO box), so it won’t be backed up at all.

The 2TB hdd is the backup drive. Each night the entire raid array is backed up to it with Back in Time, configured to take snapshots. Since it uses rsync, the backups are incremental and shouldn’t take more than a few minutes to run, depending on how much was changed during the day. Obviously as the array nears capacity fewer snapshots will be able to be kept, and once it does the idea is to replace the 2TB backup hdd with a new one, keep the old one as an archive, delete any data from the raid array that is no longer current, and start again with a fresh clean backup disk. Hopefully by then it will be a 3 or 4TB disk and they can keep more snapshots!

The file system on the backup HDD will be NTFS. This is because it supports hard links and is readable by the Windows machines, which is important for my parents when they go to retrieve files from the archive.

Final notes before we get to the nitty gritty

I had a bit of trouble getting the drive bays lined up with the ports that the OS reported they were attached to. This is important because if mdadm tells Dad that the sata disk on port x has failed, I need him to be able to know that it’s the disk in bay x. Unfortunately on the motherboard I used, Ubuntu assigns them like so:

0 – 1
1 – 3
2 – 2
3 – 4

(motherboard port – ubuntu port)

So while your motherboard may be better designed than mine, don’t assume they are in the same order. The links to the follow-up articles are below:

Nokia N900 Buyer Review

I’ve had to think long and hard about this review. The N900 is unquestionably flawed, but it’s a leading edge device, and prescient in so many ways. It shows promise of things to come, and that promise is exciting. So does one knock it as a failed attempt at reclaiming the smart phone crown, or praise its foresight and anxiously await N900+1? Considering Nokia’s stance on the device it would perhaps be unfair to call it an attempt to retake the smart phone crown, as they never positioned it as such. But it does not deserve unreserved praise either.

To get my own personal bias out of the way – I want to love the N900. It’s a Linux-based smart phone built on open source software that doesn’t try to hide its roots. I’m a Linux geek and open source enthusiast. I dislike walled gardens such as the iPhone App Store and the artificial restrictions placed on the iPhone, so a 3GS was never an option. Android is a bit too tied to Google’s services (a company which already knows much more about me than I would like to admit), and while Nokia are certainly trying to push their Ovi suite of services, they would be foolish to make it difficult for you to use competing services. My credibility as a reviewer drops somewhat given my lack of experience in using Android, and quality time with an iPhone. I’ve had a play on devices owned by friends, but that’s not enough to get to know the ins and outs of a device.

So it’s with a bit of trepidation that I review the N900. My only real frame of reference is the aging Symbian S60 – an OS that has served us well, but is now past its use-by date and hardly the ideal operating system to compare it to.
Continue reading

Identity Management

(Warning: if you’re not an IT nerd this blog post may make rather dry reading)

Identity Management is a pretty big topic these days – some might say it’s the new IT buzzword. From an organisational perspective it is highly desirable for users to have to remember as few passwords as possible, as this reduces the need to them to write them down. Centralised management and provisioning of user rights also provides more certainty and reduces overheads.

With the use of authentication services such as Facebook Connect, Windows Live ID, and Google accounts becoming more widespread on the web, we’re starting to see the web trending away from the “one identity per service” model towards fewer identity providers proving authentication services for other sites.

Recently I’ve been asked to investigate SAML-base single sign on solutions, so I’ve collected some of my thoughts in this blog post. Please note that this is based on my own research and should not be considered authoritative in any way!

The Web Perspective

One of the problems with the web today is the sheer number of usernames and passwords that people have to remember. You need to create an account for almost every online service you use, as sites need access to certain information about you in order to provide a useful service, and they need a way to ensure that you keep the same identity on the site the next time you visit. E-Commerce is a very significant example of an area where this is needed as you can’t accept payments without a fair bit of information.

Microsoft tried to solve the problem with their passport service back in 1999 (actually it may have been even earlier). The idea was that your “passport” could be used to sign in to other passport-enabled sites, and could contain enough information to allow ecommerce transactions to take place without having to enter your details every time. The problem, in typical Microsoft fashion, was that this service was a centralised Microsoft service – they wanted to hold all the information. It should have come as no surprise then that adoption was rather limited, and fortunately as a result the current Windows Live ID service is a different beast.

What was needed was an open model not tied to a particular service, and that model is OpenID. All the aforementioned services support (or have committed to supporting) OpenID, which is in layman’s terms an open way of logging in to one site using credentials from another. So what this means is that theoretically you could use your Facebook account to login to any site that supports logging in with OpenID.

“Brilliant! Now I can use one identity for everything!”

There’s a small problem though.

The major Identity Providers (holders of your information) all want to be providers, but they don’t want to be consumers (i.e. accept logins from other sites). So while you can log on to Gmail with your Google ID, and digg.com with your Facebook ID, you can’t login to Facebook with your Google ID or Gmail with your Windows Live ID. We’re a long way from the OpenID dream of being able to sign in to any service with any ID, and there’s little stopping it but branding and marketing. But we are at least moving towards needing fewer logins, as smaller sites tend to be happy to accept logins from the major providers, and OpenID adoption is growing so it’s not all bad.

Organisational Needs

Large corporate networks mainly want a single place to manage user access to company resources. They also generally want their users to have as few passwords to remember as possible, and to have to enter their passwords only when really necessary. LDAP solves the first problem by providing that central repository of user information which services can outsource their authentication to, and most applications that would be used on a large network can do this. It doesn’t solve the second problem however, as the user still has to type their password for each service. But at least it’s the same password.

OpenID works well for the web where the services are available to anyone with an email address. Basically they don’t care who the user is as long as they’re the same person. However the Identity Management needs of organisations are somewhat different. You generally don’t want to grant any old OpenID access to a company network, however you may want to grant employees or members of other organisations access to certain resources. What is needed therefore is a framework which  refers to a centralised directory service, provides single sign on, and can provide access to users of other trusted organisations.

The solution to this is “Security Assertion Markup Language”, or SAML. SAML introduces the concepts of an Identity Provider (provider of assertions) and Service Provider (consumer of assertions). What happens in a SAML authentication session is that the user’s web browser tries to access the app, gets redirected to the login page of their Identity Provider, which returns a token to the browser upon login. The browser then forwards the token to the service provider which verifies the request and grants access. The best diagram I’ve seen which explains this process is on Google’s SAML reference implementatin page for Google Apps.

The Identity Provider part (IDP) is the easy bit. The software is available (Shibboleth and SimpleSAMLphp are two examples) and once you get your head around the concepts and set it up correctly you can point it at a directory service and go. The problem currently is at the Service Provider (SP) end (the part labelled ACS in Google’s diagram), as few services actually support SAML. Google Apps is one of the first notable examples, and I’m hoping that adoption of Google apps will solve the chicken and egg problem by driving adoption of SAML and providing the install base for other software developers to jump on board and add SAML to their services.

Software such as Novell Access Manager (which supports SAML) attempts to get around the problem by effectively acting as a gateway to the service, and blocking access to unauthenticated users. That way the service doesn’t have to support SAML and you can only get to the service if you have permission, however I don’t know how the target web service is supposed to handle authentication if it needs to know who you are (for example to edit a wiki). I think the logical way would be for it to insert login credentials in the HTTP request, but hopefully this will become apparent when I start playing with it.

Conclusion

OpenID isn’t perfect, and like any username/password scheme it is particularly vulnerable to phishing attacks (only the stakes are higher as a successful attack results in access to multiple sites). The battle between the major providers to be the provider of your identity also threatens to reduce the benefits. But regardless of the risks it seems like a step forward for the web.

For organisations that need single sign on and a federated trust model, SAML seems to be the way to go. But it requires much broader adoption by software developers and service providers before it will truly eliminate multiple logons in organisations. Heck, many don’t even support LDAP yet.

Printing dd stats / status

I keep Googling this, when really I should have committed it to memory by now. It’s extremely useful when cloning hard drives, which takes a long time with modern disks (a 160gb hard drive takes about an hour over eSATA @ ~50MB/s).

Anyway while there are more efficient methods, dd is simple and it works.

So given the following dd command:

dd if=/dev/sda of=/dev/sdb bs=32M

We can find out the pid (process ID) with the following command:

ps ax | grep dd

Which in my case gives:

12147  tty2    R+     6:17 dd if /dev/sda of /dev/sdb bs 32M

i.e. 12147.

To get dd to output the stats, we send it a SIGUSR1 signal:

kill -SIGUSR1 12147

Resulting in:

3205+0 records in
3204+0 records out
107508400128 bytes (108 GB) copied, 2111.05 s, 50.9 MB/s

I should thank the author of the following link, it’s the result I get whenever I google this:

http://prefetch.net/blog/index.php/2006/06/11/printing-dd-status/