LibguestFS GLLUG Talk

Over the years there have been a handful of GLLUG members that have given so many interesting talks that I'll always turn up to watch them - and Richard Jones is definitely in that short list.

The website does an excellent job of explaining: "libguestfs is a library for accessing and modifying virtual machine (VM) disk images. Amongst the things this is good for: making batch configuration changes to guests, viewing and editing files inside guests (virt-cat, virt-edit), getting disk used/free statistics (virt-df), migrating between virtualization systems (virt-p2v), performing partial backups, performing partial guest clones, cloning VMs and changing registry/UUID/hostname info, and much else besides." but it doesn't quite convey how cool it is to spin up access in to a windows machine in a handful of seconds and then dump out the registry key you're looking for - all from a Linux command line.

Oh, and even if you didn't turn up (tsk tsk) you can read all about the libguestfs gllug talk here.

Like this post? - Digg Me! | Add to! | reddit this!

The Book of Xen – Short Review

Although I've been a big fan of virtualization for many years I've mostly been a VMWare man. UML was good for the time but VMWare workstation and GSX always seemed to be better solutions - and they had the benefits of dealing with Windows. At $WORK we looked at using Xen for our new development environment but it never felt very finished, little things like needing to compile your own dhcp client in order to get PXE booting working always felt very wrong.

But now we're looking to move away from VMWare server for certain parts of our infrastructure everything's back on the table so I went looking for a guide through the lands of Xen in the modern world - and I think I found an excellent one in The Book of Xen.

The book takes you through all the aspects of using Xen that you'd expect, from installing it, configuring the guests (DomU in Xen terminology) to making the most out of the networking options and local storage possibilities. Where it goes that extra mile is in sections like 'Beyond Linux', which guides you through using NetBSD and Solaris with Xen, Profiling and benchmarking under Xen and Lessons from the trenches, in which the authors (who run a Xen hosting service) tell you about their real-world aches and pains.

Apart from the chapter on the commercial Citrix XenServer, which I can understand the inclusion of but isn't useful to me, there was something interesting in every chapter. After working through the book I have a good understanding of what needs attention in a Xen hosting setup and what might be weaknesses. All I need now is a similar book for KVM so I can avoid doing all my own research!.

An excellent guide to Xen that brings a lot of useful material into one place - 7/10

Like this post? - Digg Me! | Add to! | reddit this!

London DevOps – March 2010

This month was the first of the London DevOps tech talks. Organised by R I Pienaar and masterfully shepherded on the evening by Chris Read about thirty sysadmins (and some developers, project managers and scrum masters) met for a series of impromptu discussions, beer and pizza

While there was no formal schedule for the evening Chris led the group in a fishbowl, seeding some ideas and then watched the conversations bloom. We went through some tool chain issues, trending, log analysis, how Splunk is the best thing since sliced bread with bacon in it and how Centos does some very interesting things with the data they collect. It was the first fishbowl I'd ever attended and it was actually a lot of fun, especially when people suggested RDF and SPARQL for a common data store.

A short break was taken when the pizza arrived and a number of interesting conversations broke out, how little admin time Apache Solr seems to need (and how odd it is to use rsync and shell scripts to sync out changes), how Redis and CouchDB are making certain problem domains easier to deal with and how the BBC has so many cool people hidden away were among those I ambled in to.

ThoughtWorks kindly donated beer, pizza and most importantly the venue - and for that we should say thank you. Getting a decent venue is always difficult for a new group. Although it's early days the group feels like it's got potential, the conversations were interesting, we don't all agree on where we should be heading and what we need next but the atmosphere was friendly and open. Hopefully these meets will last longer than SAGE-WISE did, with all the developer focused events in London it's nice to get to one that's a little closer to what I do.

Like this post? - Digg Me! | Add to! | reddit this!

BSD Magazine – A decent read

While looking for an OpenBSD baseball cap on the BSD stalls at FOSDEM I was given a couple of issues of the BSD Magazine to flick through - and it's a lot better than I'd hoped.

As most of the UK Linux magazines have become very desktop focused it's nice to see some actual low-level code - packaging for OpenBSD, writing sound drivers for your NetBSD NSLU2, custom Jabber components and basic GDB were all in the two issues I skimmed. While it's not the dearly departed Sysadmin Magazine, and it could do with an editor or two - much as I could, it is a decent read and I'm considering a subscription.

Like this post? - Digg Me! | Add to! | reddit this!

Grocery Arrival Excitement?

Many years ago, in the first dotcom boom, I worked for a website performance monitoring company. I was one of the early employees (developer number 3 and sysadmin number 2) and I remember being in a meeting with the company CEO who was telling us about a new pitch we were doing for $SUPERMARKET, they were going to try this new idea of shopping online and then delivering it to your door.

The worst part of it was that they didn't just want monitoring, they wanted a full transaction engine (with some basic OCR), a product I can probably get away with confessing that we didn't have at the time of the sales pitch. We all knew the deal, if we didn't get it life was going to be very hard there for the next six months, so we all knuckled under. The road was long, difficult and uphill in the snow in both directions but eventually we got to the day of the pitch. Which we aced in an astounding display of luck - the new app sometimes got itself in to a little bit of a state if their website had a failure - which it did about 20% of the time. They loved the demo and wanted us to give them full coverage while they did maintenance work. If we pulled it off then we'd pretty much get the deal, none of our competition at the time could match the features, it was just the uptime that was a little worrying.

So we went out and bought a dozen small desktops, monitors and networking kit, installed them all in our spare store room, put some tables and chairs in and had a company meeting. The management were completely open about what was happening, they took questions and then asked how far we'd go to help. We covered the whole weekend from Friday night to Monday morning. Nearly the entire company chipped in, from three letter titles to sales to dev to systems to HR. We had eyes on the machines over the whole period, including when the Solaris admin, the only person to let us down, didn't make his time slot. Out of all the transactions the worst was beans, they had a new version of the code on some of the servers and it'd return very odd results for beans and break the transaction runner in horrible ways. I'll never forget the 4am calls asking what we do when they offer you a lawn-mower instead.

I placed my first ever order online with the $SUPERMARKET yesterday and hopefully it should arrive in the next couple of hours. The interface may have changed and so many of its users take the service for granted that it's a little humbling to realise how much the Internet's changed so very many things. I guess this post's about a combination of things, the best job I ever had (the company was sold in the end to one of it's competitors. I left happy in the knowledge that we ate their lunch until they gave up trying to compete and bought us), how dedicated staff can be in the right environment, why you should push the boundaries of your industry and how sometimes even cans of beans can be exciting.

I had to put a single can in the order to complete the circle. Here's to hoping they don't charge me for a lawn-mower.

Update: They didn't deliver on the night, there was a "problem with the payment" so they took the money out, using the same details and delivered it two nights later. I'll class this one as a draw.

Like this post? - Digg Me! | Add to! | reddit this!

Spreadsheets Vs Post-It Notes

I'm a fan of documentation, over the years I've ended up supporting more than one business critical system that has less documentation than you get from a cat /dev/null.

The only downside, and I've been bitten by a couple of things like this over the last week is the case of the spreadsheet vs the post-it note - if you have a lovely, well formatted and information dense spreadsheet that says "A is 1" and when you get to the server room the switch has a post-it, in bad scrawl, that says "B is 2" which do you believe?

Like this post? - Digg Me! | Add to! | reddit this!

Hadoop Talk – SkillsMatter 2009

After an embarrassing tale of misunderstanding, wrong locations and blind luck I recently ended up at the Introduction to data processing with Hadoop and Pig talk over at SkillsMatter - and it was excellent.

For those that don't know about Hadoop, it's an OpenSource Java framework for data-intensive distributed applications. It enables applications to work with thousands of nodes and petabytes of data. Hadoop was inspired by Google's MapReduce and Google File System (GFS) papers. I was aware of the basics but even in an hour I learned enough to know where to look for more details. Pig on the other hand is (to me) like SQL but for Hadoop, it's a lot easier to use than writing your own Java apps and simpler (and actually possible) for non-developers to read than the reams of classes required for custom jobs.

The speaker was excellent, the presentation was well timed, fluid, concise, paced just the way I like it and other than the question session the evening was very enjoyable. You can find the Hadoop slides online.

Like this post? - Digg Me! | Add to! | reddit this!

JRuby Cookbook – Short Review

First a disclaimer, I'm not a heavy Ruby or Java guy. Most of my coding for the last couple of years has been perl and shell - because I write little things that I need right now and those two languages excel at that (CPAN is still THE decision clincher).

I recently became involved in a side project that is written in Ruby and Java though and in an excellent timing coincidence a friend returned my previously unread copy of the JRuby Cookbook. The book isn't an introduction to either Java or Ruby (there are already excellent online and dead tree resources for that) but it shows where the two can meet and how to get started at those points. It's not really a book to read back to front but it is a good approach for a cookbook.

If you're curious as to how dynamic languages on static language VMs can complement each other this is a good book to flick through. Score - 6/10 - it's not the book for me right now but it does show a lot of entry points I'll probably come back to later.

Like this post? - Digg Me! | Add to! | reddit this!

Dynamic Motion on Google Earth

It's very easy to become quite blase or even cynical about new technologies but sometimes a project grabs your attention and coaxes out a "that's very cool", the real time augmented Google Earth had that effect on me.

How long will it be before you can roll back an overlay by X weeks and see what happened in that game last Thursday or check the traffic on your new route at 7am on every Friday for a couple of weeks?

Like this post? - Digg Me! | Add to! | reddit this!