Posts

Going Back to University



A couple of weeks ago I had the enormous pleasure of returning to Exeter University where I studied for my degree more years ago than seems possible.  Getting involved with the uni again has been something I've long since wanted to do in an attempt to give back something to the institution to which I owe so much having been there to get good qualifications and not least met my wife there too!  I think early on in a career it's not necessarily something I would have been particularly useful for since I was closer to the university than my working life in age, mentality and a bunch of other factors I'm sure.  However, getting a bit older makes me feel readier to provide something tangibly useful in terms of giving something back both to the university and to the current students.  I hope that having been there recently with work it's a relationship I can start to build up.

I should probably steer clear of saying exactly why we were there but there was a small team from work some of which I knew well such as @madieq and @andysc and one or two I hadn't come across before.  Our job was to work with some academic staff for a couple of days and so it was a bit of a departure from my normal work with corporate customers.  It's fantastic to see the university from the other side of the fence (i.e. not being a student) and hearing about some of the things going on there and seeing a university every bit as vibrant and ambitious as the one I left in 2000. Of course, there was the obligatory wining and dining in the evening which just went to make the experience all the more pleasurable.

I really hope to be able to talk a lot more about things we're doing with the university in the future.  Until then, I'm looking forward to going back a little more often and potentially imparting some words (of wisdom?) to some students too.

New Thinkpad W530

It's been quite a while since I got my last laptop upgrade at work, coming up to 5 years in fact.  We have a 4 year refresh programme so I'm a little overdue but have just been given a shiny new Thinkpad W530 from Lenovo.  This seems to be our current standard issue machine for "developers" which is our way of saying "power users".  I'm part of the software development business and hence get one of these.  The up side of course is the latest and most powerful technology at a reasonably high specification, the downside is they're really quite big and heavy and the power brick - well it really is a brick.  I'll spare giving a full review of the laptop itself as there are plenty of them out there already and you'll know how to find them, however, there are one or two things I wanted to say about the machine and in particular regarding my preferred use of Linux rather than the software it comes pre-installed with.

Here's the specification highlights of the machine I've got (there is a bit of variation available with the W530):

  • Intel(R) Core(TM) i7-3720QM CPU @ 2.60GHz (5187.87 bogomips in Linux)
  • 16GB DDR3 1600MHz
  • 500GB (7200rpm) HDD
  • 15.6" 1920x1080
  • 9 Cell Battery
  • Wireless N
  • NVidia Quadro K1000M
  • Front Facing Web Cam, Mini Display Port, VGA Out, Headphone, 2x USB3, 2xUSB2.0, Smart Card Reader, Firewire,DVD Burner, GBit Ethernet
It came with firmware version 2.07 which was only 3 months old but had already been superseded by two newer versions when I got it earlier this week (there is a firmware readme available).  The newer versions fixed a couple of well known issues with screen corruption under Linux and the fan always running at full speed (and hence being noisy).  So I downloaded and applied the updated version before I did anything else.


The next thing I did was tweak a few settings in the BIOS to my liking and install Fedora 18 with the KDE desktop.  The installation went very smoothly using the Integrated graphics card on the Ivy Bridge CPU.  The W530 has Optimus built in for efficient switching between the integrated card and discrete NVidia card for a great combo of power/performance, it is however, designed for Windows and Linux support hasn't quite caught up yet although there is an open source option available - which I'm yet to try.  Post installation I installed the latest NVidia drivers available from the RPM Fusion repository (304.64) ready to switch to using the graphics subsystem in discrete only mode.  The advantage of this is greater graphical processing power and also the ability to use external display devices.  The integrated graphics card is only able to drive the laptop screen and doesn't output via the VGA port or display port.  The down side to the NVidia card is a greater power draw so reduced battery life.  Also, at the time of writing the Nouveau driver doesn't support the Quadro K1000M card so you're forced into using the proprietary driver.  This situation can only improve over time and hopefully Optimus support will grow in Linux too but I'm not holding my breath on that one given NVidia's attempt to put support into the Linux kernel was refuted by the kernel developers last year due to it not being GPL code.

Away from the graphics subsystem which was always going to be the most difficult thing under Linux on this machine, the rest of  it appears to be very well supported.  There are a few bits and pieces I haven't quite got around to trying in the couple of days since I got it but my impression is generally quite good.  Speed, as you would expect is very good although nowhere near my home machine which is a similar specification but contains an SSD instead of HDD.  Consequently, I put the speed boost I see at home down to this more or less entirely.

I've also moved away from Gnome (I don't get on with Gnome 3) and gone back to using KDE once again which I had moved away from 5 years ago when I installed my previous laptop as KDE 4 was pretty shocking at the time as well.  I've used KDE a lot more than I have Gnome in terms of years of elapsed usage but I did get on very well with Gnome 2 for the past 5 years and I'm sure I'll miss it.  That said, I can't see myself ever moving to Gnome 3 unless the developers go back on their current manifesto of treating users like idiots.  It'll be interesting to see how the Mate desktop progresses and whether XFCE picks up as well given they both have benefited from Gnome 3's unfortunate design decisions and have a much smaller community of users and developers than either Gnome or KDE.

In general then, I'm pleased with the new machine.  It's up and running to my liking in a very short period of time.  The graphics are bound to be a pain until I get used to relying on the nvidia-settings utility once again.  However, the other benefits it brings in terms of larger memory and greater processing power over my old machine are probably worth it.

Upgrading Fedora 17 to 18

It's been a while since I talked about upgrading Fedora but back in late 2009 and early 2010 I wrote about upgrading my home PC from one version of Fedora to the latest version at the time:

These were about the time (shortly after) the Fedora project introduced its latest piece of tooling, used for upgrading between distribution versions, called preupgrade.  Several years down the line and my PC and the more complex configuration than average that caused a few issues at the time have changed.  Preupgrade has been changed too, dropped in fact.  The Fedora project have moved away from it in favour of producing a new tool called Fedup (yes, yes, nice name).

It really was about time that preupgrade was replaced, as far as I know it was only ever produced as a short-term solution to upgrading Fedora without the need for booting from CD/DVD/USB/PXE and going through the Anaconda upgrade process.  It was a proof of concept if you like.  It's also been openly described within the project as "just a set of hacks" and other such non-complementary terms.  So why the need for something new?  Well I think that's already been best described elsewhere buy the guys behind Fedup so I'll just point you at a little background information on Fedup instead.

Having tried it first at work to upgrade a virtual machine from F17 to F18 I found that Fedup actually works pretty well, in fact to the user it doesn't really appear to do much differently to preupgrade.  You simply start the tool, tell it what to upgrade to, wait while it downloads the packages, then reboot to perform the upgrade, all done.  I recently tried it at home as well and it worked perfectly, no issues at all, which seems to be the experience of all the others users I've talked to about using it as well.  So far then it's a bit thumbs-up and thanks to the Fedup and Fedora teams for making it all so easy for us.

The one pitfall is that you can't upgrade straight from F16 to F18.  Traditionally with Fedora you can jump a version so you've always been able to upgrade directly from any current Fedora version that hasn't gone end of life yet to the very latest without going through intermediate versions.  Between F16 and F18 that wasn't possible though so if you're still on F16 you'll need to preupgrade your way through F17 before doing a Fedup to F18 if that's where you want to get to.  However, from F17 onwards Fedup should support the same version jumping as preupgrade did, it's simply that Fedup isn't available until F17 that causes this particular issue.



Broadband Woes

At the end of November 2012 my until then rock-solid reliable Netgear DG834PN ADSL router started reporting being disconnected from the Internet. I know this because being a decent router it has the facility to email its logs to me periodically so I have a record of everything that goes on with my Internet connection going back for years. I eventually worked out that the disconnections were being caused by phone calls connecting to the house which can happen even when we're out because we have an answering machine. So then to work, on diagnosing whether the fault lies with my equipment or with one of the service providers (Plusnet using BT line).

First I tried connecting my Netgear to the master phone socket since usually it is connected to an extension running around the house. No joy. The Netgear exhibited exactly the same behaviour from both sockets and disconnected when phone calls came in or went out of the house (only on connection, not when the phone was ringing).

Next is to try some different micro filters. No joy. I was still getting disconnected in spite of trying a couple of different ones while connected to the master socket including a genuine BT micro filter I purchased specifically to get a decent quality one for this test.

Now I've got this far I'm starting to think it's a line fault and prepare to do battle with Plusnet to get them to assess my line and perhaps send out a BT engineer to fix the problem. I go on-line to their help system. Their phone system is OK being in the UK but you have to go through a million-and-one press this to do that options. In order to raise a fault you have to answer a bunch of questions to confirm you've made all the checks that I would take for granted. Master phone socket, tick. New micro filter, tick. New router, now hang on a minute here. To the best of my (comparatively extensive) knowledge this question isn't right, routers aren't effected by phone calls, the entire point of an ADSL micro filter is to filter the line so you don't get noise during phone calls.

It seems Plusnet would not allow me to raise a line fault with them unless I had gone through all of these questions. I thought I couldn't honestly answer that I had tried a new router in spite of the fact I couldn't see how a new one would solve the problem. I decided to humour the web page and try a different router. I put the call out on social media and quickly tracked down someone a few streets away from us who had a spare I could try.

The router I borrowed was a cheap and nasty get the job done type router sent out by TalkTalk to their customers when they sign up. No wonder the guy I borrowed it from didn't use it and preferred his trusty old Netgear much like myself. However, it worked! No more disconnections when the phone connected and that was the case for whichever micro filter I used and whether I was connected to the master phone socket or the extension. So then, problem solved, it's my router after all.

Unable to keep the router I had on loan I did a bit of research into which router I should buy next to replace my Netgear. I'm a bit of a Netgear fan-boy, I have a ReadyNAS as well so it was hard to admit, based on hundreds of reviews from dissatisfied customers, their routers are now crap. I thought about going straight to an FTTC connection but wanted to get something fixed in the short term so a cheap router to keep us going for a while is the order of the day. I couldn't decide what to get and noticed Plusnet “give” you a router for a small fee to cover the postage. I decided to go with one of these.

A couple of weeks later, after I had reminded Plusnet to send me the router, it seems they received the order and took my money but somehow forgot to put it in the post, I was the proud new owner of a router made by Technicolor (or Thompson) called the TG852n. Wow wow wow, it really is quite the most appalling box of circuits, wires and firmware I've ever had the displeasure of being insulted by. It's the sort of thing I imagine an electronics company something akin to Frugal Electronics Inc. would produce, corners cut everywhere which can only be to save development cost, and credits the user with absolutely intelligence whatsoever.

I tweeted about what a bunch of crap itis and Plusnet responded wondering what I thought the issues were with it. It was difficult to explain in a 160 character response so I picked a couple of annoyances amongst plethora of wrongness that is the TG852n and sent them back. Here's my top 5:
  1. The user interface is restricted, very restricted. You cannot change anything but the very simplest of options. The first thing I wanted to do was switch it over to using my usual subnet range, nope, can't do that, you're fixed at the address range you're given. That is, unless you use the command-line interface. Most people reading this will know that I'm a fan of the command-line more than most being Mr Linux, but I really don't want to learn another one just to configure my router at home, give me a user interface that works. This brings me onto...
  2. You can telnet to the router. I didn't have to do anything to do this, you can just log in with the admin user name and password. I don't want that port open on my router thanks very much although it is comforting to know the router is a capable device scuppered only by an incapable user interface. What if I were to accidentally route port 23 from the Internet? No, just no.
  3. On the subject of open ports, port 80 appears to be open from the Internet. I don't know why, I didn't tell it to do that. Stop it. I haven't checked what other ports might have been left conveniently hanging for someone to try and get in. Remember I said I have logs going back years, I know how often my IP address is port scanned and password cracked. It happens most days.
  4. Firmware updates are more or less not supported. You certainly can't update the firmware via the user interface (surprise surprise) and if you do want to manually update it then you have to do it via TFTP (presumably via PXE boot) from a machine on your home network. Nope again, no thanks, that's rubbish. It seems, however, that Plusnet can push firmware updates to your router. I don't want that either, get off my network[1], I'll manage my end and you stick to managing yours. Finding firmware updates is nearly impossible too, they're not supplied by the manufacturer so you're left to trawl the Plusnet forums looking for posts where someone has linked to a more up-to-date version than you currently have.
  5. It's a four port Ethernet router, it has one Ethernet light. Apparently, I'm too dumb to need to see whether traffic is being routed to a certain port on the router via the convenient mechanism of a green light provided by almost every router on the planet[2]. I'm left with one light that might wave about a bit if some traffic happens to pass through the router to the Ethernet ports at some point. Oh, and they're 100MB/s ports too, but what do you expect for a free router I guess?

The final straw for me in the TG832n story and why I've decided to stop using it is that it also disconnected when the phone rang the other day. It reconnected shortly afterwards and everything was fine. However, it's now in a state where sometimes it will disconnect if the phone rings and sometimes it will not. Any which way, it always shows a red “Internet” light and a green “Broadband” light which implies that I have an active ADSL connection but that I am not dialled through to Plusnet properly. Another victory for this router then, with the red light on and glowing brightly I've got full Internet access from any of my wired or wireless devices in the house. So the router is lying to me as well[3]. I could try a firmware update, but that would be difficult.

I appreciate I'm probably not the target audience for this router. Its cut down and simplified user interface that buries what it considers “advanced” options in completely unfathomable places is probably targeted more at the likes of the silver surfer, someone who just wants to plug it in and get connected to the Internet without worrying about it any more than that.  However, the TalkTalk supplied router I have on loan (a Huawei HG532) is also a little confusing to use but it blows the Plusnet one out of the water in terms of the configuration options offered via the web GUI and would surely confuse the heck out of all but the most technical users.

Before I decided to try the Plusnet standard router my research was leading me towards Asus and (as I said before) away from Netgear. I've not tried any Asus networking kit before but I've like their motherboards for years in my home builds and we have a Transformer Prime tablet at home which is excellent. I've yet to receive this one but hope it solves the disconnection problem as well as being something that doesn't exhibit quite so many of the things I've been ranting about above. If it doesn't solve my connection issues then there may be more to say here yet.

[1] I don't really expect that Plusnet would be able to get onto my network but it's a distinct possibility.

[2] Yes, I know it's just a hub and isn't switching the traffic so is effectively broadcasting all the traffic anyway in which case one light is sufficient but I'd rather have 4.

[3] Another possibility is that the light stays red to suggest there has previously been a problem. The wafer-thin documentation actually says when the Internet light is red “Connection to Internet failed. Restart your router and see page 7. if the problem persists, open your web browser and see page 12.”

<update 5th Feb 2013>
I'm now using an Asus DSL-N12U that I purchased and reviewed at Amazon.
</update>

Combining Different Versions of Dojo

During a project I've been working on for the past couple of months my office mate and I needed to work out how to combine a recent version of Dojo and use it on a page where an old version of Dojo was already present.  Our reasons were simple, we were writing a Dojo Mobile app and hence required the use of a new version of Dojo.  However, we were displaying content within our app from a product that embeds an older version of Dojo.

With not much in the way of documentation on how to do this available, I asked our local Dojo guru for some advice and he pointed me towards the "noGlobals" option in dojoConfig.  This is the key to getting things working as it stops the new version of Dojo from writing to the global name space i.e. when you include the old version of Dojo it sets up a global variable called "dojo" if you simply include the new version of Dojo it will also write to the "dojo" global variable.  That means which ever version of Dojo you include last will always be the one you're using when you reference the "dojo" variable.  With the "noGlboals" option in place (which is adhered to by Dojo 1.7 and above I believe) you can continue to use old versions of Dojo from the global name space in conjunction with locally scoped newer versions of Dojo.

It appears there's still not a great deal of examples out there on how to do this so this is my attempt to put together a simple example to show how it's done by combining the ancient Dojo 1.3.3 with the latest at the time of writing Dojo 1.8.1:

<html>
<head>
<title>Dojo Mix-Up</title>

<!-- 
  lets start by including a really old version of Dojo 
-->
<script type="text/javascript" 
  data-dojo-config="isDebug: false, parseOnLoad: false" 
  src="dojo-release-1.3.3/dojo/dojo.js">
</script>

<!-- 
  configure Dojo to load without trampling the global namespace
-->
<script language="JavaScript" type="text/javascript">
  var dojoConfig = {
    noGlobals : true,
  };
</script>

<!-- 
  load the latest version of dojo
-->
<script type="text/javascript" 
  data-dojo-config="isDebug: false, parseOnLoad: false"
  src="dojo-release-1.8.1/dojo/dojo.js">
</script>

</head>

<body>

<!--
  Shove in a couple of divs to write to
-->
<div id="oldDojo"></div>
<div id="newDojo"></div>

<script type="text/javascript">
  // output our global namespace Dojo version (1.3.3)
  dojo.byId("oldDojo").innerHTML = dojo.version;
  
  // output our new Dojo version (1.8.1)
  require(["dojo/dom", "dojo"], function(dom,dojo){
    dom.byId("newDojo").innerHTML = dojo.version;
  });
</script>

</body>
</html>

HTC Desire Cyanogen Mod

I have the HTC Desire phone that to all intents and purposes is a decent phone even by todays standards and was an excellent phone when it was released some 2½ years ago at the time of writing.  That is, with the rather fundamental flaw for a phone that boasts an App Store - a complete lack of internal storage space.  It was built with 512MB flash storage which after HTC had muddled with it was cut to a mere 150MB for users to install applications.  That wasn't so bad back in the day, but these days you just have to install a standard set of Google apps and a few other utility applications and you're done, no more room.

The plight of the HTC Desire was giving me enough grief that I decided to do something about it by flashing CyanogenMod on it.  This like any third-party phone hack is a risky business but I figured if it failed miserably I could always restore the original HTC firmware or more likely get rid of the phone and move to something like the Samsung Galaxy S3 instead.  It turns out Cyanogen is quite the saviour of the HTC Desire and since the instructions for installing it were somewhat less than clear I wrote down what I did based on a Cyanogen forum post and a wiki page, so here it is...

I re-iterate what the forum post states:

#include <std_disclaimer.h>
/*
* Your warranty is now void.
*
* I am not responsible for bricked devices, dead SD cards,
* thermonuclear war, or you getting fired because the alarm app failed. Please
* do some research if you have any concerns about features included in this ROM
* before flashing it! YOU are choosing to make these modifications, and if
* you point the finger at me for messing up your device, I will laugh at you.
*/

Backup
I found it really useful to take a backup of things that were on the phone and put them to one side for safe-keeping both in case something went a little wrong but also to help with restoring the phone when running the new OS.  There are utilities you can use to backup your contacts (or sync them with your Google account), backup your SMS messages, even backup your saved games if you're that worried about carrying over your Angry Birds records to the new phone.  I also took a complete copy of the SD card so I had all the pictures and other things that had accumulated on there as well.

My Phone
BRAVO PVT1 SHIP S-ON
HBOOT-0.93.0001
MICROP-031d
TOUCH PANEL-SYNT0101
RADIO-5.11.05.27
Aug 10 2010,1752:18

Why don't you just S-OFF?
All the instructions talk about S-ON and how to go about gaining S-OFF without really saying anything about what this is.  It turns out that HTC implement a self-protection feature on the phone that is designed to prevent phone hacks and third-party installations that they call S-ON (presumably security on?).  With this switch set in the "on" position your phone cannot be modified.  Unfortunately for HTC they left a security flaw on some of their devices that means this switch can be altered to the "off" position.  Hence, the first thing that must be done to an HTC phone is use some software to take advantage of this and set S-OFF.

Use Revolutionary to set S-OFF
You need to find out some information about your phone similar to the info I've put above.  This will be different between models but for the HTC Desire you simply turn the phone off, then turn it on again while keeping the volume-down key depressed until the information is displayed.

The guys responsible for the software at revolutionary.io insist that you give them some information on your phone which turns out to be a sensible idea since they can attempt to ensure that this beta software doesn't screw up your device or that if it does they can prevent it being installed on similar devices in the future.  You simply go to their web site, fill out a form and download the software.

Put the phone into USB Debugging mode (Settings -> Applications -> Development -> USB debugging) and run the software.  It needs to be run as root under Linux.  It does it's thing and then reboots the phone, you'll now see you've got S-OFF set instead of S-ON if you view your phone information with the volume-down button trick once again.

One other note about revolutionary, being an open source enthusiast, is that they don't provide the source code.  Doing so would expose the security flaw that the software takes advantage of and thus would allow HTC to patch their phones accordingly and stopping this sort of modification instantly.

Use ClockworkMod to Back Up your current HTC firmware
Revolutionary gives you the option of installing ClockworkMod which is definitely worth doing.  ClockworkMod is an improved ROM manager and boot loader with some more advanced features over the stock HTC loader that comes with the phone.  One of these advantages is the ease of which a full backup of the phone's flash memory can be taken.  This appears to be called a NANDroid backup if you come across that term.  I assume that it's essentially just booting into a small program from the boot manager without accessing/mounting the flash device such that the entire flash device can be consistently backed up - a bit like a partimage or dd copy of an unmounted hard disk or partition under Linux.

Boot the phone into the bootloader (this is the same volume-down while holding power when the phone is turned off trick).  Use the volume keys to navigate up/down the menus and the power button to select the desired menu item.  You need to head for the "recovery" option which will take you through to ClockworkMod.  Select the "backup and restore" option to create a backup of the phone as it stands right now - this will give you a fighting chance of restoring the phone as-is should something go pear-shaped or for any other reason you decide to return to the stock HTC firmware.

The backup is placed on the SD card so you need to boot the phone again, access the SD card from your computer with a USB cable and store the backup image safely away somewhere - this is really important since the SD card will be wiped later along with your backup unless you copy it somewhere else first!

Get CyanogenMod Binaries onto the Phone
I grabbed the following files but there may be later versions now:

The first of these is the CyanogenMod package that will be your new operating system on the phone.  The other two packages are Google apps installed on top of the new OS.  I forget why you need the Google ones, but presume you can't get them from the Play Store on Android.

You'll need to copy these into the root directory of the SD card, still using the stock HTC firmware.

Install CyanogenMod
Boot the phone back into the recovery mode with the volume-down trick and selecting the "recovery" option to access ClockworkMod once again.  This time select the "Wipe data/factory reset" option and let that do its thing.  Then select the "Wipe cache partition" and let that complete as well.  Then you need to select the "Choose Zip from SD card" option and select the CyonagenMod zip file.  This will install the OS onto your phone after which you can choose the "Choose Zip from SD card" option again to select the Google apps zip and once again for the other Google apps.

During the installation you should create an ext data partition on the SD card that will be used a little later after the installation has completed (keep reading).  I have the stock 4GB class 2 micro SD card that came with the HTC Desire still in my phone so I opted to carve that up to use 1GB for data and keep 3GB free for use as a normal Android SD card partition.

Reboot
Reboot the phone and simple as that, you've got rid of HTC and you're booted into something that feels a lot closer to what you get on a Samsung phone i.e. a fairly stock version of Android which in this case is is 2.3 Gingerbread.

Install and Configure S2E
This app is the primary reason for installing Cyanogen, it's not the OS that's so much the saviour of the HTC Desire but the fact that using an app such as S2E (simple2ext) you can format part of your SD card for use as internal storage and thus completely resolving the tiny storage problem.  S2E is only compatible with Cyanogen and as far as I'm aware you can't do anything equivalent in the HTC stock firmware.

Once the phone has booted into Cyanogen for the first time, just fire up the Google Play store and find the S2E app.  Install it to the phone and run the configuration part of the app.  This lets you pick which parts of your installation you want to move to the ext partition you created on the SD card.  You can move over whatever you think you need to and when you're done just reboot the phone once again.

The S2E configuration will have installed a little script into the boot sequence of the phone such that on this reboot (i.e. the next reboot after running the S2E configuration) the script run and move the data you have selected onto the SD card.  It will only do this each time after you've run the S2E configuration and made changes.  These reboots can take quite a while since you could be moving quite a lot of data from your internal storage onto the SD card so be patient while it completes and boots back into Cyanogen.  Once you've done that you're good to go with using the phone again.

Final Thoughts
Having worked through all that, you'll need to tailor your phone to how you like it once again.  Restore the backed up SMS messages, contacts, files, etc.  You'll need to completely re-download any apps you liked on the HTC, configure those, set up widgets, etc, etc.  All standard stuff really, except this time you'll actually have some space in which to install things.

I'm currently using Cyanogen 7.2.0.1 which seems stable enough, it has crashed the phone a couple of times, just after I installed it but now it seems to have settled down and is running absolutely fine.  Running apps from the SD card using S2E can seem to make the phone rather laggy too so tweaking the settings in the S2E configuration to give a good balance of data stored on the SD and on the internal memory seems to be wise, along with tweaking other settings such as the read buffer.  I dropped the read buffer from the standard 2MB down to 512KB and that seems to have removed the lag.  Another way of reducing lag would be to update the SD card, say to one of the latest class 10 cards in which case it'll probably be quicker than the internal memory anyway!  I've not done that as yet and seem to be running just fine as I am for now.

Building a New PC

This post is related to a couple of my recent posts, the New PC Build post that explains the kit I ordered for this build with a little reasoning thrown in for good measure, and the New PC Install Notes post where I dumped a few of my thoughts about the software installation.  However, this post is specifically about the hardware build.

This is what I started out with, a set of shiny new boxes and an old PC which would be gutted back to the case and have only the case reused.


It's a nice Antec case and I couldn't find anything much better around today so didn't see the point in buying another one even if it did mean I had to strip it out before I started.  This is the shot with the case stripped back and all the stuff I used to have in there strewn across the bed I was doing the build on.


I only had 1 case fan in my previous build and decided to upgrade to include 2 fans in this build by including this Antec fan.  It seems to me that a lot of the new cases you buy these days have moved away from the ATX specification for front-to-back airflow as they have all sorts of fans and vents all over the place.  I quite like the idea of front-to-back airflow to give a nice continuous stream of air to the parts inside.


Here's the fan mounted in the front of my old case.  It was supplied with some little rubber grommets for screwless fixing into any case.  However, my case already had the purple clips you can see in this picture in place so I had no need for the grommets and the fan just clipped into place very quickly and easily.


Next up it was the power supply.  Here's the one I chose while still shrink-wrapped.


It turns out this is probably the most over-packaged PSU in the world.  The box is designed to give the wow factor when you open it, the likes of which are usually reserved for high-end consumer products such as a squeezebox, it has a fold-out design in a glossy box to reveal a very neatly packed PSU with all the cables hidden underneath.


The fan fitted pretty quickly and easily as well, the build was going well so far.


One of the things I use my PC for most is for photos.  I decided when putting together a spec for the new machine to include an internal card reader for convenience.  I also wanted to mount a USB 3 socket on the front of the box so this little reader from Akasa seems to fit the bill perfectly.


Here's the card reader after being unpacked and fitted.  It's much like fitting an old floppy drive, being a 3½" device and you simply connect it up to power and in this case the USB 3 motherboard header and it's job done.  I've not included a floppy drive in this build so it also makes the front of the case look less bare.


One of the indulgences of this build, given they're expensive against a traditional hard disk, is the inclusion of an SSD drive.


Here's the content of the box, you get a molex to SATA converter (presumably in case you're fitting it to some ancient power supply) as well as the fixing screws, a SATA cable and conveniently a 2½" to 3½" mounting bracket - which I needed as my case doesn't have any 2½" mounts in it.  You also get the manual and CD but those are still sealed ;-)


This is round the back of the SSD after fitting, inside the case.  You can see it's a pretty simple affair as you'd expect with just a SATA and power connectors on the back.  You can also see how the 2½" to 3½" mounting bracket works from this picture.  The bundle of wires at the top of the picture is coming from the back of the Akasa USB 3 card reader and slot.


There's plenty of better pictures of the Asus P8Z77-V LX board out there, but this is my board fresh out of the box and ready to be fitted into my case.


This is the board in place in my old case.  It was really very easy to fit as I'd removed an old Asus board before fitting this one and all the mounting holes were in exactly the same place.


Here's the new Ivy Bridge 3570K CPU out of the box with some Arcitc Silver 5 thermal compound.  Interestingly the stock cooler doesn't look too bad and is certainly much easier to fit than previous coolers I've worked with.


As I said above, I was pleasantly surprised to see the stock cooler being so easy to fit.  It was (as has been usual for coolers) a screwless installation with the four plastic fasteners secured in place with a simple firm (but not too firm) press towards the motherboard.


Here's the memory in box, ready to be fitted.  As usual for DIMMs it was a pretty undramatic installation process as they slotted in very nicely next to the CPU.  These don't have any huge fins on them for cooling and have a very simple heat sink arrangement making them low profile so I had no worries about overhanging coolers interrupting the fitting of the DIMMs since I would be using all four slots in the above picture.


All in all it was a pretty painless installation process and I've been using the machine for a couple of weeks already without any real issues.

Natural Language Processing Course


Over the first few months of this year I have been taking part in a mass online learning course in Natural Language Processing (NLP) run by Stanford University.  They publicised a group of eight courses at the end of last year and I didn't hesitate to sign up to the Natural Language Processing course knowing it would fit very well with things I'm working on in my professional role where I'm doing more and more with text analytics and continuing my work in speech to text.  There were others I could easily have signed up for too, things like security or machine learning, more or less all of them are relevant for something I'm doing.  However, given the time commitment required I decided to fully commit to one course and the NLP one was to be it.

I passed the course with a grade of 85% which was well above the required 70% pass mark.  However, the effort and time required to get there was way more than I was expecting and quite a lot more than the expected time the lecturers (Chris Manning and Dan Jurafsky) had said.  From memory it was an 8 week course with 10 hours a week required effort to complete the work. As it went on the amount of time required went up significantly, so rather than the 80 hours total I think I spent more like 1½ times that at over 120 hours!

There were four of us at work (that I know of) who embarked on the course but due to the commitment of time I've mentioned above only myself and Dale finished.  By the way, Dale has written an excellent post on the structure and content of the course so I'd suggest reading his blog for more details on that stuff, there's little point in me re-posting it as he's written such a good summary.

In terms of the participants on the course, it seems to have been quite a success for Stanford University - this is the first time they have run courses in this way it seems.  The lecturers gave us some statistics at a couple of strategic points throughout the course and it seems there were around 40,000 people registering an interest, of which around 5000 were watching the lecture material and around 2000 completed the course having taken part in the homework assignments.

I'm glad I committed as much as I did.  If I were one of the 5000 just watching the lectures and not doing the homework material I don't think I would have got as much out of it, but the added time required to complete the homework was significant so perhaps there's a trade-off here?  It's certainly the first time I've committed this much of my own personal time (it took over the lives of myself and Dale for quite a few weeks) as I was too busy at work to spend many business hours working on the course so it was all done in evenings and weekends.  That's certainly one piece of feedback I gave at the end of the course, Stanford could make the course timing more flexible but also allow more time for the course to be completed.

My experience with the way the assignments were marked was a little different to the way Dale has described in his post.  I was already very familiar with the concepts of test, development and held-out sets (three different sets of data used when training NLP systems) so wasn't surprised to see that the modules in the course didn't necessarily have an exact answer to them or more precisely that the code your wrote to perfectly analyse some data on your local system may not get full marks as it was marked against a different data set.  This may seem unfair but is common practice in all NLP system training that I know of.

All in all, an excellent course that I'm glad I did.  From what I hear of the other courses, they're not as deeply involved as the NLP course so I may well give another one a go in the future but for now I need to get a little of my life back and have a well earned rest from education.

New PC Install Notes


This post documents how I installed Linux and Windows side-by-side in a dual boot configuration.  This isn't something particularly difficult to do but I wanted to note down what I did so I remember in years to come.  Also, my new PC build contains some very up-to-date hardware (such as UEFI and an SSD drive) and with the combination of the many changes and updates to Fedora 17 (F17) and Windows 7 SP1 (Win7) it made the installation a first of a kind for me in quite a few different ways.

This post may well be updated in the future if I do further installation tweaks and optimisations to the system.  There's also clearly a lot more you can do in terms of system set up than I've written here, installation of drivers in Win7 very much being one of those.  This is simply intended as an installation and base optimisation guide.

Create USB Boot Sticks

Due to a fault with the DVD writer I ordered, it's on its way back to the retailer for replacement.  Not one to let this stop an installation going ahead I wrote install images for F17 and Win7 onto USB stick.

F17 is still in beta with the full release delayed until 29th May at the time of writing.  I tried writing and booting from the KDE Spin Beta Live image but it stopped with a kernel panic during the boot process after the grub screen.  I found using a later copy of the boot.iso (which is the same as a netinst.iso) from the distribution nightly builds solved the problem so clearly whatever the bug is on the beta image it's already been solved.

Creating a USB stick for Fedora is pretty easy these days, no messing around on the command line, I decided to use the liveusb-creator utility.  It fires up a GUI from which you basically just select the iso image you want to write and the USB device to write to, then hit "go" and it does the magic for you.

For me, writing the Windows stick was a little more complicated as it requires a Windows system to run the stick writing utility with the somewhat strange and long winded name of the Windows 7 USB/DVD Download Tool.  Fortunately, I had a Windows 7 VM laying around so I used that and writing the image to USB stick was similarly easy as it was for Fedora.  Simply choose the image file to write, the target device to write it on, then hit "go" and it does the magic for you.

Install Fedora 17

Once I worked around the kernel panic bug I mentioned above and booted with a later F17 image, the installation process was a fairly familiar affair.

With the new PC being a UEFI box there was no need to add the GPT partition when partitioning the SSD. I simply created one large root partition and a 4GB swap partition, leaving half the disk unallocated for the Win7 install later.

Since I didn't boot from the KDE spin image, I swapped out the default Gnome desktop (I've tried hard for a long time to Like Gnome 3 but I just don't so I'm moving back to KDE) and did a lot of package selection using the installer to save me messing around later but also to optimise the amount of data downloaded since I would be installing over my ADSL connection.

Once tip when installing Windows after Linux these days is to ensure the os-prober RPM gets installed.  This was done by default for me.  This package allows grub to detect the presence of other operating systems and add boot entries for them in the grub menu.  It'll come in very handy later on...!

SSD Optimisations for Linux

Even with a distribution as current as Fedora 17, the settings chosen for SSD usage are really rather sub-standard.  There are a lot of handy tips and guides out there for which settings you should change or add to the system to enhance both the performance and longevity of your SSD.  I decided to go with the rather comprehensive information provided for the Arch Linux distribution as they have a great wiki page on SSD optimisation.


Update /etc/fstab

I've only got one SSD in my system at the moment, I've removed all my old hard disks with no intention to use them right now as all my data is stored on my NAS.

Add the "noatime" and "discard" options to SSD partitions.  The discard option is the really super-important one as it turns on TRIM.

Mount /tmp as tmpfs by adding a line similar to the following:
  tmpfs /tmp tmpfs nodev,nosuid,size=75% 0 0

Doing this allows the system to write temporary files to RAM instead of the SSD.  This will improve performance (RAM is faster than SSD) and reduce writes to the SSD (improving the life of the SSD).  I've added an option to increase the allowable size of the tmpfs file system to 75% of RAM from the default of 50%.  This means I can run compilations or other intensive tasks in RAM without ruining the SSD and get the performance benefits too.  With 16GB main RAM, this will allocate up to 12GB for my /tmp directory.  In normal usage I wouldn't expect to use anywhere near this but it's nice to have it there for when I'm not running too many apps but doing something else such as compiling RPMs.

Change the Scheduler

The recommendation for an SSD is to move away from the default cfq IO scheduler and switch over to the noop scheduler.  This can be done in a variety of ways that are documented in the Arch Linux wiki page I've linked above.  Since I've only got 1 disk in my machine and it's an SSD I changed the scheduler option using grub.

For Fedora 17 this consisted of an edit to the /etc/default/grub file and adding elevator=noop to the existing GRUB_CMDLINE_LINUX stanza.  Then rebuilding the grub configuration with the command:
grub2-mkconfig > /boot/grub2/grub.cfg

Optimise Kernel Parameters

With 16GB main RAM I'm not expecting to do much in the way of swapping.  However, I did add a couple of lines to /etc/sysctl.conf to make the system less likely to do so:
  # make the system less likely to swap
  vm.swapiness=1
  vm.vfs_cache_pressure=50


Install Windows 7

Similarly to the F17 install, the Win7 install proceeded with little in the way of drama.  I did select the advanced install option when given the chance but that was just to ensure Win7 installed into the free space I had left on the SSD rather than rudely splat my shiny new F17 installation.

And yes, for those of you wondering why I'm installing Win7 at all, especially dual boot rather than in a VM, it's simply for those times when only Windows will do... so gaming mostly I should think.

There's not much else to do with Windows after installation.  Unlike current Linux distributions it's said to  detect the presence of an SSD drive and apply the appropriate optimisations automatically.  Whether this is entirely true or not I suspect I'll never be any the wiser to.

Update the Boot Loader

So Win7 didn't kill off my F17 installation which is always a bonus, but it does assume divine rights over the entire system so writes its boot loader all over the existing grub installation allowing you only to boot windows with no menu options for anything else - not ideal.  Now I need to re-install grub which I've always found easiest to do by booting a rescue Linux CD and using a chroot to the installed OS.  This has changed slightly with the inclusion of grub 2 so here's what I did.

Boot the F17 USB stick once again but this time choose the "troubleshooting" boot option and select to boot the "rescue environment".  Red Hat based systems have always done a great job of rescue environments so you'll boot into a text based system that asks you if you want various things turned on in the rescue environment such as networking (although they assume you want that these days) and if you want to attempt detection of installed Linux systems (which you do).

Drop out to the shell prompt and chroot to /mnt/sysimage.  Then rebuild the grub config (this is where os prober installed above comes in very handy) to include an entry for booting Win7.  Then re-install grub.  Job done.  Once you've booted to the rescue shell, the commands are something along the lines of:

  chroot /mnt/sysimage
  grub2-mkconfig > /boot/grub2/grub.cfg
  grub2-install /dev/sda
  exit
  exit
  reboot

This assumes you want to install grub to /dev/sda of course.  You'll need to exit (or ctrl+d) twice, once to get out of the chroot and once more to return to the rescue interface.  Then choose to reboot from there as it'll cleanly unmount your file systems and do a better job of clearing up than if you simply rebooted the system yourself.

New PC Build

It's been quite some time (about eight years) since I last built a PC for myself and I've been promising to build a new one for quite a while, now I've finally got round to ordering some parts.  Not that I need to, but I've justified the expense as a treat to myself after our annual bonus came through, having never really spent the bonus on anything particularly exciting before.

This is a short post detailing what I've gone for and why, not to show off, but so in years to come I can look back (as I have before) on what was available at the time and I'll have a record I can get to should I forget the detail of which parts are in my current PC.

I'm still the sort of geek that likes to build my own computers.  I like to do the research and put them together but it's still definitely the best way to get a good deal and the only way to get each component to be the exact one you want.  I also still like to have a PC for some reason, I have a laptop for work so I've already got something mobile and a PC just feels like the right solution for home use.

So with no further ado, on order are:

Case: I'm sticking with my old Antec SLK3700 case, okay so strictly speaking this isn't on order.

Processor: Intel Core i5-3570K (£175)
Sandybridge was a real game changer when it came out and firmly handed back the power to the hands of Intel  in the processor market.  I've been waiting for Ivybridge to come out for a while, either so I could buy a cheaper Sandybridge or plump for the Ivybridge if the price difference wasn't too great.  Well, for those who know their processors you'll see I've gone for the Ivybridge option.  Currently it's only 20-odd quid more than the equivalent Sandybridge processor and I think the extra expense is worth it to get the better on-chip graphics capabilities and minor improvements in speed and energy efficiency.

Motherboard: Asus P8Z77-V LX (£94)
I'm a sucker for a good Asus board, I've used them in most of my PC builds so that's where I start looking when I want a new machine.  I nearly went for the LE version of this board, it was another 20 quid, but I decided I wouldn't be using the extra features it provides (surround-sound audio and extra sata sockets) so I pocketed the difference and went for this one instead.

Memory: Kingston (4x4GB) DDR3 1600MHz XMP HyperX (£65)
I very nearly went for some Corsair low profile DIMMs but was swayed by the vendor support list for DIMMs for the motherboard chosen above.  I'm still in shock that you can get 16GB RAM for 65 quid!

Storage: Intel 120GB 520 Series SSD (£140)
I've got 4 hard disks in my PC at the moment, one of which is not connected, another is hardly used and the remaining 2 are coupled together in a striped RAID array in an attempt to get some sort of speed out of them.  Hard disks really are the bottleneck in your PC these days so I've decided to shove in a top notch SSD from Intel.  They're the only manufacturer to offer a 5 year warranty and also came highly recommended (under the Hitachi brand, Intel and Hitachi work together on their SSDs) from one of my respected colleagues in the storage department at work.  120GB should be ample for my storage requirements on the PC, the disk will be split to dual boot both Windows 7 64-bit (for those occasions when only Windows will do and a bit of gaming) and the main stay of Fedora x86_64.  My data other than the operating system already lives on a NAS.

PSU: CoolerMaster 600W Silent Pro Modular (£63)
Unfortunately my current 620W ATX power supply only has a 4 pin CPU connector rather than the currently common 8 pin EPS so I've begrudgingly had to fork out for a new PSU.  This CoolerMaster one gets some decent reviews for being quiet, efficient and delivering good consistent power within the tolerances required by ATX.  The 600W rating will also leave me with some overhead should I decide to whack in a high-end GPU at some point in the future.

Thermal Paste: Arctic Silver 5 (£5)
I've got some really old unbranded thermal paste knocking around somewhere but decided to invest in some decent stuff for this build so the Arctic Silver was the way to go.

DVD Writer: Samsung S222AL 22x with Lightscribe (£15)
I've got 2 optical drives at the moment, both are IDE and with the new breed of motherboards (or not necessarily even the ones that are particularly new) IDE is long since dead so I've opted to get a Sata DVD writer for this build.  Similar to the memory, I'm amazed you can pick something like this up for 15 quid!

Front Case Ports: Akasa USB 3.0 Card Reader (£20)
My current case has a couple of USB ports on the front, I thought it would be useful to throw a couple of USB 3.0 ports to the front of the case too.  This unit also has a built-in multi-card reader so I'll no longer have to hunt for my USB SDHC reader every time I want to copy pictures from my camera.

Case Fan: Antec TrueQuiet 120 (£7)
A 12cm fan for the front of my case, I've got a slot to fit another one in so I thought why not given the heat output I'd expect from this build.

Keyboard and Mouse: Logitech Desktop MK120 (£13)
My current keyboard is PS/2 and has seen better days.  I still like a simple keyboard with none of these funny curves or multimedia keys you can get these days so went for this cheap set from Logitech.

The obvious note in this build is a lack of a GPU.  As I mentioned above, I've left overhead in the power supply to put in a GPU in the future should I choose to do so.  I'm going to do this build and run on the GPU built into the processor.  I'll be interested to see what the performance of the HD 4000 is for my needs, if it's sufficient then great, otherwise I'll be tempted towards an NVidia GTX-560 card.  I guess it all depends on if I do a little more gaming then I do currently (which is next to none on the PC) and whether the HD 4000 is up to the job.

The other thing I've got my eye on is an up-rated cooler from the stock cooler supplied with the 3570K.  I quite like the look of the Corsair H60 Hydro should the need arise.