Friday, 13 September 2019

Installing Tensorflow GPU on Fedora Linux

Following on from my previous notes on building Tensorflow for a GPU on Fedora, I find myself back at it again.  I recently upgraded my GPU at home and time has moved on too so this is my current set of notes for what I'm doing with Tensorflow on Fedora.  This method, however, differs from my previous notes in as much as I'm using the pre-built Tensorflow rather than building my own.  I've found that Tensorflow is so brittle during the build process it's much easier to work with pre-built binaries and set up my system to match their build.

In my previous blog post I benchmarked the CPU versus GPU using the Keras MNIST CNN example and so I thought it would be interesting to offer the same for this new install on my home machine.  The results are  :
  • 12 minutes and 14 seconds on my CPU
  • 1 minutes and 14 seconds on my GPU
That's just over 9.9 as fast on my GPU as my CPU!

Some info on my machine and config:
  • Custom Built Home PC
  • Intel Core i5-3570K CPU @ 3.40GHz (4 cores)
  • 16GB RAM
  • NVidia GeForce GTX 1660 (CUDA Compute Capability 7.5)
  • Fedora 30 Workstation running kernel 5.2.9-200.fc30.x86_64
Background Information for NVidia Drivers
Previously, I've always used the Negativo17 repository for all my NVidia driver and CUDA needs.  However, the software versions available there are too up-to-date to allow Tensorflow GPU to be installed in a way that works.  This repository provides CUDA 10.1 where as Tensorflow, currently at version 1.14, only supports CUDA 10.0.  So we must use another source for the NVidia software that provides back-level versions.  Fortunately, there is an official NVidia repository providing drivers and CUDA for Linux, so let's use that since it also works quite nicely with the RPM Fusion repositories as well.  Hence, this method relies purely on RPM Fusion and the official NVidia repository and does not require or use the Negativo17 repository (although it would be possible to do so).

Install Required NVidia Driver
The RPM Fusion NVidia instructions can be used here for more detail, but in brief simply install the display drivers:
  • dnf install xorg-x11-drv-nvidia akmod-nvidia xorg-x11-drv-nvidia-cuda
There are some other bits you might want from this repository as well such as:
    • dnf install vdpauinfo libva-vdpau-driver libva-utils nvidia-modprobe
    Wait for the driver to build and reboot to get things up and running.

    Install Required NVidia CUDA and Machine Learning Libraries
    This step relies on using the official nvidia repositories with a little more information available in the RPM Fusion CUDA instructions.

    First of all, add a new yum configuration file.  Copy the following to /etc/yum.repos.d/nvidia.repo:

    [nvidia-cuda]
    name=nvidia-cuda
    enabled=0
    gpgcheck=1
    gpgkey=http://developer.download.nvidia.com/compute/cuda/repos/fedora27/x86_64/7fa2af80.pub
    exclude=akmod-nvidia*,kmod-nvidia*,*nvidia*,nvidia-*,cuda-nvidia-kmod-common,dkms-nvidia,nvidia-libXNVCtrl

    [nvidia-machine-learning]

    name=nvidia-machine-learning
    baseurl=http://developer.download.nvidia.com/compute/machine-learning/repos/rhel7/x86_64/
    enabled=0
    gpgcheck=1
    gpgkey=http://developer.download.nvidia.com/compute/machine-learning/repos/rhel7/x86_64/7fa2af80.pub
    exclude=libcudnn7*.cuda10.1,libnccl*.cuda10.1



    Note that the configuration above deliberately targets the fedora27 repository from NVidia.  This is because it is the location at which we can find CUDA 10.0 compatible libraries rather than CUDA 10.1 libraries that will be found in later repositories.  So the configuration above is likely to need to change over time but essentially the message here is that we can match the version of CUDA required by targeting the appropriate repository from NVidia.  These libraries will be binary compatible with future versions of Fedora so this action should be safe to do for some time yet.


    With the following configuration in place we can now install CUDA 10.0 and the machine learning libraries required for Tensorflow GPU support and all of the libraries get installed in the correct places that Tensorflow expects.

    To install, run:
    • dnf install cuda libcudnn7 libnccl

    Install Tensorflow GPU
    The final piece of the puzzle is to install Tensorflow GPU which is now as easy as:
    • pip3 install tensorflow-gpu

    Friday, 23 August 2019

    Migrating to Gnome 3

    I'm a massive laggard in the move to a Gnome 3 desktop.  Colleagues and friends have been using it for years and to be honest, I've never been comfortable using it.  But, that changed recently and I've actually grown to quite like the new desktop environment I find myself working in on a daily basis.  So I've made a full-blooded leap to a modern desktop.

    Way-back when I started using Linux as a serious desktop alternative to Windows (in about 2000-2001 ish) I was running Gnome.  I migrated away from that to KDE 3 and switched to Gnome 2 when KDE 4 was released as I didn't like the changes they had made and the new KDE 4 desktop was horribly buggy and unstable in my experience.  (Maybe there's something about brand new desktops and my not taking a liking to them?)  When Gnome released Gnome 3 I absolutely hated the user experience and used XFCE for a while before settling on the MATE desktop which I've been using for quite a few years now.

    Trying out Gnome 3 again recently and I was pleasantly surprised that the desktop has progressed significantly since those first few releases I couldn't get along with.  But it's the addition of extensions that are the final straw in my move as I've found with just the right mix I can craft a desktop that gives me a nice balance between the new world and the old, much more familiar, world.

    So, the real purpose of this post is to share the extensions I've discovered.  I'll document these below in brief but would also be interested to find others that are useful:

    Applications Menu
    This was right at the very top of my list of requirements for Gnome 3 usability.  It simply puts an old school applications menu in the top bar, a bit like your old fashioned Windows start menu or similar from other desktops.  I am, however, finding I use this very little now as the search hot-key in Gnome 3 does seem to be a quicker way of finding and starting programs.

    Frippery Bottom Panel
    This is another of my top requirements for Gnome 3 usability.  It gives you a panel at the bottom of the screen (D'uh) that allows you to switch easily between applications you have running.  It also has a small workspace switcher which is why I like the Frippery version of this type of extension versus some of the others that don't have a workspace switcher capability.

    Top Icons Plus
    Either the Top Icons or the Top Icons Plus extension that I'm using here seem so ubiquitous for Gnome 3 users I wonder why on earth they're not a default option, aside from the fact the Gnome 3 developers do seem to retain their keen vision on what a modern desktop should look like and "old" system tray icons are not part of that outlook.  This extension, if you're not already using it, allows you to see system tray icons such as the ones used by Virt Manager or Slack, for example.

    GPaste
    A clipboard management system that has a nice integration with the Gnome 3 panel.  I was previously using apps like ClipIt or Parcelite that do pretty much the same job.

    Lock Screen
    This adds a button to the gnome panel that, when clicked, locks your desktop.  This would be the same as pressing Win+L on the keyboard.  I was in the habit of using a graphical button on MATE so having this back in Gnome 3 gives me the experience I'm used to.

    No TopLeft Hot Corner
    I find the Gnome 3 facility to show activities when you mouse to the top left corner really annoying and it detracts from my productivity when it happens automatically.  Fortunately, this extension disables that feature.  It does make it more awkward to reach activities with the mouse (I'd have to click the applications menu first then select "Activities Overview") but I more or less always use the Windows key anyway.

    Places Status Indicator
    This adds the old Gnome 2 style places menu to the Gnome 3 panel.  I find I flip between using this menu to start navigating directories and just starting Gnome Files and going from there.  Any which way, having this menu back on my desktop just makes it feel a bit more familiar and comfortable.

    Remove Dropdown Arrows
    The Gnome 3 panel insists on having an arrow indicator to show items that pull down a menu when clicked.  These menus seem obvious to me and the arrows look rubbish and take up space, so this extension gets rid of them completely.  Happy days.

    Suspend Button
    I run from a laptop most of the time and use the suspend feature every time I "shut down" my laptop.  Bizarrely, there's no graphical facility (that I can find) in Gnome to suspend my machine.  This extension adds a nice button to the status menu that immediately suspends my machine.  Perfect.

    System Monitor
    Adds little graphs to the Gnome panel that show resource usage.  The extension is pretty configurable but I have it showing CPU, memory and network utilisation.  This allows me to keep an easy eye on my machine and how loaded it is at the current time.  Extremely useful for spotting those occasional rogue apps that start eating an entire core of my CPU.

    Media Keys
    I haven't decided how useful this one is going to be yet and it's currently turned off.  However, when listening to Music through services like Amazon Music from a web browser it's nice to be able to control the audio without having to revert back to the browser ever time.  This extension simply adds a few buttons to the Gnome panel to control your media.  Handy if you haven't got the physical buttons on your keyboard too.

    Do Not Disturb Button
    I generally leave this extension disabled but it's useful to have installed and running when presenting or screen sharing.  It saves any embarrassing situations of people being able to read your notifications while they're looking at your screen.  Basically, it simply stops notifications being displayed, they're still received so you can go read them later.

    Blog edited with more extensions added on 28th August 2019:
    Frippery Panel Favourites
    I'm not quite sure how I missed this from my original list as it's an extension I've been using more or less since day one in Gnome 3.  It takes your favourite menu and adds this as a set of icons to the top of the Gnome Panel.  Makes for extra quick access to your commonly used apps.

    Some more extensions have been brought to my attention since writing the list above.  I've tried out all of the ones mentioned to me but these additions (below) are the ones that seem to have stuck.

    Caffeine
    This extension sits fairly well alongside the Do Not Disturb Button extension in my original list.  This one simply disables the screen saver and auto suspend.  Hence, in conjunction with Do Not Disturb, will make a good presentation or screen sharing environment.

    GTile
    This is a genius little extension that allows you to easily resize your windows in order to tile them across your display.  I love the side-snapping in Gnome 3 that allows you to size a window to half the screen size.  In my older desktops I also had corner snapping to size a window to a quarter of the screen, Gnome 3 doesn't have this by default.  However, GTile adds an icon to your Gnome Panel that, when clicked, allows you to size to any area of your screen across a pre-defined grid - you can even change the grid size.  Brilliant for usability with lots of on-screen windows at the same time.


    Friday, 18 January 2019

    Self-Signing SSL/TLS Certificates

    Things have changed a bit since I last looked into setting up a Certificate Authority (CA) and using that to self-sign my own certificates, not least that the use of the Common Name (CN) field appears to have changed. Chrome in particular seems to insist on the use of the Subject Alternative Names (SAN) extension rather than (or in addition to) using the CN field. So these are my notes on how to set up your own CA and use that to sign certificates. I'm conscious this is bound to go out of date so at the time of writing I'm working with Firefox 64, Chrome 71 and OpenSSL 1.1.1.

    Setup
    First of all, create a config file along the lines of the following and call it anything you like but for these notes I'm going to call it ssl.conf.  Note, if you want to you can start with a different template or look at your own openssl.cnf file which on Linux is commonly found at /etc/pki/tls/openssl.cnf.

    [ req ]
    default_bits       = 4096
    distinguished_name = req_distinguished_name
    req_extensions     = req_ext
    
    [ req_distinguished_name ]
    countryName                 = Country Name (2 letter code)
    countryName_default         = GB
    stateOrProvinceName         = State or Province Name (full name)
    stateOrProvinceName_default = England
    localityName                = Locality Name (eg, city)
    localityName_default        = MyCity
    organizationName            = Organization Name (eg, company)
    organizationName_default    = MyOrg
    commonName                  = Common Name (e.g. server FQDN or YOUR name)
    commonName_max              = 64
    commonName_default          = localhost
    
    [ req_ext ]
    subjectAltName = @alt_names
    
    [alt_names]
    DNS.1 = localhost
    

    You can change any of this template and indeed you'll need to change the common name for the certificates you're generating. The CN can be changed either on the command line during certificate creation or by changing the default in ssl.conf. You will also need to change the list of names under the "alt_names" section, this list should contain one line for each host name your machine might be known as. The list starts at DNS.1 for the first entry, then you can add DNS.2 for the second entry and so on.

    NOTE: the specification and a lot of the documents available in this space indicate that an IP address can be used in the CN.  My testing seems to indicate that while this is the case, certificates produced in this way will be rejected by modern browsers.  Hence, you should list only hostnames as the CN but IP addresses still appear to be acceptable in as "alt_names".

    Create a Certificate Authority
    You'll need a certificate and key file to act as your own CA:

    openssl genrsa -out RootCA.key 4096
    openssl req -x509 -new -nodes -key RootCA.key -sha256 -days 3650 -out RootCA.pem -config ssl.conf

    You can inspect the certificate with:
    openssl x509 -in RootCA.pem -text -noout

    Create a Certificate Signing Request (CSR)
    Now you have a CA you can create a CSR that can be used with your CA certificate to generate a client certificate:

    openssl genrsa -out server.key 4096
    openssl req -new -key server.key -out server.csr -config ssl.conf

    You can inspect the certificate with:
    openssl req -text -noout -verify -in server.csr

    This time it's really important to ensure your host names are listed under the "X509v3 Subject Alternative Name" section of the certificate.

    Generate a Signed Certificate
    You can now use the CSR to create a signed certificate that can be used to serve up content over a secure connection:

    openssl x509 -req -in server.csr -CA RootCA.pem -CAkey RootCA.key -CAcreateserial -out server.pem -days 3650 -sha256 -extensions req_ext -extfile ssl.conf

    Note: if you want to create a different format of certificate here you can simply replace server.pem in the above command with something like server.crt, for example.

    You can inspect the certificate with:
    openssl x509 -in server.pem -text -noout

    Again, it's really important to ensure your host names are listed under the "X509v3 Subject Alternative Name" section of the certificate.

    Use the Certificate Server Side
    You can now put your server.pem and server.keyfiles to work and serve up content over a secure connection.  There's too many ways to do that to list here but it could be used with a web server to serve HTTPS or a websockets server to serve some sort of socket connection as a couple of examples.

    If you want to inspect the certificate that's being used on the server, replace <hostname> and <port> in the command below:

    openssl s_client -connect <hostname>:<port> | openssl x509 -noout -text

    Use the Certificate Client Side
    My use case here is with a web browser and so you'll want to import your <i>RootCA.pem</i> into your browser environment.  There are two main ways of achieving this, you can either:
    1. Import directly to the browser
    2. Import to the key store on your operating system
    It's quicker and easier to import directly to the browser but this will of course only cover that one browser on your system whereas if you use the operating system method then any application that consults the OS for certificates will see your CA certificate.

    For Firefox, go to "View Certificates" in the preferences; click the "Authorities" tab and then the "Import" button; select your <i>RootCA.pem</i> file and click OK.

    For Chrome, go to "Manage Certificates" in the settings; click the "Authorities" tab and then the "Import" button; select your <i>RootCA.pem</i> file; click the check boxes to trust the certificate and click OK.

    Friday, 16 November 2018

    Building Tensorflow GPU on Fedora Linux

    <update on Sept 13th 2019>
    I have written another post on how to install (rather than build) Tensorflow GPU for Fedora that uses a different and much simpler method.  See Installing Tensorflow GPU on Fedora Linux.
    </update>

    First off, let's say that there are easy ways of configuring Tensorflow for GPU usage such as using one of the docker images.  However, I'm a bit old school for some things and having always done so I've recently got Tensorflow going on my machine using my GPU.  Tensorflow CPU support is quite easy to do and generally works quite nicely using the pip install method.  GPU support, I've always found, is quite a bit more difficult as there are a whole bunch of things that need to be at just the right level for everything to work i.e. it's quite brittle!

    What follows are my notes (it's in the name of the blog) for how to build Tensorflow from scratch to enable GPU support and I do this on Fedora Linux.  If you want to know why it's worth bothering going to this effort, I've tested the Keras MNIST CNN example as a bench mark.  It takes:
    • 11 minutes 7 seconds on my CPU
    • 2 minutes 55 seconds on my GPU
    That's just over 3.8 as fast on my GPU as per my CPU so for large jobs this will be huge!

    Some info on my machine and config:
    • Lenovo P50 Laptop
    • Intel Core i7-6820HQ CPU @ 2.70GHz (4 core with hyper threading)
    • 32GB RAM
    • Nvidia Quadro M1000M (CUDA compute capability 5.0)
    • Fedora 28 running kernel 4.18.18-200.fc28.x86_64
    Install Required Nvidia RPMs
    You need to get everything Nvidia and CUDA installed on your machine first.   I quite like the Negativo17 repository for Nvidia on Fedora Linux and so I use this but you could also go with RPM Fusion or even download everything directly from Nvidia.  For me, right now, I have this little lot installed:
    cuda-9.2.148.1-2.fc28.x86_64
    cuda-cli-tools-9.2.148.1-2.fc28.x86_64
    cuda-cublas-9.2.148.1-2.fc28.x86_64
    cuda-cublas-devel-9.2.148.1-2.fc28.x86_64
    cuda-cudart-9.2.148.1-2.fc28.x86_64
    cuda-cudart-devel-9.2.148.1-2.fc28.x86_64
    cuda-cudnn-7.2.1.38-1.fc28.x86_64
    cuda-cudnn-devel-7.2.1.38-1.fc28.x86_64
    cuda-cufft-9.2.148.1-2.fc28.x86_64
    cuda-cufft-devel-9.2.148.1-2.fc28.x86_64
    cuda-cupti-9.2.148.1-2.fc28.x86_64
    cuda-cupti-devel-9.2.148.1-2.fc28.x86_64
    cuda-curand-9.2.148.1-2.fc28.x86_64
    cuda-curand-devel-9.2.148.1-2.fc28.x86_64
    cuda-cusolver-9.2.148.1-2.fc28.x86_64
    cuda-cusolver-devel-9.2.148.1-2.fc28.x86_64
    cuda-cusparse-9.2.148.1-2.fc28.x86_64
    cuda-cusparse-devel-9.2.148.1-2.fc28.x86_64
    cuda-devel-9.2.148.1-2.fc28.x86_64
    cuda-docs-9.2.148.1-2.fc28.noarch
    cuda-gcc-7.3.0-1.fc28.x86_64
    cuda-gcc-c++-7.3.0-1.fc28.x86_64
    cuda-gcc-gfortran-7.3.0-1.fc28.x86_64
    cuda-libs-9.2.148.1-2.fc28.x86_64
    cuda-npp-9.2.148.1-2.fc28.x86_64
    cuda-npp-devel-9.2.148.1-2.fc28.x86_64
    cuda-nvgraph-9.2.148.1-2.fc28.x86_64
    cuda-nvgraph-devel-9.2.148.1-2.fc28.x86_64
    cuda-nvml-devel-9.2.148.1-2.fc28.x86_64
    cuda-nvrtc-9.2.148.1-2.fc28.x86_64
    cuda-nvrtc-devel-9.2.148.1-2.fc28.x86_64
    cuda-nvtx-9.2.148.1-2.fc28.x86_64
    cuda-nvtx-devel-9.2.148.1-2.fc28.x86_64
    nvidia-driver-cuda-libs-410.73-4.fc28.x86_64

    You might wonder about some of the above, particularly why you might need a back level version of GCC.  When Fedora 28 has a quite capable GCC version 8 why on earth would you want version 7?  The answer lies in my comment about things being difficult or brittle, it's quite simply that CUDA doesn't yet support GCC 8 so you do need a back level compiler for this

    Install NVidia NCCL
    This library isn't available through an RPM installation or the Negativo17 repository and so you must:
    • Go to the Nvidia NCCL home page 
    • Click the link to download NCCL (requires an Nvidia developer login account)
    • Agree to the Terms and Conditions
    • Download the NCCL zipped tar file that matches your CUDA version (9.2 for this blog post)

    At the time of writing the file required is nccl_2.3.7-1+cuda9.2_x86_64.txz

    I simply untar this file into /usr/local and create a symbolic link as follows:
    • cd /usr/local
    • sudo tar -xf /path/to/file/nccl_2.3.7-1+cuda9.2_x86_64.txz
    • sudo ln -s nccl_2.3.7-1+cuda9.2_x86_64.txz nccl


    Install the Bazel Build Tool
    You're going to need a build tool called Bazel which isn't directly available in the Fedora repositories (that I know of at least) but fortunately there's a version in a copr repository you can use as documented run the following commands:
    •  dnf copr enable vbatts/bazel
    •  dnf install bazel
    Get a Copy of Tensorflow Source
    For this it's just as easy to use git as it is anything else.  You can directly clone the 1.12 release of Tensorflow into a new directory by running:
    • git clone --single-branch -b r1.12 https://github.com/tensorflow/tensorflow tensorflow-r1.12
    • cd tensorflow-r1.12
    Simply replace r1.12 in the above commands if you want to use a different Tensorflow release.

    Run the Tensorflow Configure Script
    This step is actually quite simple but you'll need the answers to some questions to hand, simply run:
    • ./configure
    I accept all the default options with the exception of:
    • "location of python" set to /usr/bin/python3 since Fedora still uses Python 2.7 as the default version at /usr/bin/python
    • "build TensorFlow with CUDA support" set to Yes
    • "CUDA SDK version" set to 9.2 (this value should match the cuda version you have installed and at the time of writing 9.2 is the current version from the Negativo17 repository)
    • "location where CUDA 9.2 toolkit is installed" set to /usr
    • "cuDNN version" set to 7.2 (similar to the cuda version above, this value should match the cuda-cudnn package version and 7.2 is the current version from the Negativo17 repository)
    • "NCCL version" set to 2.3
    • "location where NCCL 2 library is installed" set to /usr/local/nccl
    • "Cuda compute capabilities you want to build with" set to 5.0 (but this value should match the CUDA compute capability of the GPU in the machine you're building for)
    • "which gcc" set to /usr/bin/cuda-gcc (to use the back level GCC version 7)


    Fix Bazel Config
    The above config command writes a file but the location isn't compatible with the latest version of Bazel.  Presumably this issue will be fixed at some point in the future, it's not an issue with Bazel 0.18 and below as far as I'm aware, but has just become an issue on 0.19.  Simply copy the config to the correct place:
    • cat tools/bazel.rc >> .tf_configure.bazelrc
    Build Tensorflow with GPU Support
    This took around an hour to complete on my machine:
    • bazel build --config=opt --config=cuda //tensorflow/tools/pip_package:build_pip_package
    • bazel-bin/tensorflow/tools/pip_package/build_pip_package /tmp/tensorflow_pkg
    The first step is the long one for the build, the second simply builds the python wheel file.

    Install Tensorflow with GPU Support
    You've got your wheel file so simply install and enjoy:
    • pip3 install tensorflow-1.12.0-cp36-cp36m-linux_x86_64.whl 
    Run Some Code
    The first time I attempted to run some code to test I got an error:
    • failed call to cuInit: CUDA_ERROR_UNKNOWN
    This can be solved by making sure you have the nvidia-modprobe package installed.  Alternatively, you can run the little script below the following explanation.

    This seems to be some sort of permissions issue and running the following simple script to output the GPUs available on my machine but as root seems to have fixed the above issue i.e. put the following into a script, run that script as root, then any time you want to run code as an unprivileged user the above issue is fixed and the code will work:
    from keras import backend as K
    K.tensorflow_backend._get_available_gpus()

    If the above works then you can try out the Keras MNIST CNN example code.

    Monday, 5 November 2018

    VueJS Example for IBM App ID

    I was recently working on a project in VueJS that needed an authorisation layer added to it.  It turns out there aren't any existing examples of how to do this anywhere, unusually not even on Stack Overflow.  So I set about writing one and thought I would share it.  My work was based upon some other useful examples and information, particularly a blog post from the IBM Cloud blog.

    Before I go any further, the code samples are available and documented on GitHub as follows:

    1. IBM App ID API Server
    2. App ID VueJS Client

    The code is deliberately split into two such that:
    1. the API Server is used to demonstrate how to secure an API on the server side.  This is done with the WebAppStrategy of App ID which is simply an implementation of a strategy package for passportjs.  The code here isn't anything particularly new over existing examples you can find on the web but it's necessary in order to fully demonstrate the capabilities of the client code.
    2. the VueJS Client is used to demonstrate two things:
      1. how to secure a VueJS route for which I can currently find no example implementations on the web
      2. how to call an API that has been secured by App ID by passing credentials through from the client application to the API server
    The API Server should be relatively trivial to get up and running as it's a standard NodeJS API implementation using Express.  If you refer to the WebAppStrategy and the blog post I mention above then you'll see the sample code I've come up with is broadly the same i.e. an amalgamation of the two.

    The VueJS Client code can be simple to get up and running as well but it's probably more important to understand how it was created such that you can apply the same principles in your own application(s).  For this then, the explanation is a little longer...

    Start by running the VueJS command line client (cli) to create a bare project and for the sample to make sense you will need to add VueX and Router components using the tool:
    vue create vue-client
    Then understand the 3 modifications you need to make in order to have a working set of authenticated routes.

    1. A store for state. 
    It doesn't really matter how you achieve this in VueJS, you can use any form of local state storage.  The example code I have come up with uses VueX and a modification to the store.js code you get from the client above.  The idea of this is such that the client application can cache whether the user has already authenticated themselves.  If they have not then the client must request authentication via the server.  If they have, then all the credentials required for making an authenticated call to a server-side API are already available in the browser.  Essentially, this is a speed-up mechanism that stops the client from requesting client credentials on each API call since the session store for the authentication actually lives on the server side when using App ID.

    2. A new VueJS Component
    This is the component whose route is to be protected via authentication.  In the case of the example code below the standard vue cli "About" component has been used and modified slightly to include an authenticated call to the server API.  The thing to note here is that the credentials from the client side must be sent over to the server with each API call.  Using the fetch API as per the below to implement your GET request means you have to add the credentials: 'include' parameter.

    <template>
      <div class="about">
        <h1>This is a protected page</h1>
        <h2>hello: {{ hello }}</h2>
      </div>
    </template>
    
    <script>
    export default {
      data: function () {
        return {
          hello: undefined
        }
      },
      computed: {
        user () {
          return this.$store.state.user
        }
      },
      methods: {
        getProtectedAPI () {
          fetch('http://localhost:3000/protected/get-some-info',{
                credentials: 'include',
              }).then(res => res.text())
              .then(body => {
                console.dir(body)
                this.hello = JSON.parse(body).hello
              })
        },
      },
      created() {
        this.getProtectedAPI()
      }
    } 
    </script>

    3. A VueJS Navigation Guard
    You need to write a function that will be added as a VueJS middleware upon each route change.  The middleware is inserted automatically by the VueJS route code when using the beforeEnter call on a route.  This is known in VueJS as a Navigation Guard.

    function requireAuth(to, from, next) {
      // Testing authentication state of the user
      if (!store.state.user.logged) {
        // Not sure if user is logged in yet, testing their login
        const isLoggedUrl = "http://localhost:3000/auth/logged"
        fetch(isLoggedUrl, {credentials: 'include'}).then(res => res.json()).then(isLogged => {
          if (isLogged.logged) {
            // User is already logged in, storing
            store.commit("setUser", isLogged)
            next()
          } else {
            // User is not logged in, redirecting to App ID
            window.location.href=`http://localhost:3000/auth/login?redirect=${to.fullPath}`
          }
        }).catch(e => {
          // TODO: do something sensible here so the user sees their login has failed
          console.log("Testing user login failed - D'oh!")
        })
      } else {
        // User already logged in
        next()
      }
    }

    The requireAuth function does the following in plain English:

    1. Using the VueJS client side cache, test if the user is already logged in
    2. If they are not. then ask the server if the user is already logged in
      1. If they are not, then redirect them to the server login page
      2. If they are, then cache the information and load the next piece of middleware
    3. If they are, then simply load the next piece of middleware


    Each route you want to protect with the above function must have a beforeEnter: requireAuth parameter specified on the route.  When this is done, VueJS will call the requireAuth function before the component specified by the route is loaded.

    {
      path: '/protected',
      name: 'protected',
      beforeEnter: requireAuth,
      component: Protected
    }

    Note: there are methods by which you don't have do call window.location.href to redirect the user to the login page (which does seem like a bit of a nasty hack.  However, these methods require the modification of the webpack configuration and so were kept out of scope of this example for the purposes of being simple.

    Monday, 4 June 2018

    South Downs Way Walk

    I've just finished a pretty extraordinary journey, both physically and mentally, here's the story...

    The Short Version - 100 Miles, 4 Days, 2 Charities

    This bit of the blog post is a summary for the TL;DR brigade...
    • Starting in Eastbourne on 31st May
    • More or less 4 marathons in 4 days
    • Finishing in Winchester on 3rd June
    • Raising funds for:
      • The National Eczema Society
      • Parkinson's UK
    • You can still donate (please do):
    • We raised over £4500 (more than £5500 including gift aid)

    I have made all the pictures available on Flickr in my South Downs Way Walk Album.


    The Longer Version

    My best mate decided he wanted to challenge himself as a consequence of coming to terms with his 40th birthday.  He called a few of us to the pub a few months back.  We settled on the idea of walking the South Downs Way in 4 days for 2 charities.  I was nuts enough to go along with this idea and I'm writing the morning after successfully finishing the entire route with him.

    All done and in the restaurant at the finish
    In the car on the way to get started...
    The people you need to know about in this story are:
    • Matt Wettone - team leader
    • Me (obviously)
    • Andy McGrath - team super hero
    • Pete and Phil - team members
    • Stephen Warwick - honorary team member
    • Tim - Matt's dad, team logistics
    • Linda - Matt's mum, team chef

    All done and in the restaurant at the finish
    ...all done and in the restaurant at the finish

    The reason for doing this has already been much better explained by Matt than I'll manage to come up with so if you're interested in that then head over to our donation page or I've saved a copy of his text at the bottom of this post.  My personal reasons are fairly simple in as much as I'd find it hard to turn Matt down, doing something for charity is always worthwhile and challenging yourself every now and then has to be worth something as well.

    It turns out that it certainly was a challenge both physically and mentally.  I'll try to be brief below with a summary of each day...

    Day 1 - Eastbourne to Housedene Farm

    Vital statistics: 27.4 miles in 8 hours 26 minutes moving time (3.25 MPH, moving average), about 58,000 steps and 3968 feet of elevation gain.


    We planned Day 1 to be 24 miles long but ended up doing 27.4 miles.  This was due to starting further back than the documented start of the South Downs Way, we started at Eastbourne Pier, another 1.5 miles along the coast.  Also, the distance charts for the Eastern half of the walk seem to be about 10% out in terms of accuracy so we picked up another couple of miles we weren't expecting.

    Matt, Andy and I left Eastbourne in heavy rain.  I think we were all excited and apprehensive but keen to get started.  After the first couple of miles, we climbed up onto the Seven Sisters and for the next 10 miles or so found ourselves repeatedly going up and down each of the 7 cliffs.  That was really hard work, especially in poor weather, and set the tone for the rest of the day as being a really tough day.

    We pretty much dodged a bullet with the weather though, the rain cleared up by around 11am and it became overcast and warm for the remainder of the day, perfect walking weather.  The rest of the country seemingly got drowned in thundery showers that day so we were really lucky.

    We finished the day in good spirits but with Andy starting to struggle a bit with a knee problem and having picked up a blister along the route somewhere.

    Day 2 - Housedene Farm Camp Site to Amberley

    Vital statistics: 29.7 miles in 9 hours 5 minutes moving time (3.23 MPH, moving average), about 62,000 steps and 3186 feet of elevation gain.



    This route was also longer than the planned 26.5 miles we had in mind for day 2.  We picked up the trail where we left off the previous evening at Housedene Farm and walked to Amberley.  Pete joined us, meeting at the start point in the morning and intending to walk the next 2 days.

    Andy put in a heroic effort just to be ready for the start.  From the previous day, all of us thought there was a real danger he may have to pull out for at least day 2, but somehow he managed to be ready and start walking with us again.

    The weather was decent enough for walking in as much as it wasn't sunny and the temperature was probably mid teens.  However, we basically spent the entire day walking through cloud.  The most we saw all day was dew drops forming in our hair from all the moisture in the air.  The dampness did make it a little tougher but we found the biggest drawback of this weather was the lack of reward for climbing some enormous hills.  For example, we walked the length of the south rim of Devil's Dyke and none of us have the foggiest (pun intended) idea what it looks like.  Visibility was between 20 and 100 yards all day!

    At about 18 miles Pete wasn't able to continue, having aggravated an old knee injury.  We decided it was best for him to leave the trail and get picked up early so he headed to a nearby road for rescue.  At that point, Andy took the tactical decision to accompany Pete on the journey home to give himself more time and a better chance of recovery to finish the route, leaving just Matt and I.  This was definitely one of the low points for all of us in the team that day.

    Matt and I soldiered on through the cloud for another 11-12 miles after that.  We were due to be picked up at the agreed point but due to a traffic accident and resulting traffic chaos, Tim was unable to meet us until 40 minutes or so after our agreed meet time.  Matt and I decided to add some more miles to make day 3 a bit shorter and easier.


    Day 3 - Amberley to Queen Elizabeth Country Park

    Vital statistics: 25.2 miles in 8 hours 3 minutes moving time (3.13 MPH, moving average), about 54,000 steps and 3287 feet of elevation gain.


    Day 3 was a huge mental struggle for me.  We had lost Pete the day before, Andy was with us again and struggling through.  But the main problem was having eaten fish and chips for dinner the night before and not slept a wink resulting from the indigestion that followed, I was both extremely tired and feeling rather nauseous as well.  I kept going, putting faith in my physical ability (physically I felt fine), force fed myself as many calories as I could (I really didn't want to eat) and by around 2pm was starting to feel a little better.

    There were a huge number of highlights on Day 3 that really helped:

    • We were joined in a fairly last minute addition to the plan by Stephen Warwick.  It was lovely to have someone new in the team with fresh legs and a different dynamic.  Stephen's used to lengthy exercise having walked some trails in the Himalayas and also ridden from Lands End to John O'Groats.  He really helped me get my head in the right place to soldier on with some good team talks and deliberately poor maths.  "We're half way" he said at 11 miles.  "No we're not, go back and re-sit your GCSE" I replied knowing full well that we'd be the wrong side of 25 miles by the time we'd finished the day.
    • The weather was much, much better.  We could see.  It was warm, not hot.  That was lovely.
    • We'd previously walked from Buriton to Didling during training, about the last third of Day 3.  So there was a point when we reached the part of the trail we knew.  That was a huge moment for all of us.  We stopped there and celebrated with a banana each (we know how to live).  Matt and I had now walked to the point where we knew we'd walked the entire length of the remaining trail into Winchester during training and most of that done in a day.  We started to believe this might actually be possible.
    • Amy, Matt's sister, joined us at Harting Down and walked the last 8 miles or so with us.  That was another huge boost to us - thank you, Amy!
    • Linda also joined us for the last 2 miles into Queen Elizabeth Country Park.  Since she's suffering with Parkinson's, I thought that it was particularly poignant for her to join us.  I'm really glad she did. 
    • Donations were still coming in, we'd had a particularly good donation day on Day 2, and for those people who decided to wait until we started walking - many thanks - you helped keep us going and our spirits up.
    The last few miles were really tough for Andy.  Carrying his injury, he hobbled along to the finish, doubtful about whether he could complete Day 4.  Testament to Andy though, he sat down in a disabled parking space in the car park, causing much hilarity.  Other than in quite a lot of pain, I'm not sure exactly how he was feeling, but I imagine very proud of getting this far but also worried about the potential disappointment of not being able to finish the whole thing on Day 4.


    Day 4 - Queen Elizabeth Country Park to Winchester

    Vital statistics: 24.1 miles in 7 hours 12 minutes moving time (3.35 MPH, moving average), about 51,000 steps and 1972 feet of elevation gain.


    The start of day 4 was met with mixed feelings.  We had all stayed at home the previous night, our first night home for the previous 3 nights.  Seeing family and sleeping in your own bed was great.  However, Andy was unable to recover from the previous day in time and we had a message saying that he wasn't going to be able to join us for the final day.  That must have been a very hard decision to take but you'd never know it, having been delivered with the sense of humour and good grace Andy had shown throughout the walk.  We'd miss him but we did have Phil joining me and Matt with his fresh legs for the final day.

    Matt and I were buoyed by the fact we'd walked the entire length of this day during training and done so in a single day.  That feeling of confidence and the fact we knew where we were going and exactly what lay ahead at each stage of the walk certainly helped.  It made things a lot easier when we were getting tired during the day.  It's probably due to a mixture of factors but we walked a little faster on day 4 too.

    Andy parked in Winchester and walked backwards along our route to meet us and walk into town the last few miles.  Still struggling, that must have been a hard walk for him too but it was great to cross the line with the main team of 3 of us and Phil who joined for the final day.

    We arrived in Winchester a bit earlier than any of us might have expected.  But with due warning, friends and family were dotted throughout the last couple of miles to meet and greet us.  That felt amazing, picking up more people as we got closer to the end point.  We were clearly going to make it, feeling extremely tired, but unstoppable at the same time.

    We went for food and drink.  Left the building to be driven, a long way home but a shorter distance than we'd walked each day, that felt weird.

    We're done.  This is now history.  But please consider a donation, if you haven't already.


    Matt's Original Text

    "Thank you so much for visiting my page. My name is Matt and this year I will be turning the grand age of 40. I was inspired to take on a challenge to mark the occasion and hopefully do some good while I'm at it. 
    On 31st May I will be walking the South Downs Way. This is a walk that is 100 miles long and I will be attempting to cover it in 4 days. This means covering something close to a marathon each day. To help me do this I have put together a team of close friends. They will be joining me for either all (Graham & Andy) or part (Phil and Pete) of the walk. 
    Together we are raising money for The National Eczema Society and Parkinsons UK. Here's why... 
    I was diagnosed with atopic eczema aged just 3 months and I still struggle with the disease to this day. With the help of my parents, my family, close friends, some incredible doctors and some equally astounding nurses, my skin is in a far better condition that it ever was as a child. In July of 2017 my wife and I welcomed our second child into the world. Sadly he too has been suffering with eczema. 
    The work that the National Eczema Society do is vitally important in the development of new treatments, drugs and support for sufferers of this condition. I am passionate to find something that not only helps my life but helps the generations to come (including my son). 
    In September 2017 my Mum was diagnosed with Parkinson's. Prior to the diagnosis as a family we had seen her deteriorating. She was walking much slower than normal, she dragged her feet, her right arm developed a tremor, her speech was much quieter and she wasn't herself. 
    My sister got married in early September and while helping to setup the day before Mum tripped on a kerb and broke her arm. Having recognised she wasn’t herself, Mum had been having various tests. A cardiologist she’d been sent to contacted the GP to say he thought Mum should be referred to a neurologist because he thought she had Parkinson’s disease. It is very likely that not picking her feet up (due to Parkinson's) caused her to trip and fall. 
    Since then she has been given a medication, seen a Parkinsons nurse and a physiotherapist. This has meant a significant improvement and she is now standing much taller (quite a significant given how 'tall' she is), she is speaking more freely, she is interacting with all her grandchildren with a vibrancy that was missing. 
    The work that Parkinsons UK do is vital for people with this disease. They provide invaluable support and research that could not be possible without the support of donors. 
    I would be so grateful if you would like to sponsor my team and I as we embark on this challenge and help raise some money for some fabulous charities. 
    Don’t forget to gift aid your donation so we can get even more from the Taxman."

    Last, but very much not least, a great debt of thanks to everyone who has supported us.  Those of you donating your hard-earned cash to our two fantastic causes, thank you.  To our wives and families who we've abandoned to fulfil our own personal challenge, you're tired too from holding the fort while we've been gone, thank you.  Finally, particular thanks to Linda and Tim for the logistical support, you were a part of the team as well and we wouldn't have completed the journey without you.

    Thursday, 2 November 2017

    Kids Sponsored Walk with Dads

    On Saturday 11th November, my 3 year old is taking on a 3 mile sponsored walk.  We're going from Bushy Leaze woods in the next village of Beech to the Bushy Leaze children's centre in Alton.  Basically, a walk across the town in which we live.  It doesn't sound far but it's a long way for little legs with some steep climbs over rough terrain too.

    The idea is to raise funds for the centre and in particular to continue running the dad's group.  The group used to get government funding through the sure start scheme but all this funding was cut last year along with a lot of other funding the centre was able to get.  We're going to our bit to make sure the group can continue to run for future generations.  One of our friends has helped set the centre up as a charity so they can continue as much of their outreach work and things like the breast feeding clinics can continue to run too.

    You can read a little more and sponsor us at the following web page https://mydonate.bt.com/events/walkwithdad/450291

    Please feel free to share the story and donation link as you see fit.

    Thanks!

    Wednesday, 26 July 2017

    Moving to giffgaff

    Much like moving bank account, it's not very often I move mobile network.  I tend to favour staying with the same provider unless they provide me with a good reason to leave.  My current network, TalkMobile, have just done so by completely shutting down their PAYG service and so are only able to offer me a contract (that I don't want) or a rubbish deal to move to Vodafone (they're a Vodafone MVNO).

    I thought I'd write a similar post about my experience moving to the giffgaff network today to the one I wrote about Moving to TalkMobile back in 2012.  You'll see from that link that joining TalkMobile was extremely painful.  However, I'm happy to report I found the network very solid and reliable with what I can only presume is good coverage (there's always black spots, right?) on the Vodafone network.  Getting them to do anything was always painful as there's little option for self-service so it's more or less always a call to their support staff.  The staff are very polite and extremely helpful but the processes and presumably the systems they have to use seem somewhat antiquated today.

    So in comparison to my previous blog post about moving to TalkMobile, moving to giffgaff went a bit like this:

    Day 1 (yesterday)

    1. Register a giffgaff SIM a friend gave me and create a new giffgaff account all in one simple guided wizard on the giffgaff web site
    2. Request a PAC from TalkMobile
    3. Go through the number transfer process via another simple wizard on the giffgaff web site
    Day 2 (today)

    1. Observe a short outage in mobile service as my number was transferred just after lunch
    2. Happy customer

    So now I'm looking forward to years of good service from giffgaff until they provide me with a good reason to leave some time in the future.

    By the way, if you want to grab a giffgaff SIM then feel free to register via my referral link as we'll both get a free £5 credit if you do.

    Sunday, 23 April 2017

    New Thinkpad P50

    It's been a while, but true to our 4 year hardware refresh cycle, I've just received my latest laptop - a Lenovo P50.  I've been installing it with Fedora 25 since Friday and configuring and copying data over this weekend ready to swap laptops first thing this week.  I'm looking forward to trying out the new machine although I'm not quite sure why as the specs are barely different from the machine I was given 4 years ago.  It's certainly the best indication yet I've personally experienced of Moore's Law coming to a complete halt as well as many of the other specifications not improving a huge amount either.  The two most noticeable differences are likely to be the more powerful graphics chip and the inclusion of an SSD.  That said, there is twice as much RAM in this machine and I had upgraded my previous machine with an SSD as well so that particular upgrade isn't going to be noticeable for me at least.

    My previous machine was a W530 and the one I had before that was a T61p (with a T41p before that) and so I'm well used to this particular line of Thinkpad laptops.

    Here's the specifications of the machine I've got, as ever there are variants of the P50 so if you have one or are thinking of getting one the specifications could be a little different but will be broadly similar to this:

    • Intel(R) Core(TM) i7-6820HQ CPU @ 2.70GHz (5433.79 bogomips in Linux)
    • 32GB DDR4 2133MHz
    • Samsung MZNLN512 (PM871) 512GB SSD
    • 15.6" 1920 x 1080 IPS (non-touch)
    • 6 Cell Battery
    • Wireless A/C
    • NVIDIA Quadro M1000M 4 GB
    • Front Facing Web Cam, Mini Display Port, HDMI Out, Headphone, 4x USB3, Smart Card Reader, GBit Ethernet, Thunderbold, Fingerprint Reader
    So looking at those and comparing in more detail to what I had before it seems my gut feeling was pretty good.  The CPU benchmarks are more-or-less exactly the same and certainly within tolerances of error as well as other performance increases that will effect the benchmarks such as the memory clock speed.  Here's the comparison between the W530 CPU and the P50 CPU:


    The same can't be said of the GPU benchmarks though so it looks like GPUs are continuing to gain in power even when CPU speed increases have run out of steam:

    The other noticeable difference I hadn't spotted before is the battery size.  That's very apparent when you pick the machine up as it's actually a little bit thinner (probably also due to the lack of DVD/combo drive) as well as not as deep i.e. it doesn't have the big battery sticking out of the back that has been common place on this line of Thinkpad machines over the past decade or so.  I'm guessing (without having done any research on the matter) that this is probably due to improvements in battery technology so I'd think Lenovo have probably moved over to Li-ion or Li-po batteries.

    In terms of running and using the machine, it does seem very nice so far as one might expect.  It's running Fedora 25 very nicely and hasn't caused me any issues at all during setup.  I'm not really expecting any either as most if not all of the hardware seems pretty well support by Linux these days.  I think, in fact, Lenovo even offer to supply this machine pre-installed with Linux if you want.  That said, there looks to be one possible sticking point in terms of hardware support at the moment but this is very minor.  That is, the build-in fingerprint reader doesn't seem to have a driver available on Linux yet.  I did some very brief research into this yesterday and it's not clear why vendor support is lacking for the device at the moment although I did find at least one effort that has gone a fairly long way towards reverse engineering it and starting to write a driver so I would guess within the next year we'll see some sort of support for the fingerprint reader too.

    All in all then it's a good machine even though it's not a huge upgrade over my 4 year old laptop!

    Friday, 23 December 2016

    Becomming a Fedora Package Maintainer

    Those of you that know me well will know I like Linux.  Don't get me wrong, there's a lot wrong with it and the community can be a pretty harsh place where more or less everyone has an opinion they think is the right one.  I don't get a lot of chance to contribute for various reasons but earlier this year I was (finally) accepted to be a package maintainer for the Fedora project.

    Since I like Linux I do tend to use it on the desktop both at home and at work.  I do more or less everything with Linux and Open Source software including photo editing with GIMP.  It bugged me that Fedora ship a library called lensfun which is a library of lens correction data but the only piece of software using it within Fedora was Digikam.  Preferring GIMP I decided to set about packaging and supporting gimp-lensfun for Fedora.

    Becoming a Fedora Package Maintainer is a well documented process as there are joining instructions and package review guidelines for the packaging guidelines.  Whilst well documented, there's a heck of a lot of stuff you have to read through and comply with before you can submit your first package review request.  This is further complicated by not yet having any reputation within the Fedora community and so you have to build this up in other ways before your package will be recognised and one of the senior community members agrees to sponsor you as a package maintainer.

    I eventually raised my first package review request for gimp-lensfun in April 2013.  Over a period of a few weeks people throw rocks at what you've done according to the various guidelines that you either didn't see or forgot about.  This is actually really good and helpful as it ensures quality and consistency between all the new packages being accepted into Fedora.

    What I wasn't prepared for was the length of the wait.  I know this isn't an earth shattering change to Fedora but it took me over 2 years to get noticed enough for my package to be accepted and for me to be sponsored as a Fedora packager.  I think I'd probably still be waiting now but I bugged a few people on IRC to point out my review request had been sitting there for so long.

    Finally! My first package was released into the wild for Fedora 21 and 22 and I've been given the chance to give just a little back to the community.