MYTS - West Side Story

MYTS will soon be opening with West Side Story, at the Gatehouse Theatre in Stafford, for those who are a fan of the show. I'll be playing keys for this, and it will be the biggest show MYTS has ever done. It boasts an orchestra (that's right - not just a band) of 18 players, and a sensational cast. For me it's been a slight change of emphasis back to actually playing the piano rather than programming. :-) If you have a free night, and are in the area MYTS never fail to disappoint.

RPI as secure web proxy

I, like most other people in the modern age, spend a lot of time on the Internet. This time is probably, in order of usage, at work, at home, at other locations like coffee shops or other public WiFi. Like most people my own network is quite secure, but what about everywhere else?

The work WiFi is internal to the office, but how secure is that? To get on the WiFi you just need the password - there could be all kinds of devices connected. And who's to say that those devices don't have malware on them? Clearly I would like my connections to websites and so on to be protected.

Think about staying in a hotel, using the hotel guest WiFi. I'm now sharing a connection with strangers in other rooms, with God knows what intent.

The answer for me was tunneling.

This is the idea - you open a secure connection to a service running on a network you trust - in my case it's my home broadband, but could be a server in AWS if something - it's easier if it's Linux based, but essentially you can do it on anything. I have a Raspberry PI on the network at home, so that's what I went with.

You then use the ssh protocol to "tunnel" all traffic over a secured connection, via your trusted endpoint, out to the internet. The ssh client allows a proxy mode called SOCKS, which allows all network connections to be made by the proxy. So, when I hit facebook.com in my browser what's actually happening is the browser is asking the designated SOCKS proxy to establish that connection, and route traffic back. I've used the ssh command to set up that SOCKS proxy via my home network, so now all traffic is coming from the internet to my home network, and is then encrypted and tunnelled to my machine.

This works for all HTTP and HTTPS traffic, is very easy to set up, and I do it all the time now. I even have a script which sets it up on my mac for me.

The different pieces you need to configure are the PI itself, and the machine you're using. Let's start with the PI:

My home modem allows me to forward traffic from the internet to a specific host, so I have configured this to hit my Raspberry PI, and I bought a domain name for the job. I'm not telling you here what it is - sorry about that, but you know, this is a public blog... 

In the interests of not reinventing the wheel ensure your PI is good to go as an sshd server using the guide here. Note the section on generating keys - this is a really good idea as it negates the need to enter a password when setting up the tunnel.

Once you have it working you *should* be able to ssh to your PI from outside your network, if you've configured your modem correctly. If you have purchased and configured a domain for this, or used one of the dynamic IP services then this will work a treat.

If you can't get this bit working don't carry on, as this next part relies on the fact you can actually make a connection.

In order to create the SOCKS proxy tunnel you enter a command similar to this:

ssh -v -N -D8080 -o ServerAliveInterval=3 pi@yourdomainhere.com

Note the yourdomainhere.com - if you've not set up a domain you'll need the internet IP address of your modem here.

In the call here -v prints verbose information so you can see what's creating connections on the tunnel - leave it out for a quieter life. The -N stops the ssh default behaviour of executing a remote command - typically a shell, and the -D8080 is the magic which creates the tunnel. More on this in a second. The -o ServerAliveInterval=3 is a further optional parameter makes the client send a null packet to the server every 3 seconds, to keep the connection alive. Many ssh daemons kick off connections with no activity after some time, so this just stops that happening.

Now - more on that -D8080. This sets up a Dynamic proxy on port 8080. A dynamic proxy makes new connections as created on the remote host to service the requests on our local machine. SSH also allows the use of specific static routes, where a specific port on the client is routed to a specific port on the server, but we're not using that here.

I actually wrapped the above line into a script as shown here:

#! /bin/bash

scselect "Proxied"

sleep 5;

while [ 1 ]; do

  ssh -v -N -D8080 -o ServerAliveInterval=3 pi@yourdomainhere.com

  echo ssh exited... relaunching...

  sleep 5

done

This script uses the scselect command to automatically switch the Location on my mac to a location called Proxied. I have set this up as shown here in the screenshot below. Note that on a Windows machine I don't know how you'd do this in a system-wide way, but on a Mac this setting is honoured by all browsers in one hit.

The script also reconnects if the connection drops, after a 5s delay.

You can see that I have set up a SOCKS proxy on localhost, on port 8080, which matches the port in our -D parameter to ssh. If you need to use a different port that's fine - just make sure the port you put in your proxy settings match the port in your -D line.

Again, if you're doing this in your browser directly (in Windows, say) you need to find the SOCKS setting and change it in this way, and it should work just the same.

I have this script in a bin folder I can access by running terminal, and then just running ssh-tunnel. It takes over that terminal tab, which I like because I can see what's going on, and to exit it just kill the tab or CTRL-C the script. Easy.

I would strongly recommend if you're going to do this that you also take a look at your SSHD options  on the PI and remove password authentication altogether. I would also strongly recommend that you install fail2ban using this guide here. Fail2ban essentially monitors your access log file and automatically IP blocks failed login attempts. You'll likely never have any, so this means someone is trying to get into your system.

I would also do some googling on securing your PI and either set your modem to only forward port 22, or else bolt your PI down to prevent unauthorised access.

And finally....

Once you have an SSHD server on the internet you can access any of the machines on your internal connection. For instance, check out this bit of script which gives me a VNC client on my iMac INSIDE my home network. This uses the -L parameter to create a specific tunnel (rather than a dynamic one) from port 5900 locally to 5900 on 192.168.0.11. Now, what's this? 192.168 addresses are internal to my home network? That's right - this is in the context of the remote network. You can see the familiar pi@yourdomainhere.com to actually make the remote connection.

The last line is a Mac command to open a connection, but again, having made the connection you just need to open your VNC client and connect to localhost:5900 - like the dynamic proxy shown earlier you are making a LOCAL connection which is tunnelled for you.

Easy, huh! Now, go and be secure. :-)

#! /bin/bash

echo Connecting...

ssh -f -v -N -L5900:192.168.0.11:5900 -o ServerAliveInterval=3 pi@yourdomainhere.com

sleep 2; # Allow connection setup time

open vnc://localhost

Beware the Hero culture...

On paper at least a modern software project (assuming you're in the 80% of companies using Agile software development practises) follow an Agile approach, and should, in theory, have continuous integration, automated unit testing, atomic version control and release management all kinda sorted. If you're in the 20% it's possible you still are delivering great quality software but I'm willing to bet that you have more of a Hero culture.

What do I mean by Hero culture? I'll tell you. You have too many people who get off on saving the day and staying up for three days straight fixing problems rather than getting off on putting in place a rigorous process of software testing and release to ensure that go-live problems don't happen.

Too many organisations I have worked in have this problem - and what's worse is they don't recognise it.

I have been involved in two major projects recently where in both cases the bottles of bubbly were opened for the guys that stayed up all night, while the guys (like me, so no bitterness or anything) who were bleating on about the importance of repeatable unit testing, CI and so on were largely ignored, and at the end were fairly held up as cranks. That's not to say I'm not respected by my colleagues - I am - but when it comes to delivery they have a way that works for them, and what's worse is they can't see how dangerous the hero culture is.

Now, sometimes you need a few clever guys to get you through the gate, but just about every time I see extended periods of hacking in production by these very talented and expensive guys I can see how it could have been avoided. So why the hell does it keep happening?

In most cases this requirement comes about because the project has gone of the rails through poor project management, poor stakeholder management, or poor quality of deliverable. In all cases these can be trapped early and so there's no reason to be busting a gut doing 80 hour weeks when the application goes live.

To prevent that happening you absolutely must - I say again - MUST have a lean development environment, with a high degree of testing at each level, a high degree of automation in deployment, and a mechanism to track changes and issues with the application. This stuff isn't new so why are we (and many others) so bad at it?

It could be that because Sprint 0 slips we don't bother doing something that is essential so we can stay on track, but this is a falacy of the worst order. If you don't get the foundations right you need to crop later stories, not elements of the process essential for successful delivery. That may mean revisiting the business case. If you ignore the essentials you blow the business case to hell when the flakey thing goes live anyway. And probably more, because you have to release many more times with the ineffecient release process you have, because you didn't get the basics right (or something - could be VCS issues, release management, poor automated testing, or a mix - you get the idea).

The development burndown should be flat, not peaky. A release should be a BAU operation and the team goes home for the weekend afterwards without worrying about it. This is an ideal, I know, but you can get damned close. On the last agile project I ran in about one in three sprints we missed our pub lunch on a Friday. THAT'S IT!! No overnighters, no 80-hour weeks, and the quality of what we were pushing out was far in excess of anything I've seen on the projects in the two years since (working for a non Agile organisation with a serious Hero culture ethos).

I'm not having a go at the men and women that get the application out - the Heros - I'm really not. They're some of the brightest minds I've worked for. The company itself is rewarding the wrong people for the wrong things. If the board / senior management create a culture where Heroism is what's needed to get applications out then they should get called out on it, and something should fail. This may result in the city asking what the bloody hell is going on, and things will change, but too many companies don't see this as the problem and will just carry on regardless.

It's frustrating, not leastly because it's preventable, but also because it leaves people like me - people who can see a better way - looking for somewhere else to exercise these ideas. Somewhere they may actually get it!

A must-have gadget... not my normal type of post...

Every now and then a toy comes along that I just think fits the bill. It genuinely solves a problem (without inventing the market in the first place - thanks Apple for making me crave an iPad!). Examples might include Apple TV, or the Raspberry PI, or even just something mundane like a car stereo head unit that automatically sorts the EQ using test tones...

I took delivery today of another such toy.

Like just about all of you I have probably built a collection of AV equipment in my living room based on quality. I have an Arcam biwired stereo and Rotel CD that SOUND good, a Samsung TV that LOOKS good, an Apple TV which enables streaming of content from our main machine (and Netflix). The problem has always been that while these are all connected fine there's a combination of remotes, and honestly, everyone from the babysitter to my mother all need reminding how to use any of it.

Not any more - introducing the Logitech Harmony 350 - the baby of the range, but good enough for me, and a lot cheaper than the bigger remotes. It's a general purpose remote.

Now - I know what you're thinking - you've seen these before? Endlessly typing codes in from the back of some Japanese or Taiwanese translated pamphlet in the hope that just some of the functionality you need will work? Not any more - this thing has a USB interface and some software so you give it the model numbers of your kit and it configures itself.

Better than that you can see if your kit is compatible before you buy one.

Finally it has shortcut buttons for turning everyone on and setting up the correct input and so on, all in one go.

I have put the four separate remotes in a drawer now, and we have just the one remote. I know it's silly, but this means you can have good quality separate pieces of equipment and still one remote. At around £40 for the 35 it's not cheap as chips, but it works so well. It even controls the volume on the Arcam stereo!

£50 iBeacon PoC - "Welcome Home, Roger"

So, this is a trivial use of iBeacon, but I have it sussed. Using a Raspberry PI, a £12 bluetooth dongle and a ludicrously simple App for my iPhone my phone now welcomes me when I get home.

I used a few different guides for this, which I'll reference here. I didn't invent anything here, but found I had to use information from one or two guides, so I'll link them here.

Firstly - the dongle I used was one of these.

Then you need to setup the software on the Pi to drive it, for which I would recommend this guide. I would say though that this guide fell foul of actually getting the thing going. It was good for getting the software installed though. My particular combination of dongle and UUID or whatever just didn't quite work, so I then used this guide which is where I realised that I had some extraneous zeros on the end of the hciconfig command.

One *really* useful thing is to have two ssh sessions going, and have hcidump running in one of the windows. It gives you output like this:

< HCI Command: LE Set Advertise Enable (0x08|0x000a) plen 1

> HCI Event: Command Complete (0x0e) plen 4

    LE Set Advertise Enable (0x08|0x000a) ncmd 1

    status 0x0c

    Error: Command Disallowed

You can see from my sample output here that it showed up a problem - when you issue the UUID and so on you should get something like:

< HCI Command: LE Set Advertising Data (0x08|0x0008) plen 32

> HCI Event: Command Complete (0x0e) plen 4

    LE Set Advertising Data (0x08|0x0008) ncmd 1

    status 0x00

The non-zero status is a bad thing!

The final part is to set up init.d scripts to automatically start the Pi broadcasting when you reboot the Pi, and this can also be found in the second guide - very handy.

I would advise doing a sudo apt-get update and probably upgrading the Pi firmware to the latest version using rpi-update (note that this caused my Pi to go into single user mode, so have that keyboard and HDMI connection handy if you do this).

The last part was to write the app, and that's still in progress, but to test the connection I would recommend the free app by Radius Networks. There's another app referenced in the docs, but it's not free anymore.

If you want to dig Xcode out and start cutting code then that's pretty easy too, and when my little welcome app isn't so hacky I'll share the code.

Note that if you do use the UUID in the second guide above there is already a profile which will detect this in the Radius App, called Apple Locate, so it's the quickest way of checking your BLE is working properly.

Have fun!

Like a kid again - Elite: Dangerous is nearly here

When I was a nipper a game consumed my life like no other addiction had up to that point (Everquest did rather take over my life about 15 years ago). My dad came home with a copy of Elite for the Acorn Electron we had at the time, got me started on it, and awakened a geek in me that never really got back in the closet.

Elite was originally created by Dave Braben, and, with some imagination, was bloody fantastic. I played with the lights off late into the night - first on the Electron, then the Spectrum and finally my PC (actually, I have been reliving my youth with Oolite - www.oolite.org - on my Mac...).

The basic premise of the first game was a MASSIVE galaxy you could explore, flying between systems trading, bounty hunting, trying not to get killed by pirates, or being a pirate.

Here's thing thing though - the game originally was based on black and white vector graphics with no textures at all. Graphically it was good at the time, but a bit sparse, but the controls worked well, the game was reliable, and actually when played late at night at sleepovers with friends put everyone in an imaginary world flying between systems, taking notes of stock prices for the next trade...

Super.

At the time my friends and I dreamed what it would be like if you could do missions, team up with other players, or buy other ships. And now we are nearly there. Braben got a kickstarter project (https://www.kickstarter.com/projects/1461411552/elite-dangerous) to raise over £1.5m and build Elite: Dangerous. And my friends it's everything my 12 year old inner self is dying to get hold of.

The premise is the same but now it's truly massive (the whole Milky Way is mapped - I know!!) and is massively multiplayer too. You can have different ships, and take on missions. Group with friends and do joint missions together. 

Basically it's World of Warcraft meets Elite and I can't wait!! I've already warned my life she will lose me for about 6 months of my life when this game comes out and for the first time EVER I shall be installed bootcamp on my laptop so I can play it on Windows when it comes out rather than wait for the Mac version (which will follow - thanks boys!)

If you want to see more head on over to elite.frontier.co.uk.

Nuclear safety - why do we get all bent out of shape about it?

You may have seen in the media that the Office for Nuclear Regulation is considering the safety limit for the degradation of graphite bricks which protect the nuclear core of a power station. The proposal from EDF is to raise the limit from 6.2% to 8%.

What concerns me is that already anti-nuclear lobbyists are jumping on this as the government putting power generation ahead of public safety, but this is simply not the case. The ONR has a good track record of imposing good safety measures based on actual science (you know - that thing that doesn’t give a shit about public opinion, it just is) so I would urge anyone thinking of getting up in arms about this to at least wait for the science to come in.

The ONR has told EDF to commission independent scientific consultation, as it is believed that 6.2% was extremely conservative. If this is true then raising the limit is just common sense - and it's not putting anyone at risk.

The thing is - if we rush too quickly to condemn this raise based on our misguided opinion of nuclear safety we will shut down our power stations 10 years earlier than we need to, and that would just be bad for power. We have a real power generation problem looming, so turning off our stations needlessly would be a bad thing. We are looking at rolling blackouts towards the end of this decade, and I love my internet connection way too much to not get vocal when I see that coming!

Also, a quick reminder, in terms of critical illness and deaths per megawatt nuclear power is just about the safest form of power generation - including solar (making the panels is a very toxic business and people die in their manufacture).

And now I leave you to your weekend… Enjoy the sun.

Parsing Roman Numerals

I had reason to write a Roman Numeral parser - the spec was simple - pass in the number as a Roman Numeral, which I take as a String, for instance, "MCMLXXXIV", and return the decimal equivalent.

It took me a few minutes to work out a neat way of doing that, so I thought I'd share in case anyone else needs it. It doesn't check the numerals are valid, and comes without warranty, blah, blah, but it works for all my tests. It's in Java, but would be easy to convert to any other language

There's the class Parser which does the work, and the RomanNumeral enum which stores the values for each letter. There's not much error handling either.

Enjoy!

public class Parser {
    public static int parse(String romanNumerals) {
        int[] values = new int[romanNumerals.length()];
        for (int i = 0; i < romanNumerals.length(); i++) {
            values[i] = RomanNumeral.valueOf("" + romanNumerals.charAt(i)).value();
        }
        return parse(values);
    }
    public static int parse(int... values) {
        int total = 0;
        int subtraction = 0;
        for (int idx = 0; idx < values.length - 1; idx++) {
            if (values[idx] < values[idx + 1]) {
                subtraction = values[idx];
            }
            else {
                total += (values[idx] - subtraction);
                subtraction = 0;
            }
        }
        total += (values[values.length - 1] - subtraction);
        return total;
    }
    private Parser() {}
}
public enum RomanNumeral {
    I (1),
    V (5),
    X (10),
    L (50),
    C (100),
    D (500),
    M (1000);
    private int value;
    private RomanNumeral(int value) {
        this.value = value;
    }
    public int value() {
        return value;
    }
}

When is a stereo jack not a stereo jack? Know your cables!

I'm helping a young lad I've really seen grow up with his first MD post - I'm playing second keys - and we had a really interesting chat I thought it worth sharing.

I got to band call, and he was complaining that his piano sounded really tinny and thin - and indeed it did. It sounded fine on headphones, so I started tracing the wiring to the amp.

Here's the funny thing - he had a stereo splitter cable - 1/4" stereo jack to 2x 1/4" mono jacks - left and right. He'd connected the left and right to his Presonus box and the other end to the amp.

Now I know what you're thinking - I bet that's not a stereo input on the amp. You'd be right - it's a balanced input.

This led to a conversation with the young lad...

So, in a balanced TRS jack (which looks just the same as a stereo jack) the ring and tip both carry the mono signal, but crucially they are out of phase. This allows the receiving system to reduce noise more effectively, as the noise will be present and equal on both signals. There are many technical journals about differential balancing if you're interested.

The main point for this blog post is that the amp that Tom connected his keyboard to expected the ring to be a phase inversion of the tip, not a Right channel, and the processing that then took place effectively wiped out the bass entirely from the mix.

Have you ever wired up a car stereo system and got one of the speakers out of phase? I've done that, and it has the same effect. The relatively low frequency bass sound waves cancel each other out. You can get the same problem in large studios that don't have bass traps - the sound waves bounce of a wall and mix with the direct sound waves, but at a different phase, reducing (or building) the bass sound.

So, he unplugged the Right output from his Presonus and all was well. Whether or not he remembers why that worked is another matter. :-)

Incidentally, I've started using balanced line outs on my keyboards and have found a significant reduction in noise and a slight boost in input signal, so I'd recommend it.

Using a Raspberry PI as DNS and DHCP tool

This morning I set up my RPI to run DNS for my network. This means I can resolve internal machines (the PI itself and the iMac downstairs) and make use of internal DNS caching, save going out to Virgin for every DNS lookup.

It's actually trivially easy to do. If you're already using udhcpd you may want to switch, as you can do it all in one config file, but you don't have to.

Firstly apt-get the package:

apt-get install dnsmasq

You then need to edit the following lines in /etc/dnsmasq.conf. Look for the following settings, and change them for your own settings:

server

This should be the address of a good external DNS, so either your ISP one, or I've used Google's DNS'

server=8.8.8.8

server=8.8.4.4

domain

This should be the name of your internal network. As it happens mine is softfox.net. This enables you to be able to resolve addresses on your network, so in my case pi.softfox.net

domain=softfox.net

dhcp-range

If you're going to use dnsmasq for DHCP you need to edit this, but you don't have to. If you are using a different DHCP (for instance, udhcpd) you'll need to update the network DNS settings it hands out to your PI.

dhcp-range=192.168.0.100,192.168.0.199,12h

This takes the form of range start, range end, lease time. There are other options you can do for multiple ranges on different networks and all kinds of things

dhcp-host

I've not used this, but this is how you hand out a specific IP to a MAC address - could be useful. I actually have my fixed things on a fixed IP so didn't need it.

dhcp-option

Important one this - dnsmasq assumes that it's running on the router, but in my case my router is a virgin box on 192.168.0.1, so you have to set this up to point to that box.

dhcp-option=option:router,192.168.0.1

Again there are ton of things you can do here, but I have a fairly simple setup. There are examples of setting up WINS names and so on, but I have an all mac network, so don't worry about such things. You can also set up bootp and tftp for network boots from this - again, I have no need of such things.

cache-size

This sets the number of DNS address dnmasq will cache, and I would advise you to set it to it's maximum setting. It's way quicker to get the address from your own network than go out to your ISP, so I've set mine to the maximum, 10000

cache-size=10000

Stephen Wood has done a really cool post about this with more information, and some scripts to help you profile the improvements here so thanks, Steve for that, and getting me up and running so quickly.