Sunday, 31 August 2014

Node.JS Async thinking...

I've been working on a Node.JS project for the last few months. It uses all the usual suspects, including EJS, and Sequelize (http://sequelizejs.com) ORM, amongst other really useful modules.

One of the main issues you face when coming from a language like Java, as I did, is that most calls you make in Node are asynchronous. There is a single thread which is constantly making calls in the background, to which you provide callback handlers.

This is an easy enough concept to get - it means you get code like:

doMyThing(finished) {
  // do some stuff
  finished();
}
function finished() {
  // called when doMyThing() is all done
}

But this gets compounded, and more and more complicated when, for instance, doing a lot of database calls that depend on each other. Or if you have several sets of processing that need to be done in order. You end up chaining endless callbacks, which makes the code incredibly hard to read.

There is no inherent thread management in Node (that I've discovered) so what do you do if you need control over the fork() and join() calls (to quote C / POSIX terminology).

The answer I found is the async module I found here: https://github.com/caolan/async

I think there are many people using this, and it's fab! It's so fab I'm now using it when I need asynchronous control, but actually I've replaced all my for() loops with async.each - thus:

Consider this simple loop to work on the results of a DB call:

db.sequelize.findAll({where: {criteria: someCriteria}}).success(function(results) {
  // results will be provided from Sequelize as an array of objects populated from the db
  // for-loop method:
  for (var i = 0; i < results.length; i++) {
    // do something with results[i];
  }
  // async.each method:
  async.each(results, function(result, callback) {
    // do something with result
    // now tell the async library we're done with this result
    callback();
  }, function() {
    // this optional function will be called when all the results have been processed
  })
});

Forgive me if this doesn't quite have the right number of brackets and so on - I've not typed it in to check the syntax is 100%.

Note the difference - the for loop method iterates over the array, processing each element. The async.each method asynchronously calls the provided function for each element in the array, potentially calling a final method when all entries have returned. Note that the body of the handler function must contain a call to the provided callback function to tell the async library when this entry has been processed.

There are variations within the library to handle processing these in series (async.series) and a multitude of other useful things, so it has really become, for me, an essential module for programming in Node.

Thursday, 14 August 2014

D in your results? Chill out!

Just seen someone on the news really upset about working really hard and only getting a D. It's a bell curve. Get used to it! A D in this years results still makes you smarter than your parents on average, and smarter than their parents. People are getting three IQ points more intelligent each decade so don't get disheartened by it.  


We have to keep moving the goal posts to keep us getting smarter. The problem is that society tells everyone they can get an A if they work hard and it's simply not true. For the really smart people to get an A that means anything people like me have to get a D. And I did. That's just maths. Was I bummed? Yeah, but I adjusted my expectation and really focused on what I was good at and enjoyed. Am

I a world class pianist or software engineer? Nope! Am I ok with that? Yep (mostly) but I still work hard at both and am pretty damn happy where I've ended up with both. 


Conversely I DID get an A at GCSE English but I can't write for shit, which I really wish I could do. That's just how it goes. 


I hope all you results chasers get what you want, but more than that I hope that whatever happens you can look back in 10 or 20 years on the road you are setting out on and be happy where you ended up. 

Monday, 11 August 2014

3 useful tunnelling scripts for mac - SSH, VNC and AFP

I just thought I'd share these three scripts. I have previously blogged about setting up a tunnel, but I've had some people asking for these scripts specifically. The first sets up a permanent tunnel and changes location to one called Proxied - so you'll need to set this up. See my earlier post for details on this.

#! /bin/bash

handleSigint()
{
  echo SIGINT detected - reverting Location...
  scselect "Automatic"
  exit 0;
}

trap handleSigint SIGINT

scselect "Proxied"

sleep 5;

while [ 1 ]; do
  ssh -v -N -D8080 -o ServerAliveInterval=3 user@mydomain.com
  echo ssh exited... relaunching...
  sleep 5

done

Note that you'll need to change user@mydomain.com.

This script improves slightly on the original, as it catches CTRL-C and sets the location back to "Automatic".

Here's the next script - fires up screen sharing / VNC on a machine on my home network:

#! /bin/bash

echo Connecting...
ssh -f -v -N -L5900:192.168.0.11:5900 \
  -o ServerAliveInterval=3 user@mydomain.com
sleep 2; # Allow connection setup time

open vnc://localhost

Note that the 192.168.0.11 should be the machine on your network you want to connect to - note the use of the private network 192.168 here. This is the IP address my iMac at home as been assigned. Don't forget to change user@mydomain.com again.

Finally, here's a new one - this allows you to 'mount' drives from a machine on your home network. It's very similar to the VNC script, but binds the AFP port, not the VNC one:

#! /bin/bash

echo Connecting...
ssh -f -v -N -L15548:192.168.0.11:548 \
  -o ServerAliveInterval=3 user@mydomain.com
sleep 2; # Allow connection setup time

open afp://localhost:15548/

I've used a non standard port for this one to avoid conflicting with any local services you may have on this port.

You can share any service in exactly this same way - as long as you know the port number.

Enjoy!

Thursday, 7 August 2014

Theatre Ingestre present "Fiddler on the Roof"



Next week will see me playing accordion for Fiddler on the Roof, at Theatre Ingestre. An excellent cast lineup directed by Tim Downes, under the baton of Calum Robarts it should be a good one!

Ingestre Stables,  Stafford, opens 13th August.

Theatre1 is launched!



I am proud to announce the launch of Theatre1 - a new Theatre company, targeting excellence, openness and professionalism for young theatre. They are auditioning for Songs For a New World. You can sign up to audition and download the audition pack at http://theatre1.co.uk. Follow them  on twitter @t1stafford, or Facebook.com/T1Stafford.