Social Equality vs. Fairness

I had a really interesting conversation over lunch today, which extended over Facebook into a wider discussion with other friends on the merits of equality vs. fairness.

This was sparked by the news that Lloyds TSB are to offer bank accounts which obey Shariah Law, and as part of that pay no interest, but interestingly, charge no overdraft interest either. This led me to kick off a discussion around how this was unfair on non-muslims. In general, as an atheist, I see many things in society that are deemed acceptable because they have a religious basis, which are ridiculous if you actually treat all religion with the same disdain. Why, for instance, do religious bodies get tax benefits and CoE ministers are paid by the tax payer to attend the beds of the terminally ill? This is surely not fair on the rest of us?

Now it turns out these bank accounts are actually available to anyone, so in theory, anyone could move a credit card balance to an agreed overdraft facility at Lloyds and get it interest fee. So, despite the pandering to the Islamic community to get more business (too cynical?) it's actually not unfair in that everyone is treated equally in that case, but let's stick with the equality-vs-fairness theme a while longer.

It got me thinking a little about the nature of fairness and equality in society, and I wondered if maybe fairness and equality are mutually exclusive. Let me try to explain my reasoning.

I suggest that fairness cannot be measured quantitively, and as such is a purely relative and subjective measure. Equality, however, CAN be measured. Let's look at an example to try to illustrate my point. I pay a higher rate of income tax because my income is in the high rate tax payers bracket (like nearly every other white collar IT consultant I'd say). Is this fair? Not on me it isn't! A percentage based system ensures the more you earn the more you pay, but arbitrarily I am on a higher percentage. That doesn't sound fair at all. However, society has deemed a certain set of social services which must be paid for, and that only works on a raked income tax system. If I agree with paying for schools, hospitals, care for the elderly, etc. that's just the way it is. We all have equal access to these services - the rules are largely applied equally to every member of our society, but it's hardly fair. I'm paying more for the same set of services.

Let's try to make this mathematical sounding to see if rules of equality and fairness can be made simple - let us talk about applying rule A to bodies X and Y. If rule A is applied to both, irrespective of any properties of X and Y this might suggest that both bodies have been treated equally. Rule A is applied blindly without any cause to refer to the bodies to which the rule is applied - a bit like access to the NHS. The NHS is free at the point of use, and all of us can rock up at A&E if we need to. Easy. If, however, the properties of X or Y are considered in the decision as to whether or not to apply rule A this would suggest that they are not treated equally in order to attempt to treat them fairly. So you get rules which target certain individuals which are fair, but by definition not promoting equality.

Let us make this a bit more real - let us say that body X is a man, and body Y is a woman. Now let us say that childcare benefit is provided only to body Y on the basis that she is a woman - this may seem fair (and indeed is largely how our society is still geared up) but it's hardly equal.

So society decides some abstract, empirical, historical, political and perhaps religious rules for defining the properties which define different types of bodies, and then these properties are factored into the rules, making any potential for an "equal" society surely impossible. Given that we can also not be fair to everyone it feels like in our attempt to be "fair" we are actually perpetuating social inequality in the truest sense of the word.

Would it be better to only apply rules which can be applied equally to everyone, and discard the rest? Most of our laws are like this - religious people who feel compelled to wear certain clothing still have to wear crash helmets on motorbikes, for instance. Equally, rules around trade and industry do not take account of Jewish dogma regarding the Sabbath - there are many examples like this. Those people may choose not to do certain things on certain days in the week, but there's no law (nor should there ever be a law) that says I can't buy stuff and work on a Saturday.

One problem with this whole post is that back to my first example, because, actually, anyone can apply for an Islam account with Lloyds TSB the rules are applied irrespective of any religion "property" the applicant may have, and so it is an equal rule, but in this case that also sounds fair, so it's possible this is a circular argument as in this case it is both equal and fair, which may make the whole thing moot, but I think the end result is that it is not possible to treat everyone equally because we are not equal. Some people need more social support than others, some people need to pay more tax than others, and we need to treat these people unequally in order to be fair.

The only thing society then needs to work out is what is a valid set of properties on which to base the differentiating rules - I would argue that we should restrict this to empirical and scientific based properties only, otherwise we'll end up chairing arguments between religions, psychics, homeopaths and astrologers, and that can only end up in a world of shit!

SQL vs. NoSQL - some initial findings

Let me first say this post is not a scientifically controlled report - it is nothing more than some initial findings I thought it worthwhile sharing when it comes to SQL vs. NoSQL - in this case MongoDB. I know way more about SQL than I do NoSQL, but I am embarking on a project which will involve large amounts of real time data manipulation and processing for retail, and I’ve been having a play with MongoDB. There were some surprises:

1. I believed based on things I had read that a carefully structured relational database, with an optimised schema would be faster answers the questions it was designed for than any NoSQL solution. I’m not entirely sure this is true in the vast majority of cases. I actually wonder if the edge cases where this may be true make it worth using SQL unless you can really, REALLY show that it’s a good idea.

2. I had not really thought too much about the boilerplate that goes with SQL vs. NoSQL, but compare these two bits of code:




and:




These both do the same thing - use a DataGenerator to generate a bunch of test data, which in this case creates a load of first and last names, and emails. In the case of the SQL test (no ORM to be fair) there’s way more code because you have to somehow map the object domain to the data domain. Not so in the second, where you just chuck the object into the db.

This leads me to:

3. There is no “schema” per se. There are just buckets. This feels a bad thing - strong typing is good, right? Well, adding a column to a database that’s been running for a while and may have millions of records is a big deal. In a NoSQL solution you just add the new kind of object. Old objects will simply not have the new field (so one assumes you need some null checking and maybe be careful how you rebuild your Java object akin to serialisation perhaps (?) but the db is ultimately flexible.

4. Last one for now - check out the search code below. Each does the same thing - does a search for all names that start with a capital “A”:




and now the MongoDB version:



See the function chaining in the MongoDB solution? How elegant is that? Rather like Java8 streams or the kind of sequence processing you see in Clojure this is a really elegant way to process the results. Incidentally, it’s a load quicker in MongoDB than it is in Derby DB for any number of records. I’d like to say that this is because it’s a LIKE type query which is just about the most horrible thing you can do in an RDBMS, but actually, EVERY query is quicker in MongoDB. Counting the records, searching for a specific record - they’re all just quicker in my MongoDB version.

I’m sure there will be people thinking I’ve got my RDBMS tables set up badly or whatever, and while there may be optimisations I could do I’ve done nothing to either Derby or Mongo - this is just out of the box.

Interesting stuff - download MongoDB from www.mongodb.org and have a play.

My first bit of Clojure... Hello Pi.

In trying to learn Clojure I have been knocking up various silly little programs to try to understand the language. For a Java head there were various things that were quite a paradigm shift for me. I’m an old C hack so the move (back) to functional wasn’t so hard, but never having done Lisp I struggled with a few things at first.

Postfix notation

Put simply the operator is pushed first, and the operands afterwards. This looks weird at first (compare (2 + 3) with (+ 2 3) for instance, but actually, in clojure every “form” consists of an operator and it’s operands in brackets (str “abc” “def”), or (conj set-a set-b), or (+ 2 3) - make sense now? It does make long mathematical equations odd to look at, as you can see in the code fragment which calculates Pi below.

Looping

Clojure does have loops, but idiomatically good Clojure doesn’t really use them in the way you think it might. My little Pi generator below makes use of loop and recurs, which is quite standard, but it’s not really a for loop. In fact, in Clojure, there isn’t really a for loop - only a for-each loop, for processing sets or sequences.

Lazy Sequences

Not something I’ve used outside of database or file access in Java this is a way of creating a sequence of infinite length. You can define how a sequence is generated, but it is not actually processed until you take numbers from the sequence, with the (take) function, or in some other way force evaluation of members of the sequence. Very neat indeed!

The Syntax

It does make sense when you spend a while working with it, but I’m still a beginner, and sometimes find myself doing something like:

(printf myVar ” is a bit like ” + myOtherVar) 

Of course, I get some odd output as the + is evaluated and it’s .toString representation called. + in Clojure (like the other operators) is actually a Function, so you get an odd looking function reference - much like if you .toString an object in Java.

You do get used to the brackets, and code is broken down into much smaller fragments generally, which is no bad thing, but you do end up with a more fragmented programming style. It really is like going back to my first language in some ways - Logo! Remember that? Control a turtle to draw stuff on paper, and write simple functions which you then nest up in more complicated functions. Super fun!

My little Pi generator

I’m sure there is a ton of stuff wrong with this as I’m really just learning the language, so I welcome comment and guidance, but here it is anyway.

(defn calc-pi-method1
  ([] (calc-pi-method1 100))
  ([max-recurs] (loop [r 1 pi 3.0]
    (if (> r max-recurs)
      pi
      (do
        ;(println “r=” r ” pi=” pi)
        (def div (/ 4 (* (* r 2) (+ (* r 2.0) 1) (+ (* r 2.0) 2))))
        (if (even? r)
          (def newpi (- pi div))
          (def newpi (+ pi div)))
          (recur (inc r) newpi))))
))
; main
(println (calc-pi-method1) ” compared to a Math.PI value of: ” (Math/PI))

Note that the more max-recurs you pass in the closer to Pi you get, but due to limitations in the program somewhere you never quite get to Math.PI. But it’s only to try and learn the language but not be the definitive Pi generator. :-)

Laters.

SFX for How To Succeed

As well as playing the piano pad I’ve also programmed and am triggering the sound effects. This is usually a good idea where SFX appear at very specific points in the music. Now, I take an almost anal pride in this and have carefully researched the appropriate sounds, often recording or creating them myself.

So, why this post?

My telephone ringing sound effect - not the most complicated thing ever - is a perfect 1950’s bell phone ring.

But this is an American show. The ringing cadence is completely different. Also the props are trim phones not bell phones. Gah! So that’s me redoing them tomorrow. Any job worth doing… No one would notice, but I would know!

Big Data Analytics and Security

At the #RSASummit earlier this week there were three keynotes, which along with my current thinking around microservices for retail consumer data, and right action right time analytics presented an interesting orthogonal use case I’d not thought of.

The idea of Big Data Analytics in general is to make use of a sea of data (or a lake, if you prefer) to perform predictive analytics, insight, and generally use clever heuristic or other algorithms to tell you things you couldn’t have found out with a million monkeys and infinite time. Because of my current work with retail this for me has meant interacting with customers in the right way, at the right time, on the right channel.

However, the RSA boys have started preaching an alternative security model to the “Prepare to repel borders” perimeter focused security I certainly am used to. I’m no security expert so this may not be news to people in that space, but I found it really interesting what the alternative view is, and where it leads.

Essentially the security community is moving away from perimeter defence focussed to security - prevention, in other words - to analytics - detection. This is taking some time to work it’s way through the board rooms and across the golf courses of senior boards, but it’s a really important step.

Let us assume, for a start, that security breaches WILL happen. It’s inevitable. What tradiitonally happens then is a massive effort to work out what has happened, how, and how to stop it again - if the breach was even detected in the first place, and this is an important point. It is in most hackers interests to not even let you know they were there, so you leave the door open.

Introduce an analytical model - this anticipates that breaches will happen but automatically flags up when they do based on unusual activity. This means that the actual security event could be shut down way quicker than normal, and reports generated to try to close those back doors down.

Combine this with the quite inspiring take on employee freedom and you have a quite different model to the one I’ve seen in any large company I’ve been in.

In order for an analytical model to work you need the data. Currently the perimeter and blocking hard handed mechanism employed by most IT depts. means employees find workarounds - Dropbox, or SSH tunnelling - external VPNs - whatever they need to use to get their job done. The problem is none of these can be monitored by the organisation. The employees just need the tools to do their job, so introduce SSO on the web proxy, and generally allow employees to use what they want and now you can analyse usage on your network much more effectively.

This may mean you let employees use Facebook, but I put it to you that if you stop them doing this on your machine they’ll do it on their own machine, or on their own 4G dongle, and now you have no control or worse, any view on that behaviour, so if someone DOES use malware to take over control of a corporate PC there’s a lot less a chance of you seeing it.

Out of the closet...

The time has come for me to stop managing two personalities. I have, for the best part of 20 years, tried to keep my love of music, playing music, and musical theatre, separate from my love of computers, computer software and the design of same.

This has proved fruitless, and actually, I have come to realise that each benefits the other. I am one person with these two different sides to my personality, so I have decided to use this blog as the home for both. I shall continue to post my musical engagements, of course, but will also be using it as a place to write my musings on those things in software which interest me.