Jonathan Livingston Seagull

is the title of an excellent book by Richard Bach. It's well worth rereading every year or so. As usual I'm going to quote from a few pages: eat, to stay alive as long as we possibly can.
Jonathan Seagull discovered that boredom and fear and anger are the reasons that a gull's life is so short, and with these gone from his thought, he lived a long life indeed.
We choose our next world through what we learn in this one.
Heaven is not a place, and it is not a time. Heaven is being perfect.
You have less fear of learning than any gull I've seen in ten thousand years.
"Why is it", Jonathan puzzled, "that the hardest thing in the world is to convince a bird that he is free, and that he can prove it for himself if he'd just spend a little time practising? Why should that be so hard?
You don't love hatred and evil, of course. You have to practise and see the real gull, the good in every one of them, and to help them see it in themselves.
Don't believe what your eyes are telling you. All they show is limitation. Look with your understanding, to find out what you already know, and you'll see the way to fly.

Shadow Data Types

This article appeared in the December 2009 issue of the accu Overload magazine.

Shadow Data Types

Suppose we have a type called wibble defined as a concrete data type (that is, a type whose representation is fully visible) as follows:
// wibble.h
#include "grommet.h"
#include "flange.h"

typedef struct
   grommet g;
   flange f;
} wibble;

void wopen(wibble * w, int i);
void wclose(wibble * w);
The definition of wibble exposes the types involved in its representation (grommet and flange in this made up example), and hence requires a #include for those types in its header file. This exposure has a price. One cost is that any change to the grommet or flange header files, or any header files they in turn #include, at any depth, will require a recompilation of any source file that #includes wibble.h (either directly or transitively). Another cost is that client code can easily become reliant on the exposed representation rather than relying solely on the functional api. Note that in C++ you can avoid this problem by declaring your data members private.

Abstract Data Types

These costs are sufficiently high that software developers have invented techniques to hide a type's representation; to turn a type into an Abstract Data Type. An Abstract Data Type is simply a type that does not reveal its representation; a type that abstracts away its representation. In this article I'll look at two abstract data type implementation techniques: Opaque Data Types, and Shadow Data Types.

Opaque Data Types

The term Opaque Data Type is a well established term for the technique of exposing only the name of the type in its header file. This is done with a forward type declaration. This certainly has the affect of not exposing any representation!
// wibble.h
typedef struct wibble wibble;

wibble * wopen(int);
void wclose(wibble *);

Storage class restrictions

A definite downside with this approach is that clients cannot create objects.
#include "wibble.h"

void eg(void)
   wibble * ptr; // ok
   wibble value; // constraint violation
   ptr = malloc(sizeof *ptr); // constraint violation
The wibble type's representation is defined in its source file and so only code in the source file can create wibble objects. Furthermore, these wibble objects have to be returned to users as pointers. These two constraints mean the created objects cannot have auto storage class. This is a great loss since auto storage class alone of the three storage class options allows the clients to decide where the objects live which can greatly improve locality of reference.
// wibble.c
wibble * wopen(int value)
   wibble opened;
   return &opened; // very very bad
A second possibility is to create the objects with static storage class:
// wibble.c
static wibble wstorage[42];
static size_t windex = 0;
wibble * wopen(int value)
   wibble * opened = &wstorage[windex];
   return opened;
The final possibility is to create the objects with allocated storage class. That is, to create the objects dynamically on the heap:
// wibble.c
wibble * wopen(int value)
   wibble * opened = malloc(sizeof *opened);
   return opened;
The static and the allocated approaches have opposing advantages and disadvantages. Static storage is very fast and doesn't fragment the memory but the type has to decide the maximum number of objects the application will need. That might be a dependency going in the wrong direction. Allocated storage is much slower and can create memory fragmentation issues, but the application decides how many objects it needs. In short, the classic ADT technique creates an abstraction that is very opaque and pays a hefty price for this "over-abstraction". Abstracting away the representation also abstracts away the size details of a type and it is the loss of the size information that creates the storage class restrictions. The Shadow Data Type implementation technique attempts to rebalance these forces of abstraction by separating size abstraction from representation abstraction.

Shadow Data Types

The term Shadow Data Type, in contrast to Opaque Data Type, is not a well established term. The technique I'm calling Shadow Data Type has probably been around for a long time, it's just that it doesn't seem to have ever been documented anywhere and so a term for it has never become established. I've chosen the term Shadow Data Type to try and convey the idea that when you shine a light on an object it casts a shadow which reveals something of the size of the object but nothing of the details of the object. In other words, a Shadow Data Type has a "full" type declaration (rather than a forward type declaration) but one revealing only the size of type.
// wibble.h
typedef struct
   unsigned char size_shadow[16];
} wibble;

void wopen(wibble *, int);
void wclose(wibble *);
The "true" definition of the type (together with its accompanying #includes) moves into the source file:
// wibble.c
#include "wibble.h"
#include "grommet.h"
#include "flange.h"
#include <string.h>

typedef struct
   grommet g;
   flange f;
} wibble_rep;

// sizeof(wibble) >= sizeof(wibble_rep)

void wopen(wibble * w, int value)
   wibble_rep rep;
   memcpy(w, &rep, sizeof rep);
However, there are two problems needing attention.

Synchronized Alignment?

Firstly, there is no guarantee the two types (wibble and wibble_rep) are alignment compatible. We can solve this problem. The trick is to create a union containing all the basic types. We don't know which basic types have the strictest alignments but if the union contains them all the union must also have the strictest alignment.
// alignment.h
typedef union
   // one of each of all the basic types go here
   // including data pointers and function pointers
} alignment;
We redefine wibble to be a union with two members; one member to take care of the memory footprint and one member to take care of the alignment:
// wibble.h
#include "alignment.h"

typedef union
   unsigned char size_shadow[16];
   alignment universal;
} wibble;
The main problem with wibble being a union is that unions are rare. Suppose you want to forward declare the wibble type in a header. You're quite likely to forget it's a union.
typedef struct wibble wibble; // Oooops
We can fix this by simply putting the union inside a struct!
// wibble.h
#include "alignment.h"

typedef struct
   unsigned char size[16];
   alignment universal;
} shadow;
} wibble;
This is now sufficiently tricky to warrant an abstraction of its own:
// shadow_type.h

#include "alignment.h"

#define SHADOW_TYPE(name, size) \
typedef struct \
{ \
   union \
   { \
       unsigned char bytes[size]; \
       alignment universal; \
   } shadow; \
} name

// wibble.h
#include "shadow_type.h"

SHADOW_TYPE(wibble, 16);

Synchronized Size?

The second problem is hinted at by the comment in wibble.c
// sizeof(wibble) >= sizeof(wibble_rep)
This comment, like all comments, has no teeth. Ideally we'd like an assurance that if the types' sizes lose synchronization we're told about it. This can be done by asserting the relationship inside a unit test of course. The problem with this the possibility that the runtime check inside a unit-test won't get run. Or, more likely, that the unit-test simply won't get written at all. Fortunately in this case we can check the relationship using a compile time assertion. We start with the fact that you cannot declare an array of negative size:
extern char wont_compile[-1];
extern char will_compile[+1];
Now we have to select a size of either +1 or -1 if the asserted expression is true or false respectively.
// may or may not compile
extern char compile_time_assert[sizeof(wibble) >= sizeof(wibble_rep) ? +1 : -1];
Hiding this mechanism behind a macro inside a dedicated header helps to make the code more Intention Revealing.
// compile_time_assert.h
#define COMPILE_TIME_ASSERT(description, expression) \
extern char description[ (expression) ? 1 : -1 ]
// wibble.c
#include "compile_time_assert.h"
sizeof(wibble) >= sizeof(wibble_rep));
It's worth spending a few moments to think about alignment carefully. The wibble type contains a union to give us the strictest alignment. This means a single wibble_rep and a single wibble can overlay each other in either direction. If we create an array of wibbles the compiler will ensure the address of each wibble is suitably aligned. To do this it may add trailing padding to the wibble type but this padding will be reflected by sizeof(wibble). Similarly, any padding for the wibble_rep type will also be reflected by sizeof(wibble_rep). Importantly, since sizeof(wibble_rep) may be strictly less than sizeof(wibble) we cannot overlay an array of either type directly onto an array of the other type. However, we are only concerned with creating an array of wibbles since that is the type the client uses. There should never be any need to create an array of wibble_reps. Nevertheless, the .c file implementation must always do any array pointer arithmetic in terms of wibbles and never in terms of wibble_reps. Note also that using >= rather than == also allows binary compatibility with any alternative smaller representation.

Casting the shadow

Inside the source file we can create a helper function to overlay the true representation onto the clients memory (the fragment below uses the dot designator syntax introduced in c99):
// wibble.c
static inline void shadow(wibble * dst, wibble_rep * src)
   memcpy(dst, src, sizeof *src);

bool wopen(wibble * w, const char * name)
   wibble_rep rep =
       .g = ...,
       .f = ...,
   shadow(w, &rep);
Careful use of memcpy can help to make the wibble functions behave atomically from the users perspective. That is to say, the function can do the work off to the side in a local wibble_rep, and copy back into the shadow only if everything is successful. An alternative to memcpy is to cast the pointer on each access:
// wibble.c
static inline wibble_rep * light(wibble * w)
   return (wibble_rep *)w;

void wclose(wibble * w)
   wibble_rep * rep = light(w);
   rep->g = ...;
   rep->f = ...;


It makes no sense to declare a wibble object with a const modifier unless the object can be initialized.

void pointless(void)
   const wibble w; // :-(
   // ... ?
However, this is not an issue since the wibble type is opaque anyway. Nevertheless, a slight redesign can accommodate const wibble objects if desired, at the cost of copying struct objects:
wibble wopen(int value)
   wibble_rep rep = { ...value... };
   wibble w;
   memcpy(&w, &rep, sizeof rep);
   return w;
void ok(void)
   const wibble w = wopen(42);


In C it is impossible to expose a type's size without also exposing its representation. It is possible to define a type concretely and to explicitly specify its representation as being "unpublished". However, since C does not offer the C++ private keyword using the representation is always possible and remains a constant temptation. Once one piece of client code succumbs more are sure to follow and like a dam bursting the client and implementation quickly become tightly coupled and any separation is washed away. Completely hiding a type's representation behind an opaque pointer/handle removes the temptation and creates a powerful abstraction but at the price of hiding the size of the type and the consequent restriction on the storage class of client memory. A shadow data type offers a half-way house where a type is effectively split into two, with one part exposing the size and the other part holding the representation. The alignment and sizes of the two parts must correspond. Client code is then able to use all three storage class options. The implementation code takes the full load of the extra complexity mapping/overlaying between the split parts. One interesting observation is that the client code would be uneffected (other than needing recompilation) if the representation was moved back into the client side type (to try and improve performance perhaps). No mechanism is universally applicable and the shadow data type is no exception! Experience and time alone will tell if and how useful it is. Caveat emptor.

Perfect Software and other illusions about testing

is the title of an excellent book by Jerry Weinberg. As usual I'm going to quote from a few pages:
Information is neutral but people's reactions to information are rarely neutral.
I'm not in the proof business; I'm in the evidence and inference business.
A process description is just that, a description ... not of what has actually been done.
To qualify as a test, an action has to seek information that will influence an action.
Fixing under pressure tends to raise the fault-feedback-ratio.
Anyone who makes promises about the future must be some sort of devil.
Errors are made, not born.
In a moment of weakness it is difficult to resist infantile suggestions.
Good tools amplify effectiveness; if your effectiveness is negative, adding tools will amplify only the negativity.

enums in C

Enums are notoriously weakly typed in C. While pondering this the other day it occurred to me that instead of writing an enum like this:
enum suit { clubs, hearts, diamonds, spades };
you could write it like this:
struct suit
    enum { clubs, hearts, diamonds, spades } value;
The struct wrapper adds a modicum of type safety. It also allows you to forward declare suit (you can't forward declare enums). The enumerators (clubs, hearts, diamonds, clubs) are still visible and can be used when a compile time value is required (e.g., switch cases). As usual, caveat emptor.

The Mythical Man Month

is the title of an excellent book by Fred Brooks. It's well worth rereading every year or so. As usual I'm going to quote from a few pages:
The accumulation of simultaneous and interacting factors brings slower and slower motion.
Human beings are not accustomed to being perfect and few areas of human activity demand it.
Our estimating techniques fallaciously confuse effort with progress.
No part of the schedule are so thoroughly affected by sequential constraints as component debugging and system test.
Take no small slips... Trim the task.
I will contend that conceptual integrity is the most important consideration in system design.
...the total creative effort involves three distinct phases: architecture, implementation, and realization. It turns out that these can in fact be begun in parallel and proceed simultaneously.
Any software system should be grown by incremental improvement... Nothing in the past decade has so radically changed my own practice, or its effectiveness.
Unless we teach people how to design, the languages matter very little.
The Mythical Man Month is only incidentally about software but primarily about how people in teams make things.

BOSE - today you were RUBBISH

I bought a pair of BOSE noise canceling headphones at Oslo airport on 29th August. My first BOSE purchase. The left earphone has stopped working. I rang the service number and briefly explained the situation. The assistant asked for some information, some of which I'd just told her. She typed into a computer and her computer crashed. She didn't know what to do so she put me on hold. While I waited I replied to a couple of emails, made an entry to my wiki, and started reading from a book. Then she was back. She apologised. She told me that everyone's computer has crashed! She asks me for the information again. I provide it again. She types it in again. It doesn't crash this time!

Then she asks for the serial number. "Where is that?" I say. "What kind of headphones have you bought?" she says. "How can I tell?" I say. It seems you can't tell by looking at them - they have a minimalist design with only the words BOSE on the headphones. "Try looking inside the ear cushion" she says. I can see a number in the left one. I start reading it out. "No that's not it". Eh? You have a serial number in both earphones but only one is the actual one? Are you trying to be rubbish I wonder. So I look in the other ear cushion. Indeed - another number. So long and so small I can't read it. So I put my glasses on and read it out. "Ah" she says that's the such-and-such model. "They're discontinued". Why am I not surprised. She can't do a like for like replacement for that. So they'll replace it with the nearest equivalent. But before they can do that they'll post me an audio cable - could I try the new cable to see if that fixes it? it will be delivered in 5-7 working days, all I have to do is sign for it. If I'm not in when they deliver it they'll put a note through the door so I know which depot to drive to pick them up from. By now I'm losing my patience. I ask if she's really telling me I'm going to have to stay in all day to receive an audio cable that in all probability won't solve my problem and is merely creating extra weeks when I have no working BOSE headphones despite having paid over £200 for them only 2 months ago? And besides that I won't be in anyway since I'll be in another country on business. On business without my BOSE headphones. She says she'll ask someone. I'm on hold again. No, she was wrong. They'll send them regular post. They don't have to be signed for. She's very sorry but there's nothing else she can do. That's what she has to do. I'm so fed up with BOSE at this point I can't face wasting anymore of my time so I tell her to send the damn audio cable.

Do BOSE think that people willing to pay £200 for a supposed quality pair of headphones value their time so little that they're happy to waste even more of it just because the headphones don't work?

BOSE products are expensive. They are pitched as high quality products. My experience today tells me their customer service computers and their customer service process is not high quality. I don't in any way blame the girl on the phone. She's trapped inside a rubbish system, forced to follow rigid procedures like a robot.

BOSE, today you were RUBBISH.

Becoming a Technical Leader

is the title of an excellent book by Jerry Weinberg. As usual I'm going to quote from a few pages:
Underlying organic models is the fundamental idea of systems thinking: "It is impossible to change just one thing at a time."
Leaders are leaders of change in themselves.
People don't become leaders because they never fail. They become leaders because of the way they react to failure.
If you are a leader, people are your work.
There is no conflict between people and task.
Doing something new heightens awareness, which is stimulating, but also a little uncomfortable.
If you intend to grow you cannot avoid some pain.
Very few of the classic leadership studies have been done in environments even vaguely resembling the technical work situation.
The only way we can see ourselves is through other people. This inability to see ourselves as others see us is the number one obstacle to self-improvement.
Becoming a leader means shifting the focus from your ideas to the ideas of others.

30 Great Systems Thinkers

Systems Thinkers is a book by Magnus Ramage and Karen Shipp which has just caught my eye. The blurb on Amazon says:

Systems Thinkers presents a biographical history of the field of systems thinking, by examining the life and work of thirty of its major thinkers.

Annoyingly it doesn't mention who the 30 people are. A bit of googling reveals the missing information:

Early Cybernetics
  • Gregory Bateson
  • Norbert Wiener
  • Warren McCullough
  • Margaret Mead
  • W. Ross Ashby

General Systems Theory
  • Ludwig von Bertalanffy
  • Kenneth Boulding
  • Geoffrey Vickers
  • Howard Odum

System Dynamics
  • Jay Forrester
  • Donella Meadows
  • Peter Senge

Soft and Critical Systems
  • C. West Churchman
  • Russell Ackoff
  • Peter Checkland
  • Werner Ulrich
  • Michael Jackson

Later Cybernetics
  • Heinz von Foerster
  • Stafford Beer
  • Humberto Maturana
  • Niklas Luhmann
  • Paul Watzlawick

Complexity Theory
  • Ilya Prigogine
  • Stuart Kauffman
  • James Lovelock

Learning Systems
  • Kurt Lewin
  • Eric Trist
  • Chris Argyris
  • Donald Schön
  • Mary Catherine Bateson

Whole > Sum(Parts)

I was chatting to my friend Mark today. He's retired and spends his time doing exactly what he wants to which is mostly making tools and using them to make more tools and lots of beautiful things. Today he was showing me his newly finished rose cutter which allows him to create wood turning patterns you wouldn't think possible. Anyway, at one point we were chatting about shooting rabbits. He mentioned that making a gun was easy, a couple of evening's work he said! Making ammunition was more tricky though. He said you can buy all of the separate bits you need to make bullets without a license. But when you put them together then you need a license and face a jail sentence if you haven't got one. I thought was a nice example of the whole being greater than the sum of the parts.

A Tale of Two Sessions

I attended this years excellent AYE conference in Phoenix. Powerpoint style presentations are not allowed at AYE; many session are instead based on simulation and role-playing led by an experienced facilitator. The two sessions I attended on the Wednesday remain in my thoughts. Don Gray led the morning session and Steve Smith led the afternoon session. The task in both sessions was broadly similar (sorting decks of cards), but the contrast between the two sessions could hardly have been greater.

Don added arbitrary time pressure as part of the game and the participants accepted it. Steve's exercise had no time pressure. Well actually that's not quite true. I think there was time pressure but Obie who played the role of Steve's project manager shielded the workers from it very effectively. The feeling of being rushed was palpable in Don's session, whereas a feeling of general calm pervaded Steve's. There was a noticeable difference in the noise levels!

Don's exercise was initially more or less impossible to complete because of some confusion and contradiction in the requirements (quite possibly deliberate). The requirements seemed to be in much greater state of flux and quite a few participants (me included) sensed this and asked the customers various questions. This didn't help. In Steve's exercise Obie again did an excellent job of keeping things simple for the customer and the workers by being the only point of contact between them.

In Steve's session Obie explicitly said to the workers that if they didn't have anything to do at some point that was ok, they should just step back, observe and try to see where they could help. They should try not to interfere. This helped to emphasize the primary importance of the team and the overall goal.

In Don's session some people sensed the chaos building up and tried to intervene individually to help. It didn't help. All it served to emphasize was individual action rather than team co-operation. In Steve's session there was never a sense of chaos and any time any problems arose the participants were generally much more willing to follow a direction for the good of the team. They may not have agreed with the direction but you don't have to agree with a direction to support it as if it were your own. Following can often be a powerful act of leadership.

There were roughly the same number of people participating in both sessions. However, in Don's session several people disengaged from the exercise precisely because it was so chaotic. They were experiencing real discomfort. In its own way this too was an act of leadership - by removing themselves from the game they created a much better chance for the smaller team to overcome the chaos.

When technology pisses me off (5)

I went into my local Orange shop today. I simply wanted to upgrade my phone to one with a higher resolution camera. The assistant asked me some details and typed them into a computer. Nothing happened. She asked a colleague if the computer was working. He said try rebooting it. So she did. Then she asked me the same details again. And typed them in again. Still not working. Long pause with nothing happening. Decided not to get frustrated. Looked for a chair to sit down while I waited. No chairs. Sat on a low shelf instead. She asked me for the same details for a third time. This time I wrote them on a piece of paper. I decided to mention something I hadn't been asked - that I was on the Virgin tariff. That seemed to be an important piece of information and the computer was abandoned as she tried to ring someone for help. She couldn't get through. I went to sit down again. She kept trying. Eventually she got through. She passed the phone to me. The man on the other end asked for the same information for a fourth time. Then he asked me the post code of the shop I was in! I remained calm and handed the phone back to the assistant. I sat down again. The call ended. After all that I was told I couldn't upgrade my phone unless I switched to a different tariff. I was near the end of my tether by this point but I decided to at least look at the new tariffs. All more expensive than the one I'm one. I said simply "It's not going to happen. You've lost a sale." and walked out. Orange - today you were RUBBISH.

#exclude - a testing idea for C/C++

Suppose you're working on a legacy C/C++ codebase and you want to introduce some characterization tests. One seam you might consider exploiting is #includes. It's quite easy to exclude includes. For example:
# excluder.rb
include ='(\s*)#(\s*)include(.*)')
STDIN.readlines.each do |line|
 if include.match(line)
   line = "#if 0\n" + line + "#endif\n"
 STDOUT.write line
This small Ruby program reads in a source file and writes out the same source file, except the #includes are commented out. Each line like this:
#include <stdio.h>
is replaced like this:
#if 0
#include <stdio.h>

This enables you to create an "isolated" version of a source file. One where all the dependencies arising from the #includes, to any depth, are slashed in one swift cut. One where your tests clearly and visibly have to recreate all those dependencies in a custom mock environment.

This is just an idea. I might not be a good idea. I only thought of it last week. Caveat emptor.

Incomplete types oddity in C

Here's something odd from C. It's to do with incomplete types. This compiles fine in gcc:
typedef struct wibble wibble;
void f(wibble * ptr);
but this does not:
typedef struct wibble wibble;
void f(wibble array[]);
A bit of searching in the Standard reveals Function declarators
paragraph 12 - If the function declarator is not part of a definition of that function, parameters may have incomplete types...

This suggests the [] based declarator is ok. However it's not because... Array declarators
paragraph 1 - the element type shall not be an incomplete type

Which leads you ask what is an incomplete type?

6.2.5 Types
paragraph 22 - An array of unknown size is an incomplete type.

Which is interesting because it says nothing about an array of unknown type being an incomplete type. This is a shame. Particularly because of the new [] array declarator syntax in c99.

Showers are lean-agile; baths are not

In the shower a few days ago I realized that showering vs bathing is a nice example of being lean-agile or not. Imagine 10 people bathing sequentially vs showering sequentially...

With a bath you have to wait (in a queue) while the bath fills up. Not so with a shower.

If you do something else while you wait for the bath to fill up you're likely to find the bathwater too hot or too cold.

Or the tap turns itself off (I have a hot tap that does that sometimes) and the bath is still virtually empty.

Or that might be the time you discover the overflow pipe is blocked.

Or that a fully-on tap releases water faster than an overflow pipe can drain it.

With a bath the water stays in the bath and gets dirty.

With a bath the water gets cold as you lie in it.

With a bath you have to drain the water before you can refill it so the next person can bath in clean water.

With a bath you use more water.

A bath takes up more floor space.

In a shower you stand up, you stay more awake.

A bath is less encapsulated.

Bath leaves a ring of scum that has to be cleaned.

Birth of the chaordic age

is the title of an excellent book by Dee Hock. As usual I'm going to quote from a few pages:
All things, even life itself, are a seamless blending of chaos and order.
By structure I mean the embodiment of purpose, principles, people, and concept.
The results of the best organizations are apparent, but the structure, leadership, and process and transparent.
Information multiplies by transfer and is not depleted by use.
Knowledge becomes understanding when related to other knowledge in a manner useful in conceiving, anticipating, evaluating, and judging. Understanding becomes wisdom when informed by purpose, ethics, principles, memory of the past, and projection into the future.
All knowledge is an approximation.
The information age is primarily an extension of mental power.
We are at that very point in time when a four-hundred-year-old age is rattling in its deathbed and another is struggling to be born.

Out of Our Minds

is the title of an excellent book by Ken Robinson. As usual I'm going to quote from a few pages:
People are not creative in general but in doing something concrete.
Spontaneity sometimes has to be carefully planned.
Creativity is not just a matter of letting go: it involves hanging on.
Creativity is as much a process of finding problems as solving them.
Trying to put some experiences into words is like stringing clothes on a washing line when in practice they are worn one inside the other.
Creativity is a process not an event.
We classify at our peril.
Creativity is incremental.
Creativity relies on the flow of ideas.

Important - do not rub off

In one of his books Jerry Weinberg mentions a cleaner who wasn’t allowed to wipe the whiteboards. The story conjures familiar images of whiteboards full of stuff labeled “important - do not rub off”. It’s likely the stuff has lain untouched on that no-longer-white-whiteboard for weeks or months. It’s no longer important but there it stays, filling the whiteboard. A whiteboard full of stuff discourages its use in exactly the same way an empty whiteboard encourages it. And a whiteboard with the words “important - do not rub off” positively discourages use. It's really saying "keep away".

The whole purpose of whiteboards is that you wipe them. Perhaps they should be called wipeboards! Buying a wipeboard and its special pens and its special erasers and then filling it with stuff just the once is silly and expensive. If you want something permanent why not use a permanent pen on a big piece of paper stuck to the wall?

Filling a wipeboard with stuff labeled “important - do not rub off” is also silly if the stuff genuinely is still important. Someone might be desperate for a wipeboard and decide their new definitely-important stuff is more important than somebody else’s existing possibly-important stuff. So they wipe it. Wipeboards are so easy to wipe after all!

Le Chatelier's Principle says "[good] systems tend to oppose their own proper function". What is the proper function of a wipeboard? To be written on. How do you oppose being written on? Simple - wipe them completely clean every so often. Ruthlessly. Even if someone's written "important - do not rub off" on them.

With this use-clean use-clean use-clean dynamic set up the wipeboards give a clear visual impression of their dynamic use since anything on a wipeboard must necessarily have be written fairly recently. If they remain mostly unused most of the time you get to see that very clearly too.

Le Chatelier's Principle revisited

I was chatting about Le Chatelier’s principle with Hubert Matthews at an accu meeting in London yesterday evening. Hubert is a very smart guy and explained to me that losing your boarding card is a nice example, but not necessarily of Le Chatelier’s principle. What did I misunderstand?

Le Chatelier’s principle says "systems tend to oppose their own proper function". I incorrectly read this as a description of a poor system, an unorganized system, a system with no feedback, no control, (you get the idea), a system straying further and further out of balance at the slightest touch. In other words the system opposes its own proper function - but only in the sense that it never really had a chance of functioning properly in the first place! A system likely to die a quick natural death. That was the kind of systems behaviour I was looking for a name for. But I’ll come back to that.

So what is Le Chatelier’s principle really saying? As always an example helps. Every cell in your body requires a constant supply of glucose for energy. If your blood sugar level rises (e.g. after a meal) special cells in your pancreas produce insulin. Insulin causes the liver and muscles convert the extra glucose to glycogen and store it away, helping to reduce blood sugar levels. This is normal functioning. This is proper function. Ok. Now, what Le Chatelier’s Principle says (assuming I've got it right this time) is that your body will oppose this proper function! And your body does indeed start this opposition, and for a very good reason – if the insulin is left unchecked it continues to drain glucose from the blood and your blood sugar level gets too low. So when this happens different special cells in the pancreas start to produce glucagon. The glucagon works in opposition to insulin, causing the liver and muscles to convert the stored glycogen back into glucose thus increasing your blood sugar level.

In a healthy person these mini-systems operate quite properly, just like a thermostat, keeping your blood sugar level safely between fairly strict limits. Not too much (hyperglycemia) and not too little (hypoglycemia). They do this by opposing each other. It reminds me of the Goldilocks Principle. Goldilocks didn’t like the bed too soft or too hard, or the porridge too hot or too cold. Goldilocks knew her own mind.

So what have I learned from this? I’ve learned that I understand Le Chatelier’s Principle a bit better if I reword it ever so slightly to start with the word good. Good is a bit weak but I’m a bit pushed for time so it will do for now.

So what is my boarding card example an example of? Does The Inverse Le Chatelier’s principle apply? If a system does not oppose its own proper function is it more likely to be a poor system? I think there's an element of truth to that. If so we have Le Chatelier’s Principles. There are two of them, in opposition to each other, which somehow feels appropriate!

Good systems tend to oppose their own proper function.
Bad systems tend not to oppose their own proper function.

Thanks Hubert.

Predictably Irrational

is the title of an excellent book by Dan Ariely. As usual I'm going to quote from a few pages:
We don't have an internal value meter...rather we focus on the relative advantage of one thing over another
Our first decisions resonate over a long sequence of decisions.
With everything you do you should train yourself to question your repeated behaviour.
Humans are intrinsically afraid of loss.
For market norms to emerge it is sufficient to mention money.
Money, it turns out, is very often the most expensive way to motivate people.
We fall in love with what we already have.
Expectations can influence nearly every aspect of our lives.
The majority of people cheated, and they cheated just a little bit.
What a difference there is in cheating for money versus cheating for something that is a step away from cash!

Thought for the Day

From Small Memory Software by James Noble and Charles Weir.

The fundamental difference between code and data is that programmers care about code whereas users care about data.

Le Chatelier - Systems tend to oppose their own proper function

One of the principles of Systems Thinking is Le Chatelier's Principle.

Systems tend to oppose their own proper function.

A nice example of this is checking your passport and boarding card at the airport. The comedian Michael McIntyre has a very funny sketch pretending to walk through an airport. Every three or four paces he has an uncontrollable compulsion to check that his passport and boarding card are still in his breast pocket. At one point he thinks he's lost them and goes into a mild panic.

Your passport and boarding card are completely safe in the pocket. They are not going to suddenly jump out of the pocket all by themselves. Gravity is not suddenly going to go into reverse. You're unlikely to start walking around on your hands. Pretty much the only way they're going to get out of that pocket is if you take them out yourself. Which is exactly what you do to check you haven't lost them!

On more than one occasion I've seen someone unknowingly drop their boarding card while checking they still have it. This is Le Chatelier. You have a goal in mind, you act to achieve that goal, and your action creates the opposite effect!

So now you know about Le Chatelier. You know that the time you're most likely to lose your boarding card is when you check whether you've lost it. So immediately after checking it you'll remember this blog and start to worry you lost it when you checked it. You'll probably have to check it again. Very soon you'll be checking it every three or four paces like Michael McIntyre. And of course, the more you check it the more likely you are to lose it! Le Chatelier again!

UDPATE: Losing your boarding card is NOT an example of Le Chatelier's Principle. It's an example of a different Systems Law: The Law Of Unintended Consequences! See Le Chatelier's Principle Revisited for what Le Chatelier's Principle really is (I think!)

Thought for the day

I think I read this on James Grenning's site somewhere:

It’s easier to act your way into thinking differently than to think your way into acting differently.

I think the origin of this quote might be Richard Buckminster Fuller:

You cannot change how someone thinks, but you can give them a tool, the use of which leads them to think differently.

How Buildings Learn: Chapter 11 - The Scenario-buffered Building

How Buildings Learn by Stewart Brand is a really great read about the underlying processes that govern the evolution of buildings over time.

All predictions are wrong.

The product of skilled scenario work is not a plan but a strategy. Where a plan is based on prediction, a strategy is designed to encompass unforseeably changing conditions. A good strategy ensures that, no matter what happens, you always have maneuvering room.

Many a building is a brilliant (or pedestrian) design solution to the wrong design problem.

The iron rule of planning is: whatever a client or architect says will happen with a building, won't.

"Our most important responsibility to the future is not to coerce it but to attend to it" [Kevin Lynch, city theorist]

Build something smaller and more solid now that can expand in a variety of directions later.

You have to iterate [Frederick Brooks]

How Buildings Learn: Chapter 10 - Function Melts Form: Satisficing Home and Office

How Buildings Learn by Stewart Brand is a really great read about the underlying processes that govern the evolution of buildings over time.

Too eager to please the moment, over-specificity crippled all future moments. It was the image of organic, not the reality. The credo "form follows functions" was a beautiful lie. Form froze function.

Houses evidently need more low-definition space for later expansion, and it's easier to add in than to add on.

Many a remodeling contract has to announce grimly higher costs upon discovering the product of previous do-it-yourselfers.

The trick is to remodel in such a way as to make later remodeling unnecessary or at least easy. Keep furniture mobile. Keep wiring plumbing, and ducts accessible.

[By] Far the greatest rate of change comes right at the beginning, as it does with everything that lives. Finishing is never finished.

You're right down to where the building most interfaces with the people who will be living in it, and they discover that some important things were left out, and some ideas that seemed so sensible on the plans aren't going to work. Last-minute revision - the most important stage of tuning a house - comes just when time and money are shortest. Aggravated compromise is the order of the day.

Inhabitation is a highly dynamic process, little studied.

The building and its occupants jointly are the new system.

Interior designers never satisfice. They are paid to optimize, to make perfect. Perfection is frustratingly temporary.

Paradoxically, habit is both the product of learning and the escape from learning. We learn in order not to learn. Habit is efficient; learning is messy and wasteful.

"Change is suffering" was the insight that founded Buddhism.

Once in place, the organization advances best by hordes of "small wins".

Chris Alexander: "At each level of scale, it is those actually using the space who understand best how it can be made/altered to have the character of being conducive to work. ... Therefore we suggest using materials and structural systems which invite change and allow changes to accumulate."

You cannot predict or control adaptivity. All you can do is make room for it - room at the bottom. Let the mistakes happen small and disposable. Adpativity is a fine-grained process.

"Wanderer", wrote a Spanish poet, "there is no path. You lay down a path in walking." [Antonio Machado]

How Buildings Learn: Chapter 9 - How Buildings Learn From Each Other

How Buildings Learn by Stewart Brand is a really great read about the underlying processes that govern the evolution of buildings over time.

The heart of vernacular design is about form, not style. Style is time's fool. Form is time's student.

Specialized space hinders future flexibility.

However much buildings may be sold as a product, they are lived in as a process.

Small invites the metamorphosis of growth.

The difference between style and form is the difference between a statement and a language.

How Buildings Learn: Chapter 8 - The Romance of Maintenance

How Buildings Learn by Stewart Brand is a really great read about the underlying processes that govern the evolution of buildings over time.

The issue is core and absolute: no maintenance, no building.

A staggering one fifth of the sample said that the need to clean their windows had not even been considered during the design and construction of the building.

The root of all evil is water.

A building's most important organ of heath is its roof.

The worst of it is, when water comes through a flat roof, you can't tell where the leak is because the water travels great distances hidden in the roof, ceilings and walls.

The question is this: do you want a material that looks bad before it acts bad, like shingles or clapboard, or one that acts bad long before it looks bad, like vinyl siding?

What you want in materials is the quality of forgiveness.

Redundancy of function is always more reliable than attempts at perfection, which time treats cruelly.

Bricks more than any other material look like they were made to fit the human hand.

If that small stuff isn't happening all the time, you're not going to take care of it, and it isn't going to come to order.

In his book The Oregon Experiment, Alexander elaborated "Large-lump development is based on the idea of replacement. Piecemeal growth is based on the idea of repair."

Large-lump development is based on the fallacy that it is possible to build perfect buildings. Piecemeal growth is based on the healthier and more realistic view that mistakes are inevitable.

I'd like to see building designers take on problem transparency as a design goal. Use materials that smell bad when they get wet.

Maintenance, in this light, is learning.

How Buildings Learn: Chapter 7 - Preservation

How Buildings Learn by Stewart Brand is a really great read about the underlying processes that govern the evolution of buildings over time.

Preservation has become the best carrier of that moral force architecture needs if it is to have value beyond shelter.

Why are old buildings more freeing? They free you by constraining you.

It is much easier to continue than to begin. Less money is needed, as well as less time, and fewer people are involved, so fewer compromises are necessary.

How Buildings Learn: Chapter 6 - Unreal Estate

How Buildings Learn by Stewart Brand is a really great read about the underlying processes that govern the evolution of buildings over time.

Rushing is at the root of all lack of quality.

Form follows failure.

City lots...(as in parking lots). Their size is enormously influential. Small lots make for constant fine-grain adaptation instead of the sudden devastating changes that can come with large parcels. Everything depends on the pattern of ownership of the land.

American planners always take their inspiration from Europe's great cities and such urban wonders as the Piazza San Marco in Venice, but they study the look, never the process. .... Venice is a monument to a dynamic process. Not to great urban planning.

As usual, the rate of change is everything.

Chris Alexander agrees: The money is all wrong in most buildings, and it's crucial. There should be more in basic structure, less in finish, more in maintenance and adaptation.

Work done in haste is necessarily shoddy, a house of cards. On a go-fast schedule there is no margin for a single error, and error is inevitable. High risk, high loss. The opposite strategy is much surer, because the errors are piecemeal and correctable. When you proceed deliberately mistakes don't cascade, they instruct. Low risk plus time equals high gain.

How Buildings Learn: Chapter 5 - Magazine Architecture: No Road

How Buildings Learn by Stewart Brand is a really great read about the underlying processes that govern the evolution of buildings over time.

Architects offered themselves as providers of instant solutions, and only the look of a building gives instant gratification.

In the 1980's, 80 percent of the ever-growing post-construction claims against architects were for leaks. Domes leaked, always. Worst of all, domes couldn't grow or adapt. What's good about 90-degree walls: they don't catch dust, rain doesn't sit on them, easy to add to; gravity, not tension holds them in place. We are 90 degrees to the earth.

The specious old-box is old because it is profoundly adaptive.

Instead of steady accumulation, the business of contemporary architecture is dominated by two instants in time. One is the moment of go-ahead...The other is the moment of hand-over.

All the design intelligence gets forced to the earliest part of the building process, when everyone knows the least about what is really needed. A lot of the time now you see buildings that look exactly like their models. That's when you know you're in trouble.

There is real misunderstanding about whether buildings are something dynamic or something static.

Architects think of a building as a complete thing, while builders think of it and know it as a sequence.

The race for finality undermines the whole process. In reality finishing is never finished.

Unchallenged practices persist for decades.

Admiration is from a distance and brief, while love is up close and cumulative.

The needed conversion is from architecture based on image to architecture based on process.

How Buildings Learn: Chapter 4 - Houseproud: The High Road

How Buildings Learn by Stewart Brand is a really great read about the underlying processes that govern the evolution of buildings over time.

The High Road is high-visibility, often high-style, nearly always high-cost. This is the way to grow a High Road building. Take it by stages with constant minute refinement. There's a huge time lag between when you need something and when you actually get it. Trust, intimacy, intense use, and time are what made these buildings work so well.

How Buildings Learn: Chapter 3 - Nobody Cares What You Do In There: The Low Road

How Buildings Learn by Stewart Brand is a really great read about the underlying processes that govern the evolution of buildings over time.

What do people do in buildings when they can do almost anything they want?

How Buildings Learn: Chapter 2 - Shearing Layers

How Buildings Learn by Stewart Brand is a really great read about the underlying processes that govern the evolution of buildings over time.

Building on the theme of time again he quotes Frank Duffy (ex president of the Royal Institute of British Architects) "Our basic argument is that there isn't such a thing as a building. A building properly conceived is several layers of longevity of built components".

Some layers live longer than other layers.

Some layers are more stable than other layers.

Layers change at differing rates.

The rates of change over time define the layers as clearly as the individual physical changes.

Brand extends Duffy's four S's shearing classification to six S's: Site, Structure, Skin, Services, Space plan, Stuff.

Quoting Duffy again "The unit of analysis for us isn't the building, it's the use of the building through time. Time is the essence of the real design problem."

Many buildings are demolished early if their outdated systems are too deeply embedded to replace easily.

Hummingbirds and flowers are quick, redwood trees slow, and whole redwood forests even slower. Most interaction is within the same pace level.

The dynamics of the system will be dominated by the slow components, with the rapid components simply following along. Slow constrains quick; slow controls quick.

Echoing the quote from Churchill, that the building learns from it occupants and they learn from it, he provides a fascinating insight - "In classical Greece and Rome domus meant house in an expanded sense: People and their dwellings were indistinguishable: domus (as in Romanes eunt domus - Life of Brian) referred not only to the walls but also to the people within them."

How Buildings Learn: Chapter 1 - Flow

How Buildings Learn by Stewart Brand is a really great read about the underlying processes that govern the evolution of buildings over time.

At the heart of this book is the idea of change - of time. That buildings change. That people change them. That the elements change them. This, fundamentally, is the reason the parallels with software leap off every page. Understanding and managing change is arguably the fundamental aspect of understanding and managing the process of developing software. Software isn't written perfectly in a sudden flash. It takes time. It takes lots of small changes.

The author writes "My approach is to examine buildings as a whole - not just whole in space, but whole in time". He laments the aphorism "Form ever follows function" written in 1896 by Louis Sullivan (A Chicago high-rise designer) because "it misled a century of architects into believing that they could really anticipate function". The idea is to aim for software that gets better over time. Or, more accurately, that is capable of getting better over time.

Complexity Matters

Managing complexity is not just how you connect things together but to what extent the connection mechanisms scale. In Nature the scalability of the mechanisms seems to have hard physical limits. D'Arcy Thompson [1] wrote:

In nature, the effect of scale depends not on a thing in itself but in relation to its whole environment.

Everywhere Nature works true to scale, and everything has its proper size accordingly.

In short, it often happens that of the forces in action in a system some vary as one power and some as another, of the masses, distances or other magnitudes involved; the "dimensions" remain the same in our equations of equilibrium, but the relative values alter with the scale. This is known as the "Principle of Similitude", or dynamic similarity, and it and its consequences are of great importance.

Size matters too. Even the smallest mistake can kill an animal/program. In [2] Richard Gabriel writes:

build small abstractions only

build hierarchies slowly, keeping them shallow as long as possible

buildings with the quality are not made of large modular units

we must make sure the structure is correct down to 1/8th of an inch (quoting Christopher Alexander)

Fan-in/fan-out is important complexity metric. The difference in complexity between different levels of the system should be roughly the same. This is reminiscent of fractals and what Christopher Alexander calls "centers". Again from Patterns of Software:

without large structure, the design cannot hold together - it becomes merely a jumble of isolated design elements.

The nature of a system is such that at almost any granularity it looks the same - it is a system. ... When we put together an object-oriented thing, it is a system, not a program. The difference between a program and a system is precisely the characteristic of a system having many centers or ways of approaching it - from the vantage point of any sub-system, the rest of the system is a server - whereas in a program, you generally have a single way of viewing it, usually from the top down.'

Objects are important because they make it easier to create systems rather than programs and systems can handle complexity and change better than programs.

[1] On Growth and Form. D'Arcy Thompson. Cambridge University Press. ISBN 0-521-43776-8

[2] Patterns of Software. Richard Gabriel. Oxford University Press. ISBN 0-19-5100269-X

Benjamin Franklin's Prose Practice

I've finished reading Benjamin Franklin's Autobiography. The thoroughness of his deliberate prose practice is impressive:

About this time I met with an odd volume of the Spectator. It was the third. I had never before seen any of them. I bought it, read it over and over, and was much delighted with it. I thought the writing excellent, and wished, if possible, to imitate it. With this view I took some of the papers, and, making short hints of the sentiment in each sentence, laid them by for a few days, and then, without looking at the book, tried to complete the papers again, by expressing each hinted sentiment at length, and as fully as it had been expressed before, in any suitable words that should come to hand. Then I compared my Spectator with the original, discovered some of my faults, and corrected them. But I found I wanted a stock of words, or a readiness in recollecting and using them, which I thought I should have acquired before that time if I had gome on making verses; since the continual occasion for words of the same import, but of different length, to suit the measure, or of different sound for the rhyme, would have laid me under a constant necessity of searching for variety, and also have tended to fix that variety in my mind and make me master of it. Therefore I took some of the tales and turned them into verse; and, after a time, when I had pretty well forgotten the prose, turned them back again. I also sometimes jumbled my collection of hints into confusion, and after some weeks endeavored to reduce them into the best order, before I began to form the full sentences and complete the paper. This was to teach me method in the arrangement of my thoughts. By comparing my work afterwards with the original, I discovered many faults and amended them;

The Secrets of Consulting

is the title of an excellent book by Jerry Weinberg. As usual I'm going to quote from a few pages:
There's always a problem. It's always a people problem.
People want training that makes them better adapted to their present task, rather than training that makes them more adaptable to future tasks.
Consulting is not a test for the consultant, it's a service to the client.
Things are the way they are because they got that way.
Study for understanding, not for criticism.
Don't be afraid to cultivate your unconscious.
Cucumbers get more pickled than brine gets cucumbered.
The trick to earning trust is to avoid all tricks.
Never promise anything. Always keep your promise.

When technology pisses me off (3)

This time it's a digital camera. Scroll through the photos on the camera and you can see them all. Connect to a Windows XP laptop and only some of them appear. If you're involved in making technology I plead with you to at least make it do the basics correctly. If it does the basics correctly I can forgive you when rarely used extra frills don't work. What's unforgivable is when the basics don't work.

Quite by chance, I see David Hansson of Ruby on Rails fame is twittering he's having numerous problems making calls from his iPhone. Same thing. RUBBISH.

C/C++ 2 phase unit-testing again

In my previous blog entry on the 2-phase unit-testing idea the code sketch looked like this:
/* test.h */
#define TEST(name) if (run_phase(suite, name))
/* wibble_tests.c */
#include "test.h"
#include "wibble.h"
static void wibble_tests(test_suite * suite)
 TEST("fprintf failure inside wibble")
   int expected = 42;
   int actual = wibble(4, 2);
   /* assert expected == actual */
A follow on idea from this to make the string literal test names globally available. Suppose the wibble function looks like this:
/* wibble.c */
#include "wibble.h"
#include <stdio.h>

int wibble(int lhs, int rhs)
 ret = fprintf(stdout, "%d,%d", lhs, rhs);
 if (ret < 0)
And you want to mock fprintf. You can move the definition of wibble() into its own .func file like so:
/* wibble.c */
#include "wibble.h"
#include <stdio.h>

#include "wibble.func"
/* wibble.func */
int wibble(int lhs, int rhs)
 ret = fprintf(stdout, "%d,%d", lhs, rhs);
 if (ret < 0)
This allows you to isolate wibble() by #including the wibble.func file. This is quite brutal but it shouldn't affect the behaviour and brutality can be appropriate when your trying to write characterisation tests because you don't have any tests yet. The last step is to make run_phase() access the suite parameter itself (rather than being passed it) so the TEST macro can be used anywhere:
/* test.h */
#define TEST(name) if (run_phase(name))
/* wibble_tests.c */
#include "test.h"

typedef int mock_file;
static mock_file * stdout = 0;

static int fprintf(mock_file * mf, const char * format, ...)
 TEST("fprintf failure inside wibble")
   return -2;

#include "wibble.func"

static void wibble_tests(void)
 TEST("fprintf failure inside wibble")
   int expected = 42;
   int actual = wibble(4, 2);
   /* assert expected == actual */

The Machine That Changed The World the title of a book by James Womack, Daniel Jones, and Daniel Roos. It's subtitled How Lean Production Revolutionized the Global Car Wars. As usual I'm going to quote from a few pages:
In striking contrast to the mass-production plant, where stopping the line was the responsibility of the senior line manager, Ohno placed a cord above every work station and instructed workers to stop the whole assembly line immediately if a problem emerged that they couldn't fix. Then the whole team would come over to work on the problem.
Lean distribution will form the front end of a system that is driven by the needs of the customer, not by the needs of the factory.
The ultimate cause is almost always an organizational problem.
Workers at mass-production plants learn no skills.
Japanese companies do not believe they can turn around existing mass-production facilities.
In Japan...practically all hiring in companies is from the bottom only.
Anything that is new is likely to be misunderstood, typically by attempts to explain the new phenomenon in terms of traditional categories and causes.
The central problem is people - how to reward and motivate thousands of individuals from many countries and cultures so that they work in harmony.

When technology pisses me off (2)

A Buzz Lightyear DVD this time. My son Patrick has Asperger's Syndrome. He likes to watch the same film over and over again. He likes to watch the film over and over again. Not the adverts. Going to the cinema is not like watching a DVD at home. DVD's that won't let you skip the adverts when you're watching the DVD for the umpteenth time are RUBBISH

When technology pisses me off (1)

I've just come back from a great days fishing on the river Wye with my mate Brian. To get to the spot we tried to use a sat-nav. A Garmin something or other. It was Brian's. I don't have a sat-nav because on the few occasions I've used one in the past they have been universally rubbish. I'm sure when the work they're great - it's just that they've failed to work for me with sufficient regularity that they no longer get a chance. And this time it was more of the same. Brian spent about 20 minutes trying to enter GL17 9NU, the post-code of our destination. It refused to let him enter the post-code. He typed in GL1 (on a keyboard display that was not qwerty and consequently slowed you down, why?) and it jumped to a screen showing GL16 something or other. Various other options were tried. Nope. It flatly refused to play ball. I've learned that if something's not working there's no point carrying on trying to force it to work. That very rarely works. So we abandoned it and headed off in the general direction relying on good old paper map technology. Much later, quite by chance, we discovered what was going on. The previous sat-nav trip was for our first days fishing in Builth Wells, which is in Wales. Because of this it seemed to have got stuck in Wales. It was only accepting post-codes in Wales. Why doesn't it just let you enter the post-code you want to enter and then see if that post-code is in Wales or England? RUBBISH.

The fifth discipline an excellent book by Peter Senge. It's subtitled The Art and Practice of the Learning Organisation. As usual I'm going to quote from a few pages:
Surprisingly few adults work to rigorously develop their own personal mastery.
More often than we realize systems cause their own crises, not external forces or individuals' mistakes.
Living systems have integrity. Their character depends on the whole.
In systems thinking it is an axiom that every influence is both a cause and an effect. Nothing is ever influenced in just one direction.
One of the highest leverage points for improving system performance, is the minimization of system delays.
Learning is eventually always about action.
The total absence of meaningful practice or rehearsal is probably the predominant factor that keeps most management teams from being effective learning units.
No practice, no learning.
Don't push growth; remove the factors limiting growth.
Most organizations are dominated by linear thinking, not systems thinking. The dominance of the event mentality tells people that the name of the game is reacting to change, not generating change.
It cannot be stressed too much that team learning is a team skill.
You cannot change how someone thinks, but you can give them a tool the use of which leads them to think differently [Buckminster Fuller].
Sensing and acting locally is exactly how complex living systems work.
Hierarchy is antithetical to dialogue, and it is difficult to escape hierarchy in organisations.

Systems Thinking in the Public Sector the title of a book by John Seddon. As usual I'm simply going to quote from a few pages:
Standardization constrains a system's ability to absorb variety.
...the test of a good measure: does it help us understand and improve performance.
focus on costs and costs will rise, focus on flow and costs will fall.
There are two kinds of customer demand: value demand and failure demand.
Waste is man made.
People's behaviour is a product of their system.

Systems Thinking

Human beings have evolved a very strong association that cause and effect are simple and linear; that cause and effect are local in space and local in time. We are not good at seeing the non-linear effects of our actions; the effects over distance and especially over time. Here's something a friend emailed me:

Last week I found that the guys had not checked in a big chunk of library code. It was in the place where common tool-chains were getting invoked, so no one noticed. They changed something and I was seeing sudden breakages even with older versions of the code which was working. They didn't even bother to put it into the version control. I just couldn't believe, people can work this way. I immediately put it into the svn.

Here's what I replied...

Why do you think they hadn't put it in svn? Because at some level they associate the pain of putting it into svn as being greater than the pain of not putting it into svn plus living in a messy kitchen! And who can blame them when helpful people like yourself come along and clean up their mess! They are probably hardly even consciously aware that have the option of putting it into svn themselves. If they haven't put it into svn for N weeks then that is the behaviour that has become their norm. And things that don't change are not seen or thought about. That's abstraction! You are part of the vicious cycle! You are helping to perpetuate their behaviour. You are part of the system!

A more congruent approach would have been to sit next to them and talk them through what they needed to do to tidy up the mess. You absolutely mustn't do any part of it for them. Only if they actually do it themselves will they have a chance of not repeating the situation. They have to type the keystrokes at the keyboard. Remember, it is trivial for you to do it, but for them it will mean working through some level of associated pain. That is not their "fault". They are simply, in this case, less experienced than you. You simply sit next to them and calmly, patiently, ever so patiently, talk them through it in a non threatening, non blaming way.


Generativity is a tricky thing. It's when you work to achieve a desired outcome indirectly. The indirection helps to hide the desired outcome from your conscious mind. This can be important, because your mind has a tendency to interfere with the desired outcome. It's an example of Le Chatelier's Principle - Systems tend to oppose their own function.

Timothy Gallwey, a tennis coach, tells of a great example of generativity. He observed that direct verbal instructions such as ‘Keep your eye on the ball’ are ineffective because players find it hard to keep the direct advice sharp in their minds and at the same time to follow the advice. Instead Gallwey told the player to say the word "bounce" when the ball bounced in front of them and then the word "hit" as they hit the ball. Players found this alternative advice much easier to retain while playing. And since you have to watch the ball carefully to know the exact moment when it bounces and when you hit it it has the side effect of helping you keep your eye on the ball!

The Hawk and the Rabbit

When I'm consulting I sometimes ask a group what the word "agile" means to them. Their response invariably centers on the idea of speed. Agility is not primarily about speed...

A few years ago I was standing on the roof of my house (the loft was being converted - it's not something I normally do). I spotted a bird of prey (a hawk I think) circling overhead. Rabbits lived in the field opposite and I wondered if I the hawk would swoop on one - something I'd never seen before. And with that the hawk began to dive! It was clearly targeting a young rabbit sitting by the far hedge... The rabbit sensed the danger, stopped eating, and started running for its life... The hawk changed direction and kept after the speeding youngster... got very close... reached out with its talons... was about to make its kill.... And at that exact moment the rabbit changed direction. I don't mean it veered off slightly to the left or slightly to the right. I mean one moment it was running as fast as it could along the hedge and the next moment, with almost no discernible pause or reduction in speed, it was running in the opposite direction back to where it had came from. It was quite something. The rabbit was certainly fast but the hawk was faster. What saved the rabbit was its nimbleness. It's agility.

Agility isn't how fast you can keep going in the same direction; agility is how easily you can change direction.


The hawk appeared to have all the advantages during the chase. For one it had gravity on its side, and for two it only had to overcome air-resistance. The rabbit was expending a lot more energy. But the increased friction with the ground helped the rabbit make a sudden change of direction.

Desert Island Books

Appeared in the ACCU magazine CVu, Vol 21 Issue 4, Sept 2009

Introduction (by Paul Grenyer)

I've been doing this series for ten months now and every person has been brilliant. Jon's Desert Island Books is the longest yet and, I have to admit, the one I've enjoyed the most so far. As I know I've said to many people, and maybe even in a previous Desert Island Books, it's about people and Jon has demonstrated that admirably. I've known Jon for a few years and I had no idea he liked fishing! For me, Jon has always been up there with Kevlin as the best of the best of us. He not only understands how to write software, but he understands the people that write it and processes.


I'll start with my two albums. I considered choosing an album of my own. Not stuff I personally composed or sang. Heaven forbid. I couldn't sing in tune if my life depended on it. I mean a compilation of singles from different artists. I bought loads and loads of proper vinyl singles as a boy. The singles I could list. In fact I think I will. Pad it out a bit... What comes to mind.... Early OMD stuff such as Electricity. This is the day by The The. What a cracking single that is. Echo Beach by Martha and the Muffins. I liked a lot of the mod stuff too. Poison Ivy by the Lambrettas was a favourite. Pretty much anything from early Dexies Midnight Runners. Everything by David Bowie. Recently, at my daughter's school concert a very talented fifth form girl played the piano and sang True Colours by Cyndi Lauper which reminded what a great song that is. Another favourite from later on was Jeans Not Happening by The Pale Fountains. I bought plenty of singles from previous decades too. Del Shannon, The Isley Brothers, Roy Orbison. The House of the Rising Sun by the Animals. That would definitely be on the compilation album. How many singles could you fit on a compilation CD? A hundred easily. We may be here some time... Lots by the Beach Boys. Louis Armstrong's We Have All the Time in the World (recorded for the James Bond film - On Her Majesty's Secret Service in one take. He died shortly afterwards). In short I like most things with a good tune where I have a reasonable chance of discerning the lyrics by listening (the exception being Kevin Rowland from Dexies). But I decided that would be cheating. So I've mentioned it so I can discount it but at least it gets mentioned. That seems to be a standard tactic employed by previous Desert Island visitors. Instead I've opted for two regular albums instead.

Ziggy Stardust and the Spiders from Mars

By David Bowie. I just love this album. All of this album. Every single track. Every second of every track. It's hard to explain why you love an album. Perhaps its not something you should even try to explain. How can you explain how you feel about an album you love and know intimately? Other than to say you love it. And know it intimately. So I'll leave it at that.


By Radiohead. A very apt title. You'll no doubt be familiar with the haunting No Surprises which is from this album. Or maybe Karma Police - "Arrest this girl, her Hitler hairdo is making me feel ill, and we have crashed her party". What a great lyric. But neither of these is my favourite. My favourite is all 6 minutes and 23 seconds of Paranoid Android. I tweeted that I was listening to this about a week ago! In fact I've been fishing for the last two days on the River Wye so I haven't heard it for a while (I never take anything electrical with me when fishing - that would just not be right) so excuse me while I listen to it again for a moment... Rain down on me...from a great height...God loves his children...yeah...


So on to some books. Five books. I'm going to try and make choices that haven't been made by previous visitors.

World Class Match Fishing [1]

I confidently predict no one will have heard of this let alone read it. Fishing is similar to developing software in that both can involve large amounts of invisibility. But even so, unless you are keen on freshwater fishing (as I am) I don't recommend it. To explain a little (there is a point honest), match fishing is where a group of fisherman compete against each other to see who can catch the most fish (by weight) in a fixed interval of time (usually 10am - 3pm when they're the hardest to catch). Small stretches of a river (or pond/lake) are numbered and marked and the anglers draw a number to determine where to fish. Fish, like people, do not spread themselves out evenly. Quite the opposite. Consequently the vast majority of numbers (they're called swims or pegs) have no chance of winning. It's crazy really. (What has an IQ of 100? Answer 100 match fisherman! ha ha) Swims are split into sections; in a match of 60 anglers there might be 5 sections of 12 and there are smaller cash prizes for winning your section (it helps to maintain interest since, as I said, most swims have zero chance of winning). The best anglers will regularly win their sections. Kevin Ashurst was an exceptional fisherman once winning the individual World Championships. In this book he explains, often in great detail, the thinking behind his tactics and strategies when trying to win. I love it for the unselfish explanation of his secrets but even more for the insight into his clarity of thought. Good thinking is pretty rare but he had it in abundance. There are some lovely examples of the Lean principle of removing waste. To give you one example - on a river you can sometimes beat everyone in your section even if they are "better" fishermen than you simply by making sure your float is in the water longer than theirs! How can you do that? One way is simply to slow the float down!

Northern Lights [2]

For a novel I'll pick the last one I read that I could not put down once I'd started. A wonderful book, the first in His Dark Materials trilogy, it's a hypnotic mix of fantasy and reality where the fantasy is a grown-up fantasy weaving a rich picture of a parallel but definitely alternate universe with wonderful flights of imagination. The author says he prefers not to explain the meaning of what he writes instead letting the reader draw their own more personal meaning. I guess that means I shouldn't really explain the meaning I draw from it either since it might spoil your enjoyment if you decide to read it. Let's just say that the trilogy has a fairly strong and obvious athiest aspects to it. Hitch Hikers Guide to the Galaxy [3] was a close second for the novel but I've read that so many times my copy is falling apart.

The Secrets of Consulting [4]

Something closer to home for my third book. I have a lot of books by Jerry Weinberg. His most famous is probably The Psychology of Computer Programming [5] but some parts of that haven't aged particularly well. He writes that he regrets using the word psychology in its title since he is not, nor has he ever been a psychologist. Jerry may not have a certificate or official qualification but it's clear he has a great understanding of people, of the systems they are involved in, and in consulting - the art of influencing people at their request. This is my favourite Weinberg book by quite some margin and, reading between the lines, I think it is his too. I recall Kent Beck once saying he was greatly influenced by it. It's stuffed full of advice under the banner of consulting but in truth much of the book has much broader application. A lot of the book is about change which is pretty universal. It's also one of those rare books that has quality in depth. The more you read it the more you see it's hidden layers and the deeper your understanding becomes. It's well written and the advice is summarised into numerous pithy laws/aphorisms such as The Fast Food Fallacy (no difference plus no difference plus no difference plus ... eventually equals a clear difference, p.173). For a software example of that consider compiler warnings. A rare gem.

The Life of Brian [6]

It seems a shame that Desert Island visitors can't choose a couple of their favourite films. Perhaps Paul could add it as a new category? Meanwhile I'm going to have to cheat by including a film screenplay as my fourth book. Most pythonistas agree this is their finest film. The others are a bit patchy in places but every scene of Brian is a sure fire rib tickler. It's easy forget the controversy it caused when it was released. The back of the screenplay contains some reviews and, possibly uniquely, two are distinctly unfavourable! Entirely deliberate of course. I like the New Statesman's review "Hurray for blasphemy". And let's not forget the classic song "Always Look on the Bright Side of Life". That would be sage advice if you were stranded on a desert island.

The Systems Bible [11]

My last choice is is proving difficult. Lots of worthy books will miss out. Classics such as Programming Pearls [7] and The Mythical Man Month [8]. I've been reading a lot about Systems Thinking recently. Systems Thinking essentially means non-linear thinking. Human beings have evolved a very strong association that cause and effect are simple and linear; that cause and effect are local in space and time. Unfortunately the world of software is not as simple as that. Developing software takes time. Another phrase sometimes used in this context is "dynamic complexity" a term from The Fifth Discipline [9] where Peter Senge draws a useful distinction between detail complexity and dynamic complexity. The excellent An Introduction to General Systems Thinking [10] is recommended and almost got the fifth spot, but in the end I've plumped for The Systems Bible. This is a light hearted, slightly tongue in cheek book with more pithy summaries in the forms of Laws and Principles. For example Le Chatelier's Principle "Systems tend to oppose their own proper function". Another one is called The Basic Axiom of Systems-function "Big systems either work on their own on they don't. If they don't you can't make them." Very apt. It's occasionally laugh out loud funny too. Ro/Rs on page 47 for example. Well worth considering if you want to edge away from technical aspects of work for a while.