Micro refactoring - from here to there

One of my Mastery book snippets from a few days ago reads:

Masters ... are zealots of practice, connoisseurs of the small, incremental step.

I was doing the roman numerals kata in ruby and something related to small incremental steps occurred to me. Here's three tests:

def test_to_roman
  assert_equal "I", to_roman(1)
  assert_equal "II", to_roman(2)
  assert_equal "III", to_roman(3)
end

and here's the first code snippet:

def to_roman(n) 
  roman = ''
  if n == 3
    roman = 'III' 
  end
  if n == 2
    roman = 'II'
  end
  if n == 1
    roman = 'I'
  end
  roman
end

I can refactor these three if statements into a single while statement, in very small steps, as follows:

def to_roman(n) 
  roman = ''
  if n >= 3
    roman += 'I'
    n -= 1
  end
  if n >= 2
    roman += 'I'
    n -= 1
  end
  if n >= 1
    roman += 'I'
    n -= 1 
  end
  roman
end

and then:

def to_roman(n) 
  roman = ''
  if n >= 1
    roman += 'I'
    n -= 1
  end
  if n >= 1
    roman += 'I'
    n -= 1
  end
  if n >= 1
    roman += 'I'
    n -= 1 
  end
  roman
end

and then:

def to_roman(n) 
  roman = ''
  while n >= 1
    roman += 'I'
    n -= 1
  end
  roman
end

Ok. Now here's the original again:

def to_roman(n) 
  roman = ''
  if n == 3
    roman = 'III'
  end
  if n == 2
    roman = 'II'
  end
  if n == 1
    roman = 'I'
  end
  roman
end

And this time I refactor in a different direction, towards an array lookup:

def to_roman(n) 
  units = [ '', 'I', 'II', 'III' ]
  roman = ''
  if n == 3
    roman = units[3]
  end
  if n == 2
    roman = units[2]
  end
  if n == 1
    roman = units[1]
  end
  roman
end

And then:

def to_roman(n) 
  units = [ '', 'I', 'II', 'III' ]
  roman = ''
  if n == 3
    roman = units[n]
  end
  if n == 2
    roman = units[n]
  end
  if n == 1
    roman = units[n]
  end
  roman
end

And then:

def to_roman(n) 
  units = [ '', 'I', 'II', 'III' ]
  roman = ''
  if n == 3
    roman = units[n]
  end
  if n == 2
    roman = units[n]
  end
  if n == 1
    roman = units[n]
  end
  roman
  units[n]
end

And then:

def to_roman(n) 
  units = [ '', 'I', 'II', 'III' ]
  units[n]
end

In this case I can refactor from a sequence of if statements in two directions equally easy - towards a while or towards an array lookup. Ok. Now consider the situation if I was starting from a switch instead of a sequence of ifs:

def to_roman(n) 
  case n
  when 3 then 'III'
  when 2 then 'II'
  when 1 then 'I'
  end
end

From this I can easily refactor towards an array lookup but refactoring towards a while is perhaps not quite so straightforward.

I'm suggesting that a construct is useful not just in it's own right but also in relation to all the other constructs it might get refactored to or from. How easily can I move from one construct to another? Do some constructs live only on one-way roads? Do some lead you down more dead-ends than others?

I'm reminded of the family on holiday in the West Country who got thoroughly lost. Spotting a farmer leaning on a gate they stop the car, wind down the window and ask

Excuse me. Can you tell me how to get to Nempnett Thrubwell?

The old farmer looks at them and says

Well.... you don't want to start from here.

Test-gunpowder-pudding driven development

I was rereading chapter 3, Systems and Illusion in Jerry Weinberg's excellent An Introduction to General Systems Thinking yesterday. On page 56-57 Jerry writes:

If I say: "The exception proves the rule" in front of a large class, there will be a division in understanding... Some will believe I have uttered nonsense, while others will understand "The exception puts the rule to the test"

I've read the book four times. I'm a slow learner but this time something clicked and I immediately understood the earlier passage:

...the exception does not prove the rule, it teaches it.

Jerry goes on:

"Proof" in its original sense was "a test applied to substances to determine if they are of satisfactory quality."

I was struck by two thoughts when I read this. One was the parallel with testing. Of a test as a "proof". The other was the word original. I realized that when I hear the word "proof" I have a strong association with its noun meaning rather than its verb meaning. I tend to think of a proof as a finished proof that completely proves something. It's the noun-verb thing I've blogged about before. I wondered if there were any old dictionaries online so I could get a feel for how the generally accepted meaning of the word proof might have changed over time. There is. http://machaut.uchicago.edu/websters has two Webster's dictionaries. The 1828 dictionary came back with:

Proof [noun] 1. Trial; essay; experiment; any effort, process or operation that ascertains truth or fact. Thus the quality of spirit is ascertained by proof; the strength of gun-powder, ...

The 1913 one came back with:

Proof [noun] 1. Any effort, process, or operation designed to establish or discover a fact or truth; an act of testing; a test; a trial.

and a modern dictionary http://www.thefreedictionary.com/proof said:

Proof [noun]. 1. The evidence or argument that compels the mind to accept an assertion as true.

I find the difference fascinating. The 1828 and the 1913 definitions define the noun as the process whereas the modern one defines the noun as the evidence resulting from the process.

Jerry continues:

We retain this meaning in the "proofs" of printing and photography, in the "proof" of whiskey, and in "the proof of the pudding." Over the centuries, the meaning of the word "prove" began to shift, eliminating the negative possibilities to take on an additional sense: "To establish, to demonstrate, or to show truth or genuineness."

At first I didn't understand the bit about "eliminating the negative possibilities". I think it's partly to do with my ITA spelling at school. But I am persistant. Slowly it came to me. The word that did it for me in...

"Proof" in its original sense was "a test applied to substances to determine if they are of satisfactory quality."

...was the word if. To determine if they are of satisfactory quality. The proof was an act. There was the possibility of failure.

I started thinking about the word proof a bit more. I googled the phrase "the proof of the pudding". If you think this phrase is pretty meaningless then you're right - it's a shortened version of:

The proof of the pudding is in the eating.

Again it's about the possibility of failure. It reminds me of the scene in the film The Cat in the Hat (another film Patrick and I love watching) where the Cat has just made some cupcakes (with the amazing kupcake-inator). He tries one and says:

"Yeuch. They're horrible. Who want's some?"

I love that line. I also googled the word proof as related to alcohol content. The history behind the phrase is just wonderful. In the 18th century spirits were graded with gunpowder. Imagine you're buying some spirits. How would you know if an unscrupulous merchant had watered it down? You couldn't tell just by looking. What they did was pour a sample of the solution onto a pinch of gunpowder. If the wet gunpowder could still be ignited then the solution had proved itself. Don't you just love that?

So let's hear it for pudding and for spirits and for gunpowder and for tests. And for the possibility of failure.

The timeless way of building

is an excellent book by Christopher Alexander (isbn 0-19-502402-8). As usual I'm going to quote from a few pages:

A thing which has the quality without a name never fit any image exactly. What is exact is its adaptation to the forces which are in it.

The fact is that the elements themselves are patterns of relationships.

Any thing which lives can only be achieved as the end product of a process, whose force takes over and replaces the willful act of creation.

The mastery of what is made does not lie in the depths of some impenetrable ego; it lies, instead, in the simple mastery of the steps in the process, and in the definition of those steps.

How does the pattern language, which exists behind this flux, steer it, and enter into it? It hinges on the close relationship between the process of creation and the process of repair.

As in the organism, there is no sharp difference between the process of construction and the process of repair. Each process of construction helps repair some larger whole, of which it is merely a part. No thing is whole unto itself.

Within this process, every individual act of building is a process in which space gets differentiated. It is not a process of addition, in which pre-formed parts are combined to create a whole: but a process of unfolding, like the evolution of an embryo, in which the whole precedes its parts, and actually gives birth to them, by splitting.

The form of the whole, and the parts, come into being simultaneously.

This can only happen if the design is represented in an utterly fluid medium; it cannot happen in any medium where there is even the slightest resistance to change.

No building is ever perfect.

Yahtzee in ruby in cyber-dojo

I've been doing the Yahtzee kata in cyber-dojo to improve my Ruby. During the 247 increments I learned about...

• any?
• send
• [lo...hi]
• reduce
• :+
• module names start with a capital

It's also on pastie if you like a black background. You can use Cyber-Dojo to step through the diffs of every single increment!

module Yahtzee

  def self.score dice, category, n=nil
    return -1 if invalid(dice, category)
    if category == :value
      value(dice, n)
    else
      send(category, dice)
    end
  end

  def self.chance dice
    dice.reduce(:+)
  end

  def self.yahtzee dice
    dice.uniq.length == 1 ? 50 : 0
  end

  def self.value dice, n
    dice.count{|die| die == n } * n
  end

  def self.one_pair dice
    die(tally(dice).find{|n| pair?n }) * 2
  end

  def self.two_pair dice
    score_tally dice, [2,2]
  end

  def self.three_of_a_kind dice
    score_tally dice, [3]
  end

  def self.four_of_a_kind dice
    score_tally dice, [4]
  end

  def self.full_house dice
    score_tally dice, [3,2]
  end
  
  def self.small_straight dice
    dice.sort == [1,2,3,4,5] ? 15 : 0
  end

  def self.large_straight dice
    dice.sort == [2,3,4,5,6] ? 20 : 0
  end

  #- - - - - - - - - - - - - - - - - - - - - - - 

  def self.categories
    [:value, :chance, :yahtzee, :one_pair, :two_pair,
     :three_of_a_kind, :four_of_a_kind, :full_house,
     :small_straight, :large_straight
    ]
  end

  def self.invalid dice, category
    dice == nil \
      or dice.length != 5 \
        or dice.any?{|die| die < 1 or die > 6 } \
          or !categories.include? category
  end

  def self.score_tally dice, pattern
    starts?(pattern, tallies(dice)) \
      ? sum(dice, pattern.length) \
      : 0
  end

  def self.starts? short, long
    long[0...short.length] == short
  end

  def self.tallies dice
    tally(dice).map{|n| count(n) }
  end

  def self.rolls dice
    tally(dice).map{|n| die(n) }
  end

  def self.tally dice
    dice.uniq.map {|die| [dice.count(die),die] }.sort.reverse 
  end

  def self.sum dice, n
    sum2(rolls(dice)[0...n], dice)
  end

  def self.sum2 scorers, dice
    dice.select {|die| scorers.include?die }.reduce(:+)
  end

  def self.pair? n
    count(n) == 2
  end

  def self.count n
    n[0]
  end

  def self.die n
    n ? n[1] : 0
  end

end

I challenged myself to write the code in a functional style with no local variables - it's a bit 'dense' in places as a result.

Disabling professions

is an excellent book by Ivan Illich et al (isbn 0-7145-2510-3). As usual I'm going to quote from a few pages:

Finally there is is the American valuation on instrumentalism - an emphasis on doing - on doing something, almost anything when confronted with a problem. "Doing nothing in a difficult situation" was interestingly enough an item diagnostic of neuroticism on a popular American psychological test. Sometimes the emphasis on movement became so great that speed itself was emphasised - sometimes for no logical reason.

It does not strike me as inconsistent, that religion and notions of the hereafter are especially relevant when "the here" is so lousy and so short.

Man has no nature, only a history [Ortega Y Gassett]

there seems to be an absence of informal and comfortable places to gather and talk and thus a further reduction in "informal networks of help".

The business of modern society is service… Thus, service is to care which is to love and love is the universal apolitical value.

The tool defines the problem rather than the problem defining the tool.

There is no greater power than the right to define the question.

The easiest way to create a monopoly is to invent a language and procedure which will be unintelligible to the layman.

When we talk about skilled work it is not the nature of the craft, but the social relation of the worker to management that is under discussion. The full expression of the worker's skill conflicts directly with the needs of management.

One of the real satisfactions of skilled work is that, like an artist, your hands produce what your mind conceives.

The frustration of not being able to apply his skills on the job is often the motivation for doing it at home.

Working effectively with legacy code

is an excellent book by Michael Feathers (isbn 0-13-117705-2). As usual I'm going to quote from a few pages:

A few years ago, I gave my friend Erik Meade a call after I'd finished work one night. I knew Erik had just started a consulting gig with a new team, so I asked him, "How are they doing?" He said, "They're writing legacy code, man."

To me, legacy code is simply code without tests. I've gotten some grief for this definition… I have no problem defining legacy code as code without tests. It is a good working definition, and it points to a solution.

In nearly every legacy system, what the system does is more important than what it is supposed to do… Frankly, it's very important to have that knowledge of what the system actually does someplace.

When teams aren't aware of their architecture, it tends to degrade. What gets in the way of this awareness? … The brutal truth is that architecture is too important to be left exclusively to a few people.

When you have tests around the areas in which you are going to make changes, they act as a software vise [vice]. You can keep most of the behaviour fixed and know that you are changing only what you intended to.

The most subtle bugs that we can inject are bugs related to inheritance. … The language feature that gives us the most possibility for error when we lean is inheritance.

Orthogonality is a fancy word for independence… One of the startling things that you discover when you start removing duplication zealously is that designs emerge.

Sometimes it makes sense to add a variable to a class and use it to sense conditions in the method that we want to refactor.

…not all behaviours are equal in an application.

Code is pretty fragile material.

At this point in my career, I think I'm a much better programmer than I used to be, even though I know less about the details of each language I work in.

The primary purpose of a compiler is to translate source code into some other form, but in statically typed languages, you can do much more with a compiler.

Mastery

is an excellent book by George Leonard (isbn 0-452-26756-0). As usual I'm going to quote from a few pages:

You have to be willing to spend most of your time on a plateau, to keep practising even when you seem to be getting nowhere.

Our hyped-up consumerist society is engaged, in fact, in an all-out war on mastery.

Sometimes, when the moment came to go to class, I would be feeling particularly lazy. On those occasions I would be tempted to do almost anything rather than face myself once again on the mat.

Practice, the path of mastery, exists only in the present.

To see the teacher clearly, look at the students… The best teacher generally strives to point out what the student is doing right at least as frequently as what she or he is doing wrong.

A doctor practises medicine and an attorney practises law, each each of them also has a practice.

The master of any game is generally a master of practice.

The courage of a master is measured by his or her willingness to surrender. This means surrendering to your teacher and to the demands of your discipline. It also means surrendering your own hard-won proficiency from time to time in order to reach a higher or different level of proficiency.

The essence of boredom is to be found in the obsessive search for novelty. Satisfaction lies in mindful repetition, the discovery of endless richness in subtle variations on familiar themes.

For the master, surrender means there are no experts. There are only learners.

It can be argued that what is most abstract is most fundamental and often most persistent over time.

Those we know as masters are dedicated to the fundamentals of their calling. They are zealots of practice, connoisseurs of the small, incremental step. At the same time - and here's the paradox - these people, these masters, are precisely the ones who are likely to challenge previous limits.

NDC cyber-dojos

Last week Olve Maudal and I ran two cyber-dojos at the NDC conference in Oslo. The first one attracted only a few people, partly due to some confusion about where it was being held. But what it lacked in quantity it more than made up for in quality. Uncle Bob and Corey Haines seemed to invent a new style of pair programming where Uncle Bob used his left hand and Corey used his right hand! Several people posted encouraging tweets:

• @KevlinHenney: If you are at #ndc2011 today, I strongly recommend the cyber-dojo with @JonJagger and @olvemaudal this afternoon.
• @UncleBobMartin: #ndc2011 cyber-dojo was very cool. Too bad you missed it. Must not be very many programmers here
• @CoreyHaines: Terrific fun at #cyber-dojo at #ndc2010 Only 4 of us there today, you should come to the repeat on Friday after lunch!
• @occulto: Best thing today: doing a kata in #cyber-dojo with @sondreb, @unclebobmartin and @coreyhaines #ndc2011 try it on Friday!

More people attended the second one. Corey was there again, along with Michael Feathers and Jon Skeet. Once again the tweets were very encouraging.





In the evening Olve created a TurnKey Linux VirtualBox server image to make it super-easy to run your own cyber-dojo. It's less than 500MB and downloadable from http://dl.dropbox.com/u/22404698/TurnKey-CyberDojo-20110610.ova.

Mike Long has written a pdf on how to set it up.

Thankyou to Olve and Mike and to everyone who attended.

Lean on the Compiler by Hiding (C++)

My Lean on the Java Compiler by Hiding post has proved quite popular so I've been thinking about if it could also work for C++... It can. I'll use Singleton as a example again. Suppose you have a C++ class like this:

class legacy
{
public:
    legacy();
    void eg1();
    void eg2();
    ...
};

#include "legacy.hpp"
#include "singleton.hpp"

void legacy::eg1()
{
    singleton::instance()->call();
}

void legacy::eg2()
{
    singleton::instance()->call();
}

Again, you want to create a seam for singleton access. The first step is to introduce the hiding identifier in the header file:

class legacy
{
public:
    legacy();
    void eg1();
    void eg2();
    ...
private:
    class singleton;  // <-----
};

Now Lean on the Compiler. All the calls to singleton::instance() no longer compile because the global singleton class is now hidden by the local singleton class just introduced. Next, in the source file add a global variable called instance of type singleton* and refactor all occurrences of singleton::instance() to instance.

#include "legacy.hpp"
#include "singleton.hpp"

singleton * instance;

void legacy::eg1()
{
    instance->call();
}

void legacy::eg2()
{
    instance->call();
}

Then Lean on the Compiler again to make sure there are no typos. Next remove the global variable called instance from the source file and refactor the header file to this:

class singleton; // <-----

class legacy
{
public:
    legacy();
    void eg1();
    void eg2();
    ...
private:
    singleton * instance; // <-----
};

Finally, initialize the instance member in the constructor definition.

legacy::legacy()
    : instance(singleton::instance())
{
}

I haven't tried this on real legacy code either. Caveat emptor!

Lean on the Compiler by Hiding (Java)

Here's a small trick which might not be out of place in Michael Feather's excellent book. I'll use Singleton as a example. Suppose you have a class like this (Java):

public class Legacy
{
    public void eg1()
    {
        stuff(Singleton.getInstance().call());
    }
    public void eg2()
    {
        more_stuff(Singleton.getInstance().call());
    }
}

And you want to create a seam for the singleton access. The first step is to introduce two public fields:

public class Legacy
{
    public int Singleton;
    public Singleton instance;

    public void eg1()
    {
        stuff(Singleton.getInstance().call());
    }
    public void eg2()
    {
        more_stuff(Singleton.getInstance().call());
    }
}

Now Lean on the Compiler. All the calls to Singleton.getInstance() no longer compile because the Singleton class is now hidden by the int Singleton just introduced. Next refactor all occurences of Singleton.getInstance() to instance (the other identifier just introduced, you can pick any name for this one but it's type must be Singleton).

public class Legacy
{
    public int Singleton;
    public Singleton instance;

    public void eg1()
    {
        stuff(instance.call());
    }
    public void eg2()
    {
        more_stuff(instance.call());
    }
}

Then Lean on the Compiler again to make sure there's no typos. Finally refactor to this:

public class Legacy
{
    public Singleton instance = Singleton.getInstance();

    public void eg1()
    {
        stuff(instance.call());
    }
    public void eg2()
    {
        more_stuff(instance.call());
    }
}

The name lookup rules for Java allow this to work but I don't think this idea will work in C++ (for example). I haven't got time to try it now as I'm heading off to the NDC conference in Oslo. I only thought of it yesterday. I haven't tried this on real legacy code. Caveat emptor!

Update. I have a C++ version aswell.

Adapt Parameter and Preserve Signature

Working Effectively With Legacy Code by Michael Feathers (isbn 0-13-117705-2) is a great book. The very first dependency breaking technique (p326) is called Adapt Parameter. It uses this Java example:

public class ARMDispatcher
{
    public void populate(ParameterSource request) {
        String[] values
            = request.getParameters(pageStateName);
        if (values != null && values.length > 0)
        {
            marketBindings.put( 
                pageStateName + getDateStamp(),
                values[0]);
        }
        ...        
    }
    ...
}

Michael writes:

In this class, the populate method accepts an HttpServletRequest as a parameter. ... It would be great to use Extract Interface (362) to make a narrower interface that supplies only the methods we need, but we can't extract an interface from another interface. ...

and a bit later...

Adapt Parameter is one case in which we don't Preserve Signatures (312). Use extra care.

Well, here's a variation on Adapt Parameter where we do Preserve Signatures. I'll stick with the Java example, but the idea is broadly applicable...

First we create our ParameterSource interface and fill it with the signatures of the methods in HttpServletRequest that our populate method calls:

public interface ParameterSource
{
    String[] getParameters(String name);
}

then we implement the adapter for HttpServletRequest:

public class HttpServletRequestParameterSource 
    implements ParameterSource
{
    public HttpServletRequestParameterSource(HttpServletRequest request) {
        this.request = request;
    }

    public String[] getParameters(String name) {
        return request.getParameters(name);
    }

    private HttpServletRequest request;
}

Next we add an overload of populate taking a ParameterSource and implement it with an exact copy-paste:

public class ARMDispatcher
{
    public void populate(ParameterSource request) {
        String[] values
            = request.getParameters(pageStateName);
        if (values != null && values.length > 0)
        {
            marketBindings.put(
                pageStateName + getDateStamp(),
                values[0]);
        }
        ...        
    }

    public void populate(HttpServletRequest request) {
        String[] values
            = request.getParameters(pageStateName);
        if (values != null && values.length > 0)
        {
            marketBindings.put(
                pageStateName + getDateStamp(),
                values[0]);
        }
        ...        
    }
    ...
}

Now we Lean on the Compiler (315) to make sure we haven't missed any methods in HttpServletRequest that our populate method calls. Add any we missed to the ParameterSource interface. Implement them in HttpServletRequestParameterSource as one liners. When it compiles we can refactor to this:

public class ARMDispatcher
{
    public void populate(ParameterSource request) {
        String[] values
            = request.getParameters(pageStateName);
        if (values != null && values.length > 0)
        {
            marketBindings.put(
                pageStateName + getDateStamp(),
                values[0]);
        }
        ...        
    }

    public void populate(HttpServletRequest request) {
        populate(new HttpServletRequestParameterSource(request));
    }
   ...
}

And now we have a seam we can pick it at...

public class FakeParameterSource 
    implements ParameterSource
{
    public String[] values;

    public String[] getParameters(String name) {
        return values;
    }
}

Tao Te Ching

is an excellent book translated by R.L.Wing (isbn 0-7225-3491-4). As usual I'm going to quote from a few pages

In trying to understand a work like the Tao Te Ching, it is important to keep in mind that Chinese characters are not so much representations of words as they are symbols of ideas.

Because of its idea-embedded nature, the Tao Te Ching is a work that brings truth to the adage: It is better to read one book one-hundred times than one-hundred books one time.

True power is the ability to influence and change the world while living a simple, intelligent, and experientially rich existence.

Simplicity in conduct, in beliefs, and in environment brings an individual very close to the truth of reality.

When expectations are dropped, the mind expands, and reality expands along with the mind.

Nothing exists without the presence of its opposite.

Practice non-interference.

A house filled with riches cannot be defended.

Individuals who master themselves become less egocentric

Evolved Individuals strive to be intuitive, spontaneous, and simple.

Never fall completely into step with current society.

Spot the Tao Te Ching Bug

Most mornings I read one of the 81 entries in Tao Te Ching . My copy is called The Tao of Power, translated by R.L.Wing. Highly recommended. I use dice together with the chart in the picture (from page 19) to make a random selection. This morning I happened to notice that I never seem to get certain numbers. Closer inspection of the chart shows why...

Pride and revulsion

Here's a bug that bit me recently. I wrote a very small quick and dirty TDD header for C in my CyberDojo. It has a macro called CHECK_EQ which you use as follows:

CHECK_EQ(int, 42, 9*6);

Part of the macro looks like this:

#define CHECK_EQ(type, expected, actual) \
    ... \
    type e = expected; \
    type a = actual; \
    ... \

Can you see the problem? Well, suppose it's used like this:

    int e = 42;
    CHECK_EQ(int, e, 9*6);

Can you see it now? The problem is that this line in the macro

    type e = expected; \

gets expanded into this line:

    type e = e; \

Ooops. I thought about choosing a very cryptic name instead of x. Perhaps incorporating __LINE__ as a suffix. But I felt there had to be a solution that would always work. After some thought I came up with this. Brace yourself...

#define CHECK_EQ(type, expected, actual) \
   ...
   if (#expected[0] != 'x') \
   { \
       type x0 = expected; \
       type x1 = actual; \
       ... \
   } \
   else \
   { \
       type y0 = expected; \
       type y1 = actual; \
       ... \
   } \
   ...

Like Tom Duff, I feel a mixture of pride and revulsion at this discovery.

The war of art

is an excellent book by Steven Pressfield (isbn 0-446-69143-7). As usual I'm going to quote from a few pages:

Every sun casts a shadow, and genius's shadow is Resistance.

The enemy is a very good teacher [Dalai Lama]

The human being isn't wired to function as an individual.

The truly free individual is free only to the extent of his own self-mastery.

Rationalization is Resistance's spin doctor.

Nothing is as empowering as real-world validation, even if it's for failure.

That's when I realized I had become a pro. I had not yet had a success. But I had had a real failure.

The professional knows that by toiling beside the front door of technique, he leaves room for genius to enter by the back.

Whatever you can do, or dream you can, begin it. Boldness has genius, magic, and power in it. Begin it now. [Goethe]

The Self wishes to create, to evolve. The Ego likes things just the way they are.

We humans seem to have been wired by our evolutionary past to function most comfortably in a tribe of twenty to, say, eight hundred.

Kluge

is an excellent book by Gary Marcus (isbn 978-0-571-23652-7). As usual I'm going to quote from a few pages:

The best science, like the best engineering, often comes from understanding not just how things are, but how else they could have been.

One molecule of DNA polymerase does its job in a perfectly straight-forward fashion, but the other does so in a back-and-forth, herky-jerky way that would drive any rational engineer insane. Nature is prone to making kluges because it doesn't "care" whether its products are perfect or elegant. If something works, it spreads. If it doesn't, it dies out.

What can evolve at any given point in time is heavily constrained by what has evolved before.

The vast majority of our genetic material evolved in the context of creatures who didn't have language, didn't have culture, and didn't reason deliberately.

What is recent is rarely fully debugged.

Our memory is organized to focus primarily on our own experiences.

Other studies have showed that people are more likely to accept falsehoods if they are distracted or put under time pressure.

Organisms tend to value the present far more than the future.

That which is clumsy is rarely reliable.

In physics, they have laws; in biology, we have gadgets [Francis Crick]

The value of imperfections extends far beyond simple balance, however. Scientifically, every kluge contains a clue to our past; wherever there is a cumbersome solution, there is insight into how nature layered our brain together; it is no exaggeration to say that the history of evolution is a history of overlaid technologies, and kluges help expose the seams.

How do you make toast?

I'm doing two days consultancy in Cornwall. Yesterday for the folks at Research Instruments in Falmouth and today for the folks at Absolute Software in Redruth. Both are really great places to work and it's a joy visiting them.

I'm staying at the Penventon hotel. My usual breakfast routine is tea, porridge, and toast. They have a big silver toaster. You put your slices of bread into the front, it pulls them in slowly, applies a lot of heat, and then drops them down a shute.

Yesterday they came out mostly black.

I tried scraping them with a knife. I won't bother next time. They never taste good when you do that. All I did was create black dust for someone to waste time cleaning up. They didn't taste good. I didn't eat them.

I was reminded of the joke:

How do you make toast?

You burn bread and scrape the burn off.

Today I put my toast in the same as before. The toaster slowly pulled them in, applied heat, and dropped them down the chute. The slices were under toasted this time. So I put the slices in again. The toaster pulled them in again, applied heat again, and dropped them down the chute again. Just right. No knives. No scraping. No black dust. No cleaning up. I added marmalade. I ate them. Lovely.

Free

is an excellent book by Chris Anderson (isbn 978-1-9052-1148-7). As usual I'm going to quote from a few pages:

The internet is the first distribution system in history that is as well suited for the niche as for the mass, for the obscure as well as the mainstream.

Just as Moore's Law dictates that a unit of computer processing power halves in price every two years, the price of bandwidth and storage is dropping even faster.

For most of human history manure has determined how much food we had... Historians often look at the great civilizations of the ancient world through the lens of three grains: rice, wheat, and corn. Rice is protein rich but extremely hard to grow. Wheat is easy to grow but protein poor. Only corn is both easy to grow and plump with protein.

Humans are wired to understand scarcity better than abundance.

At some point in your life, you may wake up and realize that you have more money than time.

What [Alan] Kay realized was that a technologist's job is not to figure out what technology is good for. Instead it is to make technology so cheap, easy to use, and ubiquitous that anybody can use it.

Abundant information wants to be free. Scarce information wants to be expensive.

Paradoxes are the opposite of contradictions. Contradictions shut themselves down, but paradoxes keep themselves going, because every time you acknowledge the truth of one side you're going to get caught from behind by the truth on the other side.

Information is what British anthropologist Gregory Bateson described as "a difference that makes a difference."

The electricity consumed by a server now costs more over the life of that server than the server itself.

Rather than depriving life of purpose, material abundance created a scarcity of meaning.

Bounce

is an excellent book by Matthew Syed, subtitled The myth of talent and the power of practice (isbn 978-0-00-73505404). As usual I'm going to quote from a few pages:

Ericsson also found that there were no exceptions to this pattern: nobody who had reached the elite group without copious practice, and nobody who had worked their socks off but failed to excel.

It is the quality and quantity of practice, not genes, that is driving progress.

The ascendency of the mental and the acquired over the physical and the innate has been confirmed again and again.

My dad never asked me to play golf. I asked him. [Tiger Woods]

Child prodigies do not have unusual genes; they have unusual upbringings.

Top skaters fall over more during their practice sessions.

But while the adaptability of the human body is impressive, it is the plasticity of the brain that has astonished researchers.

If you don't know what you are doing wrong, you can never know what you are doing right.

These were some of the clearest findings I've ever seen. Praising children's intelligence harms their motivation, and it harms their performance. [Carol Dweck]

Lowering standards just leads to poorly educated students who feel entitled to easy work and lavish praise. [Carol Dweck]

Many of the contemporaries of Galileo (inventor of the modern telescope) really did think there was something morally dubious about the telescope; that it was taking humanity beyond the powers expressly sanctioned by God.

JavaScript: The Good Parts

is an excellent book by Douglas Crockford (isbn 978-0-596-51774-8). As usual I'm going to quote from a few pages:

This is not a book for beginners... This book is small but it is dense. There is a lot of material packed into it. Don't be discouraged if it takes multiple readings to get it. Your efforts will be rewarded.

JavaScript's popularity is almost completely independent of its qualities as a programming language.

JavaScript is the first lambda language to go mainstream. Deep down, JavaScript has more in common with Lisp and Scheme than with Java.

strong typing does not eliminate the need for careful testing.

despite its deficiencies, JavaScript is really good.

Unlike many other languages, blocks in JavaScript do not create a new scope, so variables should be defined at the top of the function, not in blocks.

An object is a container of properties, where a property has a name and a value. A property name can be any string, including the empty string. A property value can be any JavaScript value except for undefined.

An inner function also enjoys access to the parameters and variables of the functions it is nested within... This is called closure. This is the source of enormous expressive power.

That act of nothingness gives us confidence that the function does not recurse forever.

What matters about an object is what it can do, not what it is descended from. JavaScript provides a much richer set of code reuse patterns.

Much of the complexity of class hierarchies is motivated by the constraints of static type checking.

var memoizer = function(memo, formula) {
    var recur = function(n) {
        var result = memo[n];
        if (typeof result !== 'number') {
            result = formula(recur, n);
            memo[n] = result;
        }
        return result;
    };
    return recur;
};

var fibonacci = memoizer([0, 1], function(recur, n) {
    return recur(n - 1) + recur(n - 2);
});

Simple and Usable

is an excellent book by Giles Colborne (isbn 0-321-70354-5). As usual I'm going to quote from a few pages:

If you ask people they'll say everything is important and anything is feasible.

We tend to keep things, even when they're broken.

Your first design may seem like a solution, but it's usually just an early definition of the problem you are trying to solve. [Luke Wroblewski]

Broken gets fixed. Shoddy lasts forever. [Jack Moffett]

Feature lists sell so as long customers don't get a chance to use the product.

Mainstreamers want "good enough quickly;" experts want "perfect in as long as it takes."

People prefer to be pilots, not passengers.

"Seven plus or minus two." Many psychologists now believe short-term memory may be rather smaller - perhaps just four items.

Simple organization is about what feels good as you're using the software, not what looks logical in a plan.

Designing simple user experiences often turns out not to be about "How can I make this simple?" but rather "Where should I move the complexity?"

The secret to creating a simple user experience is to shift complexity into the right place, so that each moment feels simple.

Don't try to fill your user's mind with your design.

Nudge

is an excellent book by Richard Thaler and Cass Sunstein (isbn 978-0-141-04001-1). As usual I'm going to quote from a few pages:

School children, like adults, can be greatly influenced by small changes in the context.

There is no such things as a 'neutral design.'

Roughly speaking, losing something makes you twice as miserable as gaining the same same thing makes you happy. In more technical language, people are 'loss averse'... Loss aversion helps produce inertia.

Most teachers know that students tend to sit in the same seats in class, even without a seating chart.

Eating turns out to be one of the most mindless activities we do. Many of us simply eat what is put in front of us.

Social scientists generally find less conformity, in the same basic circumstances as Asch's experiments, when people are asked to give anonymous answers.

On average, those who eat with one other person eat about 35 percent more than they do when they are alone; members of a group of four eat about 75 percent more; those in groups of seven or more eat 96 percent more.

Self-control issues are most likely to arise when choices and their consequences are separated in time.

Even hard problems become easier with practice.

The best way to help Humans improve their performance is to provide feedback.

For most of their time on earth, Humans did not have to worry much about saving for retirement, because most people did not live long enough to have much of a retirement period.

public: footpath

You know you know C++ when you see a sign like this and you think, "there's a colon missing after public."

rules of thumb

don't rush

I don't think I can put it any better than Jerry Weinberg did when I interviewed him:

Things take the time they take, not the time you hope they will take. Pushing for half-time produces half-baked.

As a self-employed software coach/consultant I get to travel a lot and visit a lot of companies. At the best companies there is a palpable sense of not rushing.

think team

Software development is all about collaborative learning. I think one of the weakest points of the Agile Manifesto is it's lack of emphasis on teams. The very first word of the four "X over Y" statements is Individuals :-( XP at least takes a firm step beyond programming as an individual activity by mandating pair programming. I look at Sit Together, a practice from XP1 and I think s/Sit/X/. In other words, whatever X is, X Together.

increase visibility

Software and the process of developing it can be, to paraphrase Douglas Adams, mostly invisible. You think more clearly when you have something concrete to tie your thinking to. You manage things better when you can see them and see them constantly changing. It's no accident that Kanban boards are as popular as they are.

Things are the way they are because they got that way

If you're working on a complex codebase and you're trying to understand the complexity by looking at the codebase then you're looking in the wrong place. That's like the man in Peopleware who loses his keys in a dark street and looks for them in the adjacent street because, as he explains, "the light is better there". A codebase is the way it is because it got that way. Slowly. Over time. If you're looking at the code your looking at the effect and not at the cause. It was the developers that did it!

Things are the way they are because they got that way.

Increase visibility

My sister Alix and I were chatting about work-related stupidity we'd imagined and experienced. Strict quotas on how much time you're allowed for going to the toilet for example. Naturally the time-allowed for no 2's would be longer than the time allowed for no 1s. But how would the managers police the quotas? The answer, we realized, was specialist plumbing. Take out the old copper pipes and put in clear perspex pipes instead. Plumb them through the managers office, at eye level. Purpose built hardware probes, available at reasonable cost, would measure the volume of shit. The probes would wirelessly connect to ShitLogger (tm) an app for monitoring the volume of shit against the quotas. To log the logs, as it were.

Recency

I've written before that:

Human beings have evolved a very strong association that cause and effect are simple and linear; that cause and effect are local in space and time.

One of example of this is when contestants on Who Wants To Be a Millionaire phone a friend. It's easy to think that the friend could simply watch the program and start trying to find the answer before they're called. They can't. What you're watching happened some time in the past. The program is not live.

Another example is this blog! At the recent ACCU conference several people commented on what a fast reader I was. I often blog three or four book snippets in a week. And I am learning to increase my reading speed. And I do read a lot. But it's an illusion. I have a stock of hundreds of books I've already read and all the best bits from them are already marked. I simply take a book from my shelf and blog the best of its best bits as a book snippet. And then put the book onto the ACCU charity book-stall pile.

Bad captcha

I had this yesterday. I doubt any human could read that! As my friend Niklas Bjornerstedt tweeted

captchas are getting so hard to read that only robots will be able to solve them soon

Accu conference charity book stall

Once again a big thanks to everyone at the accu conference who took a book and made a contribution to the two charities - Water Aid and The National Autism Society this year. The total raised was a smidgen under £600 which will be split equally. Thanks also to my son Patrick (he has Aspergers Syndrome) for lending me the safe. And thankyou too to several people who brought some books for the stall during the conference. I plan to repeat the stall again in 2012.

Deliberate practice booklist

Kevlin and I have just finished the accu pre-conference tutorial. A big thankyou to all the participants. The origin of some of the quotes was a little hard to read as the printed handouts were a little small. Here is a list of the books quoted:

• The Fifth Discipline by Peter Senge
• Talent is Overrated by Geoff Colvin
• Surfing the edge of Chaos by Richard Pascale
• Quality Software Management Vol 1: Systems Thinking by Jerry Weinberg
• Authentic Happiness by Martin Seligman
• Implementing Lean Software Development by Mary and Tom Poppendeick
• Switch: How to change things when change is hard by Chip and Dan Heath
• Mindset by Carol Dweck
• The Princess Bride by Bill Goldman
• The Autobiography of Martin Luther King Jr edited by Clayborne Carson

discipline

Back to quotes table-of-contents

From Extreme Programming Explained

XP is a communal software development discipline.

From The Way of the Leader

A person who practices self-discipline and continuously develops his level of skill seldom fails in the long term.

Discipline is not intended to kill character, enthusiasm, and initiative, but to develop them.

From The Road Less Travelled and Beyond

The essence of this discipline of balancing is unlearning and "giving up" something in ourselves in order to consider new information.

All discipline is a form of submission.

From Zen Soup

Discipline is simply a matter of doing what we must, without wasting time or energy worrying whether or not we feel like it.

From The Long Walk to Freedom

I could compensate for lack of natural aptitude with diligence and discipline. I applied this to everything I did.

From Gandhi an Autobiography

Experience has taught me that silence is part of the spiritual discipline of a votary of truth.

From Good to Great

The purpose of bureaucracy is to compensate for incompetence and lack of discipline. Avoid bureaucracy and instead create a culture of discipline.

Feedback

Some feedback quotes from previous blog entries: From Agile development in the large

Quick feedback should be the first thing you introduce.

From What did you say?

We structure our world so we will not receive feedback that threatens our view.

We don't even wait to ignore feedback, but actively take steps to prevent such feedback from ever happening in the first place.

Don't concentrate on giving feedback; concentrate on being congruent - responding to the other person, to yourself, and to the here-and-now situation.

From The dance of life

All societies depend for the stability on feedback from the people. Depersonalization reduces feedback to a minimum, contributing to instability and lowering the overall level of congruence in the society.

From Understanding the professional programmer

Many programmers… work in environments in which they receive essentially no real feedback embodying the consequences of what they do. Lacking no real feedback, they lack the motivation to attempt changes, and they also lack the information needed to make the correct changes.

From Quality Software Management. Vol 2. First-Order Measurement

Software development is not primarily a manufacturing operation for we (ideally) never develop the same software twice. This uniqueness of product means that Deming's "statistical signal" - though necessary - is not sufficient for feedback control, because there often isn't enough repetition - enough stability - to generate meaningful statistics.

From Quality Software Management. Vol 4. Anticipating Change

In a feedback control system it's only our perception that determines which is controller and which is controllee.

From The systems bible

Just calling it "feedback" doesn't mean that it has actually fed back. To speak precisely: It hasn't fed back until the system changes course. Up until that point it's merely sensory input.

From Talent is overrated

Practicing without feedback is like bowling through a curtain that hangs down to knee level. You can work on technique all you like, but if you can't see the effects, two things will happen: You won't get any better, and you'll stop caring.

From the Principles of product development FLOW

Fast local feedback loops prevent the accumulation of variance.

From the Aesthetics of change

All simple and complex regulation as well as learning involve feedback. Contexts of learning and change are therefore principally concerned with altering or establishing feedback.

Context

A road sign. Up a tree. In the middle of a park.

Mozart a biography

is an excellent book by Piero Melograni (isbn 0-226-51961-9). As usual I'm going to quote from a few pages:

Mozart was great (among other reasons) because he knew how to have fun.

Some people believe that music flowed from him almost spontaneously, thanks to his genius. In reality, from earliest childhood he practiced for thousands of hours every year.

He lived only thirty-five years, but he lived them at a wolf's pace and went far in that short time.

In a letter dated 20 August 1763, Leopold Mozart relates that in many places in Germany the well water was so bad, smelly, and muddy that it was habitually mixed with wine. It was worse in Paris. Parisians drank the repugnant water of the Seine, into which the city's garbage was thrown. The Mozarts, like many others, let it settle in a pitcher for a few hours, where it formed a worrisome solid layer.

In a letter dated 1 April 1764 Leopold reported that when an eclipse of the sun had occurred, Parisians rushed into the churches to protect themselves from being poisoned by the air during the temporary disappearance of the sun's light.

Wolfgang fell seriously ill with smallpox. He was blind for nine days and hovered between life and death, for the second time, after his bout with typhus in 1765.

Mozart performed several times on the harpsichord, astonishing his listeners with the agility of his hands, his left hand in particular. Some Neapolitans, perhaps influenced by a culture that tended to superstition and like mysteries. asserted that the boy played as well as he did thanks to a magic ring that he wore. When they demanded that he take off the ring and play without it, they saw that his playing depended on talent rather than spells, and they applauded all the more.

In those days, a composer would not dream of writing the arias for an opera without consulting the singers who were to interpret them… composers were craftsmen, paid by the piece, and were completely subservient to the true superstars of the age, who were the singers.

In those days no one hesitated about applauding in the middle of a work of music.

His fingers were deformed, either because of continual keyboard exercises from childhood on or from arthritis.

Only artists capable of innovation can give a long life to their works. Innovation prompts tension, curiosity, and awe.

complexity

Back to quotes table-of-contents

From The Way of the Leader

Do the essential things well: Be proactive (do through action), Reduce complexity (concentrate effort on the essential things), Seek improvement (get the essential things done better).

From Everyday Heroes of the Quality Movement

Automating complexity is never as effective as removing it.

If you automate without first getting rid of complexity, you cast the complexity in concrete.

From Simplicity

The human brain is a very simple system that is capable of working in a complex way, rather than a complex system.

From Patterns of Software

In the modern era, we have come to favor simplicity over complexity, perfection over imperfection, symmetry over asymmetry, planning over piecemeal growth, and design awards over habitability.

From The Gift of Time

Complexity isn't an attribute; it's a relationship.

From General Principles of System Design

Complexity is a relationship between system and observer.

From The End of Certainty

A nonequilibrium system may evolve spontaneously to a state of increased complexity.

From The Systems Bible

A complex system that works is invariably found to have evolved from a simple system that worked.

From Surfing the Edge of Chaos

Recent study of evolution, both in the natural world and in computer based complex systems, has demonstrated the surprising result that the presence of parasites in a system accelerates evolution dramatically.

From Consilience

Complexity is what interests scientists in the end, not simplicity.

From Adapt - why success always starts with failure

Complexity is a problem only in tightly coupled systems.

From General Principles of Systems Design

There is a tendency for complexity in models to rise as the time between sensing and acting grows.

Photo by Kevin Wong.

Effective leadership masterclass

is an excellent book by John Adair (isbn 0-330-34785-3). As usual I'm going to quote from a few pages:

Every person and thing is only what it is in relation to others. [Lao Tzu]

It is this quality of doing things spontaneously and in an unselfconscious way, without regard to their effects upon other people's perceptions of oneself.

The natural badge of such inner humility towards all things is silence.

I believe the first test of a truly great man is his humility. [John Ruskin]

I cannot hear what you are saying because you are shouting at me. [Zulu proverb]

So many people are loath to make irrevocable decisions, are tepid in their enthusiasms. [Ordway Tead]

I do not say that the men of the 14th Army welcomed difficulties, but they grew to take a fierce pride in overcoming them by determination and ingenuity. [General William Slim]

Change and leadership are closely linked.

Leadership is of the spirit, compounded of personality and vision; its practice is an art. Management is of the mind, more a matter of accurate calculation of statistics, of methods, of time tables, and routine; its practice is a science. [General William Slim]

Leadership is bound up with culture.

As a natural leader, he [Gandhi] led by example - spinning for at least an hour every day.

We'll never survive!

One of the books for the accu charity book stall is my copy of Extreme Programming Explained by Kent Beck (1st edition).
One of the entries in its bibliography is a quote from The Princess Bride.
Buttercup and Westley are about to enter the Fire Swamp and face it's three terrors: the Flame Spurts, the Lightning Sand, and the Rodents Of Unusual Size (R.O.U.S.'s).







Buttercup says to Westley (Kent misquotes slightly here):

We'll never survive

and Westley replies:

Nonsense - you're only saying that because no one ever has.

I just love this line. And I love Kent's idea of putting a film quote in the bibliography. I love that it's in the bibliography in a section called "Attitude". I think Kent is hinting at their courage - that depending on your attitude life can be an adventure. After surviving the flame spurts Westley says to Buttercup:

Well now, that was an adventure.

And a bit further on he says:

The Fire Swap certainly does keep you on your toes.

Just before facing the R.O.U.S.'s Buttercup says:

We'll never survive - we may as well die here.

To which Westley replies:

No. No. We have already succeeded.

I love that line too. (I pretty much love every line of the film.) Again it's about attitude. As they make it out of the fire swamp Buttercup says (almost in disbelief):

We did it.

And Westley says:

Now, was that so terrible?

Attitude.
Caring about yourself.
Caring about others.
Caring about the code.