XP and culture change

Last week, at a clients site, I noticed an old, coffee-stained copy of the Cutter IT Journal. It was titled "XP and culture change", dated September 2002. Here are some quotes from it.

From Kent Beck:

Because culture embodies perception and action together, changing culture is difficult and prone to backsliding.

Is it easier to change your perception or go back to designing the old way?

From Laurent Bossavit:

A process change will always involve a cultural change.

We were also a culture of Conviviality, which you could easily mistake (as I did at first) for a culture of Communication... In Conviviality what is valued is the act of sharing information in a group setting - rather than the nature, quantity, or quality of the information thus shared.

Culture is what remains when you have forgotten everything else.

From Mary Poppendieck and Ron Moriscato:

If there were one thing that Ron's team would do differently next time, it would be to do more refactoring.

XP is a process that doesn't feel like a process.

The theory of punctuated equilibrium holds that biological species are not likely to change over a long period of time because mutations are usually swamped by the genes of the existing population. If a mutation occurs in an isolated spot away from the main population, it has a greater chance of surviving.

From Ken Schwaber:

Agile process management represents a profound shift in the development of products and software. Agile is based on an intuitive feel of what is right, springs from a completely different theoretical basis than traditional development processes, and is in sum a wholly different approach to building products in complex situations.

From Matt Simons and Chaitanya Nadkarny

A fixed-bid contract changes the very nature of the relationship between customer and vendor from collaborative to "contentious". "Embrace change" undergoes a fatal transformation into "outlaw change."

There is no way to pretend everything is fine when you have to deliver software to your customer every few weeks.

From Nancy Van Schooenderwoert and Ron Moriscato:

The advantages of pair programming hit you hard and fast. As you explain an area of code to your partner, you get a deeper understanding of how it fits into the current architecture. You're your own peer reviewer!

After pair programming for a while, we found ourselves in a situation where the entire team had worked somewhere in the module in the recent past. Code reviews became exciting idea-exchanging periods where refactoring tasks were discussed and planned.

With schedule pressure, there is a huge temptation to put off refactoring, and we did too much of that.

It's not enough for the code to work; it also has to serve as a solid base for the next wave of features that will be added.

All through the project, a frequent cause was that unit testing wasn't thorough enough.


management of the absurd

is an excellent book by Richard Farson, subtitled Paradoxes in Leadership (isbn 0-684-83044-2). As usual I'm going to quote from a few pages:
The more important a relationship, the less skill matters.
Any technique loses its power when it becomes evident that it is a technique.
People need to know they are dealing with a genuine person, not someone who is "managing" them.
It is only when the balance of power is relatively equal that truly candid communication can and should take place.
When we really listen, so that we understand the other person's perspective, we risk being changed ourselves.
Every management act in some way redistributes or reinforces power.
Ex-convicts are better able to rehabilitate prison inmates than is the prison staff. Ex-drug addicts are more successful in getting other addicts off drugs than are psychiatrists. Students learn more from each other than they do from their professors.
The introduction of highly participative systems tends to bring attacks on the stronger members, often the leaders, while more hierarchical systems bring attacks on the weaker members.
The way to judge your effectiveness is to assess the quality of the discontent you engender.
Scale is the enemy of creativity... Only prisons housing fewer than twenty inmates are likely to be rehabilitative.
The big change... held; the little ones have been much easier to resist.
We learn not from our failures but from our successes - and the failures of others.
By and large, organizations are simply not good at changing themselves. They change more often as a result of invasion from the outside or rebellion from the inside, less so as a result of planning.
Planning may not be effective at assessing the future, but it can be a good way to assess the present.
Strengths and weaknesses come dressed in the same clothing.
Children look at things we turn away from. Sometimes just pointing at what is going on is a valuable way to break through a barrier.
When people feel responsible for handling some situation in which they are, in fact, largely helpless, a dangerous combination of feelings is created: responsibility plus helplessness leads to abuse.
Training makes people more alike... Education... tends to make people different from each other.

testing legacy C/C++ when it resists

In my travels I'm sometimes asked to help a client write some unit-tests for C/C++ code which wasn't built with unit-testing in mind and so, naturally, is resisting being unit-tested. This is a classic chicken and egg situation; you want to refactor the code to get the unit-tests in place, but of course that's dangerous and painful and slow until you've got at least some unit tests in place. There are two big problems:
  • Repaying the legacy debt is likely to be a long and arduous road. There's not a lot I can do to help here except offer encouragement and to maybe remind them of the Winston Churchill quote
    if you're going through hell, keep going!
  • It often seems there's no way to get started. Clients might say something like "this can't be unit tested" when of course what they really mean is "I don't know how to unit test this". Sometimes I can suggest tricks and techniques.


One way to get started is to make the problem smaller. Suppose I have a large legacy C++ class resolutely resisting being unit-tested. I pick a method and start with that. For example, given this file, fubar.cpp

   1|#include "fubar.hpp"
   2|#include ...
   3|#include ...
    |...
1438|int fubar::f1() const
1439|{
    |   ...
1452|}
1453|
1457|void fubar::example(widget & w, int x)
1458|{
    |   ...
1598|}
1599|
1600|int fubar::f2()
1601|{
    |   ...
4561|}
4562|
I decide to start with fubar::example() which starts at line 1457 of fubar.cpp and ends 100+ lines later: I carefully cut all of lines 1457-1598 into its own new file called fubar-example
   1|
   2|void fubar::example(widget & w, int x)
   3|{
    |   ...
 141|}
 142|
and replace the cut lines from fubar.cpp with a single #include to the new file:
   1|#include "fubar.hpp"
   2|#include ...
   3|#include ...
    |...
1438|int fubar::f1() const
1439|{
    |   ...
1452|}
1453|
1457|#include "fubar-example" // <----
1458|
1459|int fubar::f2()
1460|{
    |   ...
4420|}
4421|
I'm aiming to create a unit-test for fubar::example() like this:
// here I'll dummy out everything used in fubar::example

#include "fubar-example"

// here I'll write my first unit test
However, as safe as it seems, this could cause a change in behaviour! I can easily check this. If fubar.cpp is one of the source files that compiles into something.lib then I can compare the 'before' and 'after' versions of this lib file to see if they are identical. They should be. One reason they might not be is because of things like the assert macro which uses __FILE__ and __LINE__ to report the filename and line-number. I've changed the line numbers on everything below the new #include and the lines numbers and filename inside the included file. I can fix that using the #line directive.

In the original fubar.cpp file example() started at line 1457 so fubar-example becomes:
   1|
   2|...
   3|
   4|#line 1457 "fubar.cpp"  // <----
   5|void fubar::example(widget & w, int x)
   6|{
    |   ...
 145|}
 146|
and the next method f2() started at line 1600 so fubar.cpp becomes:
   1|#include "fubar.hpp"
   2|#include ...
   3|#include ...
    |...
1438|int fubar::f1() const
1439|{
    |   ...
1452|}
1453|
1457|#include "fubar-example" 
1458|
1459|#line 1600 // <----
1460|int fubar::f2()
1461|{
    |   ...
4421|}
4422|
Now the before and after versions of the lib file are identical. Now I try to compile the test file:
// here I'll dummy out everything used in fubar::example

#include "fubar-example"

// here I'll write my first unit test
It fails to compile of course, since I can't define fubar::example() unless I've previously declared it. So I dummy it out:
class fubar
{
public:
    void example(widget & w, int x); // <----
};

#include "fubar-example"

// here I'll write my first unit test
Now it fails because the compiler doesn't know what widget is. So I forward declare it:
class widget; // <----

class fubar
{
public:
    void example(widget & w, int x);
};

#include "fubar-example"

// here I'll write my first unit test
Now it fails because fubar::example() calls a method nudge(int,int) on the widget parameter:
   1|
   2|...
   3|#line 1457 "fubar.cpp" 
   4|void fubar::example(widget & w, int x)
   5|{
    |   ...
    |   w.nudge(10,10); 
    |   ...
 145|}
 146|
So I dummy it out:
class widget
{
public:
    void nudge(int,int) // <----  
    {
    }
};

class fubar
{
public:
    void example(widget & w, int x);
};

#include "fubar-example"

// here I'll write my first unit test
Now it fails because fubar::example() invokes a macro LOG:
   1|
   2|...
   3|#line 1457 "fubar.cpp" 
   4|void fubar::example(widget & w, int x)
   5|{
    |   ...
    |   LOG(... , ...);
    |   ...
 145|}
 146|
So I dummy it out:
#define LOG(where,what)  /*nothing*/  // <----

class widget ...

class fubar
{
public:
    void example(widget & w, int x);
};

#include "fubar-example"

// here I'll write my first unit test
Maybe later I can return to the dummy LOG macro and make it less dumb but for now I'm not even compiling. One thing at a time.

Now it fails because fubar::example() declares a local std::string:
   1|
   2|...
   3|#line 1457 "fubar.cpp" 
   4|void fubar::example(widget & w, int x)
   5|{
    |   ...
    |   std::string name = "...";
    |   ...
 145|}
 146|
This one I don't need to dummy out.
#include <string>   // <----

#define LOG(where,what)  /*nothing*/

class widget ...

class fubar
{
public:
    void example(widget & w, int x);
};

#include "fubar-example"

// here I'll write my first unit test
Now it fails because fubar::example() calls a sibling method:
   1|
   2|...
   3|#line 1457 "fubar.cpp" 
   4|void fubar::example(widget & w, int x)
   5|{
    |   ...
    |   if (tweedle_dee(w))
    |   ...
 145|}
 146|
So I dummy it out:
#include <string>

#define LOG(where,what)  /*nothing*/

class widget ...

class fubar
{
public:
    void example(widget & w, int x);

    bool tweedle_dee(widget &) // <----
    {
        return false;
    }
};

#include "fubar-example"

// here I'll write my first unit test
Now it fails because fubar::example() makes a call on one of its data members:
   1|
   2|...
   3|#line 1457 "fubar.cpp" 
   4|void fubar::example(widget & w, int x)
   5|{
    |   ...
    |   address_->resolve(name.begin(), name.end());
    |   ...
 145|}
 146|
So I dummy it out, making no attempt to write the actual types of the parameters (a useful trick):
#include <string>

#define LOG(where,what)  /*nothing*/

class widget ...

class address_type
{
public:
    template<typename iterator>
    void resolve(iterator, iterator) // <----
    {
    }
};

class fubar
{
public:
    void example(widget & w, int x);

    bool tweedle_dee(widget &) 
    {
        return false;
    }

    address_type * address_; // <----
};

#include "fubar-example"

// here I'll write my first unit test
On I go, one step at a time, until finally, it compiles! Hoorah!

I'm reminded of a saying I heard (from Michael Stal). It's when there's a library you pull in but, on pulling it in, you find it has two further dependencies and you have to pull in those aswell. And they have their dependencies too. etc etc. The saying is:

you reach for the banana; you get the whole gorilla!

Except that sometimes it's worse than that. Sometimes...

you reach for the banana; you get the whole jungle!


Ok. So now it compiles. But I haven't written my first unit-test yet! So I start that:
#include <string>

#define LOG(where,what)  /*nothing*/

class widget ...

class address_type ...

class fubar
{
public:
    void example(widget & w, int x);

    bool tweedle_dee(widget &) 
    {
        return false;
    }

    address_type * address_; 
};

#include "fubar-example"

int main()
{
    fubar f;
    widget w;
    f.example(w, 42); // <----
}
Now I have a test I can actually run! Hoorah! There's no actual assertions yet, but one thing at a time. I run it. It crashes of course. The problem could be the address_ data member. The compiler generated default constructor doesn't set it so it's a random pointer. I might be able to fix that by repeating the same extraction of the constructor(s) into separate #included files. Ultimately that's what I want of course. But one thing at a time. I can definitely fix it by writing my own constructor:
#include <string>

#define LOG(where,what)  /*nothing*/

class widget ...

class address_type ...

class fubar
{
public:
    explicit fubar(address_type * address) // <----
        : address_(address)
    {
    }

    void example(widget & w, int x);

    bool tweedle_dee(widget &) 
    {
        return false;
    }

    address_type * address_; 
};

#include "fubar-example"

int main()
{
    address_type where; // <----
    fubar f(&where); // <----
    widget w;
    f.example(w, 42);
}
Now it compiles and runs without crashing! Hoorah! It is a horrible hack. Painful. But the gorilla is no longer so invisible! And if nothing else, I've got code reflecting the current understanding of my attempt to hack a way into the jungle! I've made a start. I've got something I can build on. And remember, when you say "X is impossible" what you really mean is "I don't know how to X".

P.S.
Here's another horrible testing hack for C/C++.

the starfish and the spider

is an excellent book by Ori Brafman and Rod Beckstrom (isbn 1-59184-143-7). As usual I'm going to quote from a few pages:
This is a book about what happens when there is no one in charge. It's about what happens when there's no hierarchy.
Instead of a chief, the Apaches had a Nant'an - a spiritual and cultural leader. The Nant'an led by example and held no coercive power. Tribe members followed the Nant'an because they wanted to, not because they had to... The phrase "you should" doesn't even exist in the Apache language.
Instead of having a head, like a spider, the starfish functions as a decentralized network. Get this: for the starfish to move, one of the arms must convince the other arms that it's a good idea to do so. The arm starts moving, and then - in a process that no one fully understands - the other arms cooperate and move as well. The brain doesn't "yea" or "nay" the decision. In truth, there isn't even a brain to declare a "yea" or "nay". The starfish doesn't have a brain. There is no central command.
Open systems can't rely on a police force. On the one hand there's freedom to do what you want, but on the other hand, you have added responsibility: because there are no police walking around maintaining law and order, everyone becomes a guardian of sorts.
To collect money, you generally need to have an accountant somewhere, which leads to centralization.
When you give people freedom, you get chaos, but you also get incredible creativity.
It is the right as well as the duty of every managerial employee to criticize a central management decision which he considers mistaken or ill-advised... such criticism is not only not penalized; it is encouraged as a sign of initiative and of an active interest in the business. It is always taken seriously and given real consideration. [Peter Drucker]
I taught them that communication is to be upward if it is to work at all... I taught them that top management is a function and a responsibility rather than a rank and a privilege. [Peter Drucker]
A typical GM factory in the 1980s... if an employee make a mistake or detected a problem, he could stop the line, whereupon a loud alarm would sound... The Toyota assembly line... if an employee stopped the line a pleasant "ding-dong" would sound and teams would carefully study what was going on.
It's better, as the saying goes, to be vaguely right than precisely wrong.
When we are used to seeing something in a certain way, it's hard to imagine it being any other way. If we're used to seeing the world through a centralized lens, decentralized organizations don't make much sense.

the teachings of don juan

is an excellent book by Carlos Castaneda (isbn 978-0140192384). As usual I'm going to quote from a few pages
By experiencing other worlds, then, we see our own for what it is and are thereby enabled also to see fleetingly what the real world, the one between our own cultural construct and those other worlds, must in fact be like.
There is nothing wrong with being afraid. When you fear, you see things in a different way.
Fear is the first natural enemy a man must overcome on his path to knowledge.
You dwell upon yourself too much. That's the trouble. And that produces a terrible fatigue.
Are you angry at me don Juan? I asked when he returned.
He seemed surprised at my question.
No! I'm never angry at anybody! No human being can do anything important enough for that. You get angry at people when you feel their acts are important. I don't feel that way any longer.
What will happen to the man if he runs away in fear?
Nothing happens to him except that he will never learn.
He will never become a man of knowledge. He will perhaps be a bully or a harmless, scared man, at any rate, he will be a defeated man. His first enemy will have put an end to his cravings.
And what must he do to overcome fear?
The answer is very simple. He must not run away. He must defy his fear, and in spite of it he must take the next step in learning, and the next, and the next. He must be fully afraid, and yet he must not stop. That is the rule! And a moment will come when his first enemy retreats. The man begins to feel sure of himself. His intent becomes stronger. Learning is no longer a terrifying task. When this joyous moment comes, the man can say without hesitation that he had defeated his first natural enemy.
Does it happen at once, don Juan, or little by little?
It happens little by little, and yet the fear is vanquished suddenly and fast. But won't the man be afraid again if something new happens to him? No. Once a man has vanquished fear, he is free from it for the rest of his life because, instead of fear, he has acquired clarity - a clarity of mind which erases fear. By then a man knows his desires; he knows how to satisfy those desires. He can anticipate the new steps of learning, and a sharp clarity surrounds everything. The man feels that nothing is concealed.
The freedom to choose a path imparted a sense of direction through the expression of personal inclinations.
Exertion entailed not only drama, but also the need of efficacy. Exertion had to be effective; it had to possess the quality of being properly channelled, of being suitable.
To become a man of knowledge was a task that could not be fully achieved; rather, it was an unceasing process comprising (1) the idea that one had to renew the quest of becoming a man of knowledge; (2) the idea of one's impermanency; and (3) the idea that one had to follow the path with heart.