compile time assertions in C

C has a facility for checking dynamic assertions at run-time. It's inside <assert.h> and its called assert. Now assert is a macro, so why isn't it called ASSERT? I don't know. Prior art no doubt. Anyway, assert is a dynamic runtime feature, you can only use it inside functions. 

/* not inside a function, won't compile :-( */
assert(sizeof(int) * CHAR_BIT >= 32);

That's a pity because it would be nice if I could get the compiler to check things like this automatically at compile time. I've occasionally seen an attempt at a compile-time check like this... 

#if sizeof(int) * CHAR_BIT < 32
#error People of Earth. Your attention please...
#endif

But this doesn't work. The C preprocessor is a glorified text reformatter: it knows practically nothing about C. However, there is a way to write this as a compile time assertion (and moving any error trap to an earlier phase is a Good Thing)

{ yourself }

#define COMPILE_TIME_ASSERT(expr)   \
    char constraint[expr]

COMPILE_TIME_ASSERT(sizeof(int) * CHAR_BIT >= 32);

What's is going on here? Well, the example preprocesses to... 

char constraint[sizeof(int) * CHAR_BIT >= 32];

If the expression is true, (an int is at least 32 bits), the expression will have a value of one, and constraint will be an array of one char. If the assertion is false, (an int is less than 32 bits), the expression will have a value of zero, and constraint will be an empty array. That's illegal, and you'll get a compile time error. Viola, a compile time assertion :-) You can use it inside and outside a function but you can't use it twice in the same function, as you end up with a duplicate definition. To solve that problem you could resort to some convoluted macro trickery: 

#define COMPILE_TIME_ASSERT(expr)       char UNIQUE_NAME[expr]
#define UNIQUE_NAME                     MAKE_NAME(__LINE__)
#define MAKE_NAME(line)                 MAKE_NAME2(line)
#define MAKE_NAME2(line)                constraint_ ## line

But this is pretty horrible. Also, you will probably get warnings about unused variables. Take a step back for a moment and think about why it works at all. It's because you have to specify the size of an array as a compile time constant. The formal grammar of a direct-declarator tells you this. Let's look at some bits of grammar more closely:

Constrained arrays

 direct-declarator:
  identifier
  ( declarator )
  direct-declarator [ constant-expression opt ]
  direct-declarator ( parameter-type-list )
  direct-declarator ( identifier-list opt )

I just piggy backed on this, using the constraint that the value of the constant expression cannot (in this context) be zero. A natural question (to the curious) is are there other parts of the formal grammar that require a constant expression. The answer, of course, is yes.

Constrained enums

 enumerator:
  enumeration-constant
  enumeration-constant = constant-expression

However, I can't use this because there are no useful constraints in this context.

Constrained bit-fields

 struct-declarator:
  declarator
  declarator opt : constant-expression

Reading the constraints of a bit field I see that if the width of a bit-field is zero the declaration cannot have a declarator. In other words this is legal... 

 struct x { unsigned int : 0; };

but this is not... 

 struct x { unsigned int bf : 0; };

This suggests another way to create a compile time assertion 

#define COMPILE_TIME_ASSERT(expr)   \
    struct x { unsigned int bf : expr; }

COMPILE_TIME_ASSERT(sizeof(int) * CHAR_BIT >= 32);

Trying this we again get duplicate definitions, not of a variable this time, but of the type struct x. However we can fix this by creating an anonymous struct: 

#define COMPILE_TIME_ASSERT(expr)   \
    struct { unsigned int bf : expr; }

This works. However, now you'll probably get warnings about the unused untagged struct.

There is one last bit of grammar that uses a constant-expression.

Constrained switch

 labelled-statement:
  identifier : statement
  case constant-expression : statement
  default : statement

It's well known that you can't have two case labels with the same constant. The following will not compile... 

switch (0)
{
case 0:
case 0:;
}
So, here's yet another way to create a compile time assertion. This time we don't create a dummy variable, or a dummy type, but a dummy statement. A dummy switch statement: 
#define COMPILE_TIME_ASSERT(pred)       \
    switch(0){case 0:case pred:;}

COMPILE_TIME_ASSERT(sizeof(int) * CHAR_BIT >= 32);

If pred evaluates to true (i.e., 1) then the case labels will be 0 and 1. Different; Ok. If pred evaluates to false (i.e., 0) then the case labels will be 0 and 0. The same; Compile time error. Viola. However, a switch statement cannot exist in the global scope. So the last piece of the puzzle is to put the compile time assertions inside a function. 

#include <limits.h>

#define COMPILE_TIME_ASSERT(pred)            \  
    switch(0){case 0:case pred:;}

#define ASSERT_MIN_BITSIZE(type, size)       \
    COMPILE_TIME_ASSERT(sizeof(type) * CHAR_BIT >= size)

#define ASSERT_EXACT_BITSIZE(type, size)     \
    COMPILE_TIME_ASSERT(sizeof(type) * CHAR_BIT == size)

void compile_time_assertions(void)
{
    ASSERT_MIN_BITSIZE(char,  8);
    ASSERT_MIN_BITSIZE(int,  16);
    ASSERT_EXACT_BITSIZE(long, 32);
}
1 comment:

C# struct/class differences

Events are locked?

Events declared in a class have their += and -= access automatically locked via a lock(this) to make them thread safe (static events are locked on the typeof the class). Events declared in a struct do not have their += and -= access automatically locked. A lock(this) for a struct would not work since you can only lock on a reference type expression.

Exist on stack or heap?

Value type local instances are allocated on the stack. Reference type local instances are allocated on the heap.

Can cause garbage collection?

Creating a struct instance cannot cause a garbage collection (unless the constructor directly or indirectly creates a reference type instance) whereas creating a reference type instance can cause garbage collection.

Meaning of this?

In a class, this is classified as a value, and thus cannot appear on the left hand side of an assignment, or be used as a ref/out parameter. For example: 

class Indirect
{
    //...
    public void Method(Indirect that)
    {
        RefParameter(ref this); // compile-time error
        OutParameter(out this); // compile-time error
        this = that;            // compile-time error
    }
    //...
}
In a struct, this is classified as an out parameter in a constructor and as a ref parameter in all other function members. Thus it is possible to modify the entire structure by assigning to this or passing this as a ref/out parameter. For example: 
struct Direct
{
    //...
    public void Reassign(Direct that)
    {
        RefParameter(ref this); // compiles ok
        OutParameter(out this); // compiles ok
        this = that;            // compiles ok
    }
    //...
}
Furthermore, you can reassign a whole struct even when the struct contains readonly fields! 
struct Direct
{
    public Direct(int value)
    {
        Field = value;
    }

    public void Reassign(Direct that)
    {
        RefParameter(ref this); // compiles ok
        OutParameter(out this); // compiles ok
        this = that;            // compiles ok
    }

    public readonly int Field;
}

class Show
{
    static void Main()
    {
        Direct s = new Direct(42);
        Console.WriteLine(s.Field); // writes 42
        s.Reassign(new Direct(24));
        Console.WriteLine(s.Field); // writes 24
    }
}
Note however that when you call a method on a readonly value-type field, the method call is made on a copy of the field. 
struct Direct
{
    // as above
}

class Caller
{
    public void Method()
    {
        Console.WriteLine(d.Field); // writes 42
        d.Reassign(new Direct(24));
        Console.WriteLine(d.Field); // writes 42!
    }

    private readonly Direct d = new Direct(42);
}

class Show
{
    static void Main()
    {
        Caller c = new Caller();
        c.Method();
    }
}

Always have a default constructor?

A struct always has a built-in public default constructor. 

class DefaultConstructor
{
    static void Eg()
    {
          Direct   yes = new   Direct(); // always compiles ok
        InDirect maybe = new InDirect(); // compiles if c'tor exists and is accessible
        //...
    }
}
This means that a struct is always instantiable whereas a class might not be since all its constructors could be private. 
class NonInstantiable
{
    private NonInstantiable() // ok
    {
    }
}

struct Direct
{
    private Direct() // compile-time error
    {
    }
}

Default construction triggers static constructor?

A structs static constructor is not triggered by calling the structs default constructor. It is for a class. 

struct Direct
{
    static Direct()
    {
        Console.WriteLine("This is not written");
    }
}

class NotTriggered
{
    static void Main()
    {
        Direct local = new Direct();
    }
}

Can be null?

A struct instance cannot be null. 

class Nullness
{
    static void Eg(Direct s, Indirect c)
    {
        if (s == null) ... // compile-time error
        if (c == null) ... // compiles ok
    }
}

Use with the as operator?

A struct type cannot be the right hand side operand of the as operator. 

class Fragment
{
    static void Eg(Direct s, Indirect c)
    {
          Direct  no = s as   Direct; // compile-time error
        InDirect yes = c as InDirect; // compiles ok
        //...
    }
}

Can be locked?

A struct type expression cannot be the operand of a lock statement. 

class LockStatement
{
    static void Eg(Direct s, InDirect c)
    {
        lock(s) { ... } // compile-time error
        lock(c) { ... } // compiles ok
    }
}

Can have a destructor?

A struct cannot have a destructor. A destructor is just an override of object.Finalize in disguise, and structs, being value types, are not subject to garbage collection. 

struct Direct
{
    ~Direct() {} // compile-time error
}
class InDirect
{
    ~InDirect() {} // compiles ok
}

And the CIL for ~Indirect() looks like this: 

.method family hidebysig virtual instance void
        Finalize() cil managed
{
  // ...
} // end of method Indirect::Finalize

Default field layout?

The default [StructLayout] attribute (which lives in the System.Runtime.InteropServices namespace) for a struct is LayoutKind.Sequential whereas the default StructLayout for a class is LayoutKind.Auto. (And yes, despite its name you can tag a class with the StructLayout attribute.) In other words the CIL for this: 

public struct Direct
{
    //...
}

looks like this: 

.class public sequential ansi sealed beforefieldinit Direct
       extends [mscorlib]System.ValueType
{
  //...
}

whereas the CIL for this: 

public sealed class InDirect
{
    //...
}

looks like this: 

.class public auto ansi sealed beforefieldinit Indirect
       extends [mscorlib]System.Object
{
    //...
}

Can be a volatile field?

You can't declare a user-defined struct type as a volatile field but you can declare a user-defined class type as a volatile field. 

class Bad
{
    private volatile Direct field; // compile-time error
}
class Good
{
    private volatile Indirect field; // compiles ok
}

Can have synchronized methods?

You can't use the [MethodImpl(MethodImplOptions.Synchronized)] attribute on methods of a struct type (if you call the method you get a runtime TypeLoadException) whereas you can use the [MethodImpl(MethodImplOptions.Synchronized)] attribute on methods of a class type. 

using System.Runtime.CompilerServices;

class Indirect
{
    [MethodImpl(MethodImplOptions.Synchronized)] // compiles and runs ok
    public void Method()
    {
        //...
    }
}

struct Direct
{
    [MethodImpl(MethodImplOptions.Synchronized)] // compiles ok, runtime TypeLoadException
    public void Method()
    {
        //...
    }
}

Can be pointed to?

Clause 25.2 of the C# standard defines an unmanaged type as any type that isn't a reference type and doesn't contain reference-type fields at any level of nesting. That is, one of the following:

  • Any simple value type (11.1.3, eg byte, int, long, double, bool, etc).
  • Any enum type.
  • Any pointer type.
  • Any user-defined struct-type that contains fields of unmanaged types only.
You can never take the address of a instance of a type that is not unmanaged (a fixed variable 25.3). 
class Bad
{
    static void Main()
    {
        Indirect variable = new Indirect();
        unsafe
        {
            fixed(Indirect * ptr = &variable) // compile-time error
            {
                //...
            }
        }
    }
}
If you want to fix an unmanaged instance you have to do so by fixing it through an unmanaged field. For example: 
class Indirect
{
    public int fixHandle;
}
class Bad
{
    static void Main()
    {
        Indirect variable = new Indirect();
        unsafe
        {
            fixed(int * ptr = &variable.fixHandle) // compiles ok
            {
                //...
            }
        }
    }
}
In contrast, you can (nearly) always take the address of an unmanaged instance. 
struct Direct
{
    // no reference fields at any level of nesting
}
class SimpleCase
{
    static void Main()
    {
        Direct variable = new Direct();
        unsafe
        {
            Direct * ptr = &variable; // compiles ok
            //...       
        }
    }
}
However, you have to take the address inside a fixed statement if the variable is moveable (subject to relocation by the garbage collector, see 25.3 and example above). Also, you can never take the address of a volatile field.

So, in summary, you can never create a pointer to a class type but you sometimes create a pointer to a struct type.

Can be stackalloc'd?

You can only use stackalloc on unmanaged types. Hence you can never use stackalloc on class types. For example: 

class Indirect
{
    //...
}
class Bad
{
    static void Main()
    {
        unsafe
        {
            Indirect * array = stackalloc Indirect[42]; // compile-time error
            //...
        }
    }
}
Where as you can use stackalloc on struct types that are unmanaged. For example: 
struct Direct
{
    // no reference fields at any level of nesting
}
class Good
{
    static void Main()
    {
        unsafe
        {
            Direct * array = stackalloc Direct[42]; // compiles ok
            //...
        }
    }
}

Can be sizeof'd?

You can only use sizeof on unmanaged types. Hence you can never use sizeof on class types. For example: 

class Indirect
{
    //...
}
class Bad
{
    static void Main()
    {
        unsafe
        {
            int size = sizeof(Indirect); // compile-time error
            //...
        }
    }
}
Where as you can use sizeof on struct types that are unmanaged. For example: 
struct Direct
{
    // no reference fields at any level of nesting
}
class Good
{
    static void Main()
    {
        unsafe
        {
            int size = sizeof(Direct); // compiles ok
            //...
        }
    }
}

How to initialize fields?

The fields of a class have a default initialization to zero/false/null. The fields of a struct have no default value. 

struct Direct
{
    public int Field;
}

class Indirect
{
    public Indirect()
    {
    }
    //...
    public int Field;
}

class Defaults
{
    static void Main()
    {
        Direct s;
        Console.WriteLine(s.Field);  // compile-time error

        Indirect c = new Indirect();
        Console.WriteLine(c.Field); // compiles ok
    }
}

You can initialize fields in a class at their point of declaration. For example: 

class Indirect
{
    //...
    private int field = 42;
}
You can't do this for fields in a struct. For example: 
struct Direct
{
    //...
    private int field = 42; // compile-time error
}
Fields in a struct have to be initialized in a constructor. For example: 
struct Direct
{
    public Direct(int value)
    {
        field = value;
    }
    //...
    private int field; // compiles ok
}
Also, the definite assignment rules of a struct are tracked on an individual field basis. This means you can bypass initialization and "assign" the fields of a struct one a time. For example: 
struct Direct
{
    public int X, Y;
}
class Example
{
    static void Main()
    {
        Direct d;
        d.X = 42;
        Console.WriteLine(d.X); // compiles ok
        Console.WriteLine(d.Y); // compile-time error
    }
}

Inheritance differences?

  • a struct is implicitly sealed, a class isn't.
  • a struct can't be abstract, a class can.
  • a struct can't call : base() in its constructor whereas a class with no explicit base class can.
  • a struct can't extend another class, a class can.
  • a struct can't declare protected members (eg fields, nested types) a class can.
  • a struct can't declare abstract function members, an abstract class can.
  • a struct can't declare virtual function members, a class can.
  • a struct can't declare sealed function members, a class can.
  • a struct can't declare override function members, a class can. The one exception to this rule is that a struct can override the virtual methods of System.Object, viz, Equals(), and GetHashCode(), and ToString().

Equals behavior?

classes inherit Object.Equals which implements identity equality whereas structs inherit ValueType.Equals which implements value equality. 

using System.Diagnostics;

struct Direct
{
    public Direct(int value)
    {
        field = value;
    }
    private int field;
}

class Indirect
{
    public Indirect(int value)
    {
        field = value;
    }
    private int field;
}

class EqualsBehavior
{
    static void Main()
    {
        Direct s1 = new Direct(42);
        Direct s2 = new Direct(42);

        Indirect c1 = new Indirect(42);
        Indirect c2 = new Indirect(42);

        bool structEquality = s1.Equals(s2);
        bool classIdentity  = !c1.Equals(c2);

        Debug.Assert(structEquality);
        Debug.Assert(classIdentity);

    }
}
Overriding Equals for structs should be faster because it can avoid reflection and boxing. 
struct Direct
{
    public Direct(int value)
    {
        field = value;
    }

    public override bool Equals(object other)
    {
        return other is Direct && Equals((Direct)other);
    }

    public static bool operator==(Direct lhs, Direct rhs)
    {
        return lhs.Equals(rhs);
    }

    //...    

    private bool Equals(Direct other)
    {
        return field == other.field;
    }

    private int field;
}
No comments:

ACCU C++ Countdown Pub Quiz

The ACCU conference is one of the highlights of my year. I ran a brand new session, a C++ Pub Quiz with an emphasis on fun and interaction, based loosely on the popular UK TV game show Countdown.

In the TV version, contestants play individually and have 30 seconds to find the longest word using only a small set of letters. In this version, contestants play in teams, and have ~7 minutes to write the smallest valid C++ program containing a small set of tokens.
For example, if the tokens were: 
catch -> [ ; -- foobar operator
Then a winning program (53 character program) might be: 
class c {
  c operator->(){
    foobar: try{
    }
    catch(c x[]){
        x--;
    }
  }
};

We used cyber-dojo with some custom C++17 start-points which automatically told you your program's size and score. The rules were as follows:
  • The judges decision was final
  • Only non-whitespace characters were counted
  • Programs had to compile
  • Warnings were allowed
  • Extra tokens were allowed
  • Each token has to be a single whole token. For example the . token had to be the member access token; you could not use ... ellipsis or 4.2 floating point literal


The winners and the tokens were as follows (can you find smaller programs?)
Round 1: snakes, 75 character program, dynamic_cast snafu += return switch final
Round 2: wolves,koalas tied, 54 character program, catch ; foobar operator -- [
Round 3: frogs, 62 character program, else ~ default -> using foobar 0x4b
Round 4: tigers, 44 character program, string include for auto template 42
Round 5: pandas, tigers tied, 82 character program, virtual typename x reinterpret_cast static_cast 30ul
Round 6: wolves, 64 character program, constexpr override goto wibble . this
The raccoons and lions won the conundrum rounds.

The result was very close.
In 3rd place snakes with 481 points.
In 2nd place alligators with 488 points.
In 1st place tigers with 495 points.
A big thank you to my co-presenter Rob Chatley, to all the contestants for being such good sports, and to Bloomberg for sponsoring the Quiz.

No comments:

Docker in Action

is an excellent book by Jeff Nickoloff. As usual I'm going to quote from a few pages.

The docker stop command tells the program with PID #1 in the container to halt.
Like most Docker isolation features, you can optionally create containers without their own PID namespace.
If a situation arises where the name of a container needs to change, you can always rename the container with the [docker rename] command.
There are two types of volume... The first type of volume is a bind mount. Bind mount volumes use any user-specified directory or file on the host operating system. The second type is a managed volume. Managed volumes use locations that are created by the Docker daemon in space controlled by the daemon, called Docker managed space.
When you mount a volume on a container file system, it replaces the content that the image provides at that location.
You can copy volumes directly or transitively.
The third situation where you can't use --volumes-from is if you need to change the write permission of a volume.
Remember that containers maintain IP address leases only when they're running. So if a container is stopped or restarted, it will lose its IP lease and any linked containers will have stale data.
It's not common to see minimum requirements published with open source software.
CPU shares differ from memory limits in that they're enforced only when there is contention for time on the CPU.
The union file system on your computer may have a layer count limit. These limits vary, but a limit of 42 layers is common on computers that use the AUFS system.
The most curious thing about this Dockerfile is that the ENTRYPOINT is set to a file that doesn't exist.
You have no way way to specify a bind-mount volume or read-only volume at image build time.
The examples in this chapter use the cURL command-line tool. Because this is a book about Docker, you should use cURL from inside a container.


No comments:

Siddhartha

is an excellent book by Herman Hesse. As usual I'm going to quote from a few pages.

He learned more from the river than Vasudeva could teach him. He learned from it continually. Above all, he learned from it how to listen, to listen with a still heart, with a waiting, open soul, without passion, without desire, without judgement, without opinions.
It also happened that curious people came along, who had been told that two wise men, magicians or holy men lived at the ferry. The curious ones asked many questions but they received no replies, and they found neither magicians nor wise men. They only found two friendly old men, who appeared to be mute, rather odd and stupid. And the curious ones laughed and said how foolish and credible people were to spread such wild rumours.
Is it not perhaps a mistake on your part not to be strict with him, not to punish him? Do you not chain him with your love? Do you not shame him daily with your goodness and patience and make it still more difficult for him?
Within Siddhartha there slowly grew and ripened the knowledge of what wisdom really was and the goal of his long seeking. It was nothing but a preparation of the soul, a secret art of thinking, feeling and breathing thoughts of unity at every moment of life.
From that hour Siddhartha ceased to fight against his destiny. There shone in his face the serenity of knowledge, of one who is no longer confronted with conflict of desires, who has found salvation, who is in harmony with the streams of events, with the stream of life, full of sympathy and compassion, surrendering himself to the stream, belonging to the unity of all things.
In every truth the opposite is equally true. For example, a truth can only be expressed and enveloped in words if it is one-sided. Everything that is thought and expressed in words is one-sided, only half the truth; it lacks totality, completeness, unity.
The sinner is not on the way to a Buddha-like state; he is not evolving, although our thinking cannot conceive things otherwise. No, the potential Buddha already exists in the sinner; his future is already there. The potential Buddha must be recognized in him, in you, in everybody. The world, Govinda, is not imperfect or slowly evolving along a path to perfection. No, it is perfect at every moment; every sin already carries grace within it, all small children are potential old men, all sucklings have death within them, all dying people - eternal life.
In order to learn to love the world, and no longer compare it with some kind of desired imaginary world, some imaginary vision of perfection, but to leave it as it is, to love it and be glad to belong to it.
It may be a thought, but I confess, my friend, that I do not differentiate very much between thoughts and words. Quite frankly, I do no attach great importance to thoughts either. I attach more importance to things.
I think it is only important to love the world, not to despise it, not for us to hate others, but to be able to regard the world and ourselves and all beings with love, admiration and respect.
The thing to me is of greater importance than the words; his deeds and life and more important to me than his opinions. Not in speech or thought do I regard him as a great man, but in his deeds and life.
Uncontrollable tears trickled down his old face. He was overwhelmed by a feeling of great love, of the most humble veneration.


No comments:

The Hidden Life of Trees

is an excellent book by Peter Wohlleben (isbn 1771642483). As usual I'm going to quote from a few pages.

In forked trees, at a certain point, two main shoots form, they continue to grow alongside each other. Each side of the fork creates its own crown, so in a heavy wind, both sides sway back and forth in different directions, putting a great strain on the trunk where the two parted company. ... The fork always breaks at its narrowest point, where the two sides diverge.
The process of learning stability is triggered by painful micro-tears that occur when the trees bend way over in the wind, first in one direction and then in the other. Wherever it hurts, that's where the tree must strengthen its support structure. ... The thickness and stability of the trunk, therefore, builds up as the tree responds to a series of aches and pains.
There is a honey fungus in Switzerland that covers almost 120 acres and is about a thousand years old. Another in Oregon is estimated to be 2,400 years old, extends for 2,000 acres, and weighs 660 tons. That makes fungi the largest known living organism in the world.
You find twice the amount of life-giving nitrogen and phosphorus in plants that cooperate with fungal partners than in plants that tap the soil with the roots alone.
Diversity provides security for ancient forests.
There are more life-forms in a handful of forest soil than there are people on the planet.
As foresters like to say, the forest creates its own ideal habitat.
Commercial forest monocultures also encourage the mass reproduction of butterflies and moths, such as nun moths and pine loopers. What usually happens is that viral illnesses crop up towards the end of the cycle and populations crash.
The storms pummel mature trunks with forces equivalent to a weight of approximately 220 tons. Any tree unprepared for the onslaught can't withstand the pressure and falls over. But deciduous trees are well prepared. To be more aerodynamic they cast off all their solar panels. And so a huge surface area of 1,200 square yards disappears and sinks to the forest floor. This is the equivalent of a sailboat with a 130-foot mast dropping a 100-by-130 foot mainsail.
Why do tree grow into pipes in the first place?... What was attracting them was loose soil that had not been fully compacted after construction. Here the roots found room to breathe and grow. It was only incidentally that they penetrated the seals between individual sections of pipe and eventually ran riot inside them.
Sometimes, especially in cold winters, the old wounds can act up again. Then a crack like a rifle shot echoes through the forest and the trunk splits open along the old injury. This is caused by differences in tension in the frozen wood, because the wood in trees with a history of injury varies greatly in density.


No comments:

the DevOps Handbook

is an excellent book by Gene Kim, Jez Humble, Patrick Debois, and John Willis (isbn 978-1-942788-00-3). As usual I'm going to quote from a few pages.

Make infrastructure easier to rebuild than to repair.
The average age of a Netflix AWS instance is twenty-four days.
Interrupting technology workers is easy, because the consequences are invisible to almost everyone.
In complex systems, adding more inspection steps and approval processes actually increases the likelihood of future failures.
Over the following year, they eliminated testing as a separate phase of work, instead integrating it into everyone's daily work. They doubled the features being delivered per month and halved the number of defects.
Bureaucracies are incredibly resilient and are designed to survive adverse conditions - one can remove half the bureaucrats, and the process will still survive.
When we have a tightly coupled architecture, small changes can result in large scale failure.
Our deployment pipeline infrastructure becomes as foundational for our development processes as our version control infrastructure.
If we find that unit or acceptance tests are too difficult and expensive to write and maintain, it's likely that we have an architecture that is too tightly coupled.
Any successful product or organization will necessarily evolve over its life cycle... eBay and Google are each on their fifth entire rewrite of their architecture from top to bottom.
... which can lead to the unfortunate metric of mean time until declared innocent.
The principle of small batch sizes also applies to code reviews.
80% of MTTR (mean time to recovery) is spent trying to determine what changed.
High performing DevOps organizations will fail and make mistakes more often... If high performers are performing thirty times more frequently but with only half the change failure rate, they're obviously having more failures. [Roy Rapoport, Netflix]
Spiders repair rips and tears in the web as they occur, not waiting for the failures to accumulate. [Dr Steven Spear]


1 comment:

NDC Does C++ Countdown!

It was my pleasure to run a small workshop style session at the excellent NDC-London conference. I ran a fun C++ game which parodies the popular UK TV gameshow Countdown.
  • In the TV version contestants take turns picking 9 random vowels/consonants and finding the longest word in 30 seconds.
  • In my version contestants take turns picking 7 random tokens from 5 categories: (keywords, identifiers, operators, punctuators, literals) and writing the shortest C++ program using all 7 tokens in 8 minutes.
Contestants write their code in customized cyber-dojo sessions which automatically:
  • checks which tokens have been used
  • tells you the size of the program
  • allows everyone to see all the submissions in the review
The rules:
  • tokens must be the correct type; eg you cannot write "." or ... for a dot operator
  • whitespace does not count towards the program's size
  • additional tokens are allowed
  • the program must compile
  • the program is not executed
  • warnings are allowed
In one round Phil Nash selected these 7 tokens: 
const vector tokens =
{
    ".",                  // operator
    "switch",             // keyword
    "snafu",              // identifier
    ",",                  // punctuator
    "\"salmo\"",          // literal
    "goto",               // keyword
    "!",                  // operator
};
and the winning solution (54 characters long) was: 
union X { X* x; };
X snafu() {
  l: switch (X().x,!"salmo"); goto l;
}
In another round Hulgar Frydrych selected these 7 tokens: 
const vector tokens =
{
    "catch",              // keyword
    "->",                 // operator
    "[",                  // punctuator
    ";",                  // punctuator
    "--",                 // operator
    "foobar",             // identifier
    "operator",           // keyword
};
and the winning solution (53 characters long) was: 
class c {
  c operator->(){ 
    foobar:
    try{
    }
    catch(c x[]){
        x--;
    }
  }
};
Can you create shorter versions?

No comments:

the design and implementation of cyber-dojo

At the excellent Agile on the Beach conference in Cornwall I did a presentation outlining some of the history, design and implementation of cyber-dojo. The video has just gone live on youtube.



No comments:

Continuous Delivery

Is an excellent book by Jez Humble and Dave Farley. As usual I'm going to quote from a few pages...
Software delivers no value until it is in the hands of its users.
The pattern that is central to this book is the deployment pipeline.
It should not be possible to make manual changes to testing, staging, and production environments.
If releases are frequent, the delta between releases will be small. This significantly reduces the risk associated with releasing and makes it much easier to to roll back.
Branching should, in most circumstances, be avoided.
Dashboards should be ubiquitous, and certainly at least one should be present in each team room.
One of the key principles of the deployment pipeline is that it is a pull system.
A corollary of having every version of every file in version control is that it allows you to be aggressive about deleting things that you don't think you need... The ability to weed out old ideas and implementations frees the team to try new things and to improve the code.
It should always be cheaper to create a new environment than to repair an old one.
The goal of continuous integration is that the software is in a working state all the time... Continuous is a practice not a tool... Continuously is more often than you think.
The most important practice for continuous integration to work properly is frequent check-ins to trunk or mainline.
Ideally, the compile and test process that you run prior to check-in and on your CI server should take no more than a few minutes. We think that ten minutes is about the limit, five minutes is better, and about 90 seconds is ideal.
Enabling developers to run smoke tests against a working system on a developer machine prior to each check-in can make a huge difference to the quality of your application.
Build breakages are a normal and expected part of the process. Our aim is to find errors and eliminate them as quickly as possible, without expecting perfection and zero errors.
Having a comprehensive test suite is essential to continuous integration.
You should also consider refactoring as a cornerstone of effective software development.


Labels:
No comments:

Building Microservices

Is an excellent book by Sam Newman. As usual I'm going to quote from a few pages...
Because microservices are primarily modeled around business domains, they avoid the problems of traditional tiered architectures.
Microservices should cleanly align to bounded contexts.
Another reason to prefer the nested approach could be to chunk up your architecture to simplify testing.
With an event-based collaboration, we invert things. Instead of a client initiating requests asking for things to be done, it instead says this thing happened and expects other parties to know what to do. We never tell anyone else what to do.
We always want to maintain the ability to release microservices independenty of each other.
A red build means the last change possibly did not intergrate. You need to stop all further check-ins that aren't involved in fixing the build to get it passing again.
The approach I prefer is to have a single CI build per microservice, to allow us to quickly make and validate a change prior to deployment into production.
No changes are ever made to a running server.
Rather than using a package manager like debs or RPMs, all software is installed as independent Docker apps, each running in its own container.
Flaky tests are the enemy. When they fail, they don't tell us much... A test suite with flaky tests can become a victim of what Diane Vaughan calls the normalization of deviance - the idea that over time we can become so accustomed to things being wrong that we start to accept them as being normal and not a problem.
All too often, the approach of accepting multiple services being deployed together drifts into a situation where services become coupled.
Most organizations that I see spending time creating functional test suites often expend little or no effort at all on better monitoring or recovering from failure.


Labels:
No comments:

Developer on Fire

Recently I was honoured to be interviewed by Dave Rael, the man behind Developer on Fire.
Click the image below to listen :-)


1 comment:

the design and evolution of cyber-dojo

I've talked about the design and evolution of cyber-dojo at two conferences this year. First at NorDevCon in Norwich and then also at Agile on the Beach in Falmouth. Here's the slides.



No comments:

cyber-dojo new release

The new release of cyber-dojo is out :-)



No comments:

cyber-dojo traffic-lights

My friend Byran who works at the awesome Bluefruit Software in Redruth has hooked up his cyber-dojo web server to an actual traffic-light! Fantastic. Check out the video below :-)



Byran writes
It started out as a joke between myself and Josh (one of the testers at Bluefruit). I had the traffic lights in my office as I was preparing a stand to promote the outreach events (Summer Huddle, Mission to Mars, etc...) Software Cornwall runs. The conversation went on to alternative uses for the traffic lights, I was planning to see if people would pay attention to the traffic lights if I put them in a corridor at the event; we then came up with the idea that we could use them to indicate TDD test status.
Although it started out as a joke I am going to use it at the Summer Huddle, the lights change every time anyone runs a test so it should give an idea of how the entire group are doing without highlighting an individual pair.
The software setup is very simple, there is a Python web server (using the Flask library) running on a Raspberry Pi that controls the traffic lights using GPIO Zero. When the appendTestTrafficLight() function (in run_tests.js.erb) appends the traffic light image to the webpage I made it send an http 'get' request to the Raspberry Pi web server to set the physical traffic lights at the same time. At the moment the IP address of the Raspberry Pi is hard coded in the 'run_tests.js.erb' file so I have to rebuild the web image if anything changes but it was only meant to be a joke/proof of concept. The code is on a branch called traffic_lights on my fork of the cyber-dojo web repository.
The hardware is also relatively simple, there is a converter board on the Pi; this only converts the IO pin output connector of the Raspberry Pi to the cable that attaches to the traffic lights.
The other end of the cable from the converter board attaches to the board in the top left of the inside the traffic lights; this has some optoisolators that drive the relays in the top right which in turn switch on and off the transformers (the red thing in the bottom left) that drive the lights.
I have to give credit to Steve Amor for building the hardware for the traffic lights. They are usually used during events we run to teach coding to children (and sometimes adults). The converter board has LEDs, switches and buzzers on it to show that there isn't a difference between writing software to toggle LEDs vs driving actual real world systems, it's just what's attached to the pin. Having something where they can run the same code to drive LEDs and drive real traffic lights helps to emphasise this point.


No comments:
Subscribe to: Posts (Atom)