www.hans-eric.com

Why? Because otherwise you’re bound to get bogged down by the swamp of regression testing.

Regression Testing in Incremental Development

Suppose we’re building a system using a traditional method; We design it, build it and then test it – in that order. For the sake of simplicity assume that the effort of acceptance testing the system is illustrated by the box below.

The amount of acceptance testing using a traditional approach

Now, suppose we’re creating the same system using an agile approach; Instead of building it all in one chunk we develop the system incrementally, for instance in three iterations.

An agile approach. The system is developed in increments.

New functionality is added with each iteration

Potential Releasability

Now, since we’re “agile”, acceptance testing the new functionality after each iteration is not enough. To ensure “potential releasability” for our system increments, we also need to check all previously added functionality so that we didn’t accidentally break anything we’ve already built. Test people call this regression testing.

Can you spot the problem? Incremental development requires more testing. A lot more testing. For our three iteration big  example, twice as much testing.

The theoretical amount of acceptance testing using an agile approach and three iterations, is twice as much as in Waterfall

Unfortunately, it’s even worse than that. The amount of acceptance testing increases exponentially with the number of increments. Five iterations requires three times the amount of testing, nine requires five times, and so on.

The amount of acceptance testing increase rapidly with the number of increments.

This is the reason many agile teams lose velocity as they go, either by yielding to an increasing burden of test (if they do what they should) or by spending an increasing amount of time fixing bugs that cost more to fix since they’re discovered late. (Which is the symptom of an inadequately tested system.)

A Negative Force

Of course, this heavily simplified model of acceptance testing is full of flaws. For example:

• The nature of acceptance testing in a Waterfall method is very different from the acceptance testing in agile. (One might say though that the Waterfall hides behind a false assumption (that all will go well) which looks good in plans but becomes ugly in reality).
• One can’t compare initial acceptance testing of newly implemented features, with regression testing of the same features. The former is real testing, while the second is more like checking. (By the way, read this interesting discussion on the difference of testing and checking: Which end of the horse.)

Still, I think it’s safe to say that there is a force working against us, a force that gets greater with time. The burden of regression testing challenges any team doing incremental development.

So, should we drop the whole agile thing and run for the nearest waterfall? Of course not! We still want to build the right system. And finding that out last is not an option.

The system we want is never the system we thought we wanted.

OK, but can’t we just skip the regression testing? In practice, this is what many teams do. But it’s not a good idea. Because around The Swamp of Regression Testing lies The Forest of Poor Quality. We don’t want to go there. It’s an unpleasant and unpredictable place to be.

We really need to face the swamp. Otherwise, how can we be potentially releasable?

The only real solution is to automate testing. And I’m not talking about unit testing here. Unit testing is great but has a different purpose: Unit-testing verifies that the code does what you designed it to do. Acceptance testing on the other hand verifies that our code does what the customer (or product owner, please use your favorite word for the one that pays for the stuff you build) asked for. That’s what we need to test to achieve potential releasability.

So, get out there and find a way to automate your acceptance testing. Because if you don’t, you can’t be agile.

Cheers!

Martin Fowler has an interesting post up called Utility vs Strategic Dichotomy. The point he’s making is that there are essentially two kinds of software projects, which he names utility and strategic, and that they need to be treated entirely different in the way you staff and run them.

Utility projects, he defines, are the ones that provide functions that are often necessary and crucial to run the business, but not a differentiating factor in competition with other companies.

A classic example of a utility IT project is payroll, everyone needs it, but it’s something that most people want to ‘just work’

Then he delivers hammer blow number one.

“Another consequence is that only a few projects are strategic. The 80/20 rule applies, except it may be more like 95/5.”

What Martin Fowler is saying is that a vast majority of us are doing work that is of no strategic importance to the business we support. I’m sure this statement cuts a wound in many developer’s and project manager’s self images, mine included. But I’m with Martin on this one. It’s time our sector gets its feet back on the ground and we start to realize that we’re here to support the business. Not the other way around.

Hammer blow number two.

Another way the dichotomy makes its influence felt is the role of agile methods. Most agilists tend to come from a strategic mindset, and the flexibility and rapid time-to-market that characterizes agile is crucial for strategic projects. For utility projects, however, the advantages of agile don’t matter that much.

I have a little harder swallowing this one. That the agile movement doesn’t “matter that much” while developing utility systems doesn’t ring true to me. Yes, time to customer is not as critical as in strategic projects, but that doesn’t mean that building the right system isn’t important to utility projects as well. And the agile values will help reducing cost and risk in any project, be it utility or strategic.

Cheers!

The short version of this post:

Do not spread generic types around. They are ugly and primitive.

Allow me to elaborate.

Generics are great

Before we got Generics (2004 in Java, 2005 in C#, and 2009 in Delphi) the way to enforce type safety on a collection type was to encapsulate it and use delegation.

class OrderList
{
private ArrayList items = new ArrayList();

public void Add(OrderItem item)
{
items.Add(item);
}

// Plus lots of more delegation code
// to implement Remove, foreach, etc.
}

Nowadays, we can construct our own type safe collection types with a single line of code.

List<orderitem> orders = new List<orderitem>();

That alone is good reason to love generics, and there lies the problem; many developers have come to love their generics a little too much.

Generics are ugly

The expressiveness of generics comes at a price, the price of syntactic noise. That’s natural. We must specify the parameterized types somehow, and the current syntax for doing so is probably as good as anything else.

Still, my feeling when I look at code with generics is that the constructed types don’t harmonize with the rest of the language. They kind of stand out, and are slightly negative for code readability.

OrderList orders = new OrderList();
// as compared to
List<OrderItem> orders = new List<OrderItem>();

This might not be so bad, but when we start to pass the constructed types around they become more and more like litter.

List<OrderItem> FetchOrders()
{
//…
}

double CalculateTotalSum(List<OrderItem> orders)
{
//…
}

void PrintOrders(List<OrderItem> orders)
{
//…
}

Besides being noisy, the constructed types expose implementation decisions. In our case the list of orders is implemented as a straight list. What if we found out that using a hash table implementation for performance reasons would be better? Then we’d have to change the declaration in all those places.

Dictionary<OrderItem, OrderItem> FetchOrders()
{
//…
}

double CalculateTotalSum(Dictionary<OrderItem, OrderItem> orders)
{
//…
}

void PrintOrders(Dictionary<OrderItem, OrderItem> orders)
{
//…
}

Comments redundant.

Generics are primitive

A related problem is that the constructed generic types encourage design that is more procedural than object-oriented. As an example, consider the CalculateTotalSum method again.

double CalculateTotalSum(List<OrderItem> orders)
{
//…
}

Clearly, this method belongs in the List<OrderItem> type. Ideally, we should be able to invoke it using the dot operator.

orders.TotalSum();
// instead of
CalculateTotalSum(orders);

But we cannot make that refactoring. We cannot add a TotalSum method to the List<OrderItem> type. (Well maybe we can if we make it an extension method, but I wouldn’t go there.) In that sense, constructed generic types are primitive types, closed and out of our control.

So we should hide them

Don’t get me wrong. I use generics a lot. But for the previously given reasons I do my best to hide them. I do that using the same encapsulation technique as before, or – at the very least – by inheriting the constructed types.

class OrderList : List<OrderItem>
{
  double TotalSum()
  {
    //…
  }
}

So, the rule I go by to mitigate the downsides of generics, is this:

Constructed types should always stay encapsulated. Never let them leak through any abstraction layers, be it methods, interfaces or cl

That is my view on Generics. What is yours?

Cheers!

P.S.

Just to be clear, the beef I have with Generics is with constructed types alone. I have nothing against using open generic types in public interfaces, although that is more likely to be a tool of library and framework developers.

You may have heard of The Boy Scout Rule. (I hadn’t until I watched a recorded presentation by “Uncle Bob”.)

You should always leave the campground cleaner than you found it.

Applied to programming, it means we should have the ambition of always leaving the code cleaner than we found it. Sounds great, but before we set ourselves to the mission we might benefit from asking a couple of questions.

1. What is clean code?
2. When and what should we clean?

Clean Code

What is clean code? Wictionary defines it like so:

software code that is formatted correctly and in an organized manner so that another coder can easily read or modify it

That, by necessity, is a rather vague definition. The goal of being readable and modifiable to another programmer, although highly subjective, is pretty clear. But what is a correct format? What does an organised manner look like in code? Let’s see if we can make it a little more explicit.

Code Readability

The easiest way to mess up otherwise good code is to make it inconsistent. For me, it really doesn’t matter where we put brackets, if we use camelCase or PascalCase for variable names, spaces or tabs as indentations, etc. It’s when we mix the different styles that we end up with sloppy code.

Another way to make code difficult to take in is to make it compact. Just like any other type of text, code gets more readable if it contains some space.

Although the previous definition mentions nothing about it, good naming of methods, classes, variables, etc, is at the essence of readable code. Code with poorly named abstractions is not clean code.

Organized for Reusability

Formatting code is easy. Designing for readability, on the other hand, is harder. How do we organize the code so that it is easily read and modifiable?

One thing we know, is that long chunks of code is more difficult to read and harder to reuse. So, one goal should be to have small methods. But to what cost? For instance, should we refactor code like this?

SomeInterface si = (SomeInterface)someObject;
doSomething(si);

Into this?

doSomething((SomeInterface)someObject);

What about code like this?

string someString;
if (a >= b)
  someString = “a is bigger or equal to b”;
else
  someString = “b is bigger”;

Into this?

string someString = a >= b ? “a is bigger or equal to b” : “b is bigger”;

In my opinion, the readability gains of the above refactorings are questionable. Better than reducing the number of rows in more or less clever ways, is to focus on improving abstractions.

Extract ’til you drop

Abstractions are things with names, like methods or classes. You use them to separate the whats from the hows. They are your best weapons in the fight for readability, reusability and flexibility.

A good way to improve abstractions in existing code is to extract methods from big methods, and classes from bulky classes. You can do this until there is nothing more to extract. As Robert C. Martin puts it, extract until you drop.

That leads me to my own definition of clean code.

• It is readable.
• It is consistent.
• It consists of fine grained, reusable abstractions with helpful names.

When to clean

Changing code is always associated with risk and “cleaning” is no exception. You could:

• Introduce bugs
• Cause build breaks
• Create merge conflicts

So, there are good reasons not to be carried away. A great enabling factor is how well your code is covered by automatic tests. Higher code coverage enables more brutal cleaning. A team with no automatic testing should be more careful, though, about what kind of cleaning they should do.

Also, to avoid annoying your fellow programmers it’s best to avoid sweeping changes that may result in check-in conflicts for others. Restrict your cleaning to the camp ground, the part of the code that you’re currently working on. Don’t venture out on forest-wide cleaning missions.

If the team does not share common coding rules, there’s a risk that you end up in “Curly Brace Wars”, or other religious software wars driven by team members’ individual preferences. This leads to inconsistency, annoyance and loss of energy. The only way to avoid it is to agree upon a common set of code rules.

With all that said, I have never imposed “cleaning” restrictions on my teams. They are allowed to do whatever refactorings they find necessary. If they are actively “cleaning” the code it means they care about quality, and that’s the way I like it. In the rare cases excessive cleaning cause problems, most teams will handle it and change their processes accordingly.

Cheers!

According to Tiobe Programming Community Index as of April 2010, Java is no longer the most popular programming language. C is; Not due to a sudden increase in popularity of C, but rather a cause of Java’s ongoing decline.

After more than 4 years C is back at position number 1 in the TIOBE index. The scores for C have been pretty constant through the years, varying between the 15% and 20% market share for almost 10 years. So the main reason for C’s number 1 position is not C’s uprise, but the decline of its competitor Java. Java has a long-term downward trend. It is losing ground to other languages running on the JVM. An example of such a language is JavaFX script that is now approaching the top 20.

It’s important to point out, thought, that only the language part of Java is losing ground. As Tiobe Software indicates in their comment, the JVM is as hot as ever with all kinds of cool language evolution occurring.

Another interesting note to make about the april index is that Delphi is once again among the top ten. I’m really happy about that, although it means my old prophecy of a hasty decline kind of puts me in the corner (although I’ve already taken it back).

A third point is the giant leap of Objective-C, from 42:nd to 11:th most popular language. This is a combination of iPhone’s popularity and the fact that Objective-C is the only OO alternative allowed on that platform, besides C++.

Cheers!

In my previous post I discussed the problem that method pointers aren’t able to store references to anonymous methods. Unfortunately that limitation makes anonymous methods less useful since they can’t be used to set up event handlers on the fly, like so:

// Wouldn't it be great
// if we could do something
// like this?

Memo1.OnKeyPress :=
  procedure(Sender: TObject; var Key: Char)
  begin
    Key := Chr(Ord(Key) + 1);
  end;

One thing I wasn’t aware of when I wrote that previous post is that while method pointers can’t be used to store anonymous methods, the opposite is true: Method references can indeed store both anonymous methods and instance methods. Many thanks to Barry Kelly who pointed it out to me in a comment.

Could this be the way to go? Shall we all start using method references instead of method pointers when we declare the events of our components from now on? Let’s take a look shall we? But just so that we’re clear with what we mean – there are many ways to say the same thing – here are the definitions I’m using in this post.

First of all, when I say method I mean either a function or a procedure, and it can be any of the types below. This is different from the common definition stating that methods are subroutines associated with either a class or an object.

Plain methods

These are functions and procedures declared outside the context of a class.

procedure PlainProcedure;
begin
  // Has no Self pointer
end;

Instance methods

Functions or procedures associated to an object instance

Procedure TSomeClass.InstanceProcedure;
begin
  // Self is an instance of TSomeClass
end;

Class methods

Functions or procedures associated to a class. In other languages also called static methods.

class procedure TSomeClass.ClassProcedure;
begin
  // Self is TSomeClass
end;

Anonymous methods

Functions or procedures declared as they are assigned in an execution context. These methods capture the surrounding context and may use local variables and arguments even if they’re out of execution scope.

SomeItems.Sort(
  function(Item1, Item2: TSomeItem): Integer;
  begin
    if Item1.Value > Item2.Value then
      Result := 1
    else if Item1.Value < Item2.Value then
      Result :=  -1
    else
      Result := 0;
    end
  );

For these four types of methods there are three reference types to store pointers for later invocation of the methods.

Plain method pointer

type TPlainMethodPointer =
  procedure(ASender: TObject);

Method pointers

type TMethodPointer =
  procedure(ASender: TObject) of object;

Method references

type TMethodReference =
  reference to procedure(ASender: TObject);

This compatibility graph shows what method types can be stored with the respective reference type.

From this graph we can see that the Method reference type is the unifying type that can store all types of method references. It seems like the way to go is to embrace Method references and render the other reference types obsolete. Is it possible? The answer is yes but no.

Take the event example of my last post. The VCL can not be changed to utilize Method pointers for backward compatibility reasons (like the widespread use of TMethod casts) so there’s nothing to do about it there. But what about our own components, and components created from now on?

type 
  TNotifyMethod = reference to procedure(Sender: TObject);

  TMyComponent = class(TComponent)
    ...
  published
    property OnChange: TNotifyMethod read FOnChange write FOnChange;
  end;

Well, it works fine as long as you set up the event handlers (OnChange in the above example) dynamically, but the method reference type events do not play well with the Delphi 2009 IDE. The object inspector doesn’t recognize possible event handlers and won’t create one automatically I you double click the event property. If you try to force the assignment by giving the name of an existing event handler method explicitly, the IDE throws an ugly Invalid Property Error Dialog.

As much as I’d like to be able to assign anonymous methods as event handlers to my components’, I’m not prepared to sacrifice the IDE integration. Hopefully CodeGear will fix this issue in future releases but until then anonymous methods will remain less useful than they could be.

Cheers!

As I noted in my previous post, anonymous methods is a big new feature in Delphi 2009 for the Win32 platform. While “closures” is natural and much appreciated in other languages, most notably Ruby, the Delphi community is still a bit reluctant and hesitant. Segunda Feira put words on it in his post on the subject.

I am still not convinced that this convenience is worth getting all that excited about – It has not enabled anything that was previously impossible, nor even especially difficult either to implement or to understand

[…]

anonymous methods should be used only when you absolutely have to, and so far I have not yet seen an example where anonymous methods are the only way to achieve anything

I agree that more often than not a problem can be solved with equal elegance using a pure object orientated approach, but there are situations where anonymous methods may actually be the better alternative. One situation that comes to mind is the setting up of test fixtures.
Say, for instance, that we want to test the event firing mechanism of a component. An easy way to set this up could be like the following.

procedure TestSomeComponent.TestOnChange;
var
  SUT: TSomeComponent;
  OnChangeFired: Boolean;
begin
  OnChangeFired := False;
  // Set up the fixture
  SUT := CreateTheComponentToTest;
  SUT.OnChange :=
    procedure(ASender: TObject)
    begin
      OnChangeFired := True;
      CheckSame(SUT, ASender, 'The Sender event argument should be set');
    end;
  // Perform operation
  SUT.Text := 'Some text';
  // Check the result
  CheckTrue(OnChangeFired, 'Setting the Text property should fire the OnChange event');
end;

The above code checks that the component’s OnChange event is fired when the Text property is set, and that the component is set as the Sender reference to the event handler. Except for being more compact, the biggest advantage to using anonymous methods in this scenario is that we avoid the obscure test smell and that we don’t have to clutter our test class with an instance variable (like FOnChangeFired) to identify if the event was really fired or not.

The only problem is: it doesn’t work. Why? Well, because the OnChange property is most likely of an instance method reference type (i.e. TNotifyEvent), meaning it doesn’t accept a references to an anonymous method even if it has the same signature.

type TNotifyEvent = procedure(ASender: TObject) <strong>of object</strong>;

For our code to compile we need to redeclare TNotifyEvent and remove the “of object” keywords and instead use the method reference type.

type TNotifyEvent = <strong>reference to</strong> procedure(ASender: TObject);

But of course, that’s not an option. It would mean that the traditional way of setting up an event handler (by using an instance method) would not work.

I see a definite problem with how the Delphi language explicitly forces you to distinguish between instance methods (and class methods for that matter) and anonymous method references, even though they share the same signature.
This is most unfortunate since I feel that the situations where we’d like to support both kinds of event handlers are quite common. And with the current semantics we have to use code duplication in order to achieve that. Like the two Synchronize methods of the built in TThread class.

In Delphi 2009 an overloaded variant of the TThread.Synchronize method was introduced, one that make use of anonymous methods. Here are snippets from that code:

type
  TThreadMethod = procedure of object;
  TThreadProcedure = reference to procedure;
...
  class TThread = class
    …
    procedure Synchronize(AMethod: TThreadMethod); overload;
    procedure Synchronize(AThreadProc: TThreadProcedure); overload;
    ...
  end;

The two methods have almost identical implementations. The only real difference is the type of the argument.

procedure TThread.Synchronize(AMethod: TThreadMethod);
begin
  FSynchronize.FThread := Self;
  FSynchronize.FSynchronizeException := nil;
  FSynchronize.FMethod := AMethod;
  FSynchronize.FProcedure := nil;
  Synchronize(@FSynchronize);
end;

procedure TThread.Synchronize(AThreadProc: TThreadProcedure);
begin
  FSynchronize.FThread := Self;
  FSynchronize.FSynchronizeException := nil;
  FSynchronize.FMethod := nil;
  FSynchronize.FProcedure := AThreadProc;
  Synchronize(@FSynchronize);
end;

I may be overly sensitive, but code like that really disturbs me. Unfortunately it gets worse. If we follow the execution chain down into the Synchronize class method that is invoked by the two, we find this.

class procedure TThread.Synchronize(ASyncRec: PSynchronizeRecord; QueueEvent: Boolean = False);
var
  SyncProc: TSyncProc;
  SyncProcPtr: PSyncProc;
begin
  if GetCurrentThreadID = MainThreadID then
  begin
    if Assigned(ASyncRec.FMethod) then
      ASyncRec.FMethod()
    else if Assigned(ASyncRec.FProcedure) then
      ASyncRec.FProcedure();
    end else
      …
end;

It would be a lot nicer if the two reference types were joined under a common reference type. And I can’t see why it couldn’t be done. When I look at the “of object” keywords I get a feeling that the language is leaking compiler implementation through the language interface; information that is indifferent to the developer. What matters from a callers’ perspective is the method signature, not whether the method has a self pointer or not.

I hope CodeGear recognizes this problem and find a way to clean this assymetry from the language. Anonymous methods would be so much more useful if they do.

Cheers!

I just downloaded and installed the HTML-Kit editor to see if it can help me in the WordPress Theme development projects I just started. Until now I’ve been relying upon the built in editor of the admin interface, handy for small changes but not for building themes from scratch.

Anyway, HTML-Kit was supposedly an excellent HTML editor and at a first glance it seemed good enough. That was until I tried to hit the ” (double quotation marks) key. Instead of inserting the expected character into the html document I was currently editing, HTML-Kit threw me a new MDI window containing a fresh HTML document. As if I’d pressed Ctrl-N(ew) or something.

What was going on? Well, it turned out that the default shortcuts of HTML-Kit was interfering with my custom keyboard layout. (This has happened to me with other editors as well.) My layout takes advantage of the Alt Gr button to move common special characters, like the ” character, from the top row to the base row, all for the sake of speed and ergonomy.

On my keyboard, to strike the ” key I have to press Alt Gr + L, which is actually Alt Gr + N since I base my layout on Dvorak, and since pressing Alt Gr is the same as pressing Alt + Ctrl this combination was coming in the way of the built in Alt + Ctrl + N, which means New HTML document i HTML-Kit. (Do you follow?)

The fact that HTML-Kit didn’t allow me to type a common HTML character made the editor seemingly useless. But, that happened to not be the case. HTML-Kit has a great feature that allows you to define your own shortcuts. So, I did just that: define a new Alt + Ctrl + N to just output my double-quote. (I had to do the same for right brackets, } )

I’d much prefer it if application creators wouldn’t define Alt + Ctrl shortcuts at all, for the reasons given above, but if they absolutely have to, HTML-Kit’s level of customizability is the next best thing.

Jay Fields has a post up today where he makes a distinction between Developers and Hackers.

Time after time I see requirements gathered and presented in a way that is totally disconnected from the business problem that’s being addressed. There are two ways to handle the situation.

1. Write the best code you can.
2. Talk to the business.

Hackers generally go for the first choice, which doesn’t guarantee failure. In fact, a good hacker can still deliver code quickly enough that the business will be happy. Often, he gets it right on the 2nd or 3rd try.

But, a developer can deliver what the business is looking for the first time. A(n often) quick conversation with the business ensures that the developer knows what to work on and how it will benefit the business. This dialog often leads to a superior solution implementation where the business gets what it wants and the developer is able to do it in the most efficient way (e.g. I don’t need a beautiful website, I need a command line application).

It’s essentially the same distinction I tried to make a while ago, although I used the term Programmer instead of Hacker. Maybe if I’d used the term Hacker as well, I might have avoided some of the heat I took for those definitions 🙂

Interestingly Jay too suggests — although he doesn’t say it outright — a definition where human interaction and taking the Customer’s view is what distinguishes a developer, rather than coding skills.

Cheers!

In the keynote where Alan Cooper proclaimed that Agile processes are bad for developing quality software, he made another provoking statement: that Open-Source is ultimately a symptom of management failure. His point is that with the right enthusiasm and commitment to your products, why would anyone go and work in an Open-Source project on their spare time?

Well, there are plenty of good reasons for doing Open-Source pro bono work; it’s a great way to get experience and widen perspectives for instance, but still, Alan has a point. Many of us are not as content as we could be with our regular work. Instead we’re seeking satisfaction elsewhere.

So, what should our employers do to get our full attention? Here’s my list.

• Creative freedom
  Give me a chance to contribute, to be innovative and creative. Let me spend a part of my time doing research and follow paths that interest and inspires me. Google is a great example of a company that understands the importance of this.
• Personal Development
  As Ron Jeffries has said, “the river is moving, and if we don’t keep rowing, we are going to drift back downstream.” I think self improvement is a spice of life. If my company provides me with all the books I need (and want), and lets me attend courses and conferences of my choice, chances are I’ll stay with it for life.
• Ownership
  People usually do a better job, are more careful and thorough, if they own the thing they’re working on. This is true for software as well. Make me a business partner and I’ll optimize my work according to your business goals.
• Appreciation
  The human race seems to be immensely better at criticizing than at giving appreciation. Yet, this is what we all crave, and – it has a great impact on how we see our employers. A rewarding salary is one form of showing appreciation, but I also need the outspoken forms.
• Closures
  No one can go on for ever without reaching a finish line. I want to work in a team that is long-term efficient and gets to get done often. Help me divide and I’ll conquer for you.

That was my list, what’s on yours?

Cheers!