Sunday, August 7, 2011

Model-Based Testing in Agile Development Cycles – Part I

Today I’m strafing away from Model-Based Integration testing as I’d like to ramble a bit about using Model-Based Testing in an Agile development cycle. For those few of you who haven’t heard about Agile, here’s my crude and narrow look at it.

Agile development
Agile development is often referred to as Scrum, however Scrum is just one Agile methodology, many more exists. In Agile development work is broken up into short sprints of roughly two weeks duration. At the beginning of a sprint you sit down with your team (size 4-7 roughly) and plan what you believe you can achieve during the next two weeks. The main idea is that when you complete a sprint you are done. In contrast, way too often in waterfall you hear the developer/tester saying I’m done (or “almost done”) and what they mean is that the code is done, but they need to run tests, validate performance, security, stress, etc. That does not fly in Agile, done means done, your code is ready for production. The idea of course, is that you do not have to go back and revisit your work at a later point in time – it is Agile because you do not drag a huge backlog of items you need to do when you enter the next sprint.

Okay, enough about Agile – there are lots of other (and better) sources out there online [1] and in books [2].

Model-Based Testing and Agile
Recently I’ve gained some experience applying Model-Based Testing in an Agile development cycle. It’s tricky business and you have to balance your time carefully. There are definitely some pros of applying Model-Based Testing in Agile, but one has to be very careful not to focus too much on modeling through the sprint - instead I suggest taking the time early on in the sprint to formalize a model, which servers as a good aid in discussions as well as a great reference tool later on.


The first thing to keep in mind is that Agile development is acceptance oriented, which means the first thing you do in the beginning of a sprint is to define what your acceptance criteria are. Often you specify these in terms of acceptance test scenarios which are critical sun-shine scenarios that validates that the system is living up to the functional requirements. These can be provided by the product owner (from Scrum, this is the person responsible for the product meeting the customer’s expectations). The interesting thing is that these scenarios often are described as a sequence of actions that results in some desired verifiable outcome. One example could look like:

Now this looks a lot like something a model could produce, right? The cool thing is that we can leverage these scenarios by modeling each action as a rule – observe:
    static class FinancialApplicationModel
    {
        [Rule(Action = "CreatePayment")]
        static void CreatePayment()
        {
        }

        [Rule(Action = "PostPayment")]
        static void PostPayment()
        {
        }

        [Rule(Action = "ApplyEntry")]
        static void ApplyEntry()
        {
        }

        [Rule(Action = "UnapplyEntry")]
        static void UnapplyEntry()
        {
        }

        [Rule(Action = "ReversePayment")]
        static void ReversePayment()
        {
        }

        [Rule(Action = "VerifyLedgerEntry")]
        static void VerifyLedgerEntry()
        {
        }

        [Rule(Action = "VerifyFinancialApplication")]
        static void VerifyFinancialApplication()
        {
        }
    }

Initially this constructs an unbounded model that contains no state space, exploration yields a senseless graph:

Not much of a model, but it’s a great starting point. By steadily introducing state variables and model conditions, a model starts to emerge from these actions. We introduce some counters to keep track of the model state:
    static class FinancialApplicationModel
    {
        static int Payments;
        static int Entries;
        static int Applications;

Next step is to add some logic. The following are implicit rules in financial applications:
  1. You must create a payment before you can post it
  2. You must have posted a payment before you can apply it
  3. You must have applied a payment before you can unapply it
  4. You must create a payment before you can reverse it
Implementation of these rules is left as an exercise for the reader (Note that for clarity I’ve limited all integers to a max value of 1) - or you can refer to [3]. When you are done and explore the model you should get something that closely resembles:

So let’s take a deep breath and a cup of coffee and revisit what we have done. We took four acceptance scenarios that were provided by the product owner, then we extracted the sequential actions into model rules and introduced four logical conditions, and out of the blue a model emerges – that’s pretty cool!

Now the above model is what I like to refer to as the initial model, and it’s a great starting point for discussions between the product owner, developers and testers in the team early on in the sprint! It formally captures all of the explicit and implicit requirements.

Conclusion
When it comes to requirements, the earlier on you uncover or clarify them, the better! In Agile development cycles, modeling is a powerful aid in your design discussions and can help you uncover implicit requirements of the system.

In the next post we will look into examples of implicit requirements and have a look at the generated test cases.


2 comments: