Model-Based Testing in Agile Development Cycles – Part II

Last post I left you hanging after a model had emerged from our initial acceptance requirements. The model we generated looked like:

Today’s post is about what you can use this model for in agile development, and what kind of test cases it produces.

Design discussions

As I already pointed out in the previous post, models are great in discussions. Often when you explain the model to the product owner he/she will start noticing quirks. For the example model we developed, one such quirk is that the system is not allowed to reverse a payment if said payment has been involved in a financial application. Although it is a requirement in the system, it was not mentioned in the acceptance criteria because it is not a sun-shine scenario. Design discussions based on models will help you uncover these implicit requirements.

It’s easy to solve these quirks when you are at the modeling stage, whereas maintaining traditional scenario based tests can require a lot of effort when requirements change. A benefit from modeling in Agile is that you uncover these implicit requirements early in the development cycle.

In our case we can fix the problem either by adjusting the model to avoid taking the ReversePayment action from states where the payment has been involved in a financial application. But more often than not, we would actually like to have a test case that checks that this implicit requirement is working – that is we would like the model to actually try doing it, but to expect a failure. We update the rule to:

        [Rule(Action = "ReversePayment/result")]
        static bool ReversePayment()
        {
            Condition.IsTrue(Entries > 0);
            if (Applications > 0)
                return false;

            Entries--;
            return true;
        }

Two things happen

a)      The return value of ReversePayment from the model rule is matched against the return value of the adapter layer when executing tests.

b)      The state space will not change if financial applications exist, making the rule a self-looping action.

Our new model looks like:

Notice how the return value of the ReversePayment rule differs between state S4 and S8, and how the latter is a self-looping action. This is of course because in state S8 the payment is involved in a financial application. Because we modeled the return code we will see that the test generation will construct scenarios where we test the implicit requirement is enforced by the system.

Tests

Okay, enough design discussions. Let’s look at the actual outcome! Exploring the test case generation we find:

What you immediately notice is that we recover our 4 original acceptance scenarios (encircled in orange). I encourage you to go back to part I and check the scenarios match. We also find tests covering our implicit requirement. In total we got 11 test cases out of the model even though it was built from just 4 acceptance tests. So not only should you find all your acceptance scenarios in the generated test suite, but you should expect that the model does a more thorough job as well! Actually, if you find that not all of your acceptance criteria are represented by the generated test cases, you should go back and revisit your model to look for any bugs in the design.

Conclusion

Not only does the model serve as a great input to design discussions, it will also automatically produce all your acceptance scenarios. That means when the model adapter layer and the feature you are testing are complete, you simply run your Model-Based test suite as your acceptance criteria.