Model-Based Testing in Agile Development Cycles – Part II

Last post I left you hanging after a model had emerged from our initial acceptance requirements. The model we generated looked like:

Today’s post is about what you can use this model for in agile development, and what kind of test cases it produces.

Design discussions

As I already pointed out in the previous post, models are great in discussions. Often when you explain the model to the product owner he/she will start noticing quirks. For the example model we developed, one such quirk is that the system is not allowed to reverse a payment if said payment has been involved in a financial application. Although it is a requirement in the system, it was not mentioned in the acceptance criteria because it is not a sun-shine scenario. Design discussions based on models will help you uncover these implicit requirements.

Model-Based Testing in Agile Development Cycles – Part I

Today I’m strafing away from Model-Based Integration testing as I’d like to ramble a bit about using Model-Based Testing in an Agile development cycle. For those few of you who haven’t heard about Agile, here’s my crude and narrow look at it.

Agile development

Agile development is often referred to as Scrum, however Scrum is just one Agile methodology, many more exists. In Agile development work is broken up into short sprints of roughly two weeks duration. At the beginning of a sprint you sit down with your team (size 4-7 roughly) and plan what you believe you can achieve during the next two weeks. The main idea is that when you complete a sprint you are done. In contrast, way too often in waterfall you hear the developer/tester saying I’m done (or “almost done”) and what they mean is that the code is done, but they need to run tests, validate performance, security, stress, etc. That does not fly in Agile, done means done, your code is ready for production. The idea of course, is that you do not have to go back and revisit your work at a later point in time – it is Agile because you do not drag a huge backlog of items you need to do when you enter the next sprint.

Okay, enough about Agile – there are lots of other (and better) sources out there online [1] and in books [2].

Model-Based Testing and Agile

Recently I’ve gained some experience applying Model-Based Testing in an Agile development cycle. It’s tricky business and you have to balance your time carefully. There are definitely some pros of applying Model-Based Testing in Agile, but one has to be very careful not to focus too much on modeling through the sprint - instead I suggest taking the time early on in the sprint to formalize a model, which servers as a good aid in discussions as well as a great reference tool later on.

Flexibility of model based testing in practice

We all hear people arguing, model based testing is much better than traditional testing. "Why" you ask? "Well, it's much more flexible" comes the answer. And you sit back and think, "hmm, that really didn't answer my question?".

So let me try to answer your question - why are models more flexible? Let me give you a real life example of a case where model based testing proved to be flexible. I was working on a model for a new feature for the system under test. We started out designing and implementing model based testing on this particular feature the way it was implemented. It so happens that the feature contains a list of items (we call it a journal, when you confirm your entries you post the journal), and each item has associated a set of attributes (we call these dimensions - they are generic and used for analysis later on). We had developed a model for testing the posting functionality of this journal, and the model created roughly 300 test cases. Now it so happens that the dimension attributes can be set to a blocked state preventing posting the journal - which we had modeled.

Now somebody comes along and says: "You know what? This is simply used for forecasting, there is no need to prevent posting of blocked dimensions." At exactly this time you take a deep breath and brace youself for an argument on how the requirements were setup before you realize, well... this is not a problem at all, we used model based testing. We went back and looked at the model, and literally changed one line of model code changing the expected result from the PostJournal action to always be true. Viola, we had fixed 300 test cases.

In conclusion, model based testing allows for design changes at a later stage of development, as changes only need to be introduced on the model level. This is an huge benefit, especially because during the end game the pressure starts rising to finish before the deadline, and it is exactly at that time you do not have time to change 300 test cases because a requirement changed. Oh, and by the way, do requirements change in late game? Of course they do, that's the whole reason we invented SCRUM instead of the waterfall model (but that's for another post).