Model-Based Integration Testing – Part II (State encapsulation)

Last time we saw how we could perform Model-Based Integration Testing of a FileSystem model and a Network model using extension models. One problem with this approach was that the first model had to expose its internal state to the other model. In this post I’m going to talk about model state space encapsulation.

A step in the right direction is to do model encapsulation. Let’s start out by obtaining the same model results but having the FileSystem model encapsulating its state. Simply change accessibility of all internal state variables to private:

        private static int drives, currentDirectory;

        private static SequenceContainer<int> directories = new SequenceContainer<int>();

Now of course our Network model won’t compile, as it was referencing the FileSystem state variables directly. We have to realize what the common concept is here – the Network layer is supposed to mimic a directory structure and to do this it must have an interface to create a drive in the FileSystem models state space. One easy way to obtain this is to increase the accessibility of the CreateDrive rule to public, so the Network model can directly call a rule in the FileSystem model:

        [Rule(Action = "MapNetworkDrive()")]

        static void MapNetworkDrive()

        {

            FileSystem.CreateDrive();

        }

Model-Based Integration Testing – Part I

First let me introduce the concept of integration testing. Integration testing opposed to unit testing is all about the big picture, and is designed to verify that components of an application are working together correctly. Take Windows as an example. The operating system has literally tons of components, and many of these interconnect.

One such example could be when you map a network drive on your computer. The network layer integrates into the file system by creating a virtual drive, while the file system integrates into the network layer by reading network paths. The actual network location is integrated into your file system and displayed as a drive icon under My Computer. Even though the networking layer and file system layer are both tested in isolation, there is no guarantee that the two components will work together once they are connected to each other. A ton of problems could occur in this integration! Integration testing is all about finding such issues.

Model

Let’s start by modeling a file system. We model the following:

    static class FileSystem

    {

        ...

        [Rule(Action = "CreateDirectory()")]

        static void CreateDirectory()

        {

            ...

        }

        [Rule(Action = "ChangeDirectory(directory)")]

        static void ChangeDirectory(int directory)

        {

            ...

        }

       

        [Rule(Action = "CreateFile()")]

        static void CreateFile()

        {

            ...

        }

        [Rule(Action = "CreateDrive()")]

        static void CreateDrive()

        {

            ...

        }

        [Rule(Action = "FilesOnDrive(drive)/result")]

        static int FilesOnDrive(int drive)

        {

            ...

        }

    }

The idea here is of course that we have a file system, with a default drive (say “C:\”). We can create new directories and in these we can create files, we also have a validation function that counts the number of files on a drive.

T-wise combinatorial Model-Based testing – Part II

In the previous post we saw how Model-Based Testing can be used to generate combinatorial input to the SUT. This is very nice, because we can capture this behavior in a generic way, and easily extend it, and the model will automatically generate the necessary combinations.

One of the oddities we observed, however, was that the model generated equivalent test cases where the parameter order was swapped. For pair-wise testing this is an annoyance because the model generates double the number of necessary tests, but for higher orders of t this leads to big problems as the duplications scale as n x t! (that’s t-factorial!), where n is the number on unique tests and t is the dimensionality of the combinatorial test generation.

T-wise combinatorial Model-Based testing – Part I

One of the strengths of Model-Based Testing is the ability to generate combinatorial inputs from a model. Say for example we are using a black-box testing approach on a scientific implementation of the inverse of a multi-dimensional matrix function:

               

The SUT is designed to compute the inverse of f(x,y,z) for any value of x, y, z. We may state the test invariant that

            

Which states that the matrix product should not differ from the identity matrix by more than some small residual error. The nice thing about matrix inverse is the relative simplicity in verify the correctness of the implementation, because direct matrix multiplication is simple.

The actual setup is somewhat constructed, but it serves as a good example. The point is that we are testing an implementation on a multi-dimensional function (in this case 3-dimensional). Keep in mind that the SUT could be any function that requires more than one input.

We Are Going To Berlin

We got some great news to share! Apparently the ETSI board liked our abstract on Model-Based Integration Testing so much they decided to grant us a 20 minute presentation slot at the 2011 Model-Based Testing User Conference in Berlin on October 18-20^th (which I previously announced on the blog).

Seeing how this idea on Model-Based Integration Testing has been blue-stamped by an authority, I thought it would be a good idea to share some of the details of our framework over a series of upcoming blog posts. The plan is to reveal some more details here that can be presented in 20 minutes. Then I will be able to refer participants to this blog for more details.

The basic idea of the framework is to generate model based tests that span multiple feature boundaries within the SUT. We obtain this behavior using a producer/consumer pattern on “Universally Recognized Artifacts” (URAs) which are cleverly chosen pieces of information inside the SUT that “resides on the boundaries of system features”.

For now, we are very excited about this, and looking forward to a great conference!