Monday, July 18, 2011

Model-Based Integration Testing – Part I

First let me introduce the concept of integration testing. Integration testing opposed to unit testing is all about the big picture, and is designed to verify that components of an application are working together correctly. Take Windows as an example. The operating system has literally tons of components, and many of these interconnect.

One such example could be when you map a network drive on your computer. The network layer integrates into the file system by creating a virtual drive, while the file system integrates into the network layer by reading network paths. The actual network location is integrated into your file system and displayed as a drive icon under My Computer. Even though the networking layer and file system layer are both tested in isolation, there is no guarantee that the two components will work together once they are connected to each other. A ton of problems could occur in this integration! Integration testing is all about finding such issues.

Model
Let’s start by modeling a file system. We model the following:
    static class FileSystem
    {
        ...

        [Rule(Action = "CreateDirectory()")]
        static void CreateDirectory()
        {
            ...
        }

        [Rule(Action = "ChangeDirectory(directory)")]
        static void ChangeDirectory(int directory)
        {
            ...
        }
       
        [Rule(Action = "CreateFile()")]
        static void CreateFile()
        {
            ...
        }

        [Rule(Action = "CreateDrive()")]
        static void CreateDrive()
        {
            ...
        }

        [Rule(Action = "FilesOnDrive(drive)/result")]
        static int FilesOnDrive(int drive)
        {
            ...
        }
    }

The idea here is of course that we have a file system, with a default drive (say “C:\”). We can create new directories and in these we can create files, we also have a validation function that counts the number of files on a drive.


The model implementation can be found in appendix [1]. Exploring this model yields:

Likewise we model a network layer
    static class Network
    {
        static int networkDrives;

        [Rule(Action = "MapNetworkDrive()")]
        static void MapNetworkDrive()
        {
            Condition.IsTrue(networkDrives < 1);
            networkDrives++;
        }

        [Rule(Action = "DisconnectNetworkDrive()")]
        static void DisconnectNetworkDrive()
        {
            Condition.IsTrue(networkDrives > 0);
            networkDrives--;
        }
    }

Exploration yields:


Model integration
This is all nice and tidy, but how do we integrate the models? The following picture shows two types of integrations, integration through extension (Ax extends A) or integration by passing control over to another model (Model A passes over to B). When talking about extension models, these are models that are aware of the internal state of another model, and can modify it directly. When passing control over to another model, model B has no knowledge of model A, but model A must explicitly call a rule to transfer control to model B (we will elaborate on this in future posts).


Well, the easiest way to integrate our two models is to use extension models. Let’s see how this can be done! In the config.cord file add the following section:
machine FileSystemModel() : FileSystem where ForExploration = true
{
    construct model program from FileSystem
    where scope = "OS.FileSystem" //The value of the namespace switch can be a .Net namespace or a fully-qualified class name.
}

machine NetworkModel() : Network where ForExploration = true
{
    construct model program from Network
    where scope = "OSNetwork.Network" //The value of the namespace switch can be a .Net namespace or a fully-qualified class name.
}

machine NetworkFileSystemModel() : Main where ForExploration = true
{
    NetworkModel ||| FileSystemModel;
}

Now you can explore the model “NetworkFileSystemModel” which will interleave the actions from the two models. However, the models do not “know” of each other, so there will be no synergy effect – let’s fix that! In the Network model simply change the Map and Disconnect network drive rules to modify the state of the FileSystemModel:
    static class Network
    {
        [Rule(Action = "MapNetworkDrive()")]
        static void MapNetworkDrive()
        {
            Condition.IsTrue(FileSystem.drives < 1);
            FileSystem.drives++;
            FileSystem.directories.Add(0);
            FileSystem.currentDirectory = 0;
        }

        [Rule(Action = "DisconnectNetworkDrive()")]
        static void DisconnectNetworkDrive()
        {
            Condition.IsTrue(FileSystem.drives > 0);
            FileSystem.drives--;
            FileSystem.directories.Clear();
        }
    }

Of course now we have duplicated logic, which we ought to refactor, but I will leave that as an exercise for the reader. The explored model looks like:

We can see how mapping a network drive is picked up by the FileSystem model, and again when disconnecting the drive all files and directories are lost. The obvious down side of this approach is that the models must be located in the same project. A less obvious problem is that the Network model is modifying the internal state representation of the FileSystem model, this means that the Network model must make sure it does not leave the FileSystem model in any bad state, like forgetting to clear the directories and files when disconnecting a network drive.

This becomes even more problematic if a third model is created that also wants to extend the FileSystem model, now all three models must be aware of each others presence, otherwise we could get potential conflicts. This obviously calls for a better solution, but I’ll keep the tension for next week…

3 comments:

  1. Hello Simon,
    I am following your blog articles and its very informative.Nice job

    ReplyDelete
  2. Hi, Thanks for an interesting article. Unfortunately, I could not download the model implementation. It appears the URL of [1] http://api.ge.tt/0/93GrQ46/0/blob/download does not work.

    ReplyDelete
    Replies
    1. Hi, thanks for the feedback. I've updated the link, so that it should work now.

      Delete