Applying Model-Based Testing for Fault-Injection in cloud services – Part III

2-services model

Interestingly enough, when modeling more than one service, the model will also generate tests that verify services continue functioning even when others go down. Essentially what we are checking here is that we don’t have any unnecessary dependencies between the services, and that a service won’t hang because another one is down.

When designing models including more than one service the state-space can rapidly explode. Consider adding just another service: Payment Service into the model which we depend on upon check-out.

We extend our model by adding:

    public class PaymentService : ModelService

    {

        public override string Name { get { return "Payment"; } }

        public void ProcessPayment()

        {

            ConditionRunning();

        }

    }

And changing the model to:

    static class WebshopModelProgram

    {

        static ShoppingCartService cartService = new ShoppingCartService();

        static PaymentService paymentService = new PaymentService();

        [Rule(Action = "AddToCart()")]

        static void AddToCart()

        {

            cartService.AddItem();

        }

        [Rule(Action = "ProceedToCheckOut()/result")]

        static bool ProceedToCheckOut()

        {

            if(cartService.Count() == 0)

                return false;

            cartService.Empty();

            paymentService.ProcessPayment();

            return true;

        }

The explored model looks like:

Hmm, that’s a very big expansion for adding just one more service. The problem is of course due to combinatorics, with the current model implementation we support all of the following fault states:

Combination #	Service 1	Service 2
1	Started	Started
2	Started	Killed
3	Started	Restored
4	Killed	Started
5	Killed	Killed
6	Killed	Restored
7	Restored	Started
8	Restored	Killed
9	Restored	Restored

That implies that for every single application state we want to model, the combined state space contains nine states! But are all these combinations necessary? Well, if we really want to be exhaustive they are, but again, modeling is all about simplifying the SUT. One limitation we could put in place would be having only one service at a time going down – we would still get to test that all services are functioning when any service going down, we just miss out on the interdependencies where a service malfunctions because two other services are down. Furthermore, if we have other means to test that services are brought back up correctly, we could merge the started and restored fault codes.

A limited model could look like:

For my very basic model the state space size behaves like this:

Schema	Number of states
No limits	114
Kill limit	94
Merged start/restore	58
Kill limit, merged start/restore	38

Depending on what you test a wide set of options is available, these are just some simple limitations, one could also implement multiple models with different schemas. One could be testing all services with a kill limit and merged start/restore states to gauge the overall robustness of the system, while another could be focused on one or more core services (for example the authentication service) with no limitations to exhaustively test critical services.

That takes care of the modeling. Now for the implementation of the SUT – we need a PaymentService

class running as a service. We can add a new WCF Web Role service together with our original shopping cart service, by right-clicking the Windows Azure project and selecting “New web role project”. This will create a new service for our cloud based shopping portal. In this service add a reference to the shopping cart service (notice this creates a dependency between the services) and write the following code in the interface:

    [ServiceContract]

    public interface IPaymentService

    {

        [OperationContract]

        bool ProceedToCheckout();

    }

and in the service implementation:

    public class PaymentService : IPaymentService

    {

        ShoppingCartServiceClient shoppingCartClient = new ShoppingCartServiceClient();

        public bool ProceedToCheckout()

        {

            if (shoppingCartClient.GetCount() == 0)

                return false;

            shoppingCartClient.Reset();

            return true;

        }

    }

Now for the adapter implementation – using our initial ManagedService class we can easily accommodate multiple services! In the base class ServiceTestClass we change to:

    [TestClass]

    public class ServiceTestClass : VsTestClassBase

    {

        // TODO: Modify to point to correct folder for service

        const string ShoppingCartServicePath = @"C:\Users\simej\My Documents\Visual Studio 2010\Projects\ShoppingCartService\WCFServiceWebRole1";

        const string PaymentServicePath = @"C:\Users\simej\My Documents\Visual Studio 2010\Projects\ShoppingCartService\PaymentService";

        // TODO: Modify to point to correct port for service

        const int ShoppingCartServicePort = 37207;

        const int PaymentServicePort = 37116;

        private static bool initialized = false;

        [TestInitialize]

        public void Initialize()

        {

            if (!initialized)

            {

                // Kill any stray instances of services

                ManagedService.KillAll();

                // Setup an instance of the shopping cart service

                Webshop.AddService(new ManagedService(ShoppingCartServicePath, ShoppingCartServicePort));

                Webshop.AddService(new ManagedService(PaymentServicePath, PaymentServicePort));

                initialized = true;

            }

            Webshop.Initialize();

        }

    }

In the adapter class Webshop we also add a reference to the new payment service together with the following code:

        private static PaymentServiceClient paymentClient = new PaymentServiceClient();

        public static bool ProceedToCheckOut()

        {

            return paymentClient.ProceedToCheckout();

        }

We can now generate test cases for our 2-service system based on our model. Notice that the model does not actually need to know which rules are exercising which services this is delegated to the adapter layer (which is the usual way we abstract). The generated tests will be able to exercise, kill and restore the two services we implemented to test the dependency we created between the services.

Conclusion

Having a framework in place for handling services allows us to easily add more service components to the SUT without having to do much additional implementation in the adapter layer, while we retain the ability to test service dependencies.

As with any model care must be taken to avoid state space explosion, and we have explored means for putting limitations into the framework and what downsides these cause.