Sunday, June 19, 2011

Applying Model-Based Testing for Fault-Injection in cloud services – Part II

Okay, so I have to admit I’m mixing things up a bit here. This week I wanted to go to a 2-service model, but I realized that would be jumping ahead of things. Instead I want to take the next steps in actually implementing the shopping cart service and the model interface to this service, as this has some interesting challenges.

First the service…

Shopping cart service
The shopping cart service is a Windows Azure WCF Service Web Role project. It implements a very crude interface. You can add an item to the cart, reset it (empty it), retrieve the number of stored items and proceed to checkout (which includes emptying the cart). The interface is:
    public interface IShoppingCart
    {
        /// <summary>
        /// Empties the shopping cart
        /// </summary>
        [OperationContract]
        void Reset();

        /// <summary>
        /// Adds the item with given name to the shopping cart
        /// </summary>
        /// <param name="item">Name of item to add</param>
        [OperationContract]
        void AddItem(string item);

        /// <summary>
        /// Returns the number of items currently in the cart
        /// </summary>
        /// <returns>Number of items in cart</returns>
        [OperationContract]
        int GetCount();

        /// <summary>
        /// Continues to payment input, and empties the shopping cart
        /// </summary>
        /// <returns>False if no items are in the cart</returns>
        [OperationContract]
        bool ProceedToCheckout();
    }

The service can be started from within Visual Studio where you can debug it, or you can generate an Azure ASP.NET Web Role that consumes it.

Writing tests against a service is piece-of-cake. You simply add a service reference to the test project and create a static instance of the service client, which you can then call. The adapter layer of our model based tests will simply call the appropriate service method for each action, with the exception of the “KillService” and “RestoreService” rules that are special.

The implementation of a shopping cart service can be found in the appendix [1]. 

These actions are a bit more tricky to implement...


Encapsulating services
The ability to kill and restore a service however, is not something that is available for services hosted by Visual Studio. Going back to our original post, the idea was to inject faults into the service at certain points during test execution as decided by the model exploration. It is thus a critical component of the framework to manage the service and influence it directly (i.e. not through the service interface, as it is impossible to restore the service this way). We need an interface to explicitly start and kill the services.

Please note that because we need explicit control over services, we must insist that all services are running on the local machine. This should however not be a significant limitation, as you would need to run all services in your development environment anyway.

To support this functionality we need to explicitly control the process that launches the service, so that the test execution controls the starting and stopping of services. When developing WCF Services in Visual Studio the hosting process is the “ASP.NET Development Server” – we can encapsulate this process in a manager service class allowing us to explicitly start and kill the service.

To do this we need to understand how Visual Studio launches its services. The underlying executable is “WebDev.WebServer40.exe” (at least for VS2010), and it is located under “C:\Program Files (x86)\Common Files\microsoft shared\DevServer\10.0\” (or the equivalent 32bit path). It is launched using the following syntax:
               WebDev.WebServer40.exe /port:<port> /path:<path to service> /vpath:<virtual path>

Virtual path is normally just “/”. With this in mind a class for managing a service is developed (inspiration from: Magnus Johansson):
    public class ManagedService : IDisposable
    {
        // TODO: Update to your location of WebDev.WebServer40.EXE
        const string devServerProcess = @"C:\Program Files (x86)\Common Files\microsoft shared\DevServer\10.0\WebDev.WebServer40.EXE";
        const string serviceVirtualPath = @"/";

        private Process process;
        private string servicePath;
        private int servicePort;

        public static void KillAll()
        {
            FileInfo fi = new FileInfo(devServerProcess);
            string name = Path.GetFileNameWithoutExtension(fi.FullName);

            // Kills all process instances
            foreach (Process proc in Process.GetProcessesByName(name))
            {
                try
                {
                    proc.Kill();
                }
                catch (Exception e)
                {
                    Debug.WriteLine("Error: " + e.Message);
                }
            }
        }

        public bool Running()
        {
            return !process.HasExited;
        }

        public void Kill()
        {
            try
            {
                process.Kill();
            }
            catch (Exception e)
            {
                Debug.WriteLine("Error killing service: " + e.Message);
            }
        }

        public void Start()
        {
            try
            {
                process.Start();

                // Sleep to allow the service to get up and running
                // TODO: Should wait till an end-point can be established, then proceed, instead of using sleep
                Thread.Sleep(100);
            }
            catch (Exception e)
            {
                Debug.WriteLine("Error starting service: " + e.Message);
                Debug.WriteLine("Arguments: " + process.StartInfo.Arguments);
            }
        }

        public ManagedService(string path, int port)
        {
            servicePath = path;
            servicePort = port;

            CreateProcess();
            Start();
        }

        private void CreateProcess()
        {
            ProcessStartInfo startInfo = new ProcessStartInfo(devServerProcess);
            startInfo.Arguments = String.Format("/port:{0} /path:{1} /vpath:{2}", servicePort, "\"" + servicePath + "\"", "\"" + serviceVirtualPath + "\"");
            process = new Process();
            process.StartInfo.UseShellExecute = false;
            process.StartInfo.CreateNoWindow = true;
            process.StartInfo = startInfo;
        }

        public void Dispose()
        {
            Kill();
        }
    }

With this implementation we have available ManagedService.Start() and ManagedService.Kill(). We can now develop a custom base class for our test execution which sets up the necessary services before test execution begins:
    [TestClass]
    public class ServiceTestClass : VsTestClassBase
    {
        // TODO: Modify to point to correct folder for service
        const string ShoppingCartServicePath = @"C:\Users\simej\My Documents\Visual Studio 2010\Projects\ShoppingCartService\WCFServiceWebRole1";
        // TODO: Modify to point to correct port for service
        const int ShoppingCartServicePort = 34431;

        private static bool initialized = false;

        [TestInitialize]
        public void Initialize()
        {
            if (!initialized)
            {
                // Kill any stray instances of services
                ManagedService.KillAll();

                // Setup an instance of the shopping cart service
                Webshop.AddService(new ManagedService(ShoppingCartServicePath, ShoppingCartServicePort));
                initialized = true;
            }

            Webshop.Initialize();
        }
    }

Webshop is the adapter layer, where all model actions are implemented, and contains a static List<ManagedService> which holds all the consumed services.

Make sure to adjust the path to the service as well as the port to match that in the actual service configuration (you can read this off of the ASP.NET Development Server instance running your service).

Also update your config.cord file to use the new base class:
    switch TestClassBase = "ServiceTestClass";

It is important to understand that all of this only handles starting and stopping of the service, you still need to add a service reference to the test project and instantiate that to communicate with the actual service (after you have started it, of course).

The implementation of the managed service, base test class, adapter and model based tests can be found in the appendix [2].
Test execution against service implementation
Running the generated tests from “Part I” shows:
 
As we see, 3/9 tests failed because ProceedToCheckout returned False instead of True. The root cause of these failures is that my implementation of the shopping cart service stores the cart state in memory, which is lost upon killing the service. The tests are actually finding real product bugs.

Conclusion
We saw how we can manipulate the state of services by encapsulating the “ASP.NET Development Server” allowing the model based tests to explicitly bring services down and back up, effectively allowing us to inject fault states into the cloud based system, which revealed a buggy implementation.

No comments:

Post a Comment