Domain Specialisation and Applications of Model-Based Testing

Abstract

Software testing, one of the most important methods for quality assurance, has become too expensive and error prone for complex modern software systems. Test automation aims to reduce the costs of software testing and to improve its reliability. Despite advances in test automation, there are some domains for which automation seems to be difficult, for example, testing software to reveal the presence of security vulnerabilities, testing for conformance to security properties that traverse several functionalities of an application such as privacypolicies, and testing asynchronous concurrent systems. Although there are research works that aim to solve the problems of test automation for these domains, there is still a gap between the practice and the state of the art. These works describe specific approaches that deal with particular problems, generally under restricted conditions. Nevertheless, individually, they have not made noticeable impact on the practice in test automation for these domains. Therefore, there is a need for an integrated framework that binds specific approaches together in order to provide more complete solutions. It is alsoimportant for this framework to show how current test automation efforts, tools and frame works, can be reused. This thesis addresses this need by describing a general model-based testing framework and its specialisation for the testing domains of security vulnerabilities, privacy policies and asynchronous systems.The main characteristic of the general framework resides in the separation between behavioural (control) and data generation specifications. This framework is defined on the basis of labelled transition systems and context free grammars. Labelled transition systems allow behavioural models to be kept simple and tractable while extended context free grammars allow for the generation of data values that later will be used in the execution of test cases.The extended grammars in the data generation models contain a representation of the global state of the system, which allows for example, the history of the execution of test cases to influence the generation of subsequent data values.Besides the general pattern described in the behavioural and data generation models, each specialised testing domain requires models that represent particular characteristics of the system and the testing objectives for that domain. Vulnerability testing requires a model that describes the properties of a system that make it vulnerable and another one that describes what are considered to be the malicious intentions of an attacker. Privacy policies testing,requires the addition of a model that describes the conditions under which the execution of a defined operation is restricted or permitted. An important characteristic of the privacy policies described in this thesis is that they include the concept of obligations, this is, actions that require to be performed before the execution of the test case is considered successful. This framework considers test cases that fulfil bounded obligations. In testing asynchronous systems, this thesis focuses in a defined subclass of systems in which actions can be partitioned into controllable and observable actions where executionof controllable actions is decided by the testing framework and observable actions designate the response of the system to the controllable stimuli. Different from other approaches for asynchronous systems, this thesis uses sets instead of queues to keep tracking of expected observable responses. This allows the present approach to deal with imperfect communication channels and with delays and loss of information, where the order of the observations is not important. The practical applicability of the approaches presented in this thesis is demonstrated in several case studies from various domain applications, namely web-based applications, financial exchange protocols and operating systems. Particularly, the case study on operating systems demonstrates the integration of the general approach with an existing testing framework. This case study describes advantages, disadvantages and trade-offs of such integration.

Date of Award	19 Jun 2010
Original language	English
Supervisor	Padmanabhan Krishnan (Supervisor)