Software testing, one of the most important methods for quality assurance, has become too expensive and error prone for complex modern software systems. Test automation aims to reduce the costs of software testing and to improve its reliability. Despite advances in test automation, there are some domains for which automation seems to be diﬃcult, for example, testing software to reveal the presence of security vulnerabilities, testing for conformance to security properties that traverse several functionalities of an application such as privacypolicies, and testing asynchronous concurrent systems. Although there are research works that aim to solve the problems of test automation for these domains, there is still a gap between the practice and the state of the art. These works describe speciﬁc approaches that deal with particular problems, generally under restricted conditions. Nevertheless, individually, they have not made noticeable impact on the practice in test automation for these domains. Therefore, there is a need for an integrated framework that binds speciﬁc approaches together in order to provide more complete solutions. It is alsoimportant for this framework to show how current test automation eﬀorts, tools and frame works, can be reused. This thesis addresses this need by describing a general model-based testing framework and its specialisation for the testing domains of security vulnerabilities, privacy policies and asynchronous systems.The main characteristic of the general framework resides in the separation between behavioural (control) and data generation speciﬁcations. This framework is deﬁned on the basis of labelled transition systems and context free grammars. Labelled transition systems allow behavioural models to be kept simple and tractable while extended context free grammars allow for the generation of data values that later will be used in the execution of test cases.The extended grammars in the data generation models contain a representation of the global state of the system, which allows for example, the history of the execution of test cases to inﬂuence the generation of subsequent data values.Besides the general pattern described in the behavioural and data generation models, each specialised testing domain requires models that represent particular characteristics of the system and the testing objectives for that domain. Vulnerability testing requires a model that describes the properties of a system that make it vulnerable and another one that describes what are considered to be the malicious intentions of an attacker. Privacy policies testing,requires the addition of a model that describes the conditions under which the execution of a deﬁned operation is restricted or permitted. An important characteristic of the privacy policies described in this thesis is that they include the concept of obligations, this is, actions that require to be performed before the execution of the test case is considered successful. This framework considers test cases that fulﬁl bounded obligations. In testing asynchronous systems, this thesis focuses in a deﬁned subclass of systems in which actions can be partitioned into controllable and observable actions where executionof controllable actions is decided by the testing framework and observable actions designate the response of the system to the controllable stimuli. Diﬀerent from other approaches for asynchronous systems, this thesis uses sets instead of queues to keep tracking of expected observable responses. This allows the present approach to deal with imperfect communication channels and with delays and loss of information, where the order of the observations is not important. The practical applicability of the approaches presented in this thesis is demonstrated in several case studies from various domain applications, namely web-based applications, ﬁnancial exchange protocols and operating systems. Particularly, the case study on operating systems demonstrates the integration of the general approach with an existing testing framework. This case study describes advantages, disadvantages and trade-oﬀs of such integration.
|Date of Award||19 Jun 2010|
|Supervisor||Padmanabhan Krishnan (Supervisor)|