Testing Web Services Composition: a Mapping Study

distributed under the Creative Commons Attribution License unported 3.0, which permits unrestricted use, distribution, and reproduction in any medium, provided that original work is properly cited. Abstract E-business systems are known for their frequent changes in business requirements, and traditional software development engineering approaches have difficulties in keeping up with this dynamicity. The use of service oriented architecture in software development has become popular as it provides a solution to frequent changes to business environments in a heterogeneous network. In service oriented architecture, new systems are quickly developed by combining services developed and owned by different organizations, and one way of realising this architecture is via Web services. Although much research effort has been put into the discovery, invocation and composition of services testing Web services has only begun to attract interest from both researchers and industry players. This paper aims to provide a mapping study of current Web services composition testing researches conducted by other researchers. Research papers on testing of Web services composition were gathered from various scholarly databases using provided search engines within a given period of time. The research papers were then classified according to issues addressed by them. The aim is to get a broad overview of the current state of research in Web services composition testing. By looking at the areas focused by existing researchers, gaps and untouched areas of Web services composition testing can be discovered.


Introduction
Service oriented architecture is an architectural strategy that enables Web applications to be built using services, and it is estimated that 80% of software applications will be using this architecture by the year 2015 (Vaughan, 2010).Traditional software development paradigm develops software applications by gathering requirements, converting the requirements into specifications and finally implementing the specifications into executable codes.This traditional paradigm of developing software applications has difficulties in satisfying business customers needs whenever new business requirements are required of the system.In a service oriented architecture paradigm, developers are split into three entities which are the application builders (service requesters), service brokers (publishers) and service developers (providers).Service developers develop services and application builders develop new systems by combining the services.Service brokers allow developers to publish their services and application builders to find the necessary services.Therefore, instead of designing and coding new systems, system development via service oriented Communications of the IBIMA 2 architecture is done via discovery and composition (Tsai, 2005) and one way of implementing this architecture is by using Web services.
Web services are Internet-based modular application that uses the Simple Object Access Protocol (SOAP) for communication and transfers data in XML through the Internet (Offutt and Xu, 2004).It is another distributed computing technology (like CORBA, RMI, and EJB) that allows the creation of client/server application.However unlike CORBA, RMI, EJB, Web services allow easy integration and interoperability between heterogeneous applications.Web services are platform and language independent, meaning that a client program written in C running on Windows can communicate with a Web service application programmed in Java running on Linux.This ensures that a company's application can talk to its business partner's application, even though they are written in different languages and written on different platforms.
Recent researches study how to specify (in a formal and expressive enough language), compose (automatically), discover, secure, and ensure the correctness of Web services (Tabatabaei, 2010).Although much research attention has been given to areas of Web service discovery and composition, the same cannot be said about Web services testing.Web services testing is an important aspect of Web services as it provides a way to ensure the trustworthiness of a Web service-based system.If a client is not able to trust a service from a service provider, the client will not use it (Bai et al., 2008).Testing is also necessary to ensure that the service chosen best satisfies the user's needs and of functionalities and quality (Bai et al., 2007).
This study is interested in finding the state of research in Web services composition testing.As there are not many relevant high-quality primary studies relevant to Web services composition testing, a systematic mapping study is recommended (Kitchenham and Charters, 2007) (Petticrew and Roberts, 2006).This paper aims to provide an overview of the state of research in the area of Web services composition testing with the intention of providing future research area by looking at the gaps and untouched areas.
This paper is organized as follows.Some background information on Web services testing and composition are presented in the preliminaries in Section 2. Section 3 describes the research method in conducting the mapping study.Section 4 reports the result of the study.Section 5 describes threats to validity.Section 6 concludes the work and suggests future research direction.

Web Services Composition
As mentioned earlier, application development via service oriented architecture involves searching for services that are able to fulfil the needs of the requester.However, whenever a single Web service is unable to satisfy the requirements, several Web services are composed to meet the requirements of the requester.In Web service composition, a composition schema or process definition defines the business logic of a composite Web service.A composition engine executes the business logic of a Web service composition by invoking services defined in the schema.Each composition execution is called a composition instance or process instance.Orchestration schema is a part of the composition schema that specifies the order of service invocation within a composition.BPEL and OWL-S (W3C Working Group, 2004) are two main languages created for composing Web services.BPEL is a syntax based composition language that allows manual composition of Web services whilst OWL-S is semantic based that allows a more automated approach to Web service composition.Other models used to 3 Communications of the IBIMA compose Web services include workflows, graphs, and Petri nets.

Web Services Testing
Testing is the process of executing a program with the intent of finding errors.It involves activities such as specifying test cases, generating test data, monitoring test execution, measuring test coverage, validating test results and tracking system errors.Several research papers have been written on testing of Web services addressing areas such as testing SOAP messages, WSDL interfaces, and publish, find and bind capability.Web services composition testing involves testing the extended interaction between the service provider and requester as well as the composition schema which defines the business logic of the composite service.Existing commercial tools such as Oracle BPEL process manager and Active BPEL Designer support manual functional testing that allows testers to test BPEL descriptions and service compositions based on requirements.However existing tools/approaches are not sufficient to reveal structural errors of BPEL processes (Liu, 2008).

Research Objective
The objective of the review paper is to summarize current state of the art Web services composition testing approaches based on the following main questions: The above questions are instrumental in ensuring sufficient effort and time is taken to perform this mapping study.

Sources of Information
The mapping study was conducted from April to December 2010.The literature search was carried out by searching in a set of selected repositories.The repositories used were Scopus, IEEE Xplore Digital Library, ACM Digital Library, Google Scholar and Springer Link.Although the total number of papers returned was high, most of the papers were redundant.The same paper was returned by different repository search engine.Therefore, eliminating this redundancy is a necessary step in this systematic review.

Search Criteria
Based on the research questions in section 2.1, a set of keywords and its synonyms were defined as search strings.As research in Web services composition testing is still new, no specific year range was included in the search.Table 1 describes the keywords used as well as its synonyms.BPEL was used as one of the synonyms for composition as it is a prominent language for service composition.It is an XML-based specification that models Web service behavior in a business process interaction (Peltz, 2003).Using the above keywords and synonyms, a search string was created as follows: ("Web service" OR "Web services" OR "SOA") AND ("composite" OR "composition" OR "BPEL") AND ("testing" OR "test") The above search string was adapted for different repository search engine as the searching mechanism differs from one search engine to another.

Study Selection
A set of inclusion and exclusion criteria were defined for the selection process.These criteria were then used in the mapping study process during searching.Figure 1 illustrates the mapping study process.

Fig 1: Mapping Study Process
The systematic review process started with defining research question as stated in section 2.1.Next step was to deϐine the searching keyword.The keywords are listed in Table 1.Using the keywords, searching was carried out on the selected repositories using the provided search engine.
Once the list of research papers was obtained, the papers were excluded according to the title, abstract and conclusion.At the same time, the same papers returned by different repositories were removed to eliminate redundancy.Finally, reference analysis was conducted to ensure that referenced papers were not missed out.Reference analysis is important as some of the publications might have been missed out during the keywords based search using search engines and during exclusion based on title and abstract.By looking at the reference at the end of each paper, relevant papers that are missed earlier can be included in the mapping study process.

WSC Testing Research Paper Areas
A total of 45 research papers related to Web services composition testing were returned by the mapping study searching process.
The research papers were then categorized and classiϐied into 6 groups which are survey and systematic literature reviews, test framework, test generation, regression testing and mutation testing.Figure 2 illustrates the breakdown of research papers according to the six categories.The publications were also classified based on different forum types, similar to systematic review conducted by (Endo and Simao, 2010).The forum types are workshops, conferences and symposiums, journals and book chapter.A large percentage of the publications come from conferences and symposiums (35 papers), followed by book section (6 papers).Journals and workshops contributed 2 papers each to the overall publication category.This is illustrated in Figure 4 Fig

4: Breakdown of Publication's Forum Type
The high percentage of publications coming from conference and symposiums indicates that this research area is still immature and more research needs to be done to overcome issues relating to Web services composition testing.Only two publications manage to get into journals and these publications touched on test generation issues.
The following section will discuss the findings of the mapping study for each focus area as categorized in Figure 2.

Survey/Systematic Review
There

Regression Testing
Regression testing is the "selective retesting of a system or a component to verify that modifications have not caused unintended effects and that the system or component still complies with its specified requirements" (IEEE Std 610.12-1990, 1990).
Interest in regression testing is starting to emerge with two recent publications in Web services composition regression testing.This is shown in Table 4.

Mutation Testing
Mutation testing is a technique where two or more mutant programs are executed together with the same test cases in order to determine the ability of the test cases to detect the mutants (IEEE Std 610.1990).Similar to regression testing research in mutation testing to support web services composition testing is also gaining attention with three recent publications.Table 5 lists down the authors involved, the approaches and specification used.

Threats to Validity
The validity of this mapping study is threatened by the following issues: • The research papers were obtained by keyword searching and reference analysis.
Exclusions were made by reading the title, abstract and conclusions.However, there is a possibility that there exist papers that were missed due to the above searching and exclusion method.
• Judgmental errors in classifying the papers into each category.

Conclusion and Future Work
This review paper has described the systematic mapping process, discussed the results of the mapping study and threats to the validity of the study.The systematic mapping process was described in terms of the research questions defined, searching keywords used, the exclusion and inclusion criteria.The results of the study was classified into several categories and analyzed.The paper has shown the areas of research within Web services composition testing that have been done by answering the questions that were defined initially.
Most of the research papers are from conference and paper proceedings (76%), which is an indication that the research area is still immature.More work need to be done in order to improve the current state of research in Web services composition testing.One research direction would be to focus on Semantic Web services composition testing.Although there are a few testing researches done on OWL-S, other Semantic Web services specification ought to be addressed as well.
The author is currently investigating Web services composition testing in WSMO (W3C Working Group, 2005) which is a Semantic Web services framework.

1.
What are the research areas in Web services composition testing addressed by other studies?2. What are number of publications per year and what is the trend?3. Which forum type includes papers on Web services composition testing?

Figure 3
Figure3shows the number of research publications made per year for each paper category.Overall, research publications on