Towards Computer System Validation : An overview and Evaluation of Existing Procedures

The medical device industry in Europe is one of the sectors actively regulated by directives. Medical device manufacturers face the challenge of implementing the statutory regulations. In the context of current trends regarding the digitalization of enterprises, among other things, a focus is on the computer system validation (CSV). The present research shows why the CSV in the medical device industry is necessary, which different validation approaches exist, and which tasks and activities are to be carried out within the CSV. One focus of this research is the critical consideration of the problems associated with CSV for small and medium-sized enterprises (SMEs). As a result of this research, it can be stated that the identified literature sources are very homogeneous, and the validation approaches do not show any significant differences.


Introduction
Software products are increasingly becoming the central component of complex electronic devices that control or support technical or ____________________________________________________________________________________ ______________ Marius Schönberger and Tatjana Vasiljeva (2018), Journal of Innovation Management in Small and Medium Enterprise, DOI: 10.5171/2018.512744devices are also being widely deployed in clinical environment, which was enabled by, for example, shrinking technologies, portability, or increasing interconnectedness (Alemzadeh et al., 2013).Therefore, software plays an important role in the critical functions of medical devices.However, from 2002 to 2010, medical devices based on software resulted in over 537 recalls affecting more than 1.5 million devices.During this period, 11.3% of all recalls were due to software failures (Fu, 2011).As Appendix 1 shows, a total of 195 recalls for medical devices, resulting from software errors, were reported to the US Food and Drug Administration (FDA) only in 2017 (FDA, 2018).It is apparent that medical devices are often subject to a high number of errors with potentially catastrophic effects on the patients (Alemzadeh et al., 2013).According to AAMI (2016), medical errors are involved in more than 250,000 deaths each year in the US and account for nearly one in ten deaths.Finally, the recall of medical devices poses a problem that affects the entire healthcare sector (Foe Owono, 2015; Maisel et al., 2001).
Against this background, it seems that with the current technological possibilities of the software development, a considerable increase in the complexity of the devices and thus major challenges for the reliability, patient safety and security emerge (Faris, 2006;Fu, 2011;Alemzadeh et al., 2013).Therefore, the computer system validation (CSV) is a major means of avoiding such defects and resultant recalls (Bhusnure et al., 2015) and, according to the FDA, a requirement of the quality system for medical device manufactures (FDA, 2002).The analysis of different literature on the topic of CSV shows that the implementation of CSV is not only necessary due to legal regulations, but also from economic, social and technological aspects (e.g.Vogel, 2011;Thirumalai and Sinha, 2011;Faris, 2006;Fu, 2011).However, the regulations only determine that a CSV has to be carried out and which system or software type is to be taken into account in the company (e.g.FDA, 2002; ISO, 2016), the exact scope of the validation as well as a structured approach are not specified.This leads to the challenge, that all medical device manufacturers shall determine what needs to be validated and how much validation is enough to ensure that regulatory requirements are met (Hrgarek, 2008).
This key problem forms the basis for the present research and, thus, the question arises, whether standardised procedures for the implementation of the CSV exist for the medical device industry.Although there are various guidelines, e.g. the Gamp 5 Guide, ISO 26262-8 Section 11 or IEEE Std.829, they cannot always reflect the existing diversity of industries, size differences, or special requirements of, in particular, small and medium-sized medical device manufacturers.In addition, the implementation of these directives in SMEs is often hampered by limited resources, such as the lack of existing staff capacities or insufficient funding (Nguyen, 2009;Razak et al., 2009;Buschfeld et al., 2011).With regard to the aforementioned key problem, the following research questions are to be answered within the scope of this research: 1. Which SME-specific approaches to CSV in the medical device industry are dealt with in the scientific literature, and what differences or similarities do these approaches have?2. What concrete recommendations for action can be derived from the given CSV procedures in the medical device industry for SMEs? 3. What are the implications for future research in the area of CSV?
This research is structured as follows: First, the necessary terminological basics are explained in section two.In section three, the underlying research methodology is described.In section four, SME-specific approaches to CSV will be collected and analysed based on a literature review.In this context, the results are compared and evaluated with regard to the research questions.
Finally, the contribution concludes with a device production process must be validated for its intended use.This requirement applies to any software used to automate, for example, device design, manufacturing or complaint design, as well as to any computer system used to create, modify and maintain electronic records or electronic signatures (FDA, 2002).

Research Methodology
As described at the outset of this research, major challenges exist in the medical device industry regarding the search for appropriate CSV approaches.To investigate this complex problem, a literature review is used to determine existing approaches for SMEs in the medical device industry.However, the heterogeneity of the companies established in the medical device industry (Schönberger and Čirjevskis, 2017) makes it more difficult to represent branch-specific procedures and approaches to CSV.Thus, the survey of a complete and up-to-date overview of the literature is hardly realizable.Therefore, the aim of this research is to identify central literature on the problem of the CSV, thus, the results of the literature review listed (see Table 1) in this chapter do not claim to be exhaustive.Rather, a general orientation of the literature is to be presented.The structure of the literature review, which is based on the methods of Schönberger et al. (2014) and Mikelsone and Liela (2015), is described below.

Problem formulation
The literature review will provide an overview of the content-thematic orientation of the literature.The objectives of the review are to identify relevant literature on CSV in SMEs in order to compare and analyse validation approaches.Furthermore, this analysis is intended to show whether recommendations for the implementation of validation approaches exist and, if available, whether they counteract the challenges already described for the implementation of validation projects.

Literature Research
For the implementation of a systematic literature review, the search area was narrowed down according to the following criteria: • Content limitation: The literature considered should refer as comprehensive as possible to various approaches to CSV in the medical device industry.For a better comparability of the approaches, the term "SME" is used universally without any specific industry reference within the medical device industry.Furthermore, the software and computer systems described in the literature should be business information systems, e.g.enterprise resource planning systems or software development environments, and not software systems used in medical devices, as these are not the focus of the CSV.

Literature Analysis
Various criteria are used for the comparison and the analysis of the collected literature.
On the one hand, the literature has to be addressed to SMEs or has to be oriented towards SMEs, on the other hand, the literature must show a systematic approach to CSV and recommendations for action for SMEs.A selection and definition of formal and didactic comparison criteria, for example the year of appearance, page numbers, learning objectives or best practices, will not be undertaken.

Results of the Literature Research
At the beginning of the study, a total of 1,231 sources of literature were identified in which the term "computer system validation" was mentioned (see Appendix 2 In the second phase of the literature review, eleven relevant literature sources were identified after the excluding of duplicates by using the aforementioned keywords.These results were structured according to author, title, publication type, year and focused sector (see Table 1).Following this, the identified literature was examined and selected for its relevance with regard to the research topic and the previously defined limitation criteria.For this purpose, the abstracts, indexes and introductions of the publications were analysed.Below, the contents of the identified literature sources, shown in Table 1, are briefly explained and summarised  Tracy and Nash (2002) do not focus on the medical devices industry, the described approaches can be applied to companies in the named industry.
In a first step, the CSV procedures are examined superficially, and the individual phases of the procedures are analysed and compared to the phases of the general approach.
The results of the analysis and the comparison of the procedures are shown in Figure 1.
The grey fields represent missing process steps compared to the general approach as well as to the other procedures.The comparison shows that the validation approaches have a common basic pattern, although the individual procedures vary in the number of phases, the description of the phases and the temporal sequence.As can be seen, there are differences as well as similarities regarding the beginning and the end of each validation process.While the EU Guideline provides for the preparation of a validation master plan within the planning phase of the  For a more detailed comparison of the pr cedures, the tasks and activities, which are required for carrying out the CSV, are co sidered in addition to the listing of the s perordinate phases and the chronological sequence (see Figure 1).For this purpose, the tasks occurring within the individual valid tion approaches were analysed in more and assigned to the phases of the general validation approach (see Figure 2).
In this context, a total of 43 tasks and activ ties were identified.Through the assignment, commonalities within the procedures could be determined.In the planning phase procedure requires the description of the equipment, systems and processes to be validated, as well as the consideration of standard operation procedures.Although the definition of a master validation plan is only provided by Bhusnure et al.For a more detailed comparison of the procedures, the tasks and activities, which are required for carrying out the CSV, are considered in addition to the listing of the superordinate phases and the chronological sequence (see Figure 1).For this purpose, the tasks occurring within the individual validation approaches were analysed in more detail and assigned to the phases of the general validation approach (see Figure 2).
In this context, a total of 43 tasks and activities were identified.Through the assignment, commonalities within the procedures could be determined.In the planning phase, each procedure requires the description of the equipment, systems and processes to be validated, as well as the consideration of standard operation procedures.Although the definition of a master validation plan is only provided by Bhusnure et al. (2015) and Yogesh et al. (2015) (see Figure 1), this task is also required within the planning phase by all authors.In the phase of specifying the user requirements, the development of test procedures as well as the provision of information on the computer system, project documentations and task responsibilities are required.Necessary tasks within the design qualification phase are the documentation of the hardware and software required for val dation as well as the development of a design qualification document.Although the phase of the factory and site acceptance testing is proposed by the EU Guideline, the therefore required tasks are only defined by Tracy and Nash (2002), Esch et al. (2007), and Hrgarek (2008).This can be explained by the fact that the test of the equipment and the docume tation of this test is more frequently required in laboratories.In this context, Huber (2005) also recommends the implementation of these tasks (see Table 2).Within the install tion qualification phase, the documentation of instructions for verifying the installation of hardware and software is mainly provided.Within the installation qualification phase, the documentation of instructions for verif ing the installation of hardware and software is mainly provided.Furthermore, the testing of the operational and performance functions as well as the documentation of these tests are essential tasks of the operational and performance qualification phases.Hrgarek (2008).This can be explained by the fact that t of the equipment and the documentation of this test is more frequently required in laboratories.In this context, Huber (2005) the implementation of these tasks (see Table 2).Within the installation qualification phase, the documentation of instructions for verifying the installation of hardware and software is mainly provided.Within the installation qualification phase, the documentation of instructions for verifying the installation of hardware and software ore, the testing of the operational and performance functions as well as the documentation of these tests are essential tasks of the operational and performance qualification phases.During the qualification phase, the implementation of t tasks as well as measures for ongoing validation are recommended by Tracy and Nash (2002) Criticism of the procedures for computer system validation in the medical device industry and SMEs The following weaknesses of existing validation approaches can be identified from the results of the comparison that provide an explanation for the existing challenges in CSV in SMEs as well as answers to the formulated research questions.

No references to SMEs
As described before, approaches to CSV in SMEs should be identified by means of literature analysis.However, in none of the identified literature sources, references are given to the adoption or the use of validation approaches in SMEs.Thus, it seems that although SMEs form the backbone of the European economy, they remain widely unobserved (Schönberger and Kleinert, 2016;Schubert et al., 2007).Therefore, it is surprising that this clientele is hardly the focus of current research projects and that only a few research papers, regarding the use and implementation of approaches to CSV, can be identified (see Table 1), despite the fact that research focusing on computer systems in SMEs, mainly business information systems, has been recommended by the research community for several years (e.g.Schönberger und Kleinert, 2016; Eckl et al., 2010; Devos et al., 2014).Due to the fact that the approaches cannot be delimited by the procedures in large companies, SMEs continue to face the challenge of selecting and applying a suitable process for CSV.

No differences to general approach
The comparison of the approaches to CSV and the tasks identified in this context have shown that there are no major differences to the general approach suggested by the European Commission (2015).Although some procedures differ in the structure from the general approach, a detailed analysis has shown that the tasks and activities of the identified procedures can be attributed to each phase of the general approach.Thus, the identified and evaluated validation approaches do not provide any clear delimitation criteria against general approaches.As a result, European SMEs also face the problem of establishing the requirements and tasks of the EU Guideline for CSV in their own individual corporate structures.

No description of recommendations for action
As initially described, the introduction of an approach to CSV is a difficult and timeintensive task for SMEs because, compared to larger enterprises, there is often a lack of specific knowledge, time and the necessary staff.For the implementation of an approach to CSV, clear instructions are needed for SMEs.However, the authors described and explained the individual steps within the validation process as well as the activities and tasks for the implementation of the validation, but clear and specific recommendations for action are missing.Although Esch et al. (2007) clearly summarise necessary recommendations for responsibilities, activities, and documents in the context of the CSV, however, it is not always apparent in which validation phase the recommended activities and documents have to be carried out or developed respectively.Moreover, neither tasks nor activities specifically addressed to SMEs could be identified.Thus, the already discussed heterogeneity of the medical device industry can be verified (see chapter 2.2), which makes the standardisation of a procedure for CSV more difficult and, therefore, it is also not always possible to provide general recommendations for SMEs.

Limitation and outlook
This research has tried to close the research gap initially mentioned by using a qualitative study.Based on a literature analysis, procedures for CSV were identified and the neces- sary tasks and activities were analysed.In summary, the literature collected on the subject area is very homogeneous.In particular, the identified literature sources reveal a clear shortcoming with regard to the delimitation to general validation approaches.Based on the results and problem areas of CSV in SMEs achieved and discussed in this research, the need for an SME-specific approach to CSV is demonstrated.Furthermore, it should be noted that the aspects examined in this research on the investigation of approaches to CSV in SMEs has not yet been addressed in other research papers.This research has several minor limitations.As the research focus was European SMEs in the medical technology industry, literature was used to analyse and evaluate CSV approaches that could refer to or be transferred to this industry (see Figure 1).By considering all approaches to the CSV collected during the literature review, recommendations for action could possibly be identified.However, it must be checked here whether these recommendations for action can be transferred to the medical device industry.Furthermore, the results obtained from this research would gain further in quality through a wider literature analysis with regard to the expansion of the search area.
Finally, and to answer the third research question, this work provides several connecting factors for further research work.The elaboration of SME-specific recommendations for CSV could be an approach to further research.Furthermore, this work could serve as the basis for the development of an SMEspecific validation approach, which may be preferred to the European Commission's proposal.Further research will also result from the introduction of the new Medical Device Regulation, which replaces the existing directives 90/385/EEC and 93/42/EEC directive in Europe (European Union, 2017).As changes to the regulations may have an impact on the processes of CSV, a reverification of the validation approaches and thus further research becomes necessary.

Figure 1 :
Figure 1: Comparison of approaches to computer system validation.
(2015)   gesh et al.(2015)  (see Figure1), this task is also required within the planning phase by all authors.In the phase of specifying the user requirements, the development of test procedures as well as the provision of info mation on the computer syst documentations and task responsibilities are required.Necessary tasks within the design Innovation Management in Small and Medium Enterprise ____________________________________________________________________________________ Marius Schönberger and Tatjana Vasiljeva (2018), Journal of Innovation Management in Small and Medium 512744 : Comparison of approaches to computer system validation.

8 ____________________________________________________________________________________),
During the re-qualification phase, the implementation of change management tasks as well as mea ures for ongoing validation are reco mended by Tracy and Nash (2002), Esch et al. (2007), and Bhusnure et al. (2015).In the Journal of Innovation Management in Small and Medium : Comparison of approaches to computer system validation.qualification phase are the documentation of the hardware and software required for validation as well as the development of a design t.Although the phase of the factory and site acceptance testing is proposed by the EU Guideline, the therefore required tasks are only defined by Tracy and Nash (2002), Esch et al. (2007), and , Esch et al. (2007), and Bhusnure et al. (2015).In the ____________________________________________________________________________________ ______________ Marius Schönberger and Tatjana Vasiljeva (2018), Journal of Innovation Management in Small and Medium Enterprise, DOI: 10.5171/2018.512744 Tatjana Vasiljeva (2018), Journal of Innovation Management in Small and Medium Enterprise, DOI: 10.5171/2018.512744

Table 1 :
Results of the literature review using all keywords ____________________________________________________________________________________ ______________ Marius Schönberger and Tatjana Vasiljeva (2018), Journal of Innovation Management in Small and Medium Enterprise, DOI: 10.5171/2018.512744Esch et al. ____________________________________________________________________________________ ______________ Marius Schönberger and Tatjana Vasiljeva (2018), Journal of Innovation Management in Small and Medium Enterprise, DOI: 10.5171/2018.512744