Towards Computer System Validation: An overview and Evaluation of Existing Procedures

Introduction

Software products are increasingly becoming the central component of complex electronic devices that control or support technical or business processes. Software has thus become a significant economic factor and occupies an important position not only in business, science and technology, but also in the healthcare sector (Schönberger, 2014). Thereupon, electronic and computer-based devices are also being widely deployed in clinical environment, which was enabled by, for example, shrinking technologies, portability, or increasing interconnectedness (Alemzadeh et al., 2013). Therefore, software plays an important role in the critical functions of medical devices. However, from 2002 to 2010, medical devices based on software resulted in over 537 recalls affecting more than 1.5 million devices. During this period, 11.3% of all recalls were due to software failures (Fu, 2011). As Appendix 1 shows, a total of 195 recalls for medical devices, resulting from software errors, were reported to the US Food and Drug Administration (FDA) only in 2017 (FDA, 2018). It is apparent that medical devices are often subject to a high number of errors with potentially catastrophic effects on the patients (Alemzadeh et al., 2013). According to AAMI (2016), medical errors are involved in more than 250,000 deaths each year in the US and account for nearly one in ten deaths. Finally, the recall of medical devices poses a problem that affects the entire healthcare sector (Foe Owono, 2015; Maisel et al., 2001).

Against this background, it seems that with the current technological possibilities of the software development, a considerable increase in the complexity of the devices and thus major challenges for the reliability, patient safety and security emerge (Faris, 2006; Fu, 2011; Alemzadeh et al., 2013). Therefore, the computer system validation (CSV) is a major means of avoiding such defects and resultant recalls (Bhusnure et al., 2015) and, according to the FDA, a requirement of the quality system for medical device manufactures (FDA, 2002). The analysis of different literature on the topic of CSV shows that the implementation of CSV is not only necessary due to legal regulations, but also from economic, social and technological aspects (e.g. Vogel, 2011; Thirumalai and Sinha, 2011; Faris, 2006; Fu, 2011). However, the regulations only determine that a CSV has to be carried out and which system or software type is to be taken into account in the company (e.g. FDA, 2002; ISO, 2016), the exact scope of the validation as well as a structured approach are not specified. This leads to the challenge, that all medical device manufacturers shall determine what needs to be validated and how much validation is enough to ensure that regulatory requirements are met (Hrgarek, 2008).

This key problem forms the basis for the present research and, thus, the question arises, whether standardised procedures for the implementation of the CSV exist for the medical device industry. Although there are various guidelines, e.g. the Gamp 5 Guide, ISO 26262-8 Section 11 or IEEE Std. 829, they cannot always reflect the existing diversity of industries, size differences, or special requirements of, in particular, small and medium-sized medical device manufacturers. In addition, the implementation of these directives in SMEs is often hampered by limited resources, such as the lack of existing staff capacities or insufficient funding (Nguyen, 2009; Razak et al., 2009; Buschfeld et al., 2011). With regard to the aforementioned key problem, the following research questions are to be answered within the scope of this research:

1. Which SME-specific approaches to CSV in the medical device industry are dealt with in the scientific literature, and what differences or similarities do these approaches have?
2. What concrete recommendations for action can be derived from the given CSV procedures in the medical device industry for SMEs?
3. What are the implications for future research in the area of CSV?

This research is structured as follows: First, the necessary terminological basics are explained in section two. In section three, the underlying research methodology is described. In section four, SME-specific approaches to CSV will be collected and analysed based on a literature review. In this context, the results are compared and evaluated with regard to the research questions. Finally, the contribution concludes with a summary of findings and an outlook on further research activities in section five.

Basic Terminology

SMEs in Europe

SMEs play an important role in Europe’s society and economy: The vast majority (99.8%) of all enterprises in the European Union are SMEs, 93% of these are micro enterprises with fewer than ten employees. SMEs employ more than half of the European workforce (66.8%) and are estimated to be responsible for 57.4% of the value added (Muller et al., 2016). In view of the fact that there is no single definition for SMEs nor any clear demarcation of SMEs and large enterprises (Schönberger and Kleinert, 2016), for this research the definition according to the proposal by the European Commission is used (European Commission, 2003). According to this recommendation, an enterprise that has fewer than ten employees and an annual turnover or annual total balance sheet not exceeding two million euros is defined as a micro enterprise. Small enterprises are companies that have fewer than 50 employees and an annual turnover or annual total balance sheet not exceeding ten million euros. Companies are referred to as medium-sized enterprises if they employ fewer than 250 employees and have an annual turnover not exceeding 50 million euro or an annual total balance sheet not exceeding 43 million euro (European Commission, 2003).

Medical device industry in Europe

The medical technology sector is one of the most innovative industries in Europe (Klein, 2016). According to the research of MedTech (2015), around 7.5% of the total expenditure in health care is spent on medical technology in 2015. Furthermore, 95% of the 25,000 medical technology companies in Europe are SMEs employing more than 575,000 people (MedTech, 2015). With approx. 30% of the worldwide expenditure on medical devices, Europe holds one of the biggest markets for medical technology (Klein, 2016). The medical devices market in Europe is one of the sectors actively regulated by directives (Foe Owono, 2015). As sometimes the lives of patients depend on the proper functioning of medical devices, new devices have to undergo a stringent regulatory process before they are ready to enter the market (Klein, 2016). Two prominent regulatory bodies are responsible for defining, updating and verifying compliance with these regulations for medical devices in Europe: The European Medical Device Directive (93/42/EEC) regulates which medical devices can be sold in the European market, while the FDA uses the 21 CFR 820 Quality Systems Regulation to prescribe medical device manufacturers and how to establish and maintain a quality assurance system (Francum, 2014). Although the FDA is primarily responsible for the US market, many European medical device manufacturers are following FDA regulations, even if they do not sell their medical devices in the US.

Computer system validation in the medical device industry

The term validation is used in the corresponding software-related literature in very different meanings and must first be clearly defined and delimited. Sommerville (2011) describes the term validation as follows: “[…] validation is intended to show that a system both conforms to its specification and that it meets the expectations of the system customer”. The DIN ISO 9000:2015 norm defines validation as a “confirmation, through the provision of objective evidence, that the requirements for a specific intended use or application have been fulfilled” (ISO, 2015). In this context, CSV is a requirement of the quality system regulation. “Validation requirements apply to software used as components in medical devices, to software that is itself a medical device, and to software used in production of the device or in implementation of the device manufacturer’s quality system” (FDA, 2002). According to the DIN norm mentioned before, any software used to automate any part of the medical device production process must be validated for its intended use. This requirement applies to any software used to automate, for example, device design, manufacturing or complaint design, as well as to any computer system used to create, modify and maintain electronic records or electronic signatures (FDA, 2002).

Research Methodology

As described at the outset of this research, major challenges exist in the medical device industry regarding the search for appropriate CSV approaches. To investigate this complex problem, a literature review is used to determine existing approaches for SMEs in the medical device industry. However, the heterogeneity of the companies established in the medical device industry (Schönberger and Čirjevskis, 2017) makes it more difficult to represent branch-specific procedures and approaches to CSV. Thus, the survey of a complete and up-to-date overview of the literature is hardly realizable. Therefore, the aim of this research is to identify central literature on the problem of the CSV, thus, the results of the literature review listed (see Table 1) in this chapter do not claim to be exhaustive. Rather, a general orientation of the literature is to be presented. The structure of the literature review, which is based on the methods of Schönberger et al. (2014) and Mikelsone and Liela (2015), is described below.

Problem formulation

The literature review will provide an overview of the content-thematic orientation of the literature. The objectives of the review are to identify relevant literature on CSV in SMEs in order to compare and analyse validation approaches. Furthermore, this analysis is intended to show whether recommendations for the implementation of validation approaches exist and, if available, whether they counteract the challenges already described for the implementation of validation projects.

Literature Research

For the implementation of a systematic literature review, the search area was narrowed down according to the following criteria:

• Content limitation: The literature considered should refer as comprehensive as possible to various approaches to CSV in the medical device industry. For a better comparability of the approaches, the term “SME” is used universally without any specific industry reference within the medical device industry. Furthermore, the software and computer systems described in the literature should be business information systems, e.g. enterprise resource planning systems or software development environments, and not software systems used in medical devices, as these are not the focus of the CSV.
• Linguistic limitation: According to the research background, European SMEs from the medical device industry are the focus of the literature review. Therefore, for a better comparability, only English literature is considered for this study. Due to significant differences between the definition of SMEs in the US and the European Commission’s proposal (SBA, 2017), literature sources are excluded, which refer to the definition of SMEs in the US, as this makes it difficult to compare the English literature.
• Limitation of the type of publication: In order to reach the best possible and appropriate results through the literature review, research papers, conference papers, proceedings, monographs, books as well as dissertations and habilitation treatise are examined and evaluated.

As the focus on SMEs implies the discovery of only a few sources, there is no limitation regarding the time of the publication. A time limitation could thus possibly prevent the identification of older relevant sources of literature. Nevertheless, a consideration of current literature is being aspired.

Literature Analysis

Various criteria are used for the comparison and the analysis of the collected literature. On the one hand, the literature has to be addressed to SMEs or has to be oriented towards SMEs, on the other hand, the literature must show a systematic approach to CSV and recommendations for action for SMEs. A selection and definition of formal and didactic comparison criteria, for example the year of appearance, page numbers, learning objectives or best practices, will not be undertaken.

Research findings

Results of the Literature Research

At the beginning of the study, a total of 1,231 sources of literature were identified in which the term “computer system validation” was mentioned (see Appendix 2). However, this search has not yet been restricted to the medical device industry, SMEs or standardised procedures. The following literature databases were searched for literature research: Google Scholar, Elsevier, Scopus, Ebsco Academic Search, Sciencedirect, Springer Link and Sage Journals. In a second step, a comprehensive keyword search was carried out in order to achieve better results for the literature review. Therefore, the following keywords were used: Software, Computer, Validation, Software Validation, Computer Validation, Medical Device, Medical Device Industry, SME, Business Information System, Validation Process, Software Validation Process, Computer Validation Process, Validation Approach, Software Validation Approach and Computer Validation Approach. Finally, for the collection of other relevant literature, the bibliographies of the already determined results were analysed.

In the second phase of the literature review, eleven relevant literature sources were identified after the excluding of duplicates by using the aforementioned keywords. These results were structured according to author, title, publication type, year and focused sector (see Table 1). Following this, the identified literature was examined and selected for its relevance with regard to the research topic and the previously defined limitation criteria. For this purpose, the abstracts, indexes and introductions of the publications were analysed. Below, the contents of the identified literature sources, shown in Table 1, are briefly explained and summarised

Table 1: Results of the literature review using all keywords

The contents of the identified literature refer to

• risk management approaches to CSV (Charan and Vishal Gupta, 2016; von Culin, 2011; Hrgarek, 2008; Tracy and Nash, 2002; European Commission, 2015; FDA, 2002),
• the use of the GAMP 5 standard for CSV (Tracy and Nash, 2002; Hrgarek, 2008; European Commission, 2015; Charan and Vishal Gupta, 2016),
• new approaches to CSV (Tracy and Nash, 2002; Hrgarek, 2008; McDowall, 2016),
• comparison of various approaches or methods to CSV (Tracy and Nash, 2002; McDowall, 2016),
• general guidelines to CSV (Bendale et al., 2011; Esch et al., 2007; European Commission, 2015; FDA, 2002; Yogesh et al., 2015; Huber, 2005),
• approaches for CSV of laboratory information systems (Tracy and Nash, 2002; Esch et al., 2007), and
• ethical security measures in the context of CSV (Bhusnure et al., 2015).

The identified approaches to CSV were most frequently published in journals and focused on the medical device industry, pharmaceuticals and laboratories sectors. The results of the identified research contributions have some similarities: The majority of the authors describe a risk-based approach to CSV or indicate that risks are to be identified and assessed during the CSV (Charan and Vishal Gupta, 2016; von Culin, 2011; Hrgarek, 2008; Tracy and Nash, 2002, Esch et al., 2007). Some authors see the GAMP 5 standard as a suitable approach to CSV that can be applied in companies without major adaptation (Tracy and Nash, 2002; Hrgarek, 2008; Charan and Vishal Gupta, 2016). Others refer to general CSV approaches given by the European Commission (2015) or the FDA (2002), for example Bendale et al. (2011) or Yogesh et al. (2015).

In order to allow a comparison of the different approaches to CSV, the literature sources listed in Table 1 were analysed more precisely. To ensure better comparability of procedures and as this research focuses on the European sector, the approach of the European Commission (2015) has been chosen as a general approach that complies with the GAMP 5 standard. Therefore, the approach of the FDA (2002) will not be considered further. Based on the analysis of the literature sources, it has been shown that the authors Bendale et al. (2011), Huber (2005), McDowall (2016), and von Culin (2011) do not show any specific approaches or procedures, but rather describe the principles of CSV. The approach of Charan and Vishal Gupta (2016) was also excluded because they used the GAMP 5 standard and, thus, it corresponds to the general approach. Finally, the approaches of the authors Bhusnure et al. (2015), Esch et al. (2007), Hrgarek (2008), Tracy and Nash (2002) and Yogesh et al. (2015) were selected for the comparison. Although Bhusnure et al. (2015), Esch et al. (2007), and Tracy and Nash (2002) do not focus on the medical devices industry, the described approaches can be applied to companies in the named industry.

In a first step, the CSV procedures are examined superficially, and the individual phases of the procedures are analysed and compared to the phases of the general approach. The results of the analysis and the comparison of the procedures are shown in Figure 1. The grey fields represent missing process steps compared to the general approach as well as to the other procedures. The comparison shows that the validation approaches have a common basic pattern, although the individual procedures vary in the number of phases, the description of the phases and the temporal sequence. As can be seen, there are differences as well as similarities regarding the beginning and the end of each validation process. While the EU Guideline provides for the preparation of a validation master plan within the planning phase of the CSV (European Commission, 2015), this task is defined as an independent process step by Bhusnure et al. (2015), Yogesh et al. (2015), and Tracy and Nash (2002). Moreover, the approaches of Bhusnure et al. (2015), Hrgarek (2008), and Esch et al. (2007) go beyond the conventional validation and describe further phases for error correction, system retirement or vendor audits. Furthermore, the approaches of Bhusnure et al. (2015), Tracy and Nash (2002) and Esch et al. (2007) end with the phase of the revalidation which complies with the EU Guideline. In this context, the revalidation or re-qualification is defined as the performance of a controlled manner to maintain the validated status after any change of the computer system during its operational use (European Commission, 2015; Esch et al., 2007). Within the approach of Esch et al. (2007), the re-qualification is divided into the two steps system in progress and change control. The approach of Yogesh et al. (2015) ends with the phase of the performance qualification, which is justified by the fact that at the end of this phase, the software has been checked for its intended purpose and, thus, the validation process has ended. Finally, it is conspicuous that the approach of Hrgarek (2008) establishes the installation qualification as a validation-inducing phase and has no preceding phases.

Figure 1: Comparison of approaches to computer system validation.

For a more detailed comparison of the procedures, the tasks and activities, which are required for carrying out the CSV, are considered in addition to the listing of the superordinate phases and the chronological sequence (see Figure 1). For this purpose, the tasks occurring within the individual validation approaches were analysed in more detail and assigned to the phases of the general validation approach (see Figure 2).

In this context, a total of 43 tasks and activities were identified. Through the assignment, commonalities within the procedures could be determined. In the planning phase, each procedure requires the description of the equipment, systems and processes to be validated, as well as the consideration of standard operation procedures. Although the definition of a master validation plan is only provided by Bhusnure et al. (2015) and Yogesh et al. (2015) (see Figure 1), this task is also required within the planning phase by all authors. In the phase of specifying the user requirements, the development of test procedures as well as the provision of information on the computer system, project documentations and task responsibilities are required. Necessary tasks within the design qualification phase are the documentation of the hardware and software required for validation as well as the development of a design qualification document. Although the phase of the factory and site acceptance testing is proposed by the EU Guideline, the therefore required tasks are only defined by Tracy and Nash (2002), Esch et al. (2007), and Hrgarek (2008). This can be explained by the fact that the test of the equipment and the documentation of this test is more frequently required in laboratories. In this context, Huber (2005) also recommends the implementation of these tasks (see Table 2). Within the installation qualification phase, the documentation of instructions for verifying the installation of hardware and software is mainly provided. Within the installation qualification phase, the documentation of instructions for verifying the installation of hardware and software is mainly provided. Furthermore, the testing of the operational and performance functions as well as the documentation of these tests are essential tasks of the operational and performance qualification phases. During the re-qualification phase, the implementation of change management tasks as well as measures for ongoing validation are recommended by Tracy and Nash (2002), Esch et al. (2007), and Bhusnure et al. (2015). In the follow-up phase, the documentation of Failure Mode and Effects Analysis (FMEA) is defined by Bhusnure et al. (2015) and Hrgarek (2008) as the final validation task. As the follow-up phase of the CSV is not provided by each of the approaches, there are only a few similarities. Within all validation approaches, the documentation of the user requirements specification as well as the design, installation, operational and performance qualification is required.

Figure 2: Overview of the phase-related task and activities for computer system validation.

Criticism of the procedures for computer system validation in the medical device industry and SMEs

The following weaknesses of existing validation approaches can be identified from the results of the comparison that provide an explanation for the existing challenges in CSV in SMEs as well as answers to the formulated research questions.

1. No references to SMEs

As described before, approaches to CSV in SMEs should be identified by means of literature analysis. However, in none of the identified literature sources, references are given to the adoption or the use of validation approaches in SMEs. Thus, it seems that although SMEs form the backbone of the European economy, they remain widely unobserved (Schönberger and Kleinert, 2016; Schubert et al., 2007). Therefore, it is surprising that this clientele is hardly the focus of current research projects and that only a few research papers, regarding the use and implementation of approaches to CSV, can be identified (see Table 1), despite the fact that research focusing on computer systems in SMEs, mainly business information systems, has been recommended by the research community for several years (e.g. Schönberger und Kleinert, 2016; Eckl et al., 2010; Devos et al., 2014). Due to the fact that the approaches cannot be delimited by the procedures in large companies, SMEs continue to face the challenge of selecting and applying a suitable process for CSV.

2. No differences to general approach

The comparison of the approaches to CSV and the tasks identified in this context have shown that there are no major differences to the general approach suggested by the European Commission (2015). Although some procedures differ in the structure from the general approach, a detailed analysis has shown that the tasks and activities of the identified procedures can be attributed to each phase of the general approach. Thus, the identified and evaluated validation approaches do not provide any clear delimitation criteria against general approaches. As a result, European SMEs also face the problem of establishing the requirements and tasks of the EU Guideline for CSV in their own individual corporate structures.

3. No description of recommendations for action

As initially described, the introduction of an approach to CSV is a difficult and time-intensive task for SMEs because, compared to larger enterprises, there is often a lack of specific knowledge, time and the necessary staff. For the implementation of an approach to CSV, clear instructions are needed for SMEs. However, the authors described and explained the individual steps within the validation process as well as the activities and tasks for the implementation of the validation, but clear and specific recommendations for action are missing. Although Esch et al. (2007) clearly summarise necessary recommendations for responsibilities, activities, and documents in the context of the CSV, however, it is not always apparent in which validation phase the recommended activities and documents have to be carried out or developed respectively. Moreover, neither tasks nor activities specifically addressed to SMEs could be identified. Thus, the already discussed heterogeneity of the medical device industry can be verified (see chapter 2.2), which makes the standardisation of a procedure for CSV more difficult and, therefore, it is also not always possible to provide general recommendations for SMEs.

Limitation and outlook

This research has tried to close the research gap initially mentioned by using a qualitative study. Based on a literature analysis, procedures for CSV were identified and the necessary tasks and activities were analysed. In summary, the literature collected on the subject area is very homogeneous. In particular, the identified literature sources reveal a clear shortcoming with regard to the delimitation to general validation approaches. Based on the results and problem areas of CSV in SMEs achieved and discussed in this research, the need for an SME-specific approach to CSV is demonstrated. Furthermore, it should be noted that the aspects examined in this research on the investigation of approaches to CSV in SMEs has not yet been addressed in other research papers.

This research has several minor limitations. As the research focus was European SMEs in the medical technology industry, literature was used to analyse and evaluate CSV approaches that could refer to or be transferred to this industry (see Figure 1). By considering all approaches to the CSV collected during the literature review, recommendations for action could possibly be identified. However, it must be checked here whether these recommendations for action can be transferred to the medical device industry. Furthermore, the results obtained from this research would gain further in quality through a wider literature analysis with regard to the expansion of the search area.

Finally, and to answer the third research question, this work provides several connecting factors for further research work. The elaboration of SME-specific recommendations for CSV could be an approach to further research. Furthermore, this work could serve as the basis for the development of an SME-specific validation approach, which may be preferred to the European Commission’s proposal. Further research will also result from the introduction of the new Medical Device Regulation, which replaces the existing directives 90/385/EEC and 93/42/EEC directive in Europe (European Union, 2017). As changes to the regulations may have an impact on the processes of CSV, a re-verification of the validation approaches and thus further research becomes necessary.

References

1. AAMI (2016) ‘Medical errors involved in nearly 1 in 10 U.S. deaths, study says,’ Association for the Advancement of Medical Instrumentation, 10 May 2016. [Online], [Retrieved February 15, 2018], http://www.aami.org/newsviews/newsdetail.aspx?ItemNumber=3612.
2. Alemzadeh, H., Ravishankar, K. I., Kalbarczyk, Z. and Raman, J. (2013) ‘Analysis of Safety-Critical Computer Failures in Medical Devices,’ IEEE Security and Privacy Magazine, 11(4), 14-26.
3. Bendale, A., Patel, N., Damahe, D. P., Narkhede, S. B., Jadhav, A. G. and Vidyasagar, G. (2011) ‘Computer software validation in pharmaceuticals,’ Asian Journal of Pharmaceutical Sciences and Clinical Research, 1(2), 27-39.
4. Bhusnure, O. G., Kendre, S. P., Kaudewar, D. R., Bankar, I. N., Gholve, S. B. and Giram, P. S. (2015) ‘Computer validation and ethical security measures for pharmaceutical data processing,’ World Journal of Pharmaceutical Research, 5(1), 1092-1134.
5. Buschfeld, D., Dilger, B., Hess, L., Schmid, K. and Voss, E. (2011) ‘Identification of future skills needs in micro and craft (-type) enterprises up to 2020,’ Final report, January 2011, Cologne, Hamburg, Vienna.
6. Charan, H. Y. and Vishal Gupta, N. (2016) ‘Gamp 5: A quality risk management approach to computer system validation,’ International Journal of Pharmaceutical Sciences Review and Research, 36(1), 195-198.
7. Devos, J., van Landeghem, H. and Deschoolmeester, D. (2014) ‘Information Systems for Small and Medium-Sized Enterprises. State of Art of IS Research in SMEs,’ Heidelberg et al.: Springer.
8. Eckl, V., Trettin, L., Engel, D., Rothgang, M. and Espig, T. (2010) ‘E-Business in micro and small Enterprises: A chance for growth or a sheet anchor for survival?,’ Baltic Management Review, 5(1), 27-46.
9. Esch, P. M., Donze, G., Eschbach, B., Hassler, S., Hutter, L., Saxer, H. P., Timm, U. and Zühlke, R. (2007) ‘Good Laboratory Practice (GLP) – Guidelines for the validation of computerised systems,’ The Quality Assurance Journal, 11(3‐4), 208-220.
10. European Commission (2003) ‘Commission recommendation of 6 May 2003 concerning the definition of micro, small and medium-sized enterprises,’ 2003/361/EC, Brussels.
11. European Commission (2015) ‘EU Guidelines for Good Manufacturing Practice for Medical Products for Human and Veterinary Use, Annex 15: Qualification and Validation,’ Brussels, 30 March 2015.
12. European Union (2017) ‘Regulation (EU) 2017/745 of the European Parliament and of the Council on medical devices, amending Directive 2001/83/EC, Regulation (EC) No 178/2002 and Regulation (EC) No 1223/2009 and repealing Council Directives 90/385/EEC and 93/42/EEC,’ Brussels, 5 April 2017.
13. Faris, T. H. (2006) ‘Safe and Sound Software. Creating and Efficient and Effective Quality System for Software Medical Device Organizations,’ Milwaukee: ASQ Quality Press.
14. FDA (2002) ‘General Principles of Software Validation. Final Guidance for Industry and FDA Staff,’ January 11, 2002. [Online], [Retrieved February 15, 2018] http://www.fda.gov/downloads/ MedicalDevices/DeviceRegulationandGuidance/GuidanceDocuments/ucm085371.pdf.
15. FDA (2018) ‘Medical Device Recalls. FDA database of medical device recalls’ [Online], [Retrieved February 15, 2018] http://www.accessdata.fda.gov/scripts/cdrh/cfdocs/cfRES/res.cfm.
16. Foe Owono, G. (2015) ‘Impact of EU medical device directive on medical device software,’ Dissertation, Minneapolis: Walden University.
17. Francum, J. (2014) ‘The European Medical Device Directive as it compares to 21 CFR 820,’ 10 May 2014. [Online], [Retrieved February 15, 2018] http://www.gxp-cc.com/news/fda-european-regulations-for-life-sciences/2014/03/10/the-european-medical-device-directive-as-it-compares-to-21-cfr-820/.
18. Fu, K. (2011) ‘Trustworthy medical device software,’ Public Health Effectiveness of the FDA 510(k) Clearance Process: Measuring Postmarket Performance and Other Select Topics: Workshop Report, Washington, D.C., July 2011. IOM (Institute of Medicine), National Academies Press, 97-118.
19. Hrgarek, N. (2008) ‘A management approach to software validation requirements,’ Central European Conference on Information & Intelligent Systems (CECIIS) 2008. Varaždin, Croatia, September 24-26, 219-226.
20. Huber, L. (2005) ‘Qualification and validation of software and computer systems in laboratories. Part 3. Installation and operational qualification,’ In: De Bièvre, P. and Günzler, H. (eds.) ‘Validation in Chemical Measurement,’ Berlin, Heidelberg: Springer, 140-144.
21. ISO (2015) ‘Quality management systems – Fundamentals and vocabulary,’ International Organization for Standardization, ISO 9000:2015(en), 2015. [Online], [Retrieved February 15, 2018] https://www.iso.org/obp/ui/#iso:std:45481:en.
22. ISO (2016) ‘Medical devices – Quality management systems – Requirements for regulatory purposes,’ International Organization for Standardization, ISO 13485:2016(en), 2016. [Online], [Retrieved February 15, 2018] https://www.iso.org/obp/ui/#iso:std:iso:13485:ed-3:v1:en.
23. Klein, T. (2016) ‘The medtech revolution: The European medical technology industry,’ In: Martorell, J. M., Barberà, A., Farré, A., Bertalan, M., Norah, G., Biosca, I., Estrada, G., Klein, T. and Shahin, T. (eds.) ‘Catalonia life sciences and healthcare outlook,’ April 2016, 80-91.
24. Maisel, W. H., Sweeney, M. O., Stevenson, W. G., Ellison, K. E. and Epstein, L. M. (2001) ‘Recalls and safety alerts involving pacemakers and implantable cardioverter-defibrillator generators,’ The Journal of the American Medical Association, 286(7), 793-799.
25. McDowall, R. D. (2016) ‘Welcome to the Brave New World of CSV?,’ Advances in Pharmaceutical Analysis, LCGC Europe, 29(2), 93-96.
26. MedTech (2015) ‘The European Medical Technology Industry In Figures’ [Online], [Retrieved February 15, 2018] http://www.medtecheurope.org/sites/default/files/resource_items/files/ MEDTECH_FactFigures_ONLINE3.pdf.
27. Mikelsone, E. and Liela, E. (2015) ‘Literature review of idea management: Focuses and gabs,’ Journal of Business Management, 2015, No. 9, 107-121.
28. Muller, P., Devnani, S., Julius, J., Gagliardi, D. and Marzocchi, C. (2016) ‘Annual Report on European SMEs 2015/2016. SME recovery continues. Final report,’ November 2016. [Online], [Retrieved February 15, 2018] http://ec.europa.eu/DocsRoom/documents/20264/attachments/1/ translations/en/renditions/native.
29. Nguyen, T. H. (2009) ‘Information technology adoption in SMEs: an integrated framework,’ International Journal of Entrepreneurial Behavior & Research, 15(2), 162-186.
30. Razak, I. H. A., Kamaruddin, S., Azid, I. A. and Almanar, I. P. (2009) ‘ISO 13485:2003: Implementation reference model from the Malaysian SMEs medical device industry,’ The TQM Journal, 21(1), 6-19.
31. SBA (2017) ‘Table of Small Business Size Standards. U.S. Small Business Administration,’ updated 01 October 2017. [Online], [Retrieved February 15, 2018] https://www.sba.gov/ contracting/getting-started-contractor/make-sure-you-meet-sba-size-standards/table-small-business-size-standards.
32. Schönberger, M. (2014) ‘Der professionelle Einstieg in die erfolgreiche App-Entwicklung,‘ In: Aichele, C. and Schönberger, M. (eds.) ‘App4U. Mehrwerte durch Apps im B2B und B2C,‘ Wiesbaden: Springer, 87-132.
33. Schönberger, M. and Čirjevskis, A. (2017) ‘Successful IT/IS projects in healthcare: Evaluation of critical success factors,’ Journal of e-health Management. Vol. 2017.
34. Schönberger, M. and Kleinert, T. (2016) ‘Results of a qualitative survey about the application of business information systems in German craft enterprises – first findings of an ongoing research project,’ Proceedings of the European Conference on Information Systems (ECIS) 2016, June 12-15, Istanbul, Turkey.
35. Schönberger, M., Kleinert, T., Dumont, T., Fettke, P. and Loos, P. (2014) ‘Softwareauswahl in kleinen und mittelständischen Unternehmen. Gegenüberstellung und Bewertung von Vorgehensmodellen zur Auswahl betrieblicher Anwendungssoftware,’ Proceedings of the Multikonferenz Wirtschaftsinformatik (MKWI) 2014, 26-28 February, Paderborn, Germany, 1979-1991.
36. Schubert, P., Fisher, J. and Leimstoll, U. (2007) ‘ICT and innovation in small companies,‘ Proceedings of the European Conference on Information Systems (ECIS) 2007, St. Gallen, Switzerland, June 7-9, 1226-1239.
37. Sommerville, I. (2011) ‘Software Engineering,’ 9th edition, Boston: Pearson Education.
38. Thirumalai, S. and Sinha, K. K. (2011) ‘Product recalls in the medical device industry: an empirical exploration of the sources and financial consequences,’ Management Science, 57(2), 376-392.
39. Tracy, D. S. and Nash, R. A. (2002) ‘A Validation Approach for Laboratory Information Management Systems,’ Journal of Validation Technology, 9(1), 6-14.
40. Vogel, D. A. (2011) ‘Medical Device Software. Verification, Validation, and Compliance,’ Boston, London: Artech House.
41. Von Culin, R. (2011) ‘New Approach to System Validation,’ Applied Clinical Trials, 20(2), 32-37.
42. Yogesh, P., Kamlesh, M., Mohini, B., Phad, R., Ismail, S. and Shivam, L. (2015) ‘Computer system validation: a review,’ World Journal of Pharmaceutical Research, 4(9), 444-454.

Appendix

Appendix 1: Total medical device recalls resulting from software errors in 2017. Own elaboration using the FDA Medical Device Recall Database. Recall date from 01/01/2017 to 31/12/2017 (FDA, 2018).

Appendix 2: Total number of literature sources that have been identified in various literature databases with regard to the term “computer system validation”. Own elaboration using the databases Google Scholar, Elsevier, Scopus, Ebsco Academic Search, Sciencedirect, Springer Link and Sage Journals. Databases accessed 11 February 2018