Improving the Use of E-learning in Health Care Curricula : Presentation of best Practices

E-learning is nowadays included in different health care curricula. However, this learning and teaching approach is still relatively underused in the health care study programmes. Hence, the goal of this article is to present three e-learning best practices in health care, which were validated in our previous projects/studies.First best practice, e-learning practice contents for the development of critical thinking, presents the approach which resulted in significant (p<0.001) raise in the number of discussion posts (+12.2%) for which the opinions and conclusions of the participants were justified with valid arguments. Furthermore, also the survey results showed that students achieve critical thinking skills with these contents. Second best practice, inclusion of the health care students in the development of e-contents, also resulted as a valid approach for improving the use of e-learning in health care curricula, which was confirmed by the expert opinion and e-contents final users. Finally, third best practice, our approach for evaluation and comparison of e-learning and traditional pedagogical elements value by health care students and teachers, was confirmed in the interview with the management as a valid approach that helps the management to identify whether previous elearning initiatives and activities were successful. Best practices, presented in this paper, represent the evidence that the use of modern information and communication technologies should be considered in the future as a serious requisite in health care study programmes. The broader community interested in the e-learning initiative should prepare a list of similar best practices, validate its effectiveness and prepare guidelines for their optimal implementation..


Introduction
In the last decade, e-learning has become increasingly pervasive in higher education (Zacharis 2011) and life-long learning (Atanasijević- Kunc et al. 2009).Efficient and effective e-learning solutions are present in many areas.Different studies confirm the increasing number of e-learning users and use cases (Pustišek & Kos 2003;Hess & Gunter 2013).Despite the fact that elearning is generally accepted as it allows the use of innovative and flexible approaches to learning and teaching in health sciences (Moule 2006), it is still relatively underused in the health care study programmes.Elearning has great potential because it facilitates learner-centered learning which can be adapted to different learning styles and desired levels of knowledge (Zacharis 2011).Learner-centered learning encourages learners' autonomy by promoting active learning and teaching, where creation, communication and cooperation play a prominent role in the educational process (Papič et al. 2011).Unfortunately, the ones in charge of curricula in education institutions in health care are often insufficiently aware of these advantages.
In order to support the development of elearning in the field of health care, we present some best practices that were implemented at the University of Primorska, Faculty of Health Sciences, which can help other institutions to improve their e-learning initiatives.Some of the presented best practices are based on scientific studies, which were performed at the aforementioned institution.

Best practices of e-learning implementation in health care study programmes
In this section, our e-learning best practices are presented: e-learning contents for the health care students' development of critical thinking (as follows CT) of health care students, inclusion of health care students in the development of e-contents, and measuring the acceptance of e-learning by health care students.

E-learning contents for the development of CT
Modern health care study and life-long education programmes should take into consideration that health care workers have to deal effectively with complex change, increased demands and greater accountability.Thus, individuals responsible for the preparation of formal and informal health care education should take into consideration the acquisition of CT skills.These are a prerequisite for the effective deliberation of ethical issues in personal and professional life (Matchett 2009).CT is not an approach or method to be learned, but rather a process, an orientation of the mind, and thus is more demanding and includes both the cognitive and affective domains of reasoning (Simpson & Courtney 2002).
One important challenge in the education of health care experts is the need to keep the teaching/learning process captivating and interesting by making students actively participate in their own intellectual growth (Paul 1990).Royse and Newton (2007) encourage the use of active learning environments that promote the active involvement of the participants of education.
Results of literature review, presented in our past study (Pucer, Trobec & Žvanut 2014), indicated that there are different information communication technology (ICT) based approaches for the acquisition of CT skills that are learner-centred and interactive.However, they can be time consuming to develop and costly to produce.Hence, our challenge was to develop a cost effective, simple, fun, and educational approach for designing e-learning contents for the development of CT.
Our approach for developing CT skills is based on scenarios representing different clinical situations by using different ICT resources: e.g.on-line comics, videos, virtual, and augmented reality.A scenario consists of different situations presenting various dilemmas, i.e. a problem that offers two possibilities (Metcalf & Yankou 2003).Each situation's possibility leads to a new situation without a clear solution, where students are faced with the consequences of their previous decisions.The example a scenario is presented as state diagrams in figure 1.The e-contents based on the presented approach always lead to new unfavourable situations.According to Doane (2002), a student should be invited into the abyss, into the difficulties that are part of the nursing practice.Students should grapple with the uncertainty and ambiguity, which is present in health care practice.The facts that each decision is incorrect lead the student to think twice about the situation and to reconsider possible consequences.Unfortunately, this uncertainty is often present in our everyday lives.The results of the survey, performed after the participation in the last discussion board indicated that participants perceived the econtents based on the presented approach as useful, and that they improved their CT skills.The students were asked to rate, whether the aforementioned items, related to the aforementioned elements of CT, were acquired in the e-contents.The average values of the responses were higher than 4 (on a five-point Likert scale) (for further details, please refer to Pucer, Trobec & Žvanut 2014).The developed e-contents are presented on the site inoedu.info(n.d.).Similarly, the interviews with randomly chosen students indicated the presence of the aforementioned elements.Example of Selfregulation: "These e-contents taught me on how to react in real situations.In future I will think twice before making decisions in clinical practice.
[student4]" Repeated confirmation of the validity of the presented approach through methodological triangulation represents a strong indication that the presented approach is well accepted by nursing students.Our study represents evidence that the use of modern ICT should be considered in future as a serious requisite for achieving CT skills.These educational materials can be implemented by using modern ICTs that are simple, fun, and time and cost effective.Further work in this field includes the development and test of similar e-contents for various health care and other groups.

Inclusion of nursing students in the econtents development projects
In the traditional pedagogical process, the roles of teacher and student are distinct, where students are often expected to adopt a receptive mode.Unfortunately, this practice is often used also in e-learning environments.However, different studies (e.g.McNaught, Lam & Cheng 2007; Lam et al. 2009) show that in e-learning such distinction is not justified and that students can take an active role in shaping their e-learning environment.
Although, there are many studies about the introduction of e-learning in different health care study programmes, we identified only one study (e.g.Lam et al. 2009), where pharmacy students were recruited in the shaping of e-learning environment.In this study, the positive aspects of such approach are presented: enhancing of e-learning environment and consequently of the entire course, improving student's enthusiasm due to authenticity and meaningfulness of the tasks, increasing students' selfempowerment due to the fact that shaping of e-learning environment is a "real-life" task and not merely a simulation.With the recruitment of students in e-learning design projects, the traditional student's role expands as students should acquire some teacher's competences, on the other hand, teachers, in order to keep the virtual learning environment under control, should focus on moderating the students' activities and their training.
Among the many challenges that nurse educators face today is the need to keep the teaching/learning process captivating and interesting, which can be done through the use of active learning environments (Royse & Newton 2007).Well-designed interactive elearning contents can be used to implement active learning environments (Huffaker & Calvert 2003).Furthermore, also the use of authentic learning, which engages students in real-world problems by simulating clinical practice and, thus, are more motivating for learners (Herrington, Reeves & Oliver 2010), should be considered.Hence, we decided to offer to the students the possibility to shape their e-learning environment by developing Patrik Pucer, Boštjan Žvanut and Damjan Vavpotič (2016), Journal of e-Learning and Higher Education, DOI: 10.5171/2016.515170 educative e-learning contents as a project for their BSc thesis and as a seminar in MSc course "Nursing simulations".
In 2011, we offered to the students the possibility to prepare the BSc and MSc theses.The main goal of these theses was to develop attractive and effective e-contents in different fields of nursing: health promotion, nursing informatics, basic life support.The econtents were developed with the use of the free application Courselab 2.4 (n.d.) and implemented in learning management system Moodle.Some students decided to improve the attractiveness of their e-contents with the use of web application BitStrips (n.d.).The application was used to create different characters, situations and objects in order to make the situations more vivid.The scenarios were presented as UML state diagrams, where each page of the developed e-content represents a concrete situation in a developed e-learning content, similarly as in section 2.1.In the continuation is presented an example of a student's content where potential risks for information security violations are presented as real scenarios.The student identified the scenarios during interviews with two IT experts, which presented some exemplar situations of the information security violations from their past experience (Rošeto 2013).Accordingly, using the aformentioned tools, five scenarios were developed: personal data abuse, abuse of credentials, communication of personal data through phone, surfing the web on the hospital computers, and infected USB stick.These scenarios were validated by two IT and healthcare experts and presented to nurses in an institution.An excerpt of the econtent abuse of credentials is presented in figure 2. The validity of including nursing students in the e-contents development projects was also confirmed by the following affirmation: "This is a clear evidence that nursing students can develop interesting e-contents efficient for education [IT expert 2]."

Evaluation and comparison of e-learning and traditional pedagogical elements value by health care students and teachers
Nowadays, pedagogical processes use a variety of teaching approaches, methods and tools (PE -pedagogical elements).Traditional PE (e.g.books, lectures) are being challenged by new e-learning PE based on different information technologies (e.g.ediscussion boards, digital learning material).These e-learning PE on one hand bring many advantages but on the other hand also some disadvantages (Brown 2011;Gabriela 2009;Gunnar 2009).Therefore, it is important to be able to distinguish between PE that are more or less appropriate for a certain setting.According to Haw and Keating (2006), the implementation of pedagogical processes and their PEs should be appropriately evaluated.However, existing evaluation models have two important limitations.One limitation is that they either evaluate the pedagogical process as a whole (i.e. at the level of curricula or even institutions) thus neglecting the specifics of PEs (e.g.Haw  To overcome these difficulties, we developed an evaluation model (Vavpotič, Žvanut & Trobec 2013) that facilitates a comparative evaluation of different e-learning and traditional PEs by concurrently considering students' and teachers' perspectives.The proposed model consists of three main dimensions: the opportunities for use of PE (Opp), the quality of knowledge gained by the students that use the PE (Qual) and the value of PE for students (Value).The Opp and the Qual dimensions are evaluated by teachers who use the evaluated PEs in their pedagogical processes, while the Value dimension is evaluated by students who also use these same PEs as a part of the pedagogical processes they attend.The Opp dimension is measured through a single property: the frequency of opportunities for use of the evaluated PE regardless of whether the PE is actually being used or not.More opportunities for use of a certain PE means that such PE has the potential to play a more important role in the pedagogical process than an infrequently used PE.The Qual dimension measures the impact of the PE on the quality of knowledge gained by the students that use the PE.This dimension is measured through five properties considering declarative and functional dimensions of knowledge (Biggs & Tang 2011): the completeness, consistency and usability of obtained knowledge, and the possibility to obtain complex and deep knowledge.The Value dimension is measured through five properties: frequency of use in case of a given opportunity, improvement in the speed of learning, improvement in the quality of learning, usefulness in learning and complexity.These properties are based on the diffusion of innovations theory (Rogers 2003)  simultaneously evaluated by students and teachers for each PE, we gain a comprehensive insight into the value of each evaluated PE and its comparison to other evaluated PEs.This insight forms the ground for preparation of proper actions to improve PEs and the pedagogical process as a whole.
We tested the described evaluation model at the Faculty of health sciences, University of Primorska, Slovenia at the end of the 2010/2011 academic year before the first exams.The evaluation protocol was as follows.First the PEs for evaluation were catalogued by focus groups in which the teachers involved in the evaluated pedagogical processes participated, as well.Only those PEs were catalogued that were used by teachers in their pedagogical processes.Next, questionnaires for the catalogued PEs were prepared for students and teachers who were then asked to evaluate the PEs.Finally, the results of the evaluation were presented to the school management (i.e. the dean, the vice dean, the head of the nursing department, and the president of the governance board) and the selected PEs were further analysed with the intention to prepare suitable improvement actions.
The results of one of these evaluations is shown in figure 3. The results are presented on a scatter plot divided into four quadrants (Q1-Q4).Each dot on the scatter plot represents a single PE marked by a unique number.Grey dots represent e-learning PEs, while white dots represent traditional PEs.The size of each dot represents the Qual dimension, horizontal axis represents the Opp dimension and the vertical axis represents the Value dimension.In order to improve the pedagogical process, teachers and management should focus primarily on the PEs that have a considerable positive impact on the quality of knowledge and are positioned in either Q2 or Q3.For the PEs in Q2 (e.g., Figure 3, PE 10), teachers and management should investigate whether it is possible to alter the pedagogical process to create more opportunities for the use of such PEs or to introduce new similar PEs with more opportunities for use.In the case of the PEs in Q3 (e.g., Figure 3, PE 8), further investigations are required and actions should be taken to improve their value for students by either properly presenting them to the students or by replacing them with PEs that are more acceptable from the students' perspective.Different actions should be taken for the PEs in Q1 (e.g., Figure 3, PE 13).These PEs neither have high value for students nor high opportunity for use.Consequentially, teachers should determine whether these PEs can be replaced by other PEs or removed from the pedagogical process.Finally, the PEs in Q4 (e.g., Figure 3  The proposed approach was tested in three nursing pedagogical processes based on the same curriculum.The vice dean, responsible for study programmes, confirmed that this approach helped her to identify problematic PEs, for example both the dean and the vice dean were previously unaware of the low acceptance of e-discussion boards (Figure 3, PE 14).Vice dean noted: "Obviously this approach can help us to identify less effective PE and to provide valid arguments for their improvements actions." All the members of the management confirmed that the results provided by the evaluation model gave them important and valuable information.They used the information to direct their improvement efforts with focus on selected PE.These improvement efforts for the selected PE included: promotion of PE among students, further deep analysis of PE, increasing the opportunities for use of PE, and substitution of PE with another PE.The dean concluded: "The periodical use of the presented approach would will help us to monitor the effectiveness of past pedagogical process improvements actions."

Conclusion
This paper presents some effective best practices for improving the use of e-learning in health care institutions.Nowadays, pedagogical processes use a variety of pedagogical elements, from traditional books and lecturing to e-discussions and interactive videos, each with its advantages and disadvantages.Therefore it, is important to be able to distinguish between elements that are more or less appropriate for a certain setting.The proposed model for the comparative evaluation of e-learning and the traditional pedagogical elements permits its users to observe the value of pedagogical processes in detail, to detect less suitable pedagogical elements, and to formulate focused actions for the improvement of pedagogical processes through the improvement of their pedagogical elements.____________________________________________________________________________________ Therefore, the model provides valuable information to the teachers and the management (of the pedagogical process) identifying whether previous pedagogical initiatives and activities were successful and uncovering parts that need improvements.Only pedagogical processes with optimal pedagogical elements can support the learners' development and preservation of long term knowledge and critical thinking skills.
Another important challenge of the aforementioned pedagogical processes is to keep the teaching/learning process captivating and interesting by making learners actively participate in their own intellectual growth and thus allowing them to develop critical thinking skills.There are different information communication technology based approaches for the acquisition of critical thinking skills that are learner-centred and interactive but time consuming to develop and costly to produce.Our scenario based e-learning contents are simple, fun and time and cost effective.Besides that, such e-contents are proved to be useful and help to develop critical thinking by grappling learners with the uncertainty and ambiguity, which are present in health care practice, and leading learners to reconsider possible consequences of their decisions.The use of such an approach, combined with resources like on-line comics, videos, virtual and augmented reality, should be considered as a serious requisite for the achievement of critical thinking skills.
Because the presented approach is inexpensive and does not require more than basic information and communication skills, we are including students in the development of e-contents based on the presented approach.With the recruitment of students, the traditional student's role expands as students acquire some teacher's competences and take an active role in shaping their e-learning environment.Thus, students have been developing contents from their areas of interest by using differently and innovatively authentic learning, which is sometimes very different from the "classics" used by teachers.The developed e-contents are being evaluated by IT and health care experts.The majority of the developed econtents are evaluated as useful and with valid real-world situations from various clinical practices.The development of authentic and attractive e-learning contents with the active participation of students can represent a valid added value for the entire learning programmes.However, the lack of financial resources can represent a serious barrier for similar initiatives as the inclusion of students and the evaluation of e-contents requires additional resources (e.g.students' training, mentoring, data analysis of the evaluation results).
Our best practices represent only a portion of possible initiatives that can be potentially implemented in different health care institutions, from faculties to hospitals.We propose, that the broader community interested in the e-learning initiative should prepare a list of similar best practices, validate its effectiveness and prepare guidelines for their optimal implementation.We believe that such guidelines can improve the use of e-learning in all health care institutions, improving study programmes at faculties or life-long learning programmes in hospitals.

Figure 1 :
Figure 1: An excerpt of the example of the scenario: "Administration of the analgesic"

Figure 2 :
Figure 2: An excerpt from the e-content "Abuse of credentials" (Rošeto, 2013) and the existing studies which use similar properties to measure the relative advantage and the complexity of innovations (Riemenschneider, Hardgrave & Davis 2002; Vavpotič & Hovelja 2012).As the three dimensions are ______________ Patrik Pucer, Boštjan Žvanut and Damjan Vavpotič (2016), Journal of e-Learning and Higher Education, DOI: 10.5171/2016.515170 , PE 5) that have high opportunity for use and value for students should be periodically monitored in order to identify significant changes of their position.____________________________________________________________________________________ ______________ Patrik Pucer, Boštjan Žvanut and Damjan Vavpotič (2016), Journal of e-Learning and Higher Education, DOI: 10.5171/2016.515170

Figure 3 :
Figure 3: Example of the results of the evaluation of PEs in a pedagogical process; PEs enlisted were identified by the focus group (Vavpotič, Žvanut & Trobec 2013) ______________Patrik Pucer, Boštjan Žvanut and Damjan Vavpotič (2016), Journal of e-Learning and Higher Education, DOI: 10.5171/2016.515170