Improving the Use of E-learning in Health Care Curricula: Presentation of best Practices

Patrik Pucer1, Boštjan Žvanut2 and Damjan Vavpotič3

1,2 University of Primorska, Faculty of health sciences, Izola, Slovenia

3 University of Ljubljana, Faculty of Computer and Information Science, Ljubljana, Slovenia

Academic Editor: Sebastian Ion Ceptureanu

Cite this Article as:

Patrik Pucer, Boštjan Žvanut and Damjan Vavpotič (2016)," Improving the Use of E-learning in Health Care Curricula: Presentation of best Practices", Journal of e-Learning and Higher Education, Vol. 2016 (2016), Article ID 515170, DOI: 10.5171/2016.515170

Copyright © 2016. Patrik Pucer, Boštjan Žvanut and Damjan Vavpotič . Distributed under Creative Commons CC-BY 4.0

Abstract

E-learning is nowadays included in different health care curricula. However, this learning and teaching approach is still relatively underused in the health care study programmes. Hence, the goal of this article is to present three e-learning best practices in health care, which were validated in our previous projects/studies.First best practice, e-learning practice  contents for the development of critical thinking, presents the approach which resulted in significant (p<0.001) raise in the number of discussion posts (+12.2%) for which the opinions and conclusions of the participants were justified with valid arguments. Furthermore, also the survey results showed that students achieve critical thinking skills with these contents. Second best practice, inclusion of the health care students in the development of e-contents, also resulted as a valid approach for improving the use of e-learning in health care curricula, which  was confirmed by the expert opinion and e-contents final users. Finally, third best practice, our approach for evaluation and comparison of e-learning and traditional pedagogical elements value by health care students and teachers, was confirmed in the interview with the management as a valid approach that helps the management to identify whether previous e-learning initiatives and activities were successful.Best practices, presented in this paper, represent the evidence that the use of modern information and communication technologies should be considered in the future as a serious requisite in health care study programmes. The broader community interested in the e-learning initiative should prepare a list of similar best practices, validate its effectiveness and prepare guidelines for their optimal implementation..

Keywords: e-learning, health care, best practices.

Introduction

In the last decade, e-learning has become increasingly pervasive in higher education (Zacharis 2011) and life-long learning (Atanasijević-Kunc et al. 2009). Efficient and effective e-learning solutions are present in many areas. Different studies confirm the increasing number of e-learning users and use cases (Pustišek & Kos 2003; Hess & Gunter 2013).  Despite the fact that e-learning is generally accepted as it allows the use of innovative and flexible approaches to learning and teaching in health sciences (Moule 2006), it is still relatively underused in the health care study programmes. E-learning has great potential because it facilitates learner-centered learning which can be adapted to different learning styles and desired levels of knowledge (Zacharis 2011). Learner-centered learning encourages learners’ autonomy by promoting active learning and teaching, where creation, communication and cooperation play a prominent role in the educational process (Papič et al. 2011). Unfortunately, the ones in charge of curricula in education institutions in health care are often insufficiently aware of these advantages. 

In order to support the development of e-learning in the field of health care, we present some best practices that were implemented at the University of Primorska, Faculty of Health Sciences, which can help other institutions to improve their e-learning initiatives. Some of the presented best practices are based on scientific studies, which were performed at the aforementioned institution.

Best practices of e-learning implementation in health care study programmes

In this section, our e-learning best practices are presented: e-learning contents for the  health care students’ development of critical thinking (as follows CT) of health care students, inclusion of health care students in the development of e-contents, and measuring the acceptance of e-learning by health care students.

E-learning contents for the development of CT

Modern health care study and life-long education programmes should take into consideration that health care workers have to deal effectively with complex change, increased demands and greater accountability. Thus, individuals responsible for the preparation of formal and informal health care education should take into consideration the acquisition of CT skills. These are a prerequisite for the effective deliberation of ethical issues in personal and professional life (Matchett 2009). CT is not an approach or method to be learned, but rather a process, an orientation of the mind, and thus is more demanding and includes both the cognitive and affective domains of reasoning (Simpson & Courtney 2002).

One important challenge in the education of health care experts is the need to keep the teaching/learning process captivating and interesting by making students actively participate in their own intellectual growth (Paul 1990). Royse and Newton (2007) encourage the use of active learning environments that promote the active involvement of the participants of education.

Results of literature review, presented in our past study (Pucer, Trobec & Žvanut 2014),  indicated that there are different information communication technology (ICT) based approaches for the acquisition of CT skills that are learner-centred and interactive. However, they can be time consuming to develop and costly to produce. Hence, our challenge was to develop a cost effective, simple, fun, and educational approach for designing e-learning contents for the development of CT. 

Our approach for developing CT skills is based on scenarios representing different clinical situations by using different ICT resources: e.g. on-line comics, videos, virtual, and augmented reality. A scenario consists of different situations presenting various dilemmas, i.e. a problem that offers two possibilities (Metcalf & Yankou 2003). Each situation’s possibility leads to a new situation without a clear solution, where students are faced with the consequences of their previous decisions. The example a scenario is presented as state diagrams in figure 1. The e-contents based on the presented approach always lead to new unfavourable situations. According to Doane (2002), a student should be invited into the abyss, into the difficulties that are part of the nursing practice. Students should grapple with the uncertainty and ambiguity, which is present in health care practice. The facts that each decision is incorrect lead the student to think twice about the situation and to reconsider possible consequences. Unfortunately, this uncertainty is often present in our everyday lives.

An excerpt of the example of the scenario: “Administration of the analgesic

Figure 1: An excerpt of the example of the scenario: “Administration of the analgesic”

In order to test the effectiveness of the developed approach, a quasi-experimental time series design was used as it facilitated pre- and post-testing of the study’s participants. The effectiveness of the developed e-contents was evaluated with the qualitative analysis of discussion boards and an anonymous survey with open- and closed-ended questions. The study was performed in May 2012 during the course “Ethics and Philosophy in Nursing” at the Faculty of Health Sciences, University of Primorska, Slovenia, where three scenarios were developed. Participants were 40 first-year undergraduate nursing students. In discussion boards, real clinical dilemmas were discussed. Qualitative analysis of the discussion boards, which were visited before and after the review of the developed e-contents, was performed. Elements of CT (Facione & Facione, 1994; Facione, Oxman-Michelli & Weinstein 1990): analysis, inference, interpretation, explanation, evaluation, and self-regulation were identified in the discussion boards posts. Results indicate a significant (p<0.001) improvement in the percentage of posts (+12.2%) for which the opinions and conclusions of the participants were justified with valid arguments, i.e. inference and explanation. For example:

“I believe that this solution is acceptable as it is in accordance with the 2nd principle of the Code of Ethics for Nurses [student 1].”

“According the nursing home regulation, we can not support this decision [student 2].”

Furthermore, a significant (p<0.001) reduction of posts where students’ conclusions were without proper justification (−9.7%), e.g. , i.e. inference without any explanation:

“I believe that this is not OK [student 2].”

“I don’t agree with you [student 3].”

The results of the survey, performed after the participation in the last discussion board indicated that participants perceived the e-contents based on the presented approach as useful, and that they improved their CT skills. The students were asked to rate, whether the aforementioned items, related to the aforementioned elements of CT, were acquired in the e-contents. The average values of the responses were higher than 4 (on a five-point Likert scale) (for further details, please refer to Pucer, Trobec & Žvanut 2014). The developed e-contents are presented on the site inoedu.info (n.d.). Similarly, the interviews with randomly chosen students indicated the presence of the aforementioned elements. Example of Self-regulation:

“These e-contents taught me on how to react in real situations. In future I will think twice before making decisions in clinical practice. [student4]”

Repeated confirmation of the validity of the presented approach through methodological triangulation represents a strong indication that the presented approach is well accepted by nursing students. Our study represents evidence that the use of modern ICT should be considered in future as a serious requisite for achieving CT skills. These educational materials can be implemented by using modern ICTs that are simple, fun, and time and cost effective. Further work in this field includes the development and test of similar e-contents for various health care and other groups.

Inclusion of nursing students in the e-contents development projects

In the traditional pedagogical process, the roles of teacher and student are distinct, where students are often expected to adopt a receptive mode. Unfortunately, this practice is often used also in e-learning environments. However, different studies (e.g. McNaught, Lam & Cheng 2007; Lam et al. 2009)  show that in e-learning such distinction is not justified and that students can take an active role in shaping their e-learning environment.

Although, there are many studies about the introduction of e-learning in different health care study programmes, we identified only one study (e.g. Lam et al. 2009), where  pharmacy students were recruited in the shaping of e-learning environment. In this study, the positive aspects of such approach are presented: enhancing of e-learning environment and consequently of the entire course, improving student’s enthusiasm due to authenticity and meaningfulness of the tasks, increasing students’ self-empowerment due to the fact that shaping of e-learning environment is a “real-life” task and not merely a simulation. With  the recruitment of students in e-learning design projects, the traditional student’s role expands as students should acquire some teacher’s competences, on the other hand, teachers, in order to keep the virtual learning environment under control, should focus on moderating the students’ activities and their training.

Among the many challenges that nurse educators face today is the need to keep the teaching/learning process captivating and interesting, which can be done through the use of active learning environments (Royse & Newton 2007). Well-designed interactive e-learning contents can be used to implement active learning environments (Huffaker & Calvert 2003). Furthermore, also the use of authentic learning, which engages students in real-world problems by simulating clinical practice and, thus, are more motivating for learners (Herrington, Reeves & Oliver 2010), should be considered. Hence, we decided to offer to the students the possibility to shape their e-learning environment by developing educative e-learning contents as a project for their BSc thesis and as a seminar in MSc course “Nursing simulations”.

In 2011, we offered to the students the possibility to prepare the BSc and MSc theses. The main goal of these theses was to develop attractive and effective e-contents in different fields of nursing: health promotion, nursing informatics, basic life support.The e-contents were developed with the use of the free application Courselab 2.4 (n.d.) and implemented in learning management system Moodle.

Some students decided to improve the attractiveness of their e-contents with the use of web application BitStrips (n.d.). The application was used to create different characters, situations and objects in order to make the situations more vivid. The scenarios were presented as UML state diagrams, where each page of the developed e-content represents a concrete situation in a developed e-learning content, similarly as in section 2.1. In the continuation is presented an example of a student’s content where potential risks for information security violations are presented as real scenarios. The student identified the scenarios during interviews with two IT experts, which presented some exemplar situations of the information security violations from their past experience (Rošeto 2013). Accordingly, using the aformentioned tools, five scenarios were developed: personal data abuse, abuse of credentials, communication of personal data through phone, surfing the web on the hospital computers, and infected USB stick. These scenarios were validated by two IT and healthcare experts and presented to nurses in an institution. An excerpt of the e-content abuse of credentials is presented in figure 2.
An excerpt from the e-content “Abuse of credentials” (Rošeto, 2013)

Figure 2: An excerpt from the e-content “Abuse of credentials” (Rošeto, 2013)
 

Both experts confirmed the usefulness and the validity of the developed contents:

“I believe that these e-contents can help the students and also health professionals to acquire knowledge in the field of IT security more effectively than in traditional manner [IT expert 1].”

“Presenting potential IT security violations in such attractive and effective way can help nurses to be more aware of these risks  [IT expert 2].”

The validity of including nursing students in the e-contents development projects was also confirmed by the following affirmation:

“This is a clear evidence that nursing students can develop interesting e-contents efficient for education [IT expert 2].”

Evaluation and comparison of e-learning and traditional pedagogical elements value by health care students and teachers

Nowadays, pedagogical processes use a variety of teaching approaches, methods and tools (PE — pedagogical elements). Traditional PE (e.g. books, lectures) are being challenged by new e-learning PE based on different information technologies (e.g. e-discussion boards, digital learning material). These e-learning PE on one hand bring many advantages but on the other hand also some disadvantages (Brown 2011; Gabriela 2009; Gunnar 2009). Therefore, it is important to be able to distinguish between PE that are more or less appropriate for a certain setting. According to Haw and Keating (2006), the implementation of pedagogical processes and their PEs should be appropriately evaluated. However, existing evaluation models have two important limitations. One limitation is that they either evaluate the pedagogical process as a whole (i.e. at the level of curricula or even institutions) thus neglecting the specifics of PEs (e.g. Haw & Keating 2006), or they evaluate only a single PE thus overlooking other parts of the pedagogical process (e.g., Fernández Alemán, Carrillo de Gea & Rodríguez Mondéjar 2011; Kelly et al. 2009). The second limitation of many studies is that they focus only on one group of PE users, typically only on teachers (e.g. Yuen & Ma 2008) or only on students (e.g. Richardson 2005) and examine the perspective of one group only. However, this is inadequate as teachers might prefer certain PEs even if they are not well accepted by the students, while students might favour PEs which are less effective or unpopular among teachers.

To overcome these difficulties, we developed an evaluation model (Vavpotič, Žvanut & Trobec 2013) that facilitates a comparative evaluation of different e-learning and traditional PEs by concurrently considering students’ and teachers’ perspectives. The proposed model consists of three main dimensions: the opportunities for use of PE (Opp), the quality of knowledge gained by the students that use the PE (Qual) and the value of PE for students (Value). The Opp and the Qual dimensions are evaluated by teachers who use the evaluated PEs in their pedagogical processes, while the Value dimension is evaluated by students who also use these same PEs as a part of the pedagogical processes they attend. The Opp dimension is measured through a single property: the frequency of opportunities for use of the evaluated PE regardless of whether the PE is actually being used or not. More opportunities for use of a certain PE means that such PE has the potential to play a more important role in the pedagogical process than an infrequently used PE. The Qual dimension measures the impact of the PE on the quality of knowledge gained by the students that use the PE. This dimension is measured through five properties considering declarative and functional dimensions of knowledge (Biggs & Tang 2011):  the completeness, consistency and usability of obtained knowledge, and the possibility to obtain complex and deep knowledge. The Value dimension is measured through five properties: frequency of use in case of a given opportunity, improvement in the speed of learning, improvement in the quality of learning, usefulness in learning and complexity. These properties are based on the diffusion of innovations theory (Rogers 2003) and the existing studies which use similar properties to measure the relative advantage and the complexity of innovations (Riemenschneider, Hardgrave & Davis 2002; Vavpotič & Hovelja 2012). As the three dimensions are simultaneously evaluated by students and teachers for each PE, we gain a comprehensive insight into the value of each evaluated PE and its comparison to other evaluated PEs. This insight forms the ground for preparation of proper actions to improve PEs and the pedagogical process as a whole.

We tested the described evaluation model at the Faculty of health sciences, University of Primorska, Slovenia at the end of the 2010/2011 academic year before the first exams. The evaluation protocol was as follows. First the PEs for evaluation were catalogued by focus groups in which the teachers involved in the evaluated pedagogical processes participated, as well. Only those PEs were catalogued that were used by teachers in their pedagogical processes. Next, questionnaires for the catalogued PEs were prepared for students and teachers who were then asked to evaluate the PEs. Finally, the results of the evaluation were presented to the school management (i.e. the dean, the vice dean, the head of the nursing department, and the president of the governance board) and the selected PEs were further analysed with the intention to prepare suitable improvement actions.

The results of one of these evaluations is shown in figure 3. The results are presented on a scatter plot divided into four quadrants (Q1-Q4). Each dot on the scatter plot represents a single PE marked by a unique number. Grey dots represent e-learning PEs, while white dots represent traditional PEs. The size of each dot represents the Qual dimension, horizontal axis represents the Opp dimension and the vertical axis represents the Value dimension. In order to improve the pedagogical process, teachers and management should focus primarily on the PEs that have a considerable positive impact on the quality of knowledge and are positioned in either Q2 or Q3. For the PEs in Q2 (e.g., Figure 3, PE 10), teachers and management should investigate whether it is possible to alter the pedagogical process to create more opportunities for the use of such PEs or to introduce new similar PEs with more opportunities for use. In the case of the PEs in Q3 (e.g., Figure 3, PE 8), further investigations are required and actions should be taken to improve their value for students by either properly presenting them to the students or by replacing them with PEs that are more acceptable from the students’ perspective. Different actions should be taken for the PEs in Q1 (e.g., Figure 3, PE 13). These PEs neither have high value for students nor high opportunity for use. Consequentially, teachers should determine whether these PEs can be replaced by other PEs or removed from the pedagogical process. Finally, the PEs in Q4 (e.g., Figure 3, PE 5) that have high opportunity for use and value for students should be periodically monitored in order to identify significant changes of their position.
 Example of the results of the evaluation of PEs in a pedagogical process; PEs enlisted were identified by the focus group (Vavpotič, Žvanut & Trobec 2013)

Figure 3: Example of the results of the evaluation of PEs in a pedagogical process; PEs enlisted were identified by the focus group (Vavpotič, Žvanut & Trobec 2013)

The proposed approach was tested in three nursing pedagogical processes based on the same curriculum. The vice dean, responsible for study programmes, confirmed that this approach helped her to identify problematic PEs, for example both the dean and the vice dean were previously unaware of the low acceptance of e-discussion boards (Figure 3, PE 14). Vice dean noted:

“Obviously this approach can help us to identify less effective PE and to provide valid arguments for their improvements actions.”

All the members of the management confirmed that the results provided by the evaluation model gave them important and valuable information. They used the information to direct their improvement efforts with focus on selected PE. These improvement efforts for the selected PE included: promotion of PE among students, further deep analysis of PE, increasing the opportunities for use of PE, and substitution of PE with another PE. The dean concluded:

“The periodical use of the presented approach would  will help us to monitor the effectiveness of past pedagogical process improvements actions.”

Conclusion

This paper presents some effective best practices for improving the use of e-learning in health care institutions. Nowadays, pedagogical processes use a variety of pedagogical elements, from traditional books and lecturing to e-discussions and interactive videos, each with its advantages and disadvantages. Therefore it, is important to be able to distinguish between elements that are more or less appropriate for a certain setting. The proposed model for the comparative evaluation of e-learning and the traditional pedagogical elements permits its users to observe the value of pedagogical processes in detail, to detect less suitable pedagogical elements, and to formulate focused actions for the improvement of pedagogical processes through the improvement of their pedagogical elements. Therefore, the model provides valuable information to the teachers and the management (of the pedagogical process) identifying whether previous pedagogical initiatives and activities were successful and uncovering parts that need improvements. Only pedagogical processes with optimal pedagogical elements can support the learners’ development and preservation of long term knowledge and critical thinking skills.

Another important challenge of the aforementioned pedagogical processes is to keep the teaching/learning process captivating and interesting by making learners actively participate in their own intellectual growth and thus allowing them to develop critical thinking skills. There are different information communication technology based approaches for the acquisition of critical thinking skills that are learner-centred and interactive but time consuming to develop and costly to produce. Our scenario based e-learning contents are simple, fun and time and cost effective. Besides that, such e-contents are proved to be useful and help to develop critical thinking by grappling learners with the uncertainty and ambiguity, which are present in health care practice, and leading learners to reconsider possible consequences of their decisions. The use of such an approach, combined with resources like on-line comics, videos, virtual and augmented reality, should be considered as a serious requisite for the achievement of critical thinking skills.

Because the presented approach is inexpensive and does not require more than basic information and communication skills, we are including students in the development of e-contents based on the presented approach. With the recruitment of students, the traditional student’s role expands as students acquire some teacher’s competences and take an active role in shaping their e-learning environment. Thus, students have been developing contents from their areas of interest by using differently and innovatively authentic learning, which is sometimes very different from the “classics” used by teachers. The developed e-contents are being evaluated by IT and health care experts. The majority of the developed e-contents are evaluated as useful and with valid real-world situations from various clinical practices. The development of authentic and attractive e-learning contents with the active participation of students can represent a valid added value for the entire learning programmes. However, the lack of financial resources can represent a serious barrier for similar initiatives as the inclusion of students and the evaluation of e-contents requires additional resources (e.g. students’ training, mentoring, data analysis of the evaluation results).

Our best practices represent only a portion of possible initiatives that can be potentially implemented in different health care institutions, from faculties to hospitals. We propose, that the broader community interested in the e-learning initiative should prepare a list of similar best practices, validate its effectiveness and prepare guidelines for their optimal implementation. We believe that such guidelines can improve the use of e-learning in all health care institutions, improving study programmes at faculties or life-long learning programmes in hospitals.

References

1.    Atanasijević-Kunc, M, Logar, V, Karba, R, Papić, M & Bešter, J. (2009) ‘E-learning and control-system design using a virtual and remote laboratory,’ Journal for fluid power and automation  13 (1), 144-150.

2.    Biggs, JB & Tang, C. (2011) Teaching for Quality Learning at University: what the student does (4th ed.), The Society for Research into Higher Education, Open University Press, McGraw-Hill Education, Maidenhead.
Google Scholar

3.    Bitstrips. (n.d.)  Comics starring YOU and your Friends [Online]. [Accessed 20 October 2014]. Available: http://www.bitstrips.com/.

4.    Brown, SA. (2011) ‘Seeing Web 2.0 in context: A study of academic perceptions,’ The internet and higher education, 15(1), 50—57.

5.    CourseLab. (n.d.) Welcome to CourseLab! [Online]. [Accessed 20 October 2014]. Available: http://www.courselab.com/view_doc.html?mode=home.

6.    Doane, GH. (2002) ‘In the spirit of creativity: the learning and teaching of ethics in nursing,‘ Journal of advanced nursing  39 (6), 521—528.
PublisherGoogle Scholar

7.    Facione, PA & Facione, NC. (1994) Holistic Critical Thinking Scoring Rubric, California Academic Press, Millbrae, CA.
Google Scholar

8.    Facione, PA, Oxman-Michelli, W & Weinstein, M. (1990) Critical Thinking: A Statement of Expert Consensus for Purposes of Educational Assessment and Instruction, California Academic Press, Millbrae, CA.

9.    Fernández Alemán, JL, Carrillo de Gea, JM & Rodríguez Mondéjar, JJ. (2011) ‘Effects of competitive computer-assisted learning versus conventional teaching methods on the acquisition and retention of knowledge in medical surgical nursing students,’ Nurse education today, 31 (8), 866-871.
PublisherGoogle Scholar

10.    Gabriela, G. (2009). ‘To use or not to use web 2.0 in higher education?,’ Procedia – Social and Behavioral Sciences 1(1), 478-482.
Publisher Google Scholar

11.    Gunnar, A. (2009) ‘Web 2.0, pedagogical support for reflexive and emotional social interaction among Swedish students,’ The internet and higher education 13 (4), 197-205.

12.    Haw, MA & Keating, SB. (2006) Adaptation of Curriculum Development and Evaluation of Staff Development and Client-Centered Health Education, Curriculum Development and Evaluation in Nursing, Keating, SB. (ed.),  Lippincott Williams & Wilkins, Philadelphia etc.

13.    Herrington, J, Reeves, TC & Oliver, R. (2010) A guide to authentic e-learning, Routledge, London and New York.
Google Scholar

14.    Hess, T & Gunter, G. (2013) ‘Serious game-based and nongame-based online courses: Learning experiences and outcomes,’ British journal of educational technology 44 (3), 372—385.
PublisherGoogle Scholar

15.    Huffaker, DA & Calvert, SI (2003) ‘The new science of learning: active learning, metacognition, and transfer of knowledge in e-learning applications,’ Journal of educational computing research, 29 (3), 325- 334.
PublisherGoogle Scholar

16.    Inoedu.info. (n.d.). Acquisiton of Critical Thinking with ICTs. [Online]. [Accessed 21 October 2014]. Available: http://www.inoedu.info/how-to-acquire-critical-thinking-with-ict.

17.    Kelly, M, Lyng, C, McGrath, M & Cannon, G. (2009) ‘A multi-method study to determine the effectiveness of, and student attitudes to, online instructional videos for teaching clinical nursing skills,‘ Nurse education today 29 (3), 292-300.
PublisherGoogle Scholar

18.    Lam, P, Yeung, MA, Cheung, C &  McNaught, C. (2009). Using the development of eLearning material as challenging and authentic learning experiences, Same places, Different Spaces, 26th Annual Australian Society for Computers in Learning in Tertiary Education 2009 Conference (ASCILITE), pp. 548-556. Available on:  http://www.ascilite.org.au/conferences/auckland09/procs/lam.pdf, Atkinson, R, McBeath, C. (eds.).

19.    Matchett, NJ. (2009) ‘Cooperative learning, critical thinking, and character: techniques to cultivate ethical deliberation,’ Public Integrity 12 (1), 25—38.
PublisherGoogle Scholar

20.    McNaught, C, Lam, P & Cheng, KF. (2007) ‘Using the design phase of e-learning in higher education as an authentic learning experience for students,’ International Journal of Learning 13 (12), 101-110.
Google Scholar

21.    Metcalf, BL & Yankou, D. (2003) ‘Using gaming to help nursing students understand ethics,’
Journal of nursing education 42 (5), 212-215.
Google Scholar

22.    Moule, P. (2006) ‘E-Learning for Healthcare Students: Developing the Communities of Practice Framework,‘ Journal of advanced nursing 54 (3), 370—80.
PublisherGoogle Scholar

23.    Papić, M, Zebec, L, Pogačnik, M, Bešter, J, Atanasijević-Kunc, M & Logar, V. (2011) ‘Personalized learning environment E-CHO’. EUROSIM simulation news Europe 22 (2), 17-24.

24.    Paul, R. (1990). Critical Thinking. Centre for Critical Thinking and Moral Critique. Sonoma
State University, Rohnert Park, CA.

25.    Pucer, P, Trobec, I & Žvanut, B. (2014) ‘An information communication technology based approach for the acquisition of critical thinking skills,’ Nurse education today 34(6), 964-970.
PublisherGoogle Scholar

26.    Pustišek, M & Kos, A. (2003) E-learning: Functions, Services and Solutions, E-learning: Functions, Services and Solutions, Kramar, P & Miklavčič, D. (eds), Faculty of Electrical Engineering, Ljubljana.

27.    Richardson, JTE. (2005) ‘Instruments for obtaining student feedback: a review of the literature,‘ Assessment & evaluation in higher education 30 (4), 387-415.
PublisherGoogle Scholar

28.    Riemenschneider, CK,  Hardgrave, BC & Davis, FD. (2002) ‘Explaining Software Developer Acceptance of Methodologies: A Comparison of Five Theoretical Models,’ IEEE transactions on software engineering 28 (12), 1135-1145.
PublisherGoogle Scholar

29.    Rogers, EM. (2003) Diffusion of innovations (5th ed.), Free Press, New York.

30.    Rošeto, S. (2013). Identification and development of e-content of information security in health care : diploma thesis, University of Primorska, Faculty of health sciences, Izola, Slovenia.

31.    Royse, MA & Newton, SE. (2007) ‘How gaming is used as an innovative strategy for nursing
Education,’ Nursing Education Perspectives 28 (5), 263—267.

32.    Simpson, E & Courtney, M. (2002) ‘Critical thinking in nursing education: literature review,’International journal of nursing practice 8 (2), 89-98.
PublisherGoogle Scholar

33.    Vavpotič, D & Hovelja, T. (2012) ‘Improving the evaluation of software development methodology adoption and its impact on enterprise performance,’ Computer Science and Information Systems 9 (1), 165-187.
PublisherGoogle Scholar

34.    Vavpotič, D, Žvanut, B & Trobec, I. (2013). ‘A Comparative Evaluation of E-learning and Traditional Pedagogical Process Elements,’ Educational Technology & Society 16 (3), 76-87.

35.    Yuen, AHK & Ma, WWK. (2008) ‘Exploring teacher acceptance of e-learning technology,’ Asia-Pacific journal of teacher education 36 (3), 229-243.
PublisherGoogle Scholar

36.    Zacharis, NZ. (2011) ‘The Effect of Learning Style on Preference for Web-Based Courses and Learning Outcomes,’ British Journal of Educational Technology 42 (5), 790-800.
PublisherGoogle Scholar

Shares