Maturity Model for Virtual Education

Introduction

Virtual education is part of one of the paradigms of education that have progressively reached important levels of development both nationally and internationally. The startup of the virtual methodology has become a challenge for higher educational institutions like the Universidad Distrital Francisco José de Caldas, an institution which Works for the development and consolidation of an educational offer that allows to make significant contributions to the development of the region in the academic and research field, directing its efforts towards the management of knowledge and appropriation of communication technologies by the productive, academic and social sectors (Universidad Distrital José de Caldas, 2009). In that sense, the university manages conducting the Master´s degree in Mobile Telecommunications with virtual methodology.

The university incorporates the virtual methodology as a part of the Strategic Development Plan 2007-2016 (Universidad Distrital Francisco José de Caldas, 2009) and in response to the Colombian National Education Ministry politics (Ministerio de Educación Nacional, 2010, 2013).  This program is intended to facilitate access to professionals in the area of ​​telecommunication who, for multiple reasons, cannot travel to the facilities of the University. In this sense, the university and the academic program coordination carry out a process to define some of the quality criteria affecting the good development of the academic program for the mentioned methodology. Following this idea, a study is carried out and defined as a maturity model that supports the quality processes of the program to guarantee its constant improvement.

The article starts by defining the concept of virtual education and its characteristics. The second part presents information related to the maturity model implemented in other areas of knowledge, particularly in software development. In the third part, the maturity model applicable to virtual methodology programs is presented, defining the respective criteria and maturity levels with its respective attributes and by which the model is applied to the mentioned academic program in order to validate the presented maturity proposal. The final section presents the conclusions and future work in this line of research.

Virtual Education

The Colombian National Education Ministry defines virtual education as an educative methodology that overcomes the space and time limitations, characterized by learning-teaching strategies, with “the use of telematics reds, TIC and cyberspace like the principal environment in which all or almost all, at least the eighty percent (80%) of the academic activity is carried out.” (Ministerio de Educación Nacional, 2018)

Virtual formation is based on autonomy, self-learning, learning how to learn, time management, and favoring the future graduated competences with models like telecommuting, worker 2.0, knowledge worker and networking. 

However, it is necessary to consider that the mentioned process of implementation and appropriation of the technologies for education requires a change of paradigm from the academic community, a change that can be developed in a stepped and progressive way, so that  new conditions and competences are progressively assimilated and adapted by each teacher and institution, who, in turn, will redirect the student toward the pedagogic and didactic use of information and communication technologies (ICT) in  favor of professional training. (Sousa & Gonzalez-Loureiro, 2017).

In the virtual learning environments, the conditions are created for the student to acquire new knowledge and information of new experiences, new concepts that impact the processes of analysis, reflection and appropriation, with the support of technological means and digital resources. In the same way, these virtual environments encourage the collaborative, autonomous work and self-discipline of the student, who manages his/her time to fulfill the academic objectives proposed by each academic space. Within this perspective, the teaching-learning process will be defined  by  three key characteristics as according to (Cabero Almenara & Llorente Cejudo, 2010; Dhir, Verma, Batta, & Mishra, 2017): 1) Educate for creation: A process centered in knowledge construction; 2) Educate for permanent learning: Learn how to learn; 3) Educate for environmental intelligence: Store, distribute, re-use and recuperate information.

Under this paradigm, the academic activity is not only about the teacher/tutor, but also about the student and the group of study companions. It is not only about giving information in virtual classrooms, but also supporting and helping generate information and knowledge in a continuous process of production and reproduction of information. In general, the paradigm of virtuality is based on facilitating learning and interactivity between the teacher and the student, and between the students themselves, creating an association in a network of people, located in different regions or countries, thus forming an academic Virtual Community (Cabero Almenara & Llorente Cejudo, 2010; Dhir, et al., 2017).

Program development in the Master´s degree Mobile Telecommunications with the virtual methodology counts on the following didactic and pedagogical strategies: 1) Discipline contents for each signature; 2) Database; 3) Virtual classes (use of electronic boards); 4) Virtual assessment; 5) Teachers/tutors and experts in the knowledge area; 6) LMS platform; 7) Video conference platform; 8) Virtual campus; 9) Help table. 

In this sense, education with the virtual methodology is characterized by a set of factors associated with learning processes, management processes, organizational processes and support and follow-up processes. These factors indicate and establish guidelines to evaluate all their contexts and performance, allowing to create a maturity model defined as OCTA (Organization, Content, Technology and Learning), and interacting as a system.

Maturity Model

The models of maturity are of vital importance since they allow having a clear, precise vision of the aspects to evaluate the processes of an organization. The maturity model was initially developed for the processes related to the development and implementation of software.

Maturity Model for Virtual Education

The models of maturity can vary depending on their use and purpose. Although most of them are adaptable to any situation, their common purpose is focused on process improvement methodologies in the field of software engineering (Proenca & Borbinha, 2018).

There are multiple evaluation maturity models in the educational field like: OCDMM (Neuhauser, 2004), ECM2 and E-Learning Maturity Model (Manford, McSporran, Mann & Williamson, 2003; Nazir, Pujeri & Rizvi, 2013). The effects of the present article are used as a base for the EMM model (E-Learning Maturity Model) (Marshall, 2010). The model presents 5 process categories: Learning, Development, Support, Evaluation and Organization; where each of them has a clear objective in the evaluation procedure. 

The OCTA proposed maturity model (Organization, Content, Technology and Learning (Aprendizaje in spanish)), based on an analysis of the existing models of eMM and CMMI (Arcos-Medina, Oñate-Andino, & Pástor, 2018), defines the relevant processes to diagnose and offer a process of improvement to reach the maximum level in terms of criteriaAs far as the virtual higher education is concerned, this model gives importance to the learning process and the relevance and characteristics of the appropriate content for the training process.

Each criterion has its description by level of maturity, fixing the characteristics of the proper criterion to establish and improve the processes inherent to the programs with the virtual methodology, in addition to providing characteristics such as:

• Establishing the beginning of the framework through the use of: common language, parameters, evaluation criteria and objectives that are part of a model well known by the general educative community.
• Constituting a common objective in terms of the use of IT (Information Technologies) in the pedagogic field, which is valid for any educative center.
• Adaptable to any virtual program.
• Allowing the evaluation of the continuous improvement of the use of IT by the educative center.
• Robust to be supported in the application of best practices in the pedagogical processes supported in ICT.

A maturity model allows obtaining an overview of the criteria to achieve the objectives and make an assertive diagnosis of the situation, achieving the potential for the continuous improvement of the processes, which must be aligned with the needs (Proenca & Borbinha, 2018).

OCTA Maturity Model

It is a model of maturity which is derived from the analysis of quality management practices in the internal processes of educational programs with the virtual methodology, thus allowing to evaluate the criteria of the model that are proposed in this model, such as: O= Organization and infrastructure, C= Curriculum and Contents, T=Technology and support and A= (Aprendizaje). It is worth mentioning that the mentioned criteria are aligned with the national educative politics, the education quality conditions and the criteria and good practices for content creation.

Fig. 1: Maturity Model – Own elaboration

OCTA Maturity Model Definition of Criteria

Each criterion is represented with maturity levels, which has been adapted by the maturity model CMMI, PAM DE COBIT 5.0 (Gutiérrez, 2017) between others, and are described next:

Fig. 2: Maturity Levels– Own elaboration

Students Learning

The learning process of students has a group of characteristics where active and dynamic compromise prevails, bearing in mind that each student has a different learning style and this is potentialized by the means of knowledge acquisition and construction, supported by IT tools for the students to acquire their own learning process.

The frequent interaction, whether between  teachers and students or students themselves  has a clear relationship with the use and application of IT, since it promotes rapid interaction, sharing ideas and analyzing criticism. IT helps to observe students’ performance and provides timely and accurate evaluation.

In a connection to the real world, tools are provided through IT to apply concepts in a variety of contexts that break school isolation from the real world (Requena, 2008).

Maturity models for student learning criteria have specific characteristics for each maturity level as follows:

• Initial level: There is no evidence of a formal process or discipline ready for the processes of virtual education, as well as planning an adequate space and time to start a virtual program, but there is a willingness and intention to start a teaching-learning process with the virtual methodology.
• Planned level: Identifying students’ areas of need  by academics to make the learning process independent and qualified when delivering complete resources through IT, having an informal use of standard pedagogical models.
• Defined level: Considered as a structured and integrated process, it has the support, the plans and the strategies to guide the needs of the students, as well as the formally identified pedagogical models based on the student-teacher interaction, supported by IT.
• Managed level: Having an organizational focus, determines that students’ learning resources are formally evaluated by adding a feedback process. In addition, it makes an adequate relationship between the use of IT and learning models, encouraging group work.
• Optimized level: The continuous improvement of educational effectiveness is evidenced, developing self-learning strategies, based on restructured pedagogical models to reflect the needs of the students, taking into account the innovation in pedagogical and educational models applied to e-learning.

Contents and Curriculum

It is one of the most sensitive aspects and of greater care, since the curricular plans must focus on the development of the proposed program and the fulfillment of its objectives to create  professions prepared to solve the real problems of society, as well as the development of the entire digital component. According to Simanca & Porras, “it is the essential knowledge of the different methods of instructional design, development models, tools and standards, for the creation and organization of virtual learning scenarios, which guarantee the fulfillment of the already defined objectives” (Simanca, Porras, Garrido & Hernández, 2017). Having these digital resources planned, organized, created, evaluated and applied to comply with the offer, the development of curricular content leads to an interdisciplinary work with pedagogical approaches and development of quality educational material. It is very important to emphasize that this criterion is focused on the aspect of the work of the virtual tutor and the inherent subjects to it, as well as the aspects related to the development of digital content, and standards and quality platforms for its development (Osma, Guzman & Ramirez, 2017).

The maturity model for the curricular content criterion has specific characteristics for each level of maturity as follows:

• Initial level: There is no formal process, therefore, there are no content development plans; there is no defined methodology; there is no clarity in pedagogy in the use of the virtual methodology; and the development of curricular contents does not have digital resources.
• Planned level: There is evidence of a virtual methodology; there are templates and application of protocols for the development of digital resources. Learning objectives are set.
• Defined level: Stipulated as a structured process, it has available funds for the creation of resources, and university plans as strategies to guide the needs of the students, in addition to the pedagogical model which formally identified elements of teaching-learning and students’ interaction with – teachers. Technology-support organization and infrastructure are supported by IT.
• Managed level: There is an organizational approach, the resources needed for the process are managed and the processes are evaluated regularly. There is evidence of quality characteristics such as: usability, reliability and robustness of the resources used by students.
• Optimized level: There are measurement metrics in the performance and the impact of curricular contents is monitored through the management of IT, which must be established as support for the use of the different resources needed to support virtual education. Virtual laboratories, accompanied by the use of refined, modified or new digital resources, the development of their own OVAS, innovation with new developments for the learning process, as well as an identification of the improvement options through a formal process for the re-evaluation of the content in its learning context, are all supported by IT.

Technology and Support

This criterion is related to IT and its support in the Gartner Consult with its Gartner Cycle (Gartner, 2018) model that gives a chronologic map which gives adaptations and implantations in five stages: 1) Launching, 2) Oversized expectations peak, 3) Disillusion abysm, 3) Consolidation stage and 5) Productivity stage.

This representation of maturity demonstrates a process of adaptation of IT in education, from its launch, going through an adaptation process, in addition to a critical point until reaching a consolidation (Osma et al., 2017).

It is expected that with this criterion, it will be possible to evaluate the levels in which all the technology related to education is found, from the learning systems (LMS), the 3D environments and the new trends in virtual education and their implementations.

The maturity model for Technology criterion counts with specific characteristics for each maturity level for this case.

• Initial level: Must have a technological platform for the development of program content, with limited technical support for the use of these platforms.
• Planned level: A coherent approach will be used for the development of e-learning resources, developing plans for the creation of these resources with specific objectives.
• Defined level: Where there should be a structured and integrated process, represented in the use and implementation of policies and standards for the use of IT, both for CMS and LMS, as for online support, in addition to having specialized technical and educational support.
• Managed level: There is evidence of an organizational focus and support with the necessary and accurate technological infrastructure with the ability to work on various types of devices, and with concurrency capacity for the development of MOOCs.
• Optimized level: Represented in continuous improvement, constant innovation in new technologies for the development of digital resources, as well as administrative and academic processes. It is expected that with this criterion, the levels in which all the technology related to the Master’s Degree can be evaluated, from the learning systems (LMS), to the new trends in virtual education and their implementations.

Infrastructure and Organization

In this criterion, everything related to the organizational aspect is expressed, both for the institution that manages and controls everything related to education, as well as for the program, especially with the virtual methodology, to which the model is applied, seeking a technological innovation in the organizational field, so that management and administration control, as well as University Welfare, are efficient to achieve an optimal organization with established continuous improvement processes.

The maturity model for the Organization and infrastructure criterion has specific characteristics for each level of maturity as follows:

• Initial level: There is no evidence of an established infrastructure, there is no organization of the defined academic-administrative processes, and there is evidence of the supervision and control of the administration limited to the academic quality.
• Planned level; There is evidence of an infrastructure that supports the creation of contents and academic management, as well as systems of evaluating the technological satisfaction by the different members of the academic-administrative process, as well as evidence of policies and procedures.
• Defined level: The organization has policies, an infrastructure and resources necessary for the creation of useful content with recognition of the organization. In addition, it establishes procedures for promotion, developing the vision and organizational strategy for the integration of all systems, supported by organizational programs. The processes are structured and integrated.
• Managed level: There are educational effectiveness metrics and associated goals to validate the quality of education, with organizational audits to evaluate learning performance on a regular basis, as well as an implementation of quality virtual education models based on international standards.
• Optimized level: The continuous improvement of the Organization and infrastructure criterion must show the improvement of the organizational capacity and performance, the application of technological innovation in every aspect related to the organization, and the use of standards and frameworks using the best practices to optimize the performance from service.

Practical Case of OCTA Maturity Model–master’s degree in Mobile Telecommunication

Diagnosis, evaluation and recommendation using OCTA maturity model, are applied in the Master´s degree in Mobile Telecommunication with the virtual methodology. Each criterion is described with a process assessment.

Table1: Diagnosis of the Master´s program applying the maturity model – Own elaboration

Conclusions

The design and adaptation of the maturity model for virtual education can be carried out in a formal way, developing improvements in the areas of key processes that can intervene in a virtual methodology program.

With this proposal, the definition of good practices that allow reducing the possible desertion in the programs with this methodology is sought. It is evident that factors such as the support and accompaniment of teachers are valued by students, which generates a follow-up process and support for learning processes.

In the case of the master’s degree, it is concluded that it is in Level 3: (Defined), at the time of reaching level 4: (Managed). The processes and adaptations take time to follow, and procedures must be performed correctly and clearly, so that when Level 5: (Optimized) is achieved, it is easier to manage The teaching-learning process of the Master in Mobile Telecommunications must be a constant and stipulated process, both in processes and in IT, to achieve the Mobile improvement.

The proposed model allows, on one hand, higher education institutions to implement programs using the virtual methodology, in addition to the quality criteria proposed by each government through its Ministries or vigilant bodies of education, and have a medium to evaluate the status of the academic program and, on the other hand, have a strategy to implement improvement plans until reaching the levels of maturity proposed.

Having this maturity proposal helps the future planning process and plans for the continuous improvement of the programs in the established criteria, which is a key factor in the quality and management processes of the program. There is no hierarchy between the criteria, because in the light of the process developed, each factor has a similar weight compared to the virtual methodology, thus giving an integral vision and coordination between academic, technical and administrative factors.

From this proposal and for future research, it would be convenient to establish a metric model for each of the factors and criteria defined, so that the evaluation would not only be qualitative, but also quantitative, thus having an integral evaluation model. Likewise, the model should be explored and evaluated from the different perspectives of all those involved in the training process, including teachers, students and administrators.

References

• Arcos-Medina, G., Oñate-Andino, A., & Pástor, D. (2018). Structural Analisys of CMMI with the Software Development Process, Project Portfolio Management and PMBOK. KnE Engineering, 1(2), 223-240.
• Cabero Almenara, J., & Llorente Cejudo, M. C. (2010). Comunidades virtuales para el aprendizaje. Edutec: Revista electrónica de tecnología educativa (34), 2.
• Chen, Y. C., & Wang, Y. J. (2018). Application and development of the people capability maturity model level of an organisation. Total Quality Management & Business Excellence, 29(3-4), 329-345. doi: 10.1080/14783363.2016.1184568
• Dhir, S. K., Verma, D., Batta, M., & Mishra, D. (2017). E-Learning in Medical Education in India. Indian Pediatrics, 54(10), 871-877. doi: 10.1007/s13312-017-1154-7
• (2018). Gartner Hype Cycle, from https://www.gartner.com/en/research/methodologies/gartner-hype-cycle
• Gutiérrez, J. L. G. (2017). Nivel de madurez de la alineación, planificación y organización de TIC en la ULADECH Católica usando Cobit 5. In Crescendo, 8(2), 255-264.
• Kybartaite, A., Nousiainen, J., & Malmivuo, J. (2013). Technologies and methods in virtual campus for improving learning process. Computer Applications in Engineering Education, 21(1), 185-192. doi: 10.1002/cae.20460
• Lacerda, T. C., & von Wangenheim, C. G. (2018). Systematic literature review of usability capability/maturity models. Computer Standards & Interfaces, 55, 95-105. doi: 10.1016/j.csi.2017.06.001
• Manford, C., McSporran, M., Mann, S., & Williamson, A. (2003). e-Learning quality: becoming a level five learning organization. Proceedings of the 16th NACCQ, Palmerston North New Zeeland, 343-348.
• Marshall, S. (2010). A quality framework for continuous improvement of e-Learning: The e-Learning Maturity Model. International Journal of E-Learning & Distance Education, 24(1), 143-166.
• Ministerio de Educación Nacional. (2010). Decreto 1295 – Registro Calificado. Bogotá.
• Ministerio de Educación Nacional. (2013). Competencias TIC para el desarrollo profesional docente. (978-958-750-762-1). Bogotá – Colombia: Ministerio de Educación Nacional.
• Ministerio de Educación Nacional. (2018). Decreto 1280. Colombia – Bogotá D.C.:  Retrieved from http://es.presidencia.gov.co/normativa/normativa/DECRETO%201280%20DEL%2025%20DE%20JULIO%20DE%202018.pdf.
• Nazir, M. J., Pujeri, R. V., & Rizvi, A. H. (2013). Maturity Level Definitions for the Evaluative Framework to Measure the Maturity of Skill Based Training Program with Multimedia Support in an E-Learning Environment [SBTP-MSeLE]: A Learners Perspective. J. Comput. Inf. Technol, 2(4), 1-8.
• Neuhauser, C. (2004). A maturity model: Does it provide a path for online course design. The Journal of Interactive Online Learning, 3(1), 1-17.
• Ogange, B. O., Agak, J. O., Okelo, K. O., & Kiprotich, P. (2018). Student Perceptions of the Effectiveness of Formative Assessment in an Online Learning Environment. Open Praxis, 10(1), 29-39. doi: 10.5944/openpraxis.10.1.705
• Osma, J. I. P., Guzman, J. L. R., & Ramirez, C. X. G. (2017). Modelo de gestión de servicios ITIL para E-learning. Revista Educación en Ingeniería, 12(23), 28-34.
• Porras, A. A., & Quintero, B. G. C. (2012). Aproximación modelo para el diseño de cursos virtuales. REDES DE INGENIERIA, 3(1), 96-108.
• Proenca, D., & Borbinha, J. (2018). Formalizing ISO/IEC 15504-5 and SEI CMMI v1.3-Enabling automatic inference of maturity and capability levels. [Article]. Computer Standards & Interfaces, 60, 13-25. doi: 10.1016/j.csi.2018.04.007
• Requena, S. H. (2008). El modelo constructivista con las nuevas tecnologías: aplicado en el proceso de aprendizaje. Revista de Universidad y Sociedad del Conocimiento, 26-35.
• Simanca, F. A., Porras, A. A., Garrido, F. B., & Hernández, P. C. (2017). Implementación de herramientas tecnológicas en los procesos de Enseñanza-Aprendizaje de los Triángulos. I+ D Revista de Investigación, 10(2).
• Sousa, M. J., & Gonzalez-Loureiro, M. (2017). Comparative study on skills needed by organizations and effectively developed in eLearning management courses. Universal Access in the Information Society, 16(4), 877-888. doi: 10.1007/s10209-016-0492-x
• Universidad Distrital Francisco José de Caldas. (2009). Plan Estratégico de Desarrollo 2008 – 2016.