Performance Measurement for Knowledge Management: Designing a Reference Model
Latifa Oufkir, Mounia Fredj and Ismail Kassou
ENSIAS, Mohammed V University in Rabat, Morocco
Volume 2017, Article ID 733562, Journal of Organizational Knowledge Management, 13 pages, DOI: 10.5171/2017.733562
Received date : 29 March 2016; Accepted date : 21 June 2016; Published date : 23 February 2017
Academic editor: Ahlem Soualhia
Cite this Article as: Latifa Oufkir, Mounia Fredj and Ismail Kassou (2017), “Performance Measurement for Knowledge Management: Designing a Reference Model ”, Journal of Organizational Knowledge Management, Vol. 2017 (2017), Article ID 733562, DOI: 10.5171/2017.733562
Copyright © 2017. Latifa Oufkir, Mounia Fredj and Ismail Kassou . Distributed under Creative Commons CC-BY 4.0
Knowledge Management (KM) projects are socio-technical systems that enable knowledge activities and ensure that the right knowledge gets to the right person at the right time. The crucial role of KM in achieving companies’ organizational objectives generates a great interest in KM projects and an increasing trend in KM adoption. As the number of KM applications is growing, there is a focus shift towards measuring performance of such KM projects in order to rationalize their usage. In fact, there have been several models developed for KM performance measurement in the literature. While most of these models address mainly the performance of the whole KM of the organization and do not assess each KM project separately. Other measurement models are tailored for a specific KM project. In this work, we propose a generic KM performance measurement model fitting any kind of KM projects. This model is built on existing KM models literature and enhanced with theoretical findings. In particular, it is composed of three interrelated sub-models: KM processes, socio-technical influencing KM factors and KM key performance indicators. Moreover, it follows the reference modeling design process in view of producing a high quality model.
Keywords: knowledge management model; performance measurement; reference model;
Knowledge is recognized as a key resource for organizational success. Making the best use of this resource is an important concern for both researcher and practitioner (Barthelme-trapp and Vincent 2001). To obtain all Knowledge Management (KM) benefits, it is necessary to measure the KM performance (Ragab and Arisha 2013). Since there are no standards in measuring KM performance, many approaches and models are proposed in the literature. They argue that KM performance measurement follows three phases: defining the KM objective for which the performance will be measured, modeling the components of the KM to be measured, and identifying a set of relevant measures for each component of the model (Wong et al. 2013; Oufkir et al. 2016). However, literature on the KM performance measurement still raises some critical constraints to be considered when creating a KM performance model:
- Diversity of KM applications. Along with the growing number of KM applications, performance measurement design becomes more constraining. Dealing with the diversity of KM projects requires either multiple performance measurement models or a generic one that can be applied to all settings (Del-Rey-Chamorro et al. 2003). However, no generic KM project performance measurement model has yet been proposed in the literature while a significant number of approaches deal with specific project assessment or assessment of the overall KM organization (Wong et al. 2013).
- Variety of KM models. Due to the broadness of KM field and the lack of KM standards, existing KM models vary in scope and focus. A widely adopted taxonomy identifies three trends in designing KM models (Dudezert and Agnes 2006; Wong et al. 2013; Handzic 2011): the first one is based on the understanding of knowledge concept, the second approach focuses on the knowledge flows in the organization, and the third approach considers the social-technical aspect of KM and related impact factors. Indeed, each approach is valuable in itself, but constitutes only a partial view of KM. Handzic (2011) raises the need of a unified model that incorporates the three previous views. Also, he tackles the need to develop a deep understanding of each KM core terms.
- Lack of scientifically proven KM model.The analysis of the KM models literature leads to the conclusion that little academic work exists on designing a KM model in a scientific manner. Indeed, recent studies criticize the often missing model validity check and the lack of quality concern in designing models (Matook and Indulska 2009). This is particularly true for KM model design. To the best of our knowledge, KM model design is based on both KM requirements and qualitative review of existing KM models. It is usually subject to empirical application to check model effectiveness (Handzic 2011) but is rarely subject to quality validation. Therefore, there is a need to integrate KM model quantitative valuation in designing KM models.
Thus our work focuses on the following research questions (RQ):
- RQ1: Is it possible to design a generic KM model able to assess diverse KM projects?
- RQ2: What are the key elements that a generic performance measurement KM model should cover?
- RQ3: How to design our generic KM model in accordance with the scientific design method to ensure its validity?
To address these issues, we propose a design of reference model (RM) for KM performance measurement (KMPM) that draws from both the literature review of existing KM models as well as the performance measurement requirements. The model design follows the quality driven design approach: a five step design methodology that provides a model quality assessment based on predefined RM quality criteria. Our reference model is able to assess both the overall organizational KM and any specific KM project thanks to its generic process and KPI sub models. Such reference model provides a common formalization for researchers with a global view, theoretical background and precise terminology. Practitioners also need this conceptual model to help them to better assess KM initiatives both jointly and separately and to identify those that are adequate to their context.
The remainder of the paper is organized as follows. Section 2 provides an overview of related work about knowledge management modeling. Section 3 includes an introduction of the reference modeling design and presents the adopted design steps of our model. Section 4 summarizes our contribution and points on some future work.
According to Davenport and Prusak (1998); Alavi and Leidner (2001), KM refers to methods, mechanisms and tools designed towards preserving, valuing, creating and sharing knowledge with a view to furthering the organization’s objectives . Implementing a KM project in an organization consists of implementing a socio-technical system that is able to ensure KM processes.
Reviewing literature on KM models shows that researchers suggest three major components for KM (Dudezert and Agnes 2006; Wong et al. 2013; Handzic 2011):
- Knowledge resources: refer to the whole organization knowledge (Wong et al. 2013). According to Davenport (Davenport et al. 1998), the knowledge resources include: human capital (e.g. employee staff, customer and suppliers), knowledge capital (quantity and quality of knowledge possessed by the firm) and intellectual property (the product of knowledge creation that generates value).
- KM processes: involve activities related to knowledge flow in the organization considering knowledge dynamic nature (Alavi and Leidner 2001).
- KM factors: other than KM resources and processes, researchers consider the importance of some influencing factors for the support and the success of KM initiatives (Ragab and Arisha 2013), (Ale et al. 2014; Wong et al. 2013).These factors include cultural, structural and technological aspects such as trust culture, centralization structure and IT support.
Nevertheless, the literature presents evidence of the relationship between knowledge resources and knowledge processes. Handzic (2011) argues that knowledge resource perspective is deeply anchored in the process perspective as knowledge resources are transiting through the KM processes; they represent their outputs. Besides, the performance measurement is more of dynamic knowledge matter than static knowledge (Lerro et al. 2012; Nonaka et al. 2000). Accordingly, we do not consider knowledge resources as a stand-alone perspective whereas we assume that understanding knowledge is very important in order to achieve proper knowledge process model design. Therefore, the knowledge view yields some important observations:
- Knowledge exists in many forms in the organization. At least two forms of knowledge are particularly valued: the tacit knowledge that is rooted in action, experience and involvement in a specific context; it is composed of beliefs, know-how and skills. And the explicit knowledge that is articulated, codified and communicated in symbolic form and/or natural language (Alavi and Leidner 2001). Both forms interact permanently in the enterprise through many conversion mechanisms (Nonaka et al. 2000).
- Knowledge exists in the organization in different levels (Nonaka and Takeuchi 1995; Grundstein 2012).We distinguish first the organizational level that is shared among distributed actors belonging to the same organization (e.g. knowledge incorporated within routines, models and regular and predictable behaviours). The second one is the collective knowledge which is owned by a group of persons that share a mutual identification, actions and projects (e.g. communities). The last one is the individual knowledge which is the personal and intangible knowledge. It encompasses people abilities, know-how and know-what…
The KM factors perspective emphasizes that knowledge mobilization within organization is strongly context-sensitive. It is made up of two separate attributes related to the socio-technical aspect of KM: the technical factors that consist of the project infrastructure and supporting technologies, and the social factors that depend on the dominating culture, the KM structure and the human aspect. Several authors study KM factors theoretically and empirically and come to a firm agreement on their identification (Ale et al. 2014; Wong et al. 2013), (Ragab and Arisha 2013).
Regarding the KM process perspective, literature shows a broad range of models. Nonaka et al. (2000) propose the SECI model that presents four ways for knowledge types conversions: socialization (tacit knowledge to tacit), externalization (tacit to explicit), internalization (explicit converted into tacit) and combination (explicit to explicit). SECI model introduces the BA context as a KM factor, a Japanese term meaning “place”, that refers to the specific context (place, energy and quality) needed for the effective knowledge flow. On the other hand, the knowledge circulation process (KCP) proposed by Chang (Chang Lee et al. 2005) consists of knowledge creation, knowledge accumulation, knowledge sharing, knowledge utilization and knowledge internalization processes. The basic assumption behind the KCP is the work of Alavi and Ledner (Alavi and Leidner 2001) that considers knowledge having multifaceted characteristics: object, state of mind, access to information, process and potential for influencing future action. The KCP and the SECI models are the most used models for KM performance measurement purpose. Grundstein (2012), in turn, builds a model composed of four generic processes: identification, preservation, recovery and valuation of knowledge. These processes respond to the recurrent knowledge problems recognized in the organization and provide a detailed description of knowledge flows by breaking down each generic process into more specific sub-processes. Other models presented in the literature are more empirical and depict the author’s point of view about KM activities. Knowledge management process cycle proposed by King (2009) is an example of such models.
On the basis of the previous analysis, studied KM perspectives stress on the following points:
- Most of the reviewed models were developed under a special KM perspective or as a solution to a particular organization problem.
- Reviewed models globally share the same central idea that KM processes depict the knowledge flow in the organization. However, the delimitation of process scope is controversial with the exception of core functions (creating, preserving and transferring knowledge).
- KM processes involve interactions and conversions between knowledge in all its forms: the tacit and the explicit type besides the individual, the collective and the organizational aforementioned levels.
- Two models seem particularly interesting for the performance measurement purpose among all reviewed KM process models: the SECI model (Nonaka et al. 2000) that considers KM processes as knowledge forms conversions, and the Grundstein process model (Grundstein 2012) that defines generic processes and sub processes in accordance with organization knowledge problems. Combining both approaches while designing a KM model can contribute to our requirement in twofold: it structures KM project according to Knowledge problems and knowledge forms and it facilitates the performance measurement through the KM process decomposition to the desired granularity.
- A context that brings together cultural, structural and technological factors is conducive to the deployment success of the KM. This aspect was largely discussed in the literature that shows a broad agreement on these factors identification.
On the other hand, performance measurement relies basically on performance measures which provide a benchmark of progress towards the performance achievement. Existing work on performance measurement applied to KM stresses on the importance of KPI and categorizes them in different ways (Wong et al. 2013). Indeed, Kuah et al. (2012) state that KM can be viewed as a system based on KM processes that consume inputs and produce outputs, and that these variables can be used as a proxy to measure KM performance. Del-Rey-Chamorro (Del-Rey-Chamorro et al. 2003) addresses two kinds of KM measures : core outcomes in the strategic level and performance drivers in the operational level. Goldoni and Oliveira (Goldoni and Oliveira 2010) suggest that KM metrics can be divided into process and result metrics. This suggestion aims to measure KM performance at different levels of the organization. Globally, all previous studies agree that KPI is an important element for performance measurement and that it monitors significant KM components.
Design of the KMPM
In this section, we present an overview of the reference model design. Then, we focus on a special reference model design approach that we apply to the case of knowledge management reference model design.
Generally, a Reference Model (RM) serves as a starting point for developing solutions according to a concrete problem in a scientific manner (Scheer and Nüttgens 2000). RMs refer to aggregated, generic or theoretical models that need adaptation for a specific concern (e.g. a project development, an enterprise) (Andreas and Frank 2016).
Since a low quality RM can be damaging for the organization, a considerable number of works focus on guiding the development process to achieve a better RM quality (Winter and Schelp 2006),(Andreas and Frank 2016; Matook and Indulska 2009; Thomas 2006). Reviewing research on the subject shows a list of shared RM requirements. Andreas and Frank (2016) present these requirements as a five step design methodology (cf. Fig 1).
Phase 1 is the scope definition which involves the identification of the scope and the target model. In the second phase, the literature review on the subject is gathered and analyzed. Phase three consists of the reference model construction and design in an iterative way, based on the obtained results from the prior phases. The established model needs a quantitative evaluation which is the subject of the fourth phase. Lastly, an empirical assessment is performed within a real-world environment.
Figure 1: Reference Model Design Steps
Design Phase 1: Scope of the model
Performance measurement is an ultimate purpose that should be taken into account while building the KMPM model. It should meet the three requirements (RQ 1-3) mentioned in the introduction.
Design Phase 2 : Literature review
Based on the literature analysis, we depict the most important KM key concepts within the research field as sub models. Therefore, we build a model that entails three perspectives: process sub-model, factors sub-model and KPI sub-model. We state that agreement is reached concerning KM factors. However, KM processes delimitation should be considered carefully as they represent interactions between knowledge forms. Thus, it was decided to consider the processes according to the organization knowledge problems and to proceed to their decomposition in view of knowledge forms conversion. The knowledge forms and related knowledge conversions are presented in Fig 2.
We select the Unified Modelling Language (UML) and especially the class diagram for the presentation of the reference model. For the process sub-model, the flow chart notation is used.
Figure 2: KM sub-processes and knowledge conversions
Design Phase 3: Reference model construction
The proposed RM summarizes the most important research results and provides a common KM enterprise model in its different aspects. It embodies three perspectives (cf. Fig. 3):
- KM process model: is derived from the qualitative literature analysis and based on inductive results,
- KM context model: encompasses the well known KM factors
- KPI model: this component plays an important role as it provides measures for each perspective. The final project performance indicators are the aggregation of all perspectives.
Figure 3: The KM reference model
Every KM project is an instantiation of one or more KM processes; it is influenced by the organization factors and is monitored with a set of KPIs. Fig. 5 provides an excerpt of the reference model for a sample KM process.
The Process Model Perspective
The process model is built mainly on literature models (Grundstein 2012), (Ale et al. 2014), (Nonaka et al. 2000), (King 2009), (Heisig 2003). It is composed of four generic processes that are decomposed to several sub-processes. Hence, identification process encompasses identification and location; preservation process comprises the acquisition of knowledge, its modeling and its formalization in addition to the store sub process; the recovery process contains the access, the application, the combination and the creation of knowledge; and lastly the knowledge update process (cf. Fig. 4). These sub processes represent knowledge problems and conversions between knowledge in all its forms.
Figure4: The KM process model
The KPI model perspective
The KPI perspective operationalizes the other KM perspectives. KPIs monitor the performance of a KM project according to both KM processes and KM context aspects. All measures in this study are based on existing KM performance measurement instruments and KM literature (Wolf and Tendron 2014; Robertson 2003; Hoss and Schlussel 2009; Wong et al. 2013; Don 2001; Heisig 2003; Ley et al. 2010).
For KM processes, KPIs are grouped into two categories (Hoss and Schlussel 2009; Lee and Choi 2003): input and output. Each KPI has its own set of measurement criteria: scale, range, current and target value. It can be either quantitative or qualitative.
The Context Model Perspective
KM context corresponds to the factors that impact the success of KM project. According to Gold et al. (2001), technology, culture, strategy and KM structure are the most influential factors on KM project success.
Figure 5: The KM Model Excerpt
Design phase 4: Quantitative evaluation
Within this step, we aim to assess the overall model and test its compliance with the accepted RM quality criteria. We adopt the quality function deployment (QFD) based approach as proposed by Matook (Matook and Indulska 2009), which is an adaptation of the initial QFD to the RM design context, then we adapt it to the specific adopted RM requirements. The QFD method is used originally for products and services to transform the qualitative user demand into quantitative parameters in order to deploy methods for achieving the design quality. Matook (Matook and Indulska 2009) reports that among all proposed RM assessment quantitative methods in the literature, it is the only one that provides both the RM quality attributes, an assessment method and a proposal of application.
Defined reference modeling quality attributes are:
- Generality: the degree to which an RM is usable in different cases.
- Flexibility: the ease with which the RM accomodates change from initial requirement.
- Completeness: the degree to which designed RM covers all predefined scope.
- Usability: the ease of use, adaptation and application of the reference model.
- Understandability: the clarity of concepts, purpose and structure of the RM
The QFD based approach involves developing a matrice referred as “The House of Quality” (HoQ) that incorporates RM quality attributes (horizontal dimension), RM development phases (vertical dimension), the center of the house which assesses the impact of the RM development phases on the RM quality attributes and the user perceptions (cf. Table I).
Table 1: The QFD based approach adapted from (Matook and Indulska 2009)
The quality measure is calculated based on the product of user assessment of the requirements importance and the completeness of the development phases (as reported by the RM designer). Therefore, as we follow the RM design steps for our model construction, we assume that designer ratings have some high values. Instead, full rating is not obtained unless we test the model with empirical applications and thus obtain the user importance ratings and fulfilment ratings.
This paper presents a KM reference model for knowledge management that responds to the performance measurement requirements. This study provides three main contributions: first, our research reviews the literature on RM design in order to determine the best development approach that results in a high quality RM. Our proposed model follows the identified development process. Secondly, our model is built on KM theoretical foundations. It entails three sub-models :i) a process sub model derived from the KM model literature review and enhanced with literature analysis and logical findings, ii) a KPI model that provides KM project measures according to the performance measurement need, and iii) a context sub model that represents the KM influencing factors. Thirdly, the KMPM is able to assess both the overall organizational KM and the diversity of KM projects unlike existing models that are designed either for assessing the KM of an organization or a specific KM project.
Our work is relevant in twofold: firstly through designing a KM reference model with the quality concern, this can be a significant step toward field standardization. Secondly, by filling the literature gap on KM performance measurement and proposing a KM model that assesses all kinds of KM projects.
The limitation of the study is related to the full design process application. In fact, we propose a five step design approach for the RM design. Meanwhile, the latest step which concerns empirical application is not addressed in this paper. This is due to the fact that most existing KM models claim to be empirical, so we focus on the theoretical aspect in order to broaden the model scope.
Future work may consist on empirical validation of the proposed model which is a part of an overall KM performance measurement framework. A multiple case study will be performed to obtain insights about reference model validity and the global framework reliability.
- Alavi, M. and Leidner, D.E. (2001), ‘Knowledge Management and Knowledge Management Systems: Conceptual Foundations and Research Issues,’ MIS quarterly, 25(1), 107–136.
- Ale, M., Toledo,CM., Chiotti, O. and Gali, MR. (2014), ‘A Conceptual Model and Technological Support for Organizational Knowledge Management,’ Science of Computer Programming, 95(1), 73–92..
- Andreas, J. and Frank, T. (2016), ‘Towards a Document-Driven Approach for Designing Reference Models: From a Conceptual Process Model to its Application,’ Journal of Systems and Software, 111, 254–269.
- Barthelme-trapp, F. and Vincent, B. (2001), ‘Analyse comparée de méthodes de gestion des connaissances pour une approche managériale’ Actes de la Xième Conférence de l’Association Internationale de Management Stratégique,13-15 Juin,2001,Québec, Canada,1-25.
- Chang Lee, K., Lee, S. and Kang, I.W.(2005), ‘KMPI: Measuring Knowledge Management Performance,’ Information & Management, 42(3), 469–482.
- Davenport, T.H. and Prusak, L. (1998) Working Knowledge: How Organizations Manage What They Know, Harvard Business Press, Boston.
- Davenport, T.H., Long, D.W. De and Beers, M.C.(1998), ‘Successful Knowledge Management Projects,’ Sloan Management Review, 39(2), 43–57.
- Del-Rey-Chamorro, F.M., Roy, R., Wegen, B. and Steele, A. (2003), ‘A framework to create key performance indicators for knowledge management solutions,’ Journal of Knowledge Management, 7(2), 46–62.
- Don (2001), Metrics guide for knowledge Management initiatives. [Online]. [Retrieved December 22, 2014] Available: http://www.km.gov/documents.
- Dudezert, A. & Agnes, L.(2006). Performance et Gestion des Connaissances : Contribution à la construction d’un cadre d’analyse. In Actes des Journées des IAE, Congrès du cinquantaire, 3-4 April 2006 Montpellier, France.
- Gold, A.H., Segars, A.H. and Arvind, M. (2001), ‘Knowledge management: An Organizational Capabilities Perspective,’ Journal of management information systems,18(1), 185-214.
- Goldoni, V. and Oliveira, M. (2010), ‘Knowledge management metrics in software development companies in Brazil,’ Journal of Knowledge Management, 14(2), 301–313.
- Grundstein, M. (2012), Three postulates that change knowledge management paradigm, New Research on Knowledge Management Models and Methods, Huei TH.(ed), Intech, Paris.
- Handzic, M. (2011), ‘Integrated Socio-Technical Knowledge Management Model: an Empirical Evaluation,’ Journal of Knowledge Management, 15(2), 198–211.
- Heisig, P. (2003). European Guide to good Practice in Knowledge Management – Part 4 : Guidelines for Measuring KM. [Online]. CEN Workshop Agreement. [Retrieved Mars 17,2013]. Available: https://www.k4health.org/toolkits/km/
- Hoss, R. and Schlussel, A. (2009), How Do You Measure the Knowledge Management (KM) Maturity of Your Organization? Metrics That Assess an Organization’s KM State. [Online]. USAWC.[Retrieved January 18, 2012]. http://www.digitalgovernment.com/media/Downloads.
- King, W.R. (2009), Knowledge Management and Organizational Learning, Springer US,Boston.
- Kuah, C.T., Wong, K.Y. and Wong, W.P. (2012), ‘Monte Carlo Data Envelopment Analysis with Genetic Algorithm for Knowledge Management performance measurement,’ Expert Systems with Applications, 39(10), 9348–9358.
- Lee, H. and Choi, B.(2003), ‘Knowledge Management Enablers, Processes, and Organizational Performance : An Integration and Empirical Examination,’ Journal of Management Information Systems, 20(1), 179–228.
- Lerro, A., Iacobone, F. A. and Schiuma, G. (2012), ‘Knowledge Assets Assessment Strategies: Organizational Value, Processes, Approaches and Evaluation Architectures,’ Journal of Knowledge Management, 16(4), 563–575.
- Ley, T., Kump, B. and Albert, D. (2010), ‘A methodology for Eliciting, Modelling, and Evaluating Expert Knowledge for an Adaptive Work-Integrated Learning System,’ International Journal of Human Computer Studies, 68(4), 185–208.
- Matook, S. and Indulska, M. (2009), ‘Improving the Quality of Process Reference Models: A Quality Function Deployment-Based Approach,’ Decision Support Systems, 47(1), 60–71.
- Nonaka, I. and Takeuchi, H. (1995), ‘The Knowledge-Creating Company: How Japanese Companies Create the Dynamics of Innovation,’ Long Range Planning, 29(4), 1–2.
- Nonaka, I., Toyama, R. and Konno, N. (2000), ‘SECI , Ba and Leadership : a Unified Model of Dynamic Knowledge Creation,’ Long Range Planning, 33(1), 5–34.
- Oufkir, L., Fredj, M. and Kassou, I. (2016), ‘Knowledge management performance measurement: A Generic Framework’ 12th International Baltic Conference on Databases and Information Systems (DB&IS) , 4-6 July 2016, Riga, Latvia,1-16.
- Ragab, M.A. and Arisha, A. (2013), ‘ Knowledge Management and Measurement : A Critical Review,’ Journal of Knowledge Management, 17(6), 873–901.
- Robertson, J. (2003). Metrics for knowledge management and content management. [Online]. Step two. [Retieved June 7, 2015]. Available: http://www.steptwo.com.au/papers.
- Scheer, A.-W. and Nüttgens, M. (2000), ‘ARIS Architecture and Reference Models for Business Process Management,’ Business Process Management: Models, Techniques, and Empirical Studies, 1806, 376–389.
- Thomas, O. (2006), ‘Understanding the Term Reference Model in Information Systems Research: History, Literature Analysis and Explanation’ Proceeding of the Business Process Management Workshops (BPM), ISBN: 978-3-540-32596-3, 5 September 2005,Nancy, France, 484–496.
- Winter, R. and Schelp, J. (2006), ‘Reference modeling and method construction: a design science perspective’ Proceedings of the 2006 ACM symposium on Applied computing (SAC ’06), ISBN:1-59593-108-2, 23-27 April 2006, Dijon, France, 1561–1562.
- Wolf, J. & Tendron, J.-F. (2014). Knowledge Capture & Transfer at Kraft Foods.[Online]. [Retrieved January 13, 2015]. https://www.apqc.org/knowledge-base/documents.
- Wong, K.Y., Tan, L.P., Lee,SC. and Wong WP. (2013) , ‘Knowledge Management performance measurement: measures, approaches, trends and future directions,’ Information Development, 31(3), 239–257.