Measuring ERP Success as a Function of the User Category at AUC

Organization Background

The American University in Cairo (AUC) was instituted in the year 1919 with around 400 undergraduate degree students. Now AUC has more than 5,000 undergraduate degree students, approximately 1,000 graduate degree students and more than 40,000 non-degree students. AUC is a non-profit organization despite the fact that it is a private university. Distinguished Board of Trustees volunteers their resources in support of the university (AUC Web, 2010). For decades, AUC remained the only private university in Egypt but there was also some competition from the public universities that existed back then and the quality education they offered. With the emergence of a few private universities in the region, the monopoly is not anymore existent in the literal sense. However, AUC still provides the highest quality education in the region and is the only university in Egypt to be accredited from the Association to Advance Collegiate Schools of Business (AACSB) . Moreover, AUC is also known to have the best technological teaching facilities in the country (AUC Web, 2010).

Back in the 70s, centralization of the university IT took place with the introduction of the mainframes along with the networking facilities. A Century mainframe was rented from NCR to perform all the AUC administrative and academic functions, such as the AUC payroll. Moreover, external work as well as statistical reports was done for many large organizations, such as the FAO, World Bank, FORD Foundation and the Egyptian Ministry of Health.

In the 80s, some departments at AUC, such as the Purchasing Department, Budgeting and the Accounts started computerizing its manual system and they used CUFS to do so (Aly El Arabi, 2011). In the year 1983, the IBM mainframe was bought to replace the rented NCR mainframe and a conversion process took place to suit the new IBM system (Maha Amer, 2010). Later in the year 2004, some tailor made in-house systems were developed, such as the online purchase requisition system. In 2007, top management decided that it is high time to buy an Enterprise Resource Planning (ERP) package to start integrating the “isolated islands” together and the Purchasing Department was among the first to adopt SAP.

In the year 2011, the Egyptian revolution took off and major changes were expected that could influence education in general and AUC in particular. For one thing, the IT played a major role in the revolution and the AUC alumni were very instrumental in this. The economic climate for one thing is not very promising and it is estimated to remain so for a while (until at least the revolution is over in both the literal and figurative sense). Additionally, in 2012 the AUC students managed one of the biggest strikes ever mainly because of the fees. Paying the fees and getting donations might be a threat and that could eventually affect the generous investments in IT at AUC.
 
Setting the Stage

The people and the way they act on information besides the flexibility, the integration and the overall interaction of the systems are all important for managing any information system (Martin, 2003). The classic Leavitt model emphasizes the impact of the task, technology, people and structure, showing how usually a change in one results in reactive change in the others (Leavitt, 1965).

The task is generically the information systems task represented in the different ERP modules that are so far implemented within the purchasing, budgeting, accounts, payroll and HR at AUC. The tasks naturally differ from one user to the other.

Technology is mainly the adoption of an ERP package in an educational setting and the competence of the users who are considered quite competent in dealing with the technology either new or upgraded.

People (actors) are the users of the ERP, namely the faculty and the staff (including administration and top management). The users were categorized with the help of one of the ERP Power Users (PU), namely Ms. Mona Abdallah, as follows:

1. The Steering Committee (SC) is a group of senior management from across different corporate functions (Somers and Nelson, 2001) responsible for decision making and they are in direct contact with the Board of Trustees of AUC.
2. The Project Managers (PM) is the liaison between Steering Committee and the Process Owners (sometimes also a member in the Steering Committee).
3. The Process Owner (PO) is the owners of the one of the ERP modules, e.g., Grants and Funds Management, Human Capital Management, Material Management, etc.
4. The Power User (PU) has more authority than the regular End User and usually has to integrate the work of more than one End User.
5. The End User (EU) is any person who has access to the ERP system on campus.

Structure is considered to have a positive relationship on the ERP system success (Ifinedo, 2006). In 2007, ERP was introduced at AUC in phases but until this day, the ERP is still under diffusion and is not entirely disseminated within all departments at AUC. Gradually, AUC adopted a centralized approach and this has greatly affected the direction and vision of top management.

Note here that this study is not concerned with Phase I (Lewin’s Unfreezing Phase) that is referred to as the Launching Phase in Exhibit#1 (Cooper & Zmud termed it the Initiation Phase). Nor is it concerned with the first part of Phase II (part of Lewin’s Change Phase) or specifically the Embracing Phase (the Adoption Phase as is referred to by Cooper & Zmud). Those early phases are the decision making (DM) phases of buying the ERP package and are beyond the scope of this study. This study is concerned with the Adaptation (the second part of Phase II) which ensues, but not entails, the beginning months of the actual use of the ERP that usually ends up being unsuccessful. This study is also concerned with Phase III (Lewin’s Refreezing Phase); specifically with the approval, the disciplining, as well as the final pervading phases (referred to by Cooper & Zmud as the Acceptance, Routinizationand Infusion respectively).

Case Description

In this research, a model is proposed to measure the success of the ERP as a function of segmenting its users. To do that, ten attributes were used, five users were segmented and five resultant kin models were developed corresponding to each of those five users, and therefore not only one model was used for all users. It should be noted that, those ten attributes were never all used together in one model because not all users were exposed to the same attributes— and this is what differentiated one model from the other. As a general concept after segmenting the ERP users, five corresponding models were developed to match each of those five users based on assigning only the attributes that the user is associated with. The following Exhibit displays the conceptual model which is stemmed from the DeLone and MacLean model of ERP success (1992), despite the fact that it didn’t consider the user category in its framework back then. This research starts by associating the quality vis-à-vis the System (SQ), the Information (IQ) and the Vendor (VQ) with the User Perception (UP); the Implementation Problems (IP), the Satisfaction (S) as well as the impact vis-à-vis: the Individual Impact (II), the Workgroup Impact (WI) and the Organization Impact (OI).

Exhibit # 2. The Proposed Concept of the ERP Model

DeLone and McLean (1992) had a very thorough analysis of the diverse studies that had slightly different definitions of the System Quality (SQ), Information Quality (IQ) and Vendor Quality (VQ).  Plant and Willcocks (2007), on the other hand, had a good “Culture-Related Questionnaire” measuring the VQ. In this research, Ifinedo’s (2007) method was used. Ifinedo built his questionnaire upon previous work of his (Ifinedo, 2006), the DeLone and McLean model (1992) as well as on Gable’s el al. that was written in 2003 (Ifinedo, 2007). The SQ encompassed things such as accuracy, reliability, and flexibility; whereas the IQ encompassed things such as timeliness, relevance, up-to-date and availability. On the other hand, the VQ simply measured their credibility, the training as well as the technical support they provide, along with the overall communication and relationship. However, the VQ here is separated into the SAP Egypt vendor and the outsourced vendor or consultant as suggested by the Project Manager at AUC, Mr. Ashraf el Gabry. It is noteworthy to mention that the majority of items that make up the SQ and IQ quality attributes were items in previous instruments  developed to measure the ERP success (Wu and Wang, 2000) that all had an acceptable Cronbach’s alpha. The Wu and Wang instrument was based on the classic Doll and Torkzaden user satisfaction instrument developed in the 80s (Doll and Torkzaden, 1988). In this research, the Doll and Torkzaden satisfaction instrument was partitioned and partly used to measure two quality attributes (namely, the SQ and IQ). For example, the accuracy section of the Doll and Torkzaden user satisfaction instrument is used in this research with the SQ attribute/section and the timely section is used with the IQ attribute/section.

The Spathis and Constantinides (2003) instrument to measure the Implementation Problems (IP) in the questionnaire used in this research had an adequate Cronbach’s alpha of over 0.7; the User’s Perception (UP) part is based on a combination of tested instruments mainly the Bradford and Florin (2003), as well as the Ifinedo (2007) while the Satisfaction (S) part is based on the DeLone and McLean model (1992). As noted before, most of those quality questions were taken from the Ifinedo (2007) questionnaire that had a Cronbach’s alpha of more than 0.7, and that is considered to have satisfactory internal validity (Bland and Altman, 1997).

Only measuring the Individual Impact (II) to determine the effect of the ERP on the individual creativity, productivity as well as the overall task execution and/or measuring the Organization Impact (OI) with its performance on the organization, was not considered enough by Ifinedo. He argues that the Workgroup Impact (WI) is also very vital to avoid the isolated islands of information as was the case in the past (Ifinedo, 2006). Most of the impact section in this study is based on the Ifinedo (2006) instrument although some of the items in the OI are also from Staehr who happened to base his work on the Shang and Saddon framework (Staehr, 2008).

Kallunki et al. argue that there are contradictory results when empirically measuring the ERP effects on performance because of the estimated average of two years of failure before gaining improvement in performance (Kallunki et al., 2011). In other words, some studies were carried out during those two years and thus Kallunki et al. assume failure during that stage; while others were carried out after those two years and that is why they harvest success. Note here that AUC started adopting its ERP back in 2007, while this study was initiated in 2010, i.e., after the estimated two years of unsuccessful adoption.

The classification of the users was vital because it is hypothesized that measuring the impact differed according to the category of the user. Such classification was not used for research and measurement before; usually the users were all given the same questionnaires regardless of their category (which meant having to answer questions that are not within their scope). Consequently, instead of sending the same questions to all users, the questions were segmented according to the category of the users to avoid having users answer questions that are not related to his/her work and get unrealistic and inaccurate results. For instance, the VQ was measured by the PMs only while the IQ was measured by the PUs and EUs only. As for the IP attribute, although it is the only attribute that is measured by all users, not all the IP questions were used for all the users.
 
End User (EU)

For the End User (EU), the focus is on the quality vis-à-vis: System Quality (SQ) and Information Quality (IQ) and their impact on EU as to the Individual Impact (II). Note here that the Vendor Quality (VQ) was not part of the quality in this EU model since the EUs do not have any direct interaction with the vendors. Also EUs, as defined in this paper, have no consequence on the Organization (OI) or the Workgroup (WI) and that is why those two attributes were also omitted from the impact. In this study, a sample of 11 EUs filled out the EU questionnaire.

When examining the quality concerning the SQ and IQ, noted is a very strong correlation with all of the other attributes. In other words, the quality significantly as well as positively affects the IP, UP, IP + UP combined; S as well as the EU’s II (0.702, 0952, 0.924, 0.761, and 0.713 respectively) as can be seen from the following Exhibit:

Exhibit# 3. End User (EU) Model: Quality vis-à-vis: System and Information, Effect on IP, UP, IP + UP Combined, S and II 

Please note that the coefficient of correlation can be a positive one reflecting a positive relationship,  zero signifying the absence of a relationship, or a negative one for negative relationships.

Additionally, when studying the IP and UP as a group, it can be also observed a significantly strong as well as positive correlation with the EU’s S as well as II (see Exhibit # 4).

Exhibit# 4. End User (EU) Model: Effect of IP + UP as a Group on the S and II

Power User (PU)

The Power User (PU) model is identical to the EU model. It focuses the quality only on the System Quality (SQ) and Information Quality (IQ), and their impact on the Individual (II). Note how the Vendor Quality (VQ) was also not part of this PU model since the PUs do not have any direct contact with the vendors. Also PUs, as defined in this paper, have no authority on the Organization (OI) or the Workgroup (WI) and that is why those two attributes were also omitted. In this study, a sample of 14 PUs filled out the PU questionnaire, 3 of which are authorized to do programming.

When examining the quality of the SQ and IQ, we get a strong correlation with the other attributes except for the Satisfaction, which is moderate but still significant. In other words, the quality strongly and positively affects the IP, UP, IP+UP combined and II while it only moderately correlates with the S (see Exhibit # 5).

Exhibit# 5. Power User (PU) Model Quality vis-à-vis System and Information, Effect on IP, UP, IP + UP Combined, S and II

Additionally, through studying the IP and UP as a group, noted is a very strong correlation with the S as well as the II (see Exhibit # 6).

Exhibit# 6. Power User (PU) Model: the Effect of IP + UP as a Group and Effect on S and II

Process Owner (PO)

For the Process Owner (PO) model, the focus is on the System Quality (SQ) only and its impact on the Workgroup (WI). Note that the POs, as defined in this paper, hardly use the ERP as individuals and they do not function from the level of the Organization (OI) as a whole. That is why the II and the OI attributes were omitted from the PO model. Also note that the Vendor Quality (VQ) was not part of the PO model since the POs do not have any direct contact with the vendors. In this study, a sample of 4 POs filled out the PO questionnaire.

When grouping the IP and UP, observed is a very strong and positive impact on the S as well as on the WI as can be seen from the following Exhibit:

Exhibit# 7. Process Owner (PO) Model: the Effect of IP + UP Joined on S and WI.

Project Manager (PM)

There is only one Project Manager (PM) in charge of the ERP implementation at AUC. The PM model proposed is the only one with the VQ as well as the three impacts being measured together (II, WI and OI). In this research paper, the vendors were segmented into the SAP Egypt vendor (VQ SAP) as well as the outsourced Vendor (VQ outsource) as suggested by the PM who intuitively observed a big difference. The quality as to the SQ and VQ (both SAP & outsource) is proposed to affect the Implementation Problems (IP), as well as the impact vis-à-vis the Individual (II), the Workgroup (WI) and the Organization (OI).

When observing the quality regarding the SQ, VQ SAP, VQ outsource, significant results is deduced.

Exhibit# 8. Project Manager (PM) Model Quality Attributes Joined: Effect on IP, II, WI and OI

On examining the quality vis-à-vis the SQ and VQ, there results a strong correlation with the IP and the impact vis-à-vis: II, WI and OI as can be seen from the following Exhibit:

Exhibit# 9. Project Manager (PM) Model Quality Attributes Combined

(SQ + VQ SAP + VQ out): Effect on IP and Impact Combined (II + WI + OI)

Steering Committee 

Exhibit #10. Steering Committee (SC) Model

The Steering Committee (SC) at AUC has only three members in it with one of the three being a highly technical CTO. This model had the least number of attributes as opposed to the PM model with the most number of attributes. Here only the IP and  OP are directly associated with the SC. The results were considered high with a Correlation Coefficient of 0.802 despite the small sample size. Meaning that, the SC perceives the IP to strongly influence the OI.

This is quite apparent from the pie charts displayed in the following Exhibits where the evaluation of the IP and OI as assessed by the Steering Committee (SC) is relatively good, with the SC feeling indifferent and in rare cases disagreeing about it. This pattern is quite interesting because the SC members were the ones with the most disagreement among them maybe because of the gap between them in their technical background. Note here that is impossible to test the significance of this gap between the members because it means testing a “group” of one (very technical) against two (not technical) which cannot be done statistically because of missing standard deviation in one of the groups.

Exhibit# 11. SC Evaluation of IP and OI

Current Challenges/Problems Facing the Organization

There are many studies on “best practices” for implementing ERP, all with different schools of thought. One study found that the majority of ERP implementations were either phased rollout or big bang (Neal, 2010); but in general there is no one best way to do anything. At AUC a mixture of strategies was used. The general strategy is to phase in the ERP one department at a time with one PO in charge and then within the respective department; this was done using the big bang strategy.

The major challenge when implementing ERP at AUC was the turnover among the users during the implementation phase. This is a major issue with ERP in general (Wong et al., 2005) due to the lengthy ongoing process of the implementation phases. At AUC, the ERP is still being initiated in different departments slowly but surely. So far, the ERP implementation is complete in two departments and “under construction” in the third. Ever since the ERP was initiated, the PM was changed once and more than one PO was also changed.

As previously noted, prior to buying the ERP in 2007, AUC had acquired different computerized systems across the years but none were integrated together. The ERP allowed this integration to take place but only with the departments that finalized the ERP conversion process. The major challenge would then be how effective this is and how successful.

In this section, the researcher is integrating all the users together and applying the previously analyzed results on the proposed model. First, the quality attributes were grouped together (IQ + SQ + VQ sap + VQ out) in one generic quality attribute for all the users to test its effect on the perception and problems combined (IP + UP) as well as on Satisfaction and also on a generic impact attribute that we got after grouping the three impacts together (II + WI + OI). Notably, very significant results were detected in all relations as can be seen from the following Exhibit:

Exhibit# 12. Quality Attributes Combined (SQ + VQ SAP + VQ out): Effect on IP+UP Combined, along with Satisfaction, and on the Impact Attributes Combined (II + WI + OI) for all the Users Integrated Together

Furthermore, after combining all the users together, the effect of the IP + UP combined on S and the combined Impact (II + WI + OI) was very strong portraying very high correlation among the newly grouped attributes (see Exhibit # 13).

Exhibit# 13. IP+UP Combined and Effect on Satisfaction as Well as Effect on the Impact Attributes Combined (II + WI + OI) for all the Users Integrated Together

As for the effect of the S on the generic Impact attribute (II + WI + OI), also a high positive correlation was  detected as can be seen from the following Exhibit:

Exhibit# 14. Satisfaction Effect on the Impact Attributes Combined

(II + WI + OI) for all the Users Integrated Together

Finally after integrating all the users together, the collective quality, IP+UP, S, along with the collective impact attributes are displayed in the following Exhibit for easy comparison:

Exhibit # 15. The Combined Quality, the IP+ UP Combined, the S and Combined Impact of all the Users Integrated Together 

In this study, the impact of the ERP adoption on performance was examined in an educational institute, namely the American University in Cairo (AUC), although the results could be generalized to different activities. It was hypothesized that not necessarily the different ERP users (who have different roles) correspond to the same surrogate measure of success. To accomplish this, the ERP users were first segmented into five categories ranging all the way from the top management ERP decision makers to the very end users of such systems. Ten attributes were used in the proposed model: the quality as to the System Quality (SQ), the Information Quality (IQ), the SAP Vendor Quality (VQ SAP) as well as the Outsourced Vendor Quality (VQ outsourced); the User Perception (UP); the Implementation Problems (IP); the Satisfaction; and the impact vis-à-vis the Individual Impact (II), the Workgroup Impact (WI) and the Organization Impact (OI). It was predicted that all predecessor attributes either directly or indirectly affected all successor attributes (see Exhibit #2 for the proposed concept of the ERP model). Overall, the research findings suggest that all predecessor attributes significantly affected all successor attributes.

Thus, it is suggested that future researchers test this model with more universities and organizations.
 
References

Allen, D. & Kern, T. (2001). “Enterprise Resource Planning Implementation: Stories of Power, Politics and Resistance,” in N. L. Russo, B. Fitzgerald, and J. I. Degross (eds.), Realigning Research and Practice in Information Systems Development: the Social and Organizational Perspective, Boston, MA: Kluwer, Chapter 11, pp. 149-162.
Publisher – Google Scholar 

Bland, J. M. & Altman, D. G. (1997). “Cronbach’s Alpha,” British Medical Journal, Volume 314, page 572.
Publisher – Google Scholar – British Library Direct

Bradford, M. & Florin, J. (2003). “Examining the Role of Innovation Diffusion Factors on the Implementation Success of Enterprise Resource Planning Systems,” International Journal of Accounting Information Systems, Volume 4, pages 205-225.
Publisher – Google Scholar 

Chung, B. Y., Skibniewski, M. J., Lucas, H. C. & Kwak4, Y. H. (2008). “Analyzing Enterprise Resource Planning System Implementation Success Factors in the Engineering—Construction Industry,” Journal of Computing in Civil Engineering, Volume 22, Issue 6.pages 373-382.
Publisher – Google Scholar 

Cohen, J. (1988). Statistical Power Analysis for the Behavioral Sciences, 2nd edition, Lawrence Erbaum Associates.
Publisher – Google Scholar 

Cohen, J. (1992). “Current Directions in Psychological Science,” Vol. 1, No. 3 (Jun., Published by: Sage Publications,Inc. on behalf of Association for Psychological Science, Stable URL: http://www.jstor.org/stable/20182143.
Publisher – Google Scholar 

DeLone, W. H. & McLean, E. R. (1992). “Information System Success: The Quest for the Dependent Variable,”Information Systems Research, 3 (1), pp. 60—95
Publisher – Google Scholar 

Doll, W. J. & Torkzaden, G. (1988). “The Measurement of End-User Computing Satisfaction,” MIS Quarterly, Vol. 12 No. 2, pp. 259-71.
Publisher – Google Scholar 

Ifinedo, P. (2006). “Extending the Gable et al Enterprise Systems Success Measurement Model: A Preliminary Study,”Journal of Information Technology Management, Vol. 17 No. 1, pp. 14-33.
Publisher – Google Scholar 

Ifinedo, P. (2007). “An Empirical Study of ERP Success Evaluations by Business and IT Managers,” Information Management and Computer Science, Vol. 15, No. 4, pp. 270-282.
Publisher – Google Scholar – British Library Direct

Ifinedo, P. & Nahar, N. (2006). “Quality, Impact and Success of ERP Systems: A Study Involving Some Firms in the Nordic-Baltic Region,” Journal of Information Technology Impact, Vol. 6, No. 1, pp. 19-46.
Publisher – Google Scholar 

Kallunki, J. P., Laitinen, E. K. & Silvola, H. (2011). “Impact of Enterprise Resource Planning Systems on Management Control Systems and Firm Performance,” International Journal of Accounting Information Systems, Volume 12, pages 20-39.
Publisher – Google Scholar 

Leavitt, H. J. & March, J. G. (1962). ‘Applied Organizational Change in Industry Structural Technological and Humanistic Approaches,’ March J.G., ed., Handbook of Organizations, Rand-McNally, Chicago, iii., 1965, pp. 114-117.
Google Scholar 

Lewin, K. (1947). “Frontier in Group Dynamics: Concept, Method and Reality in Social Science; Social Equilibria and Social Change,” Human Relations, Vol. 1, pp. 5-41.
Publisher – Google Scholar 

Neal, Houston, ERP Implementation Strategies — A Guide to ERP Implementation Methodology,http://blog.softwareadvice.com/articles/manufacturing/erp-implementation-strategies-1031101/, March 11, 2010.
Publisher 

Plant, R. & Willcocks, L. (2007). “Critical Success Factors in International ERP Implementations: A Case Research Approach,” Journal of Computer Information Systems, Spring 2007.
Publisher – Google Scholar – British Library Direct

Rasmy, M. H., Tharwat, A. & Ashraf, S. (2005). “Enterprise Resource Planning (ERP) Implementation in the Egyptian Organizational Context,” European and Mediterranean Conference on Information Systems, Cairo, Egypt.
Publisher – Google Scholar 

Spathis, C. & Constantinides, S. (2003). “The Usefulness of ERP Systems for Effective Management,” Industrial Management and Data Systems, 103/9, pp. 677-685.
Publisher – Google Scholar 

Staehr, L. (2008). “Understanding the Role of Managerial Agency in Achieving Business Benefits from ERP Systems,” Information Systems Journal, Vol. 20, pp. 213-238
Publisher – Google Scholar 

Wong, A., Patrick. Y. K. C., Scarbrough, H. & Davison, R. (2005). “Critical Failure Factors in ERP Implementation,”www.pacis-net.org/file/2005/395.pdf.
Publisher 

Wu, J. H. & Wang, Y. M. (2006). “Measuring ERP Success: The Ultimate Users’ View,” International Journal of Operations & Production Management, Vol. 26 Issue 8, pp.882 — 903.
Publisher – Google Scholar – British Library Direct