Introduction
This paper presents the results of a study carried out in identifying perception of audit effectiveness amongst Quality Environment (QE) auditors. Quality environment audit in Malaysia was first introduced in 1998 and formally refered to as 5S certification audit. 5S is an acronym for five Japanese words that are Seiri, Seiton, Seiso, Seiketsu and Shitsuke. In this programme, the driving force for Quality Environment (5S) program comes from people. In this respect, Shitsuke is critical to its success. Shitsuke refers to instilling good habits through training. This component is expected to create a more conducive, clean and tidy workplace. The heart of the concept is the belief that positive change occurs through investing in people as “with every pair of hands comes a free brain”. A successful Quality Environment (5S) Practice programme should contribute to a change in culture as well as a change in processes and work practice. This was demonstrated in a study conducted locally by Khamis, et al. (2009 ). In the study, 5S practices is seen as an effective technique that can improve housekeeping, environmental performance and health and safety standards in an integrated holistic way (Khamis, et al., 2009 ). The dynamic of 5S practices were further elaborated by Samuel (2010) with his 5S+ concept (a combination of 5S and Lean 5S). However, concerns have been raised in recent years about the usefulness of conventional compliance audits for continuous improvement due to inconsistency of audit processes and results, and the questionable value of compliance audits towards understanding complexities of business systems (Karapetrovic & Willborn, 2001). Karapetrovic & Willborn (2001) further suggested and advocated the implementation of a systems approach in auditing in order to add ‘‘value’’ to the audited organization. It is argued that in a well-planned and managed audit system, ‘‘competent auditors’’ must strive to identify improvements. Dereli, Baykasoglu, & Das (2007) asserted that the ‘‘quality world’’ has focused on the auditing process and tried to find effective ways to improve the quality of ‘‘quality auditing’’ and conclude that only ‘‘value-added audits’’ can be employed as a continuous improvement tool.
In communicating the outcomes of the study, this paper is presented in three main parts: the first part discusses the literature review on the QE audit, followed by a description of the methodology adopted in the study. The third part discusses the findings and its implication. Finally, the conclusion and recommendations for future research are then presented.
Quality audit is defined as the process of systematic examination of a quality system as defined in the ISO 9000 (2000). It further describes an ‘‘audit’’ as ‘‘a systematic, independent, and documented process for obtaining evidence and evaluating its objectively to determine the extent to which audit criteria fulfilled’’ (Pivka & Mulej, 2004). The extent to which the audit criteria have been fulfilled can be considered as audit effectiveness. An audit is effective when it has met its objectives. Beckmerhagen, Berg, Karapetrovic, & Willborn (2004) mentioned that by virtue of being a goal oriented activity, the audit process and its achieved outcomes (audit performance) are always compared with the planned objectives. A typical evaluation of audit performance includes the measurement of the achieved effectiveness and efficiency, and a subsequent comparison of the actual performance with the expected goals. This is consistent with ISO 9000 (2000), where effectiveness is defined as the extent to which planned activities are realized, planned results are achieved and resources effectively utilised.
Today’s organisations are compelled to obtain QS certification in response to the demands of customers and business conditions (Tsim, Yeung, & Edgar, 2002). Rajendran & Devadasan (2005) quote a study by Karapetrovic & Willborn (2000)w hich posits that in order to meet this requirement at a faster rate, quality-auditing exercise is being carried out hurriedly as a part of the QS certification process. Purely compliance audits have become a routine for both auditors and the auditee; therefore, the quality audit scope has been enlarged to areas where the organization or managers master their business (Pivka & Mulej, 2004). The question is then raised as to whether auditors distinguish the different objectives of compliance verses quality audits. Unlike compliance audit where the objective is to merely indicate that the auditee has met the quality standards, a suitably timed and properly organized quality-auditing programme will lead to continuous quality improvement (Beecroft, 1996). However, Pomeranz (1992) opines that there were two spectres that loomed over auditing: ineffectiveness and inadequate productivity.
Beckmerhagen, et al. (2004) stated that improvement in audit effectiveness may mean doing less auditing, while greater audit productivity may suggest doing more. They argue that in order to adequately measure audit effectiveness, one should evaluate not only the results (outcomes) of the audit against planned objectives, but also the audit process (extending from the planning and execution to the reporting and follow-up) and resources (including the auditor independence and competence). This conflict means that measuring audit effectiveness is indeed a more complex task than it might appear first hand. It involves the evaluation of the whole audit system, including objectives, processes andresources (Karapetrovic & Willborn, 2000).
This study attempts to adapt an approach used in a model developed by Mihret & Yismaw (2007) which outlined four grouping of factors that influence the internal audit effectiveness, namely: organisational setting, management support, internal audit quality and auditor attributes (Mihret & Yismaw, 2007, p. 472). Two of the groups i.e. internal audit quality and auditor attributes are now applied to QE audit effectiveness in this paper. This paper seeks to assess the perceptions of the factors that contribute to the Quality Environment audit effectiveness based on attributed internal audit quality.
Research Methodology
Data were gathered via a survey using self-administered questionnaires. In order to ascertain that the responses were from the practitioner’s perspective, the survey target respondents were from the QE certification auditors possessing knowledge about QE audit and who have participated in the certification audit process. The study was conducted at the audit certification unit of an organization that is responsible for promoting QE certification in Malaysia, both for public and private sectors. The survey questionnaire consisted of two main sections. The first section captured respondents’ demographic profile, followed by the second section which listed 14 statements designed to obtain the respondents’ perception towards the factors that affects QE audit effectiveness. Likert Scale (1 as strongly disagree, 3 as neither agree nor disagree and 5 as strongly agree) was used in capturing the respondents’ perception. The survey questionnaires were later distributed via email and collected from the respondents in person.
Result and Analysis
Demographic Background
The survey response rate is 63.89%; 23 out of 36 responses returned were usable for analysis. Analysis of the age group shows that only 26.1% of the total respondents were below 35 years old, and only 3 respondents did not have post-graduate qualification. The result shows that the respondents composition on position between managerial and executive position are almost balanced with 56.5% at executive level (both consultants and senior consultant level), and remaining at the management level (manager and above).
Summary of the years of working experience shows that although 21.7% (5 respondents) only had 1 to 3 years of experience in QE auditing, all of the respondents have been working for more than 5 years. This data show that all respondents have involved in QE certification auditing process. This is one of the important elements of this study in ensuring that the result is a reflection of QE audit practitioners’ perception.
Top Three Prioritizations between Management and Executive Levels
The overall mean scores for each of the 14 statements were calculated and listed from highest to lowest as displayed in Table 1. As shown in the table, the mean score ranged from 3.609 to 4.913 (on a five point scale with 1 being strongly disagree, 3 being neither agree nor disagree and 5 being strongly agree). Analysis on the mean differences between management and executive are within 0.19 points, from which can be safely concluded that there is no major gaps of perception between the two levels. Analysis by position groups, on the other hand, shows that differences between highest and lowest mean value for each groups are 1.1 points for management and 1.5385 for executive level. The lower disparity in mean values reflects consistence of responses from management as compared to the executive responses.
Table 1: Mean Score Comparison between Management and Executive Levels
(Note: 1, 2 & 3 represents the top three values of means for each group)
Tables 1 shows that both levels rating knowledge, skills and abilities have the highest mean score with management mean score of 4.9 (value of standard deviation is 0.3162), and executive level mean score of 4.9231 (standard deviation value at 0.2774). Based on this, it can be safely mentioned that both levels perceive knowledge, skills and abilities as the most important factors that contribute to the QE audit effectiveness. There was a slightly more variation in the responses of the management compared to the executives’. It seems that both management and executives agree that the most important factors for QE audit effectiveness are the knowledge, skills and abilities of the auditor.
Similar trends were also applied to the second prioritization based on mean score. This refers to the ability of QE auditor to apply QE standards criteria with management level mean score of 4.8 (standard deviation value at 0.4216), and executive level mean score of 4.6154 (standard deviation value at 0.50637). Additionally, for executive level, the ability of auditors to give recommendation for improvement also falls in the second ranking of prioritisation together with the ability to apply QE standards criteria. Both of these factors have the same mean score and standard deviation value (mean score of 4.6154, standard deviation value at 0.50637). Contrary to the first prioritization, there is a greater variation in the responses of the executive level compared to the management level. The results also imply that management rates the ability of the auditor to apply the QE standard criteria higher than the ability to recommend. Such view appears to be biased to compliance thinking rather than improvement thinking.
As for the third highest prioritization, mean score shows that management level rates auditors’ abilities to identify non-conformance of QE practices with mean score of 4.7 (standard deviation of 0.48205), whilst executive rates ability to apply knowledge on QE and other related subject matter impacting the effectiveness of QE audit (with mean score of 4.4615, standard deviation value at 0.51887). Again, the executives responses were much more varied compared to the management. The executives also do not appear to differentiate between the criteria as can be seen from similar weightage given to item 14. Management on the other hand clearly weighs item 14 (audit findings makes best improvement on QE practices contributes more to the effectiveness of QE audit) than item 9 (auditors’ ability in applying QE knowledge and other related subject matters). This balances out the earlier finding (second prioritization) of management preference for the compliance aspect of QE criteria application.
Responses Comparison of Factors Affecting QE Audit Effectiveness
Seven statements in the survey were unanimously recognized as the factors that contribute to the effectiveness of the QE audit. The seven factors are: (1) selection of auditors, (2) knowledge, skills and abilities, (3) ability of auditors in applying QE standard criteria, (4) ability to identify non-conformance of QE practices, (5) ability to give recommendation for improvement, (6) ability to apply knowledge to QE and other related subject matters, and (7) finding of auditing that will best improve QE practices. All the items were rated as agree (Likert scale value at 4) or strongly agree (Likert scale value at 5) by all the respondents.
In contrast with the seven items mentioned above, mixture responses were noted from the remaining 7 items between the two groups to the factors that affected the quality audit effectiveness. The seven items are (1) number of years in service, (2) number of auditors, (3) determination scope of audit, (4) number of audit report produces, (5) duration of producing audit report, (6) fair presentation of the audit finding, and (7) clarity of the audit finding. Mixture responses refer to the selection of response ranging from strongly disagree (Likert scale value at 1) to strongly agree (Likert scale value at 5). Having said that, the percentages of respondents who have chosen to disagree or neither to agree or disagree is minimal as shown in Table 2.
Table 2: Analysis in Differences of Perception between Levels
Conclusion
The result of prioritization, based on mean score values, indicates that the top three prioritizations of the factors perceived to contribute to the QE audit effectiveness are almost similar in both management and executive levels. However, clearly management have stronger preferences of certain aspects of QE audit over other aspects. This could be due to their experience as well as the level of responsibilities they hold. This could have implications for knowledge transfer from senior to junior auditors. Nevertheless, the results should be interpreted with caution as the sample is small and concentrated in one department. Half of the items were unanimously recognized as the factors that contribute to the effectiveness of the QE audit, whilst the other half provided mixed responses. The current study represents the perspective of QE audit effectiveness from the practitioners’ point of view, targeted towards QE certification auditors. The outcome of the study contributes to the QE audit practices which enable the practitioner to make more informed decisions in designing continuous improvement process. The study also contributes to the existing literature on QE audit and perceives factors that contribute to its effectiveness, specifically focusing on Malaysian environment. Hence, recommendations for further research should include more robust research on the perception on effectiveness, not only for QE but also for other Quality certification series audit. The researchers also suggest future attempts to focus on outlining valid and reliable criteria for measuring auditor effectiveness in conducting QE audit. Empirical study at a larger scale is needed as business environment is facing constant changes and the expectation of Quality improvement program is on the rise. Studies at a larger scale need to be conducted for appropriate generalisation of the findings.
Acknowledgment
Authors would like to acknowledge the helpful comments from the participant of 16th IBIMA Conference held in Kuala Lumpur, Malaysia. Authors also would like to thank three anonymous individuals for their assistance in enabling this study. Errors and omissions are solely the author’s responsibility.
(adsbygoogle = window.adsbygoogle || []).push({});
References
Beckmerhagen, I. A., Berg, H. P., Karapetrovic, S. V. & Willborn, W. O. (2004). “On the Effectiveness of Quality Management System Audits,” The TQM Magazine, 16(1), 14-25.
Publisher – Google Scholar – British Library Direct
Dennis Beecroft, G. (1996). “Internal Quality Audits – Obstacles Or Opportunities?,” Training for Quality, 4(3), 32-34.
Publisher – Google Scholar – British Library Direct
Dereli, T., Baykasoglu, A. & Das, G. S. (2007). “Fuzzy Quality-Team Formation for Value Added Auditing: A Case Study,” Journal of Engineering and Technology Management, 24(4), 366-394.
Publisher – Google Scholar
ISO 9000. (2000). “Quality Management System – Fundamentals and Vocabolary,” Geneva: International Organisation for Standardization.
Publisher
Karapetrovic, S. & Willborn, W. (2000). “Quality Assurance and Effectiveness of Audit Systems,” International Journal of Quality & Reliability Management, 17(6), 679-703.
Publisher – Google Scholar – British Library Direct
Karapetrovic, S. & Willborn, W. (2001). “Audit System: Concepts and Practices,” Total Quality Management, 12(1), 13-28.
Publisher – Google Scholar – British Library Direct
Khamis, N., Ab Rahman, M. N., Jamaludin, K. R., Ismail, A. R., Ghani, J. A. & Zulkifli, R. (2009) . “Development of 5S Practice Checklist for Manufacturing Industry,” Paper Presented at the Proceeding of the Wold Congress on Engineering 2009 London, UK.
Publisher – Google Scholar
Mihret, D. G. & Yismaw, A. W. (2007). “Internal Audit Effectiveness: An Ethiopian Public Sector Case Study,” Managerial Auditing Journal, 22(5), 470-484.
Publisher – Google Scholar – British Library Direct
Pivka, M. & Mulej, M. (2004). “Requisitely Holistic Iso 9000 Audit Leads to Continuous Innovation/Improvement,” Cybernetics and Systems: An International Journal, 35(4), 363-378.
Publisher – Google Scholar – British Library Direct
Pomeranz, F. (1992). ‘The Successful Audit: New Ways to Reduce Risk Exposure and Increase Efficiency,’ New York: Business One Irwin.
Google Scholar
Rajendran, M. & Devadasan, S. R. (2005). “Quality Audits: Their Status, Prowess and Future Focus,” Managerial Auditing Journal, 20(4), 364-382.
Publisher – Google Scholar
Samuel, K. M. H. (2010). “Integrated Lean TQM Model for Sustainable Development,” The TQM Journal, 22(6), 583-593.
Publisher – Google Scholar
Tsim, Y. C., Yeung, V. W. S. & Leung, E. T. C. (2002). “An Adaptation to ISO 9001:2000 for Certified Organisations,” Managerial Auditing Journal, 17(5), 245-250.
Publisher – Google Scholar – British Library Direct
