Factors affecting IT Audit Quality: an Exploratory Study

An information technology audit is defined by Pathak (2005) as “the process of collecting and evaluating evidence to determine whether an information system safeguards assets, maintains data integrity, consumes resources efficiently, and achieves organisational goals effectively.” In Malaysia, according to Mahzan and Veerankutty (2011) 16% of audit firms perform IT audit assurance services. This trend towards IT audit is likely to increase as IT audit continues to be stressed by the accounting profession and the auditing standards. For example, International Standards of Auditing (ISA 315) requires auditors to examine and ascertain the IT procedures, processes, and controls when assessing the client’s controls environment. Abstract


Introduction
An information technology audit is defined by Pathak (2005) as "the process of collecting and evaluating evidence to determine whether an information system safeguards assets, maintains data integrity, consumes resources efficiently, and achieves organisational goals effectively." In Malaysia, according to Mahzan and Veerankutty (2011) 16% of audit firms perform IT audit assurance services. This trend towards IT audit is likely to increase as IT audit continues to be stressed by the accounting profession and the auditing standards. For example, International Standards of Auditing (ISA 315) requires auditors to examine and ascertain the IT procedures, processes, and controls when assessing the client's controls environment. IT has changed the way an audit is conducted (Kim et al. 2009) and has become pervasive in the way businesses conduct their operations and record their transactions. Organisations are increasingly relying on IT internal controls to safeguard their physical assets and data. For example, IT corporate governance framework, such as COBIT has provided guidelines and procedures on governance of IT in an organisation. The primary role of an IT audit is to ensure the integrity of an organisation's information systems (Senft and Gallegos, 2008). IT audits are important organisational processes that add value to the organisation because they support the auditor's judgement on the integrity, reliability and quality of the information produced by the organisation's information systems (Gallegos et al. 2004).
To ensure the value of IT audits, organisations should implement a standard method for evaluating the quality of audits (Senft and Gallegos, 2008). There has been a stream of research into factors that make up IT audit quality (Havelka and Merhout 2007, Merhout and Havelka 2008, Stoel 2012, Havelka 2013. IT audit quality literature was developed from two strands of research. The first strand of research and the earliest IT audit quality framework study was using the inductive approach. A focus group was used to obtain and develop factors related to IT audit quality (Havelka and Merhout 2007). As this study consists of only a few respondents, it is likely that the factors generated could be biased. Havelka and Merhout expanded their study further by including more participants to validate their list (Merhout and Havelka 2008). Nevertheless, their numbers of participants are still low. Their framework called ITAP contains general categories and does not provide questionnaire tools. . The Merhout and Havelka study was later combined with research from Stoel et al (2012) that includes 140 indicators, 26 concepts and 6 broad categories. Thus, the framework is difficult to apply as a survey instrument.
The second strand of IT audit quality approach uses the deductive approach. Stoel et al. (2012) is the only known paper that developed and empirically tested a survey tool on IT audit quality. They developed the constructs by combining constructs from financial audit quality literature (Carcello, Hermanson et al. 1992, Vehn, Carcello et al. 1997 with the IT audit literature (Merhout and Havelka 2008). That paper then used factor analysis from the ICASA respondents from the United States to rank which factors have the most impact on IT audit quality. However, that research did not look at the effects that these factors have on the outcome of IT audit nor look at the relationships between these factors.
The purpose of this study is to extend previous research by developing a broad IT quality survey that is practical that could be used to understand the factors that affect IT audit quality. This paper will present preliminary descriptive results of those factors.

Literature Review and Theoretical Paradigms
Information technology (IT) has become an indispensable tool in modern business. The increased reliance on IT and the complex, evolving nature of IT systems, has resulted in the need to implement internal controls to safeguard commercial information (Stoel and Muhanna, 2011). Not surprisingly, IT in all of its facets, has received extensive scholarly attention. Recent attention on IT audit research has focused on the need for, and the conduct of, an IT audit (Al Omari, Barnes, and Pittman, 2013). To be classified as an IT audit, the examination must involve information technology, either as the specific focus of the examination (even indirectly, such as IT governance), or as the means to complete an engagement (Chong and Tan, 2012;Merhout and Halveka, 2008 Omari, Barnes, and Pittman, 2012;Parkinson and Baker, 2005), or conceptualisations surrounding the conduct of IT audits. For instance, scholars have examined the strategy and standards of IT audits (Pealrson, 2001), computer assisted tools/software used in IT audits (Gallegos, Vlosky, and Vlosky, 1992;Gillevet, 1995), the planning (Lam, 2001) and management (Van Grembergen and De Haes, 2005) of the audit. From a broader perspective, scholars have studied the effectiveness of IT audits (Alzeban and Gwilliam, 2012), the role of an IT auditor (Chaney and Kim, 2007), the types of IT audits (Senft and Gallegos, 2008), the IT audit process (Gallegos, 2002) and the training of IT auditors (Curtis et al. 2009).
One area in the IT literature that has received little scholarly attention is the quality of IT audits. This is surprising given the increasingly critical function that IT plays in organisations, the need for a clearer understanding of what constitutes quality in IT auditing is needed. Consequently, there is no definition of IT audit quality. Regardless, any notion of IT audit quality is nestled in an organisation's IT governance (Ferguson et al. 2013).

Model Framework
This section defines the IT audit quality constructs identified from the literature and also explains the associated items that reflect these constructs. We developed the dependent variable IT audit quality while the independent variable constructs are taken from Stoel, Havelka, and Merhout (2012) and Halveka and Merhout (2013). With a few exceptions, we use the same items as Stoel, Havelka, and Merhout (2012) while reclassifying a few items to the appropriate constructs. We also added items from Heroux (2012) and Havelka and Merhout (2013).
We posit that the following IT related factors; "Auditor IT Knowledge and Competencies", "Internal Control Knowledge", "Target System Complexity", and "Resources" are correlated with IT Audit Quality while controlling for accounting variables that are found to affect audit quality as mentioned by Carcello et al. (1992), Samelson et al. (2006) and Vehn et al. (1997). The following sections describe in detail the constructs and scales used.

IT Audit Quality
Although IT audit quality is not explicitly defined, it could be implied from the objectives of IT audit. We use the Delone and McLean (2003) paradigm that states that • effectiveness that is whether IT audit could assess that the organisation information system is able to meet organisational goals (Weber 1988, Merhout andHavelka 2008), • reliability that is how reliable is the IT audit conducted on the auditee (Stoel 2012), • efficiency that is how well the IT audit is able to perform while minimising the cost (Stoel 2012), • overall perception of quality that is how the IT audit is viewed (Lowensohn, Johnson et al. 2007).

Audit team's IT knowledge and competencies
To be able to detect material weaknesses in IT systems, audit teams need to have knowledge and competencies not only about accounting and auditing, but also on IT specialised knowledge. Specialised IT professional qualifications and certifications have been shown to have more likelihood of involvement with auditing on IT governance, risks and controls (Héroux and Fortin, 2012). Knowledge about IT and accounting system, are shown to be important factors in IT audit quality (Havelka and Merhout, 2013;Stoel, Havelka, and Merhout, 2012). IT audit teams require knowledge of tools and techniques to help them audit "through the computer" rather "around the computer" (Janvrin, Bierstaker et al. 2008, Janvrin, Lowe et al. 2008. Lastly, understanding of the risks involved from the technology used was identified as a factor to IT audit quality (Havelka 2013). Accordingly, we took the construct and the following scales from Stoel • specialised IT professional qualifications and certifications, • knowledge of IT and accounting system, • knowledge of CAAT (Computerassisted auditing tools), • knowledge of risks associated with technology use.

Audit team's Internal Control Knowledge
Poor internal controls are likely to cause material misstatements in the financial statements (Ge and McVay 2005). Thus, knowledge of internal controls is an important facet of IT auditing (Stoel, Havelka, and Merhout, 2012 (2012) • knowledge of internal controls, • knowledge of information security, • knowledge of data processing integrity, • knowledge of data structure controls

Target system complexity
This construct refers to how difficult it is to  (Havelka and Merhout, 2013). For this questionnaire we incorporated "Business Scale and Audit Scope" and "Auditability" into target system complexity because the construct is the function of business size and scale of the auditee, how broad the scope of the audit, the support given by the auditee and the reliability of the internal controls (Stoel, Havelka, and Merhout, 2012). These are then indicated by the following items provided by Stoel (2012): • number of geographical dispersed business units, • number of business units or processes or systems involved, • support by auditee, • how well the internal control is defined and documented.

Resources
Resources refer to the availability of audit tools, time, budget and audit staff that the audit team could command to assist their IT auditing activities (Stoel, Havelka, and Merhout, 2012). These items include: • whether computer-assisted auditing tools (CAATs) are used (Stoel 2012), • whether there is enough time to conduct the IT audit (Héroux 2012), • whether there is enough budget available to conduct the IT audit (Héroux 2012), • whether there is enough staff to properly conduct the IT audit (Havelka 2013).

Organisation profile
About three quarters (74%) of the respondents are large companies and the rest medium size companies. This is consistent with the population because we surveyed only the Top Market Cap of the Malaysian Stock Exchange. As shown in Table 2, the top 3 industries our respondents are in; the construction industry (17%), manufacturing (16%) and food and hospitality (11%).

Dependent and independent variables
Our dependent variable is the IT audit quality. The respondents rate IT audit performed in their organisation as effective, reliable and efficient (Table 3). For the independent variables (Table 3 to   Table 7), target system complexity is ranked the highest (mean of 5.7) followed by internal control knowledge (mean of 5.65), and IT knowledge and competencies (mean of 4.74).     The correlation matrix as shown in Table 8 indicates that IT knowledge and competencies are moderately positively correlated with IT audit quality (p<0.001).
All the other independent variables are shown to be significantly (p<0.05) positively correlated with IT audit quality except auditor independence and target system complexity. Nevertheless, since our sample size is small with increase in the number of respondents, the significance would likely increase as well. Our results seem to indicate that IT knowledge and competencies are important in determining the IT audit quality. There is also significant interaction between the independent variables. Firstly, audit planning and methodology is strongly correlated with auditor and audit interaction (p<0.001). Secondly, resources provided in the firm are also strongly correlated with auditor's IT knowledge and competencies (p<0.001).

Conclusion and Discussion
IT has become pervasive and critical in successful operations and management of any organisations. Thus, it has become necessary to audit the information systems of organisations. Previous research has focused on questionnaires that have too many items and may not be practical. In addition, the factors affecting IT audit quality also have not been studied.
The contribution of this exploratory paper is to identify broad constructs from the literature that affects IT audit and used it to develop a questionnaire. Furthermore, this paper presents the preliminary descriptive statistics on the relationships of the factors that affect IT audit quality. We found that auditor's IT knowledge and competencies are significantly correlated with IT audit quality. This has implication on policymakers and professional accounting bodies in improving IT audit quality.
Our research is limited by the number of respondents. This affects the statistical methods and inferences that can be drawn from the data. Nevertheless, our paper sets the stage towards more research in this area. The questionnaire used needs to be rigorously tested and validated.