Introduction
Although vendors of Enterprise Resource Planning (ERP) provide standard report with the acceptance of the customized form required by enterprises (Data Systems ERP & SAP ERP, 2015), its development process encounters some issues: (1) Standard mechanism is imperfect resulting in instable schedule; (2) Problems encountered in the development process are repeated resulting in poor performance; (3) The development personnel aren’t timely sharing information, making project being delayed; (4) Users are not familiar with its features, which makes it hard to produce benefits; (5) Despite the fact that the cooperative platform is built, developers only achieve little performance due to poor communication mechanism. Therefore, this study based on the view of collaboration design (1) reviews literature, and integrates the methods of action research to establish the prototype of report development mechanism; (2) conducts qualitative interview to modify the deficiency of the mechanism prototype; (3) applies Delphi method to summarize experts’ opinions, and creates the best project management mechanism for ERP report development, (4) reveals benefits and values about this result by case study; (5) explains the management implication of the research result and its value. This research not only decreases the time wasting and conflictions during the developing of the report, but improves report quality, even satisfies enterprises’ requirements.
Literature review
This research aims to (1) review literature about action research; (2) summarize difficulties of developing ERP report; (3) reveal collaborative design methods/modes; (4) review 3 qualitative research methods.
Procedure of Action Research
Action research (AR) is proposed by Lewin (1946); this methodology encourages people to improve society and raises the revolution, to change the working pattern and flow in their own industry. Susman and Evered (1978) regard the social activity as being well constructed and organized, so that people can apply AR spirit to manage its work and process; they suggest the following 5 steps to implement AR activity: (1) Diagnosis: define and discover problem; (2) Action Planning: propose methods and process to solve problem; (3) Action Talking: practice action; (4) Evaluating: assess performance; (5) Specifying Learning: suggest new improvement methods and steps. Malterud (2011) proposed 7 steps to practice AR activity: (1) recognize problem; (2) summarize experience; (3) arrange target; (4) construct implementation items; (5) describe process; (6) take action; (7) redefine problem.
Hjalmarsson et al. (2010) indicated the AR could assist members to find/research/resolve problem during physical action, and even improve project performance; its process includes: (1) Planning; (2) Action; (3) Observing; (4) Reflecting. Chapman et al. (2013) regards AR as an accelerator for members to learn knowledge and professional ability during collaborating, reflecting, recording, communicating, and negotiating with others. Therefore, Kemmis et al. (2014) believe the value of AR is “to execute immediately”, thus being a dynamic AR spiral. Such operation mechanism includes steps of (1) specifying learning; (2) diagnosing; (3) action planning; (4) taking action; (5) evaluating etc.
Obstacle for developing ERP Report
Though the biggest ERP Company in Taiwan- Data Systems ERP has provided report development system- “Express” for customers who implement “TIPTOP ERP”, which can reduce the ERP implementation cost and increase development flexibility to customizing forms. Although the “Express” features complete structure, which enhances the degree of rigor on customizing form, and allows users from each hierarchy to share the needed information (Data Systems ERP, 2015), project personnel in enterprise often lacks such developing capability to face obstacles in implementation.
In otherwise, SAP also design a report system – “SAP Crystal Reports”, which can produce visual graphics and dynamic charts, and even facilitating executives to accurately analyze and make decision. Meanwhile, report developers can quickly create reports through SAP Crystal’s portal in order to (1) shorten the time of delivering and developing information; (2) connect to any data source, design interactive report, and share information (SAP ERP, 2015). However, because the information between each department within enterprises is not linked, it reduces the overall development performance.
Collaboration Design Methods/Modes
The concept of collaboration design (CD) can provide an environment, which makes it easy for many people to explore and interact together, and decrease the various anomalies during project process, slash cost, and boost operational efficiency, which contains 3 design patterns: (1) mutual-collaboration; (2) exclusive collaboration; (3) dictator collaboration (Maher et al., 1996). And, based on different working patterns and attributes, scholars had proposed many models for CD. Lyu et al. (2002) stand in users’ characteristics perspective, to propose (1) horizontal collaboration: members sharing professional knowledge and resource to conduct duties; (2) vertical collaboration: members are built by designers and manufactures. In time serial perspective, Huang et al. (2003) separate CD into: (1) Parallel mode: multiple designer developing products in single platform same time; (2) Sequential mode: multiple designers developing product in single platform at different times. Furthermore, Zheng et al. (2010) based on the characteristic of time and location, propose 4 design methods: (1) face to face collaboration: collaboration in the same location at the same time; (2) synchronous distributed collaboration: collaboration in different locations at the same time; (3) asynchronous collaboration: collaboration in the same location at different times; (4) asynchronous distributed collaboration: collaboration in different locations at different times.
Qualitative Research Method
KJ method is proposed by Kawakita (1996), which can integrate complex factors to establish the relationship of mutual dependency between factors. So, it can group factors with the same attribute. Its steps are: (1) determine the theme; (2) gather data; (3) sort data into groups; (4) create header cards; (5) draw finished diagram (Cheng and Leu, 2011).
Focus Group Interviews (FGIs) allow a group of experts, by interaction and communication, to conduct confirmation, suggestion, modification on a specific subject to obtain consensus. Their steps include: (1) identification of the problem; (2) identification of population; (3) identification of moderator; (4) pretest of the interview; (5) recruiting the sample; (6) conducting the interviews; (7) analysis of the data; (8) writing the report; (9) decision making/action (Ryan et al., 2013).
Delphi Method (DM) can obtain consistency of experts’ opinions through multiple and repeated questionnaires. Bosun and Modrak (2014) indicate DM could acquire experts’ opinions and consensus by distributing the questionnaire several times. Its steps are: (1) identification of the problem and definition of the subject; (2) determining the expertise required; (3) selection of the experts, a heterogeneous response group; (4) preparation and distribution of the first questionnaire, (5) analysis of the first questionnaire; (6) a second written round, if necessary; (7) analysis of the second questionnaire; (8) having a group meeting (Hasson and Keeney, 2011).
Research Method and Design
To propose ERP report developing management mechanism, this study (1) establishes execution procedure of AR; (2) compares and decide CD method of report development; (3) proposes development mechanism prototype of CD; (4) establishes and confirms the final report development mechanism; (5) creates the development mechanism of form processes; (6) describes the management implication of this mechanism.
Establish Execution Procedure of Action Research
To make the established report development mechanism conforming with the process of AR, this study (1) integrates AR steps proposed by scholars and establishes the prototype of action research process through KJ method; (2) corrects and confirms integrated steps by FGIs; (3) proposes the best execution procedure of AR.
Although previous research results had established many concepts and processes of AR activity, to integrate these processes, this study invites a scholar to conduct KJ method, and follows the 5 steps described by Cheng and Leu (2011).
This result shows the processes, and contains the following 10 steps: (1) set study goal (identify improving objective); (2) identify current problem; (3) formulate overall plan; (4) run action (record status/results); (5) observe action process (recognize relation); (6) analyze and diagnose problem (discover cruxes); (7) re-think improvement plan; (8) assess initial purpose; (9) modify planed goal; (10) restart action cycle (1).
Furthermore, to confirm the rationality of the above 8 steps, this study follows the 9 steps proposed by Ryan et al (2013), then invites the following 2 experts to conduct twice FGIs: (1) an experienced management consultant; (2) a project management professor. In these interviews, experts: (1) exchange the order of the “Set study goal” and “Identify current problem” to meet the logic of project implementation process; (2) emphasize the importance to build knowledge sharing in “run action”; (3) combine “Analyze and diagnose problem” and “Assess initial purpose” into “Evaluate and diagnose problem” for evaluating the reliability of the result; (4) merge “Re-think improvement plan” and “modify planed goal” into “Set improvement goal” for selecting improving plan and correcting initial goal; (5) add “Restart action cycle” step after the end of processes for establishing the implementation objectives and principles in next cycle.
Through twice FGIs, the implementation procedure of AR activity is defined to be the following 8 steps to: (1) identify current problem: discover the problem of current status or preceding cycle; (2) set study goal: verify the main factor of the issue, and stipulate improvement goal; (3) formulate overall plan: plan on the detailed steps and schedule of report development; (4) conduct action: record each event, achievement, and experience during action process; (5) observe action process: observe the main phenomena of events that occurred during the process; (6) evaluate and diagnose the problem: evaluate and diagnose the implementation performance; (7) set improvement goal: focus on the main factor of various issues and propose overall improvement goals; (8) restart action cycle: filter and decide on feasible solution, then restart. Hence, this research not only obtained a clear step to conduct AR activity, but applied it to establish the developing mechanism of ERP report.
Compare and Decide Collaborative Design Method
To propose a reasonable collaborative design method, this study is based on the establishing process of AR implementation procedure and invites an expert and a professor to jointly conduct KJ method twice. After the expert and professor summarize the 11 CD methods proposed by 4 scholars, this study groups the “Collaborative Method” into 3 categories: (1) Dictator coupled collaboration (process controlling); (2) Close coupled collaboration (online operation/joint maintenance); (3) Loosely coupled collaboration (off-line operation/assigning work by specialty).
Next, this study in addition to the invitation of the two aforementioned professors also invites three ERP specialists with more than five years consulting experience and one professor taught ERP and project management to jointly conduct FGIs twice in accordance with the 9 steps described in Ryan et al (2013). It is to select, among the 3 categories of CD method, a method that is suitable for use in this study. After discussion, the members of expert group all agree that “Close coupled collaboration” method can be the foundation of report development mechanism, because : (1) many employees can use the platform at the same time; (2) have the same and consistent principal; (3) the information from the stage of customer requirement to that of technique development is instant and synchronous.
Propose the Development Mechanism Prototype of Collaboration Design
This study is based on the preceding implementation procedure of AR and CD method, to create 10 stages, 20 forms for controlling work quality control, then establishing the development mechanism prototype. Meanwhile, to ensure the rigor of this mechanism, it is to examine a variety of methodologies to support the rationality of each step and its control from.
Firstly, this study applies KT Problem Solving concept proposed by Kepner and Tregoe (1981) (assessment of the current situation, problem analysis, and decision analysis are the three systematic procedures to solve problem) to set up the following implementation forms: 1. In the 1st stage “Identify current problem”, it establishes: (1) the “Summary of current problem” table to record the issues, (2) the “Analysis of problem factor” table to explore various factors that cause problems. 2. In the 2nd stage “Set study goal”, it creates (3) “Study goal formulation” table to select improvement goal form main problem and its key factors, (4) “Summary of study target” table to describe the resources, tasks, and tools. Additionally, this research referring to the concept of Elaboration Theory Instruction proposed by Reigeluth (1992) (in project process, overall concept and objectives, and clear detail must be defined so as to present the overall framework) creates forms for: 3. In the 3rd stage “Formulate overall plan”, (5) it sets up the “Summary of plan implementation” table to clear the axis/compendium and execution method of the overall development plan, (6) through the “Action scheduling” table, it specifies the detailed schedule and steps.
Then, this study follows the concept of “Constant Comparative Method” proposed by Fram (2013) (summarize physical data, constantly compare, and construct new method) that creates forms for: 4. In the 4th stage “Physical conduct action”, it sets up (7) the “Action history log” table to record various important issues, practice, and outcome, (8) through “Summary of critical experience” table to implement knowledge sharing mechanism so as to discover critical technique. 5. Meanwhile, this study in the 5th stage “Observe action process” sets up: (9) “Observation of execution phenomenon” table to physically present various phenomena.
Furthermore, this study in accordance with the concept of “Critical Theory” proposed by Strydom (2011) to find out the essence of things through finding the contradiction and conflict implied in things or phenomena itself. In the 6th stage “Observe action process”, it sets up: (10) “Casual path of phenomenon” table to find occurrences procedure of various phenomena and the relationship between each phenomenon. In addition to this, this study referring to the concept of “Quality Control Story” proposed by Kondo (1990) (to jointly discover, improve, and solve problem) creates the following forms: In the 7th stage “Evaluate and diagnose problem” sets up: (11) “Action effectiveness evaluation” table so as to assess, through a number of guidelines, the effectiveness of report development and grasp the execution outcome and difference, (12) through the “Summary of inefficient performance” table, it shows the problem and phenomenon of the inefficiency, (13) it uses “Diagnosis of problem cause” table to highlight the problem and reason of inefficient affairs.
Next, this study in accordance with the concept of “Deduction Approach” proposed by Suter (2011) in the “Question Instruction/Problem Solving” (to find out difficulty or problem, determine its nature, propose solution, observe and verify the feasibility of scenarios) creates the following forms: In the 8th stage “Set improvement goal”, it sets up (14) the “Summary of key problem factor” table to generalize the main problem and its factors, (15) through “Acknowledgement of improvement goal” table, it proposes to improve programs of the problem, and establish goals, (16) it uses “Goal quality assessment” table to ensure the rationality and value of the improved goal, (17) it works up the “Problem improvement program” table to specify the execution practice and step for improvement program. In the 9th stage “Restart action”, it sets up (18) “Program appropriateness evaluation” table to assess the feasibility of various programs and select valuable ones.
Lastly, this study follows the concept of “Evaluation Criteria” proposed by Blackwell (1965) (to apply 7 guidelines for evaluation of various steps of the whole AR process) that creates the following forms: in the 10th stage “Restart action”, it sets up (19) “Action research evaluation” table to re-examine various steps, and, (20) “Reaction planning” table, it determines the development goal for the next stage and items.
To confirm the rationality of the 10 stages and 20 work quality control forms proposed in this development mechanism, this research invites 2 ERP experts, 1 supervisor from EPR development department of enterprise, and 1 professor taught project management to jointly conduct FGIs twice. This research had followed experts’ opinions, then corrected the field name, connection, location, similarity and definition of each stage and form. So far, the “Development Mechanism Prototype” is confirmed by experts and scholars, which contains 6 stages, 13 forms, and 57 fields.
Establish and Confirm the Final Report Development Mechanism
To ensure the rationality of this prototype, this research is based on Dalkey’s (1969) opinions; it not only invites the 4 aforementioned experts, but invites another 4 consulting EPR advisors and 4 professors to conduct twice Delphi questionnaire survey by the 8 steps proposed by Hasson and Keeney (2011).
And, this research collects the 12 questionnaires, and confirms the consistence by “Quartile difference method” suggested by So and Bonk (2010). The 2nd survey results indicate (1) 10 forms are highly consistent (Q value ≦ 0.6); (2) 2 forms are fairly consistent (Q value between 0.6 and 1); (3) 1 form is poorly consistent (Q value >1); (4) the overall consistency is 92% (=12/13).
By general principal (the overall consistency is higher than 70%), this study can stop surveying. But, to build a much reliable result, this research conducts Delphi questionnaire survey in next week, it shows similar results: (1) 10 forms are highly consistent; (2) 2 forms are fairly consistent; (3) 1 form is poorly consistent; (4) the overall consistency is 92% (=12/13). Apparently, this result is extremely rational and reliable. So, the “development mechanism draft” not only positively recognizes experts, but be trusted by enterprise.
Development mechanism of form processes
To assist enterprise understand and implement “development mechanism draft”, this research again applies FGIs to confirm the procedure and casual relationship of 6 stages and 13 forms. And then, this research translates the above result into Figure 1 “Development diagram of ERP report”.
In Figure 1, developers can base on the following steps while encountering a problem: (1) fill “1. List of current problem” to describe physical problem and requirement; (2) complete “2. List of problem factor”, “3. List of improvement goal”, “4. Introduction table of goal condition”, “5. List of implementation plan”, “6. Schedule of action process” for detailed description of the problem and target; (3) fill “ 7. Record of daily work” for sharing knowledge, experience, solution, and method; (4) explain intensive details in “8. List of core experience”; (5) write down the results of observation “9. Report of implementation detail” and “10. Path diagram of casual relationship” by “7. Record of daily work”; (6) show improvement result by summarizing “8. List of core experience” to “11. Assessment table of action performance”; (7) establish the basement to discover issues by summarizing “10. Path diagram of casual relationship” and “11. Assessment table of action performance” into” 12. Record of inefficient action”; (8) complete “13. Assessment table of improvement result” by concerning improvement achievement
Figure 1: Development diagram of ERP report
Case Study
To ensure the results are useful and feasible, this study cooperates with Immense Digitize Engineering Corporation (Chunghwa Telecom (CHT)”Cloud ERP” suppliers, enterprise management consulting firms), they are cooperates with Chunghwa Telecom (the largest telecommunications company in Taiwan) for developing SaaS ERP. During the 3 months, Immense Digitize had successfully established ERP report developing mechanism, then confirmed the research result is reliable.
Hence, this research aims to (1) describe the progress and opinion in case study: (2) state the key implementation item of ERP report development mechanism: (3) check shortcoming during development activity.
Progress and Opinion in Case Study
This research will introduce the process and results during the 3 months (78 business days). To give a detailed description of the process of the entire mechanism, this research held an educational session for ERP report development mechanism. Furthermore, this research had held communication meetings twice a week.
This research practices the 1st stage “Identify current problem” based on development mechanism in 1-7 days. In this stage, developers presented 20 issues, and recorded in “1. Summary of current problem”. And then, the 20 issues could classify into operation (5 issues), communication (2 issues), programming (5 issues), system (3 issues), others (5 issues). Furthermore, developers had discovered 53 possible factors for the 20 issues, and scored each factor, even filtered 16 major factors, recorded in “2. Analyze problem factor”.
Then, during the 8-15days, this research introduces the 2nd stage “Set study goal”. In this stage, developers proposed 16 improvement goals, and then listed in “3. Study goal formulation”. And, developers applied Likert 7 scale to score the 16 goals, and then sorted its priority. Moreover, developers wrote down 14 goals (average score higher than 4) “4. Summary of study target”, and stated related description. These descriptions contain various topics and targets, including: (1) ERP system; (2) collaborative design platform; (3) Microsoft Word; (4) Microsoft Excel; (5) iReport-5.0.0; (6) communication software; (7) Microsoft SQL Server 2013. At the same time, to enhance the stability, developers had described detailed information in “notification” field.
During 16-20 days, this research implements “Formulate overall plan”, members proposed 52 steps to practice above 14 major goals, and records the 52 steps in “5. Summary plan implementation”. In other words, to decrease development risk, avoid great mistake, and confirm essential resource, this research had stated “implementation principle” and “essential resources”. Finally, members proposed implementation schedule based on the 52 steps, and struggled to improve ERP report during 35 days. Therefore, in “6. Action scheduling”, this research not only shows the progress bar of each activity by members, but displays the situation of each related activity.
During 21-55 days, this research practices “Physical conduct action”, developers had completed collaborative duties, and shared experience and process to develop a report. Through 10 times session in the 5 weeks, developers had summarized 352 “implementation experience” into “7. Action history log”. Then, to boost developing ability and reduce mistake occurrence, developers had selected 56 development tips from “implementation experience (positive)”, and listed in “8. Summary of critical experience”.
During 56-60 days, this research starts the 5th stage “Observe action process”, developers list 102 “implementation experience (negative)” into “9. Observation from execution phenomenon”. Even, developers draw the casual relationship among 72 situations from 102 “implementation experience (negative)” in “10. Casual path of each phenomenon”.
Last, during 61-78 days, this research conducted the 6th stage “Observe action process”, developers assessed 14 goals by “completion status” and “evaluation criterion”, and analyzed the difference between result and target, even recorded the reason of differences in “11. Action effectiveness evaluation”. Moreover, developers selected 5 poor-performance goals, and listed 15 factors and reasons in “12. Summary of inefficient performance”. And, the 15 factors and reasons would be the new goal in the next cycle. Finally, developers apply the following 7 criteria: (1) efficiency; (2) quality; (3) cost; (4) time cost; (5) value; (6) resource usage rate; (7) integration, to evaluate the 14 goals, and then record 9 greatness results in “13. Diagnosis of problem cause”.
To confirm the benefit of cooperation activity, this study applies Likert 7 points scale to conduct the satisfaction questionnaire survey twice. The results say the average value of each key business project can get over 5 points (70% of 7 points) acceptance in two acceptance questionnaires. Obviously, the findings of this study are accepted by the 6 designers.
Key Implementation Item of Development Mechanism
The case company hopes all developers can timely report and record their work outcome on a daily basis. Thus, it uses this study result to develop a series of document information systems so as to become a management foundation for developing projects.
In two months development process, this study provides practical opinions and explanations. During the 1 month trial process, 26 employees from 4 project development/designing departments have proposed the following 4 opinions. Although developers/designers have proposed 4 opinions, the manager responsible for this project deems to observe the situation for a period of time prior to considering if it is necessary to modify the system so as to satisfy these enquiries. With one more month of use, in order to satisfy the enquiries of the developers, the project manager finally modifies the system offering 3 new functionalities of (1) providing definition and description of various fields on the form; (2) adding query functionality to facilitate reference and comparison, and (3) adding signing functionality to speed up the efficiency of communication on upper and lower hierarchies’ view.
In the semi-open questionnaires, it is apparent the development mechanism obtains acceptance of executives and subordinates with the use of the mechanism. And, this also means that the study results are to be accepted, trusted and used.
Apparently, this development mechanism of ERP report (contains 6 stages, 13 forms and 34 items) in Table 1, not only successfully integrated action research and cooperative design method, but was even trusted by supervisors and users.
Table 1: Key implementation item of ERP report development mechanism
Checking Shortcoming during Development Activity
This research had achieved brilliant results with the support and cooperation from supervisor, manager, and developers. However, to boost efficiency and prevent difficulties occurring, all managers and developers had collected 5 categories and 25 interference factors.
And, this research aims to introduce the importance of each category and factor. The most crucial category for developing ERP report is “understand ERP system operation and attribute”, which contains: “1. Realize system attribute”, “2. Practice report operation process”, and “3. Correctly upload report file”. Then, the “ability and experience to solve problem” category is also critical for developing report, which contains: “4. Independently developing plenty report”, “5. Clearly describe fields logic”, “6. Resolve special situation”, and “7. System operation experience”. Furthermore, how to recognize “ERP report programming resource” is still important, it contains “8. Training and education”, “9. keep practicing”, “10. Understand document description”, “11. Unify programming code”, “12. Read technical material”, “13. Learning programming reference”, “14, provide real test”. Then, the 4th category “communication mechanism” including “15. Understandable document notice”, “16. Clear requirement description”, “17. Group learning activity”, “18. Sharing and feeding back mechanism”. “19. standardize technical words”, and “20. Physically distribute duties”. Finally, “programming logic” category including “21. Independently structure program”, “22. Professional SQL function knowledge”, “23. Keep learning”, “24. Supervisor supporting”, and “25. Heavily depend partners”.
Despite the fact that “understand ERP system operation and attribute” is the most critical category, it contains little interference factors, and could be improved by practicing and carefully programming. However, though “ERP report programming resource” and “communication mechanism” are less important, they feature many factors, it says above 2 lessons are crucial for implementing collaborative development design.
Moreover, to simplify forms, this research conducts twice Delphi questionnaire survey (Likert 7 scale) to measure the importance of each form. Once the importance is higher than 6, it means it is “extremely critical form”, including: “1. List of current problem”, “3. List of improvement goal”, “5. List of implementation plan”, “7. Record of daily work”, once importance between 4.1-4.9, means it is “critical form”, containing: “2. List of problem factor”, “6. Schedule of action process”, “8. List of core experience”, “10. Path diagram of casual relationship”, “12. Record of inefficient action”, once importance less than 4, means it is “referable form”, which including: “4. Introduction table of goal condition”, “ 9. Report of implementation detail”, “11. Assessment table of action performance”, “13. Assessment table of improvement result”.
Conclusions
Despite the fact that ERP vendors had developed various systems to develop report, they still find it hard to overcome the obstacle during report developing. To slash time cost and satisfy customers’ report requirement, this research combines action research and collaborative design, then building “development mechanism of ERP report”. Enterprise can apply the mechanism to achieve: (1) controlling and monitoring report development process; (2) boosting development performance and report quality; (3) exploring problems and keeping improving mechanism; (4) sharing information and knowledge promptly; (5) Optimizing the platform.
Hence, the “development mechanism of ERP report” proposed in this study has 3 management implications: 1. In management mechanism dimension, it: (1) enables enterprise to effectively control work quality on development, and improves report development quality and efficiency; (2) helps developers cultivate their habit of constant examining, improving, and optimizing work process, 2. In management application dimension, it: (1) boosts the internal information of enterprise and knowledge becomes instant with the high level of sharing; (2) prevents the repeated mistake occurrence during the process of development, then raises management efficiency and technical force, 3. In management decision dimension, it: (1) assists executives to master schedule and problem, then timely improve and effectively control the schedule of project management; (2) cultivates developer/designer with effective self-management, and then increases the ability to solve problems. Therefore, these results not only break through the dilemma during report development, but enhance enterprises to meet the diverse needs from customers, and even promote the overall operation performance.
References
1. Bosun P. and Modrak V. (2014), ‘Using the Delphi method in forecasting tourism activity,’ International Letters of Social and Humanistic Sciences, 14, 66-72.
2. Blackwell S. (1965), Action Research: its Promise and Problems. Applied Research in Education, Littlefield: Adams & Co, 243-244.
3. Chapman H., Lewis P., Osborne Y. and Gray G. (2013), ‘An action research approach for the professional development of Vietnamese nurse educators,’ Nurse Education Today, 33(2), 129-132.
4. Cheng Y. M. and Leu S. S. (2011), ‘Integrating data mining with KJ method to classify bridge construction defects,’ Expert Systems with Applications, 38(6), 7143—7150.
5. Data Systems ERP. (2015), Express Report, available online at: http://www.dsc.com.tw/Product_Solution/Product_ERPII_1/Express/tabid/630/Default.aspx.
6. Dalkey N. C. (1969), The Delphi Method: An Experimental Study of Group Opinion, Santa Monica, CA: The Rand Corporation.
7. Fram S. M. (2013), ‘The Constant Comparative Analysis Method Outside of Grounded Theory.’ Qualitative Report, 18(1), 1-25.
8. Hasson F. and Keeney S. (2011), ‘Enhancing rigour in the Delphi technique research,’ Technological Forecasting and Social Change, 78(9), 1695-1704.
9. Hjalmarsson A., Rudmark D. and Lind M. (2010), ‘When designers are not in control—experiences from using action research to improve researcher-developer collaboration in design science research’ Global perspectives on design science research, Springer Berlin Heidelberg, 1-15.
10. Huang C. J., Trappey A. J. and Yao Y. H. (2003), ‘Design and development of a WfMC-based modularized network collaborative design system,’ Journal of Chinese Institute of Industrial Engineers, 20(4), 422-434.
11. Kemmis S., McTaggart R. and Nixon R. (2014), The Action Research Planner: Doing Critical Participatory Action Research, Springer.
12. Kawakita J. (1996), KJ-method-let chaos tell, Chuoukouronsha, Tokyo.
13. Kondo Y. (1990), ‘Key points in quality control training courses for managers,’ Total Quality Management, 1(3), 309-318.
14. Kepner C. H. and Tregoe B. B. (1981), The New Rational Manager, NJ: Princeton Research Press.
15. Li B., Zheng M., Austin S. and Thorpe T. (2010), ‘Application of Collaborative Project Management in Preconstruction Planning’ IEEE International Conference on 3rd Information Management, Innovation Management and Industrial Engineering, 3, 357-360.
16. Lyu J., Fu H. P. and Lin Y. (2002), ‘Business Excellence through Collaborative Management’ Proceedings of the Fourth Asia-Pacific Conference on Industrial Engineering and Management Systemsx, Taipei, Taiwan, Springer Berlin Heidelberg.
17. Lewin K. (1946), ‘Action research and minority problems,’ Journal of social issues, 2(4), 34-46.
18. Malterud K. (2011), Kvalitative metoder i medisinsk forskning, Universitetsforlaget, Oslo.
19. Maher M. L., Cicognani A. and Simoff S. (1996), ‘An experimental study of computer mediated collaborative design’ Proceedings of the IEEE Workshops on Enabling Technologies Infrastructure for Collaborative Enterprises (WET ICE ’96)), 268-273.
20. Ryan K. E., Gandha T., Culbertson M. J. and Carlson C. (2014), ‘Focus Group Evidence Implications for Design and Analysis,’ American Journal of Evaluation, 35(3), 328-345.
21. Reigeluth C. M. (1992), ‘Elaborating the elaboration theory,’ Educational Technology Research and Development, 40(3), 80-86.
22. SAP ERP. (2015), Crystal Report, available online at: http://www.sap.com/taiwan/solution/sme/software/analytics/crystal-bi/index.html.
23. Strydom P. (2011), Contemporary critical theory and methodology, Taylor & Francis.
24. Suter W. N. (2011), Introduction to educational research: A critical thinking approach, Thousand Oaks, CA: Sage.
25. So H. J. and Bonk C. J. (2010), ‘Examining the Roles of Blended Learning Approaches in Computer-Supported Collaborative Learning (CSCL) Environments: A Delphi Study,’ Journal of Educational Technology & Society, 13(3), 189-200.
26. Susman G. I. and Evered R. D. (1978), ‘An assessment of the scientific merits of action research,’ Administrative science quarterly, 4(23), 582-603.
