Design Science Methodology as a Framework for Artefact Creation and Risk-Informed Modelling

Introduction

Design Science Methodology (DSM) is a research paradigm primarily concerned with the development and validation of normative knowledge, particularly in the field of information technology. It is characterized by a focus on the creation of artefacts (such as models, methods, designs) with the intention of solving practical problems and improving the functional characteristics of these artefacts. Research in this area differs from the natural sciences in that it is concerned with developing clear solutions or artefacts to achieve specific goals (whereas in the natural sciences, research goals are often more theoretical and difficult to apply in practice). The main goal in this area is to produce knowledge that professionals in a particular field can use to develop practical solutions to real-world problems. As mentioned above, the central activity in DSM is the creation of artefacts, which are described in more detail later in the paper. Nevertheless, in modern conditions, its use has become more widespread in other areas, such as management, marketing, sociology, psychology. Thus, a more detailed assessment in terms of the possibilities of using this technology, improving the development of artefacts, as well as taking into account the human factor and other risks in model development remains relevant.

A significant number of scientists have been analysing the DSM and improving the possibilities of applying this paradigm. Thus, M. Muntean et al. (2022) studied the application of DSM in the development of business artefacts. They emphasized two approaches to the development of artefacts in this area: based on the answer to emerging problems and to specific questions within the business process. Special attention was paid to the creation of artefacts for sustainable development. From the point of view of ethical issues, the DSM was discussed by A. Elragal and M. Haddara (2019). The scientists noted that when conducting such studies, researchers should not discriminate on the basis of any characteristics, be objective about their own and others’ conclusions, respect the intellectual property of others and financial interests. Special attention was paid to the issues of plagiarism in such works. Authors in turn, evaluated Big Data assessments in the context of Design Science Research (DSR). The researchers used established evaluation criteria that covered scientific and practical usefulness, aimed at contributing to both DSR and science in general. They also emphasized the importance of taking into account ethical principles, especially confidentiality, when carrying out such projects. F. Jacob et al. (2022) assessed the principles, methods, contributions, and limitations of DSM in marketing. As part of their work, they demonstrated the potential of this field for marketing research, emphasizing the importance of conducting research that is scientific and applicable to practitioners. The authors believe that in the future, the quality of artefact development and the possibility of its applied use will improve in marketing and other fields.

The purpose of this study was to review the DSM, in particular the component of artefact creation and human factors’ assessment within the model. This will help to increase the efficiency of future developments of similar models among scientists.

Materials and Methods

A survey was conducted at the University of Gdansk among IT students who have completed at least one cybersecurity-related course. The survey was aimed at identifying the most relevant features of risk analysis methodologies. These students have not yet had professional experience, so their opinions are not burdened by the conditions imposed by their employer or manager in the form of methods and tools they already use in their work. Therefore, when starting their own business or implementing an information security management system, they will be guided by their own opinion when choosing a risk analysis methodology. A total of 85 students took part in the survey. It was anonymous and based on a 5-degree grading scale. The possible answers were: absolutely yes, yes, I don’t know, rather no, absolutely no, and each answer was assigned a corresponding number of points: 2, 1, 0, -1, -2. The questions asked to the respondents were related to various components associated with working with artefacts, risk assessment, and some other issues in the context of DSM. For example, questions were asked such as “is the versatility of the methodology an important feature for you?”; “is the openness of the risk assessment algorithm important to you?”; “is the risk identification function important to you?”. All of this allowed drawing conclusions about the importance of risk assessment and consideration of the human factor in research. More detailed information (the full number of questions asked and the number of “points” for each of them) is shown below in the study.

The main approach used in the study was a systematic one, which allowed analysing all the data and considering the DSM as such, its role and application possibilities. The study also used a significant number of scientific methods. For example, the analysis was used to draw conclusions based on existing data in open sources about how DSM is considered within the modern scientific literature, how modern models are built to solve certain practical problems. The historical method allowed evaluating the approaches that prevailed within this paradigm in the past. The forecasting method made it possible to draw a conclusion about the prospects for the development of the situation in the future. The graphical method allowed for the construction of tables, which are important for understanding the main results of the study. All calculations were carried out in Microsoft Excel. Nevertheless, like any scientific work, this article has certain limitations. For example, it covers a wide range of topics related to DSM, from its conceptual framework to evaluation methodology and human factors. Despite the fact that the article comprehensively describes the concepts and stages of the DSM, it does not provide clear examples of how the DSM can be applied in practice.

Results

As mentioned above, DSM is only a specific method of solving practical problems, characterized by the construction of artefacts that are actually these very methods of solving certain difficulties (Gregorio et al., 2021). It refers to a set of guidelines, principles, and systematic procedures that researchers follow when conducting design research, describes the step-by-step process and methods used to create, design, and evaluate artefacts or solutions to solve specific problems or address specific concerns; provides researchers with a structured framework to follow throughout the research process, from problem definition to artefact creation and evaluation. Design Science Research, in turn, is a broader research paradigm or approach that focuses on the creation and evaluation of innovative artefacts or solutions to real-world problems, emphasizing the creation of new artefacts such as software systems, algorithms, models, frameworks, or design theories that are intended to solve specific problems or improve a particular situation. It often involves iterative cycles of design, development, and evaluation to produce practical and relevant solutions. Thus, although both components have similar names and refer to the same method, they have different meanings in essence, which should also be assessed when analysing the work. The term Design Science can be used to summarize these two phenomena.

An artefact itself is a unique solution to a problem with a clearly described specification. Artefacts began to be created as answers to various practical difficulties, and were supposed to be a solution to various problems. It could be provided in the form of a specific object or technology, i.e., one that can be built and used in practice in the future. These should be certain objects that can be built and used in the future, but in modern conditions they can also be models that allow solving certain problems or any other options for their solution. An artefact as such must have a design, i.e., a plan for its formation and further use. The design plan is responsible for creating an object in such a way that it meets the main goals of the project, while the use plan is responsible for its further use. The creation of artefacts in research is always determined to be relevant due to the fact that scientists’ knowledge about them can never be complete, and creation plans should not be standardized. The effectiveness and success of the project (artefact) in the future can be influenced by how comprehensively and clearly the project is planned and all its aspects are discussed; how much the performers themselves know what they expect to see from this project. Nevertheless, although certain recommended stages and components are provided for the research process, they can actually take a very different form, and the proposed option is far from mandatory. The works in the context of Design Science can also be divided into separate categories or genres. They offer different approaches to conducting research, each with its own focus, process, role of theory, and evaluation methods. Nevertheless, they all have certain common features, namely the fact that an artefact has been developed, a certain problem to be solved, a goal.

The conceptual structure of DSM consists of several elements. They are shown and described below in Table 1.

Table 1:  Elements of DSM models and their description

Source: compiled by the author based on data from J. vom Brocke et al. (2020b).

As can be seen in Table 1 above, the DSM focuses on the study of real-world problems in a variety of application areas. It assesses the existing knowledge base to determine if design knowledge is available to solve the problem. If there is insufficient information available, then it aims to generate innovative methods to solve these problems based on the existing information. There are six main steps involved in this process: problem identification and motivation (defining the research problem), defining the goal to be solved, design and development (generating the artefact, which is any designed object that contributes to the research), demonstration (using the artefact in a real-world setting), evaluation (actually evaluating the effectiveness of the artefact in solving the problem), and communication. The evaluation process itself is particularly important, as it must be conducted professionally in order to assess whether the model is effective in further use. The following components can be suggested: assessment of novelty, importance, and feasibility (to what extent the model can be implemented); assessment of the solution design (simplicity, clarity, consistency); assessment of effectiveness (what benefits will be brought by its implementation in the real world). In general, evaluation can be carried out both before and after the creation of an artefact. Pre-assessment involves considering potential systems or technologies before making decisions about acquisition or implementation. It often uses a cost-benefit analysis to determine whether the technology is worth implementing. There are various assessment methodologies that fall into three main approaches: fundamental, comprehensive and meta-approach; and the process of conducting such an analysis can also be approached in different ways: reductionist, relying solely on metrics, or hermeneutic, taking into account the understanding and interpretation of metrics by decision makers. In particular, it is worth highlighting that ex ante evaluation in public sector systems can take into account factors beyond economic performance, such as human life and well-being, and often includes participatory measures and social perspectives. Ongoing evaluation, in turn, takes place after the system or technology has been implemented. Examples of such evaluation methods include pilot projects, user opinion surveys, historical data analysis, and constructs of success analysis. Contemporary approaches to evaluation take into account the context, content, and design of the process, taking into account the needs of stakeholders and a variety of measurement tools (Miah and Genemo, 2016).

The human factor is essentially the influence of people, their behaviour, decisions, and abilities on the results of model development. Taking this factor into account is important in many industries where models are used for forecasting, planning and decision-making. It is important to assess how people react to behaviour in certain situations, their reactions to risks, constraints, changes in conditions, or other factors. It is possible to assess the interaction of people with technical systems and technologies that are taken into account in the models. This may include responses to user interfaces, automated processes, and other aspects. In addition, the level of people’s skills and training, as well as certain psychological aspects, such as emotional state, and their impact on decision-making, should be assessed. Surprisingly, despite the variety of methodologies that exist, many of them do not assess the human factor, even in the area of crime, where it should play a major role. There may be many reasons for this, but the most likely is the complexity of the human factor as such its subjectivity and difficulty in analysing it. It is also possible that some professionals do not really consider it necessary to add this factor to their research work because they consider it unimportant, and some do not have enough resources or knowledge (particularly in psychology or related disciplines) to conduct research with human factors analysis.

Nevertheless, in recent years, there has been a growing focus on human factors in modelling, especially in areas where the impact of people on a system or process is important. The same applies to the lack of work describing existing capabilities for analysing risks that can arise from many causes, including human factors. The reasons for this have already been partially mentioned above: in fact, they are related to all the advantages that come with the use of the DSM, namely the possibility of using a clear systematic approach, improving the quality and efficiency of research, in particular, by creating an artefact. In this case, it has its own special configuration, which differs from the usual development of artefacts, for example, in the IT sector. In this case, the artefact should take the form of a risk model, a risk assessment tool, or some analytical tool that can be used for similar purposes. An artefact in risk analysis should serve as a tool for understanding, assessing and managing risks in a specific context, for example, within a particular company, for a particular project, within a particular business. This should allow for more innovative solutions to be offered in the risk assessment process, reduce their impact on the business process, and increase the reliability and safety of projects.

Thus, the relevance of the functions was assessed as shown in Table 2.

Table 2: Description of the question and the number of points scored as part of the survey conducted by the University of Gdansk

Source: compiled by the author.

Table 2 shows that the artefact itself should be a high-quality methodology with a special IT system, a clear algorithm that takes into account the human factor, supports risk identification and is designed for a specific industry, i.e., government agencies. The survey also shows that students are quite attentive to the issue of risk assessment and human factors in their research.

As mentioned above, a characteristic that distinguishes this methodology from others is the formation of a so-called artefact, which is essentially any constructed object or theory (system, model, method) that has been developed to solve a specific problem or meet a specific need. It plays a key role in the DSM context, as it is the primary means by which researchers address real-world problems in a given field of study, or identify opportunities. Its formation is the basis of the research, although the principles of this process will differ greatly depending on the research area. In fact, the rationale for creating artefacts in the DSM can be quite broad and serve several purposes at the same time: solving research problems, encouraging researchers to develop innovations, generating new knowledge and theories. However, this process can face certain difficulties, which can be very diverse: from the lack of a clearly defined goal before starting work, which does not allow for a solution to be formed, to ethical considerations. Of course, the process of constructing an artefact itself is also a complex process (creating a model after the relevant researchers have decided what it is, or creating a certain technology), but, in these matters, the researcher’s skills, knowledge, and practice play a crucial role. In this context, it is possible only to advise to follow the research plan and strictly fulfil all its basic requirements.

Discussion

An overall assessment of Design Science Research was conducted by J. vom Brocke et al. (2020a). The researchers explored the key concepts and models associated with Design Science Research to create a fundamental understanding of the planning, execution and sharing of knowledge generated by specific DSR projects. There is a very wide range of radically different methodologies available, but only a few of them take the human factor into account in a meaningful way. This was done, for example, in the study by J. Bell and J. Holroyd (2009). In their work, they tried to update the Health and Safety Executive (HSE) on developments in the field of human reliability assessment methods. The review identified 17 tools that could be useful for health and safety departments. While most of them were well-established methods, several new “third generation” tools were also noted that contain industry-specific data. The researchers showed that all the tools have recognized limitations, and yet they can all be used. They can still provide important data for risk assessment. The paper also divides these tools into those that should be classified as first generation and those that should be classified as second generation. First-generation tools may be suitable for sites that are just starting to quantify human risk, as they provide basic information. Second-generation tools may be more appropriate for sites that have historically used first-generation methods and now require a more in-depth understanding of risk (Vasilevski and Birt, 2021.). Only one relevant third generation tool specific to the nuclear industry was identified. Nevertheless, according to scientists, the human factor remains the most important source of threats to information systems.

The possibilities of using Design Science in the framework of business evaluation were assessed in the work of D. Dimov et al. (2023). They concluded that entrepreneurship research based on Design Science should focus on aligning the real world with theoretical constructs. This purposeful goal should be consistently applied throughout the research process, ensuring validity and soundness. This was also mentioned in the paper above, when the emphasis was placed on the fact that in modern conditions an artefact can take on a variety of guises and does not necessarily have to be a new technology. It may well be a development in the context of business principles in its various components, macroeconomic policy development, or any other economic variables.

In turn, A.R. Hevner (2007) in his work also considered the concept of Design Science and outlined three inherent research cycles within this paradigm. The first was the relevance cycle, which initiated DSM by identifying opportunities or problems in a real-world application environment. It contained research requirements and defined criteria for evaluating research results. Its output would be field tested and evaluated in the application domain, and the results of the field-testing would lead to further iterations of the relevance cycle to refine the research requirements based on practical experience. The next is a rigorous cycle that draws on the knowledge base of scientific theories, engineering methods, experience, and existing artefacts relevant to the research. In this cycle, researchers must thoroughly research and reference the knowledge base to ensure that their designs are innovative rather than routine. The cycle involves selecting and applying appropriate theories and methods to construct and thoroughly evaluate the artefact. The last is the design cycle, which involved rapid repetition between the construction and evaluation of the design artefact, creating alternative designs and evaluating them against the requirements until a satisfactory design is achieved.

The work above also proposed a methodology for developing an artefact and conducting a DSM. However, it had a different structure and focused on identifying problems and motivations, setting goals, developing the artefact, demonstrating it, evaluating it, and communicating it further. In any case, different methodologies for conducting DSM may have a right to exist, since only in practice can one understand which one is more effective (De Leoz and Petter, 2018). It is only necessary to evaluate the positive and negative aspects of each of them and select the one that would most effectively perform the functions required within the research.

Peffers et al. (2007) developed a methodology for Design Science for conducting research within the framework of information systems. Scientists tried to develop a similar methodology in terms of studying innovative systems (IS) and noted that these areas are quite new compared to other disciplines. They define the model proposed in the work as general, noting that there may be alternative approaches to its development or review, based on specific research objectives and context. The authors also speculated that other types of DS research methodologies may emerge in the future. These may include methodologies for curiosity-driven research, context -specific research streams, problem-solving in organizational contexts, improving specific research processes, or responding to unique constraints.

A study on training future teachers to design immersive educational resources within the framework of the DSM was conducted by S.O. Semerikov et al. (2022). The study provided a general definition of e-learning resources as structured digital materials with subject content and metadata, and outlined four categories of requirements for the design of such resources: general didactic requirements, specific didactic requirements, psychological requirements, and ergonomic requirements. The concept of immersiveness, which implies deep involvement of the subject in the content, was also explored. As a result, a methodology for designing immersive educational resources was developed, consisting of four interrelated components: target (learning objectives), content (educational content), technological (teaching methods and tools), and evaluation and performance (assessment and expected results). As part of the study, researchers also created learning content and examples for the development of various classes of immersive educational resources, including courses, tutorials, and electronic reference books. It is worth noting that the human factor component of the study was also not investigated in detail, despite the fact that the research was conducted relatively recently. Nevertheless, the methodology proposed by the researchers can be considered correct for use (its components are similar to those mentioned in the study above).

The future trend of research and evaluation of the human factor (in particular, in the field of system logistics) was assessed by F. Sgarbossa et al. (2020). The paper calls for a focus on human-centred industrial design, modelling with human-centred approaches, and management with a human-centred perspective. The vision paper emphasizes that considering the human factor in these systems is crucial for business success, especially in the context of digital transformation. This should encourage researchers and organizations to bridge the gap between traditional operations management and human capital measurement perspectives. In general, these statements also confirm the importance of human capital measurement, as noted in the paper above. It is worth noting that, in the future, it is worth expecting an increase in the number of works that will directly take into account the human factor within the framework of artefact creation and in the context of DSM in general.

Opportunities to improve the efficiency of factories by aligning it with the human factor were assessed by J. Li et al. (2018). In their work, they focused on the humanization of production in the context of global innovation, emphasizing the importance of improving the efficiency of factories and their scientific alignment with the human factor. Scientists describe the factory of the future as one that is humane and cares about people’s well-being and productivity. However, assessing this component of the human factor is not enough. It is important to understand that a person is a danger to any work or project implementation. They can make a mistake either because of their incompetence, inattention or stressful circumstances (at work or in their life in general). Therefore, this part of the human factor should also be taken into account when creating artefacts.

Peffers et al. (2018) in their study highlighted important points and new directions in the context of DSR research. They proposed to consider aesthetics as the third dimension of the value of artefacts, along with utility and truthfulness. This suggests that artefacts can be not only functional and accurate, but also beautiful or elegant, recognizing the importance of aesthetics in systems designed for hedonic or aesthetic purposes. In addition, scholars have identified their social impact, emphasizing it as systems are increasingly focused on end users. This intersects with the study of the human factor and its role in the development of such systems. In other words, within their research, the scientists point out the role of assessing the impact of the developed factors on the human component, and also considered it important to take it into account within the systems as a whole, which was also described in the framework of the work above.

Conclusions

The paper explores DSM as a problem-solving paradigm that focuses on creating innovative artefacts to solve real-world problems. Its projects aim to develop human knowledge and organizational capabilities through the development of new designs, models, methods, and examples. The creation of artefacts is central to this methodology, as they represent unique solutions to well-defined problems. Artefacts can take the form of objects, technologies, models, or other solutions with a clear design and use case. The process of developing an artefact consists of several stages, including problem identification, goal setting, design and development, demonstration, evaluation, and communication. Their evaluation is critical, including assessments of novelty, importance, feasibility, design quality, and effectiveness. Researchers are constantly encouraged to create artefacts as part of their research, but because the development process itself is complex, their creation is often problematic. For this reason, researchers are advised to strictly adhere to research plans in order to achieve effective results.

The paper also highlighted the importance of the human factor in modelling and evaluation, especially in areas where human influence on systems or processes is significant. However, many existing methodologies do not adequately account for it, and therefore more and more attention is being paid to addressing this research gap. The article points to a number of reasons that may cause this situation, including the complexity of assessing this factor, the need for additional funding for these purposes. Nevertheless, in recent years, attention to this factor has been increasing, and in the future, it is worth expecting the role of this component in the models to grow.

The future of this paradigm has several promising areas of development. For example, the integration of the human factor into modelling and evaluation methodologies should be a priority, especially in areas where human influence has a significant impact on the results. In addition, researchers should develop and improve comprehensive evaluation strategies that cover all stages of the research process, from problem identification to artefact creation and evaluation. Innovations in the process of artefact creation should also be encouraged to promote development through this paradigm of science and innovation.

References

• Bell J., Holroyd J. 2009. Review of human reliability assessment methods. Buxton: Health and Safety Laboratory.
• De Leoz, G., Petter, S. 2018. Considering the social impacts of artefacts in information systems design science research. European Journal of Information Systems, 27(2), 154-170.
• Dimov, D., Maula, M., Romme, A.G.L. 2023. Crafting and assessing design science research for entrepreneurship. Entrepreneurship Theory and Practice, 47(5), 1543-1567.
• Dresch, A., Lacerda, D.P. Antunes Jr., J.A.V. 2014. Design science research. In: Design Science Research (pp. 67-102). Cham: Springer.
• Elragal, A., Haddara, M. 2019. Design science research: Evaluation in the lens of big data analytics. Systems, 7(2), 27.
• Figueiredo, J., García-Peñalvo, F.J. 2022. Design science research applied to difficulties of teaching and learning initial programming. Universal Access in the Information Society. https://doi.org/10.1007/s10209-022-00941-4
• Gregório, J., Reis, L., Peyroteo, M., Maia, M., da Silva M.M., Lapão, L.V. 2021. The role of design science research methodology in developing pharmacy eHealth services. Research in Social and Administrative Pharmacy, 17(12), 2089-2096.
• Herselman, M., Botha, A. 2020. Applying Design Science research as a methodology in post graduate studies: A South African perspective. In: SAICSIT ’20: Conference of the South African Institute of Computer Scientists and Information Technologists (pp. 251-258). New York: Association for Computing Machinery.
• Hevner, A.R. 2007. A three cycle view of design science research. Scandinavian Journal of Information Systems, 19(2), 4.
• Jacob, F., Pez, V., Volle, P. 2022. Principles, methods, contributions, and limitations of design science research in marketing: Illustrative application to customer journey management. Recherche et Applications En Marketing, 37(2), 2-29.
• Keskin, D., Romme, G. 2020. Mixing oil with water: How to effectively teach design science in management education? Brazilian Administration Review, 17(1), e190036.
• Li, J., Tan, X., Li, J. 2018. Research on dynamic facility layout problem of manufacturing unit considering human factors. Mathematical problems in Engineering, 2018, 6040561.
• Miah, S.J., Genemo, H. 2016. A design science research methodology for expert systems development. Australasian Journal of Information Systems, 20. https://doi.org/10.3127/ajis.v20i0.1329
• Muntean, M., Danaiata, R.D., Hurbeana, L. 2022. Applying design science research for developing business artifacts. Procedia Computer Science, 199, 637-642.
• Peffers, K., Tuunanen, T. Niehaves, B. 2018. Design science research genres: Introduction to the special issue on exemplars and criteria for applicable design science research. European Journal of Information Systems, 27(2), 129-139.
• Peffers, K., Tuunanen, T., Rothenberger, M.A., Chatterjee, S. 2007. A design science research methodology for information systems research. Journal of Management Information Systems, 24(3), 45-77.
• Semerikov, S.O., Vakaliuk, T.A., Mintii, I.S., Hamaniuk, V.A., Soloviev, V.N., Bondarenko, O.V., Nechypurenko, P.P., Shokaliuk, S.V., Moiseienko, N.V., Shepiliev, D.S. 2022. Design methodology for immersive educational resources. Educational Dimension, 58, 176-199.
• Sgarbossa, F., Grosse, E.H., Neumann, W.P., Battini, D., Glock, C.H. 2020. Human factors in production and logistics systems of the future. Annual Reviews in Control, 49, 295-305.
• Vasilevski, N., Birt, J. 2021. Human-centered design science research evaluation for gamified augmented reality. Frontiers in Virtual Reality, 2, 713718.
• vom Brocke, J., Hevner, A., Maedche, A. 2020a. Introduction to design science research. In: Design Science Research. Cases (pp. 1-13). Cham: Springer.
• vom Brocke, J., Winter, R., Hevner, A., Maedche, A. 2020b. Special issue editorial – Accumulation and evolution of design knowledge in design science research: A journey through time and space. Journal of the Association for Information Systems, 21(3), 520-544.
• Wrycza, J., Maślankowski, B., Marcinkowski, 2014. Implementation diagrams in IT systems modeling. E-mentor, 53(1), 77-85.