Development of an Original Methodology For Assessment of Private-Public Integration Efficiency Level

Introduction

Preliminary factors assessment would allow potential participants to forecast the probability of successful implementation of innovations projects. A detailed algorithm would rate countries on the bases of the probability of successful implementation of innovation projects. This unique integrated system of indicators enables a simultaneous assessment of five basic factors that influence the success of PPP projects: institutional environment quality, existing practice of innovation projects implementation, level of readiness of government authorities, private organizations, and population for successful implementation of public-private partnership implementation.

In developing the indicators, the authors used the following literature: The development of public-private partnerships was considered in the researches of the scientists: Hodge, Greve, 2007; Ke, Wang, Chan, 2010; Llanto, Navarro, Ortiz, 2016; Marques., 2011; Moszoro, 2010; Schaeffer, Loveridge, 2002; Steijn, Klijn, Edelenbos, 2011; Wang, 2014. The studies of the institutional environment were presented in the books: Chernov, 2016; Popov, 2015. The problems of increasing the effectiveness of the private sector were studied by the scientists: Mostafavi, Abraham, Sinfield, 2014; Plohotnicova, 2014; Voronov et al, 2017; Wang, Zhao, 2014. The effectiveness of state enterprise zones and public policies was considered in the researches: Kingdon, 1984; Lewis, 1980; Millward, 2011; Peters, Fisher, 2002.

Methodology and Methods of Research

The principle of Implementation of innovative PPP projects is presented in Fig. 1.

Fig. 1: Implementation of innovative PPP projects

Author’s methodology is to nine stages:

1. Determination of the assessment indicators list;
2. Selection of experts who can assess indicators by individual countries;
3. Assessment of the level of demonstration of each indicator by using the method of peer review. Peer review of indicators;
4. Assessment coordination between experts;
5. Calculation of average values for each indicator;
6. Calculation of probability of successful implementation;
7. Assessment of PSIi integrated indicator;
8. Building rating of countries;
9. Interpretation of level of PSIi.

Stage 1. Determination of the assessment indicators list

To work out the indicators’ list, we concentrate on Russian practice, the experience is of other countries’ innovative projects implementation (e.g., Great Britain, USA, Canada, Australia, and China), and the United Nations Economic Commission for Europe’s studies on efficient and innovative management. We figure out the set of factors that significantly influence the success of innovation projects implementation in most countries with respect to Federal Law No. 224, the processes of initiation, development, and decision making on the implementation of a PPP project are fully regulated. Among the key innovations is the introduction of the Institute for evaluating the effectiveness of the project at the preparation stage, which includes an assessment of financial efficiency and socio-economic effect.

If the project is effective, finding a qualified advantage is done through comparing the net discounted cash flow for the PPP project and the state (municipal) contract, thereby choosing the most optimal form of implementation of the infrastructure project. The methodology for assessing the effectiveness of the public-private partnership project was approved by Order No. 894 of the Ministry for Economic Development and Trade of the Russian Federation on November 30, 2015. This assessment is carried out both for projects initiated by a public partner and a private one (Kovaleva, Chernov, 2017).

In the event that the project is initiated by the private partner, the private partner must present a bank guarantee to the public partner issued to comply with requirements entering in a contract in accordance with Part 9 of Art. 10 of Federal Law No. 224, in the amount not less than 5% of the volume of projected project financing. 

The approach to building the indicators system is expressed as follows:

1. Justification of indicators’ relevancy as well as availability for measuring and monitoring.
2. Correspondence with the relevancy indicator that allows excluding indicators of lower priority.
3. Presence of the universality indicator that implies possibility of applying the indicator by its general meaning for assessment.
4. Determination of the most suitable number of indicators that means their sufficiency but not excessiveness. Since we took into account that innovation projects implementation respects public as well as private interests, it was necessary to develop a list of indicators that would as full as possible enable the assessment of all complexity and expansiveness of factors that influence success of innovation projects implementation (fig. 2).

Fig. 2: Areas of factors that influence success of innovation projects implementation

Subject to the above mentioned approach, the authors selected a set of 98 indicators.

1. d) Assessment indicators grouping. In order to simplify the assessment procedure, all set of indicators was divided in 5 main groups (table 1) according to the number of the above mentioned factors.

The systematization and generalization of the provisions of the new Russian legislation, the accumulated experience in the implementation of PPP projects, as well as the analysis of expert opinions of specialists and managers implementing PPP projects, allow us to separate and group the indicators of the successful implementation of projects. Furthermore, throughout the execution of innovation projects, decision makers must respect public as well as private interests. To guarantee the interests of the public and private side, we have developed an innovative set of 98 indicators (divided into five main groups) that would enable the assessment of full complexity and expansiveness of factors that influence the success of innovation projects (table 1).

Table 1: Groups of Indicators of Assessing the Probability of Successful Implementation of Innovation Projects in a Country

Group 1 Factors of governmental impact. They show the interest of government authorities for an active solution of infrastructural issues; ability of government authorities’ representatives for innovations and acceptance of new models; level of understanding rights and obligations of parties during project implementations.

Group 2 Factors of private sector impact. Indicators that characterize the level of trust of private sector to government authorities that arrange procurement; level of private sector readiness for innovation projects implementation; undertaking project-related risks

Group 3 Factors of society impact in general. Indicators that assess the level and possibility of expression of activity by population; level of consistency of the services offered with the needs

Group 4 Factors of environment impact.  Indicators that assess the balance of the system of innovation projects supported by government authorities; quality of mechanisms of interaction between the state and private sector; level of administrative barriers and quality of measures for preventing corruption in the country; availability of financial resources for successful implementation of innovation projects

Group 5 Innovation projects implementation (results of the state and private sector integration).

Indicators that assess the level of balance of responsibility areas and risks undertaken by government authorities and private sector; level of economic and social efficiency of projects; quality and prompt implementation of projects

Assessment and interpretation of the integrated indicators

Stage 2. Selection of experts who can assess indicators by individual countries

The number of interviewed respondents in respect of a single country (Boyer and Newcomer, 2015) must follow the following criteria:

– To assess indicators listed in table 1, at least 3 experts from each area of activities (government, science, community, and business) must be invited; that will ensure the balance of interests of all interested parties.

– Respondents must have the necessary level of competence in the matter under study with 7 years in their respective areas of expertise.

Stage 3. Assessment of the level of demonstration of each indicator by using the method of peer review

We offer to assess indicators according to the expert solutions matrix (table 2) where the highest possible value is 7 points

Table 2: Expert Solutions Matrix

Stage 4. Assessment coordination between experts using Kendall’s concordance coefficient (W): if W ≤ 0.5, experts’ opinion is not agreed; if 0.5 ≤ W ≤ 0.7, experts’ opinion is agreed; if 0.7 < W ≤ 1, there is a high level of agreement of experts’ opinion. If the level of agreement is insufficient according to a person who makes a decision, the interviewing procedure must be repeated.

Stage 5. Calculation of the average point set by experts in respect of a certain indicator:

Depending on the correspondence of PSI_i integrated indicator to some range of values, it is possible to figure out the probability of successful implementation of innovation project in the country (table 3).

Table 3: Range of Values of PSIi Integrated Indicator

Stage 8. Building a rating of countries. At this stage, countries are rated by level of PSI_i integrated indicator.

Stage 9. Interpretation of the level of PSIi integrated indicator. Based on this new

technique, which is an alternative to the recommendations developed by the Russian Federation Government, we rated Russian regions by the level of probability of successful projects implementation in the innovations area (Kovaleva, Chernov, 2017),  (table 4, fig. 3).

Table 4: Results of applying the methodology

Fig. 3: Experts’ assessment of indicators groups for innovation projects

Value of PSI_i integrated indicator across Russia shows that the probability of successful implementation of innovation projects is at the medium level; the internal environment is characterized by moderate uncertainty which its degree depends on the selected variant of the project financing. Experts assess each group of indicators. Standard values for each group are based on the medium level of assessing 6 points indicators.

The PSI_i integral indicator in Russia proves the existence of the following trends:

– reforming legislative basis in the sphere of innovation projects implementation;

– developing specialized structures (Department for Investments Policy and Public-private Partnership Development at the Russian Federation Ministry of Economic Development; Department for Investments and Public-private Partnership Coordination of the Ministry of Investments and Development for the Sverdlovsk Region; National Center for PPP, Autonomous Non-profit Organization, etc.);

– implementing many innovation projects; however, most of the projects are at the approval stage (Levitin, 2010). According to the National Center for Public-Private Partnerships, 2,446 infrastructure projects, 2,200 of which are concessional projects, are approved for implementation in Russia as at the beginning of 2017;

– testing some mechanisms of projects financing (Roumboutsos, 2016; Dghingholija, 2010) and developing conceptually new financial models (Gafurova, 2013).

The analysis of foreign experience in the implementation of projects for sustainable development economy has shown that in almost all countries, the greatest number of innovation projects are carried out in the construction and energy sectors, while very strict requirements for efficiency level are imposed on the facilities (Fig. 2) (Ke, Wang, Chan, 2010;  Koppenjan, de Jong, 2017).

Conclusion

Upon results of assessing the level of PSI_i integrated indicator we may conclude the following:

1. Value of PSI_i integrated indicator corresponds to the level of range 1. This means that there are no necessary conditions for implementing innovation projects in the country. The reason is that the government and private sector are not ready for cooperation and joint participation in the provision of public services; also, there are no necessary institutes as well as intentions to establish them.
2. Value of PSI_i integrated indicator at the level of range 2. That is, the country is marked by the beginning cooperation between government and private sector; necessary institutes are forming, including the beginning of legislative system reforming process, the first specialized structures are formed, several mechanisms of financing innovation projects are tested, and the most of projects are at the approval stage.
3. Value of PSI_i integrated indicator at the level of range 3. This means that there is a process of cooperation between authorities and the private sector at the country. Necessary institutes are established; methodological recommendations are developed; individual mechanisms of financing prioritized innovation projects and programs are tested; new models of cooperation are developing; there are some examples of efficient projects implementation; most of the projects are at the approval stage.
4. Value of PSI_i integrated indicator at the level of range 4. This means that the process of cooperation between authorities and private sector is a specific feature of the country economy; there are necessary institutes; an innovation projects portfolio is formed; most of the projects are at the implementation stage; there are examples of successful projects implementation (services offered are consistent with the expectations of population; the state and private sector achieve planned economic results); population demonstrates active behavior in the matters of projects implementation.
5. Value of PSI_i integrated indicator at the level of range 5. This means that the process of cooperation between authorities and private sector constitutes the basis of socially-oriented economy of the country; there are necessary institutes that are properly functioning; an innovation projects portfolio of proper quality is formed; there are many examples of successful projects implementation (services offered are consistent with the expectations of population; the state and private sector achieve planned economic results); population provides support and demonstrates active behavior in the matters of projects implementation.

To assess the probability of successful implementation of innovation projects, the authors selected representatives of developed countries (Australia, USA, Canada, Japan, and several EU countries: Great Britain, France, Spain, Germany, Bulgaria, and Croatia) as well as representatives of developing countries (Albania, China, Iraq, and Russia). The average number of experts involved in the assessment of each country is 14. Experts are selected by criteria listed in the authors’ technique (stage 2). Calculations of concordance coefficient show that experts’ opinion is agreed to the sufficient degree (0.5 ≤ W). Basing on the assessment held, we have rated countries by the level of probability of successful implementation of innovation projects. 

Acknowledgment

Our research was supported by the Ural Federal University and the Ural State University of Economics during preparation of the manuscript.

References

1. Boyer, J.E., Newcomer E.K. (2015), ‘Developing government expertise in strategic contracting for public-private partnerships’, Journal of Strategic Contracting and Negotiation, 1 (2), 129-148.
2. Dghingholija, А.F., Shahovskaja, L.S., Morozova, I.А. (2010) Public-private partnership: the nature, forms, prospects and directions of development in the modern economy, Economic Education, Moscow.
3. Chernov, S.S. (2016) Problems of economics and management of enterprises, industries, complexes, ООО «CRNS», Novosibirsk.
4. Gafurova, G.Т. (2013) Public-private partnership: theory and practice, Kramin, T.V. (ed), Poznanie, Kazan.
5. Garmston, H., Pann, W. (2012), ‘Building regulations in energy efficiency: Compliance in England and Wales’, Energy Policy, 45, 594-605.
6. Hodge, A.G., Greve, C. (2007), ‘Public – Private Partnerships: An International Performance Review’, Public Administration Review, 67 (3), 545-558.
7. Ke, Y., Wang, Sh., Chan, А. (2010), ‘Risk Allocation in Public-Private Partnership Infrastructure Projects: Comparative Study’, Journal of Infrastructure Systems, 16 (4), 343-351.
8. Kingdon, J. (1984) Agendas, Alternatives and Public Policies, Little, Brown and Company, Boston.
9. Koppenjan, J, de Jong, M. (2017), ‘The introduction of public–private partnerships in the Netherlands as a case of institutional bricolage: The evolution of an Anglo-Saxon transplant in a Rhineland context’, Public Administration, 1-14.
10. Kovaleva, I.V., Chernov, M.V. (2017), ‘Concept of state regulation of entrepreneurial activity’, Bulletin of the Altai State Agrarian University, 12 (134), 163-167.
11. Lewis, E. (1980) Public Entrepreneurship: Toward a Theory of Bureaucratic Political Power, Indiana University Press, Bloomington.
12. Llanto, G., Navarro, A., Ortiz, K. (2016), ‘Infrastructure financing, public-private partnerships and development in the Asia-Pacific region’, Asia-Pacific Development Journal, 22 (2), 27-69.
13. Marques, R.C. (2011), ‘Risks, contracts, and private-sector participation in infrastructure’, Journal of construction engineering and management, 137 (11), 925-932.
14. Millward, R. (2011), ‘Public enterprise in the modern western world: an historical analysis’, Annual of Public and Cooperative Economics, 82 (4), 375-398.
15. Mostafavi, A., Abraham, D., Sinfield, J. (2014), ‘Innovation in Infrastructure Project Finance: A Typology for Conceptualization’, International Journal of Innovation Science, 6 (3), 127-144.
16. Moszoro, M. (2010), Efficient Public–Private Partnerships, 884, IESE Working Paper.
17. Peters, A., Fisher, P. (2002), ‘The Effectiveness of State Enterprise Zones’, Employment Research, 9 (4), 1-3.
18. Plohotnicova, G.V. (2014), ‘Innovative economics: foreign experience of development’, Proceedings of the 4th sci. and pract. conf. ‘Institutes and mechanisms of innovative development’, 23-24 Oct. 2014. Kursk, Russia, 315-319.
19. Popov, Е.V. (2015) Institutions. UrO RAN, Ekaterinburg.
20. Roumboutsos, A. (2016), ‘Public Private Partnerships in Transport Infrastructure’, Transport Review, 36 (2), 167-169.
21. Schaeffer, P., Loveridge, S. (2002), ‘Toward an Understanding of Types of Public–Private Cooperation’, Public Performance and Management Review, 26 (2), 169-189.
22. Steijn, B., Klijn, E.-H., Edelenbos, J. (2011), ‘Public Private Partnerships: Added Value By Organizational Form Or Management?’, Public Administration, 89 (4), 1235-1252.
23. Voronov, D.S., Erypalov, S.E., Pridvizhkin, S.V., Pelymskaya, I.S., Rusetskaya, E.A. (2017), ‘Assessment of Competitiveness of the Leading Russian Metal-lurgical Enterprises’, Journal of Applied Economic Sciences, volume XII, spring, 1(47), 36-49.
24. Wang, Y. (2014), ‘Evolution of public-private partnership models in American toll road development: Learning based on public’, International Journal of Project Management, 33, 684-696.
25. Wang, Y., Zhao, J.Z. (2014), ‘Motivations, Obstacles and Resources’, Public Performance & Management Review, 37 (4), 679-704.