Management and Economics (April 2008) 26, 387–393 The impacts of change management practices on project change cost performance YI ZOU1 and SANG-HOON LEE2* 1 2 FMC Technologies Inc. , Houston, USA Department of Engineering Technology, University of Houston, 304A Technology Bldg, University of Houston, Houston 77204, USA Received 18 June 2007; accepted 15 January 2008 Change cost is one of the most sensitive aspects of construction project management, but it is also one of the most difficult to control.
It has been widely recognized that construction projects that adopt change management practices generally incur lower change costs in comparison with project budgets. The relationship between change management practices and cost performance is investigated. Construction project data for this research are derived from the Construction Industry Institute Benchmarking and Metrics database. Multiple one-way ANOVA and linear regression are performed to investigate the effectiveness of individual change management practices elements and overall change management practices implementation in controlling project change cost, respectively.
The data analysis results show that individual change management practices elements have different levels of leverage in helping to control project change cost and that using change management practices is truly helpful in lowering the proportion of change cost in project actual cost. Keywords: Project management, change management, performance improvement. Introduction Many researchers have conducted statistical analyses to reveal the correlations between project management best practices and project performance, and they have provided valuable recommendations to the industry about how to improve project performance.
Among many project management best practices, change management practice is one of the most important practices (Lee et al. , 2004; Zou and Lee, 2006). Further, project change cost performance is one of the most essential metrics used as a measure of project success (Williams, 2000; Eden et al. , 2005). However, the previous studies concentrated on the overall change management practices implementation level, and none of them looked into individual change management practice elements.
In addition, project budget was generally adopted as the basis for comparison when measuring project change cost magnificence, which entails a problem of accuracy, as will be elucidated later in this paper. This paper resolves these *Author for correspondence. E-mail: [email protected] uh. edu problems, and its intention is to show construction managers how each individual element of change management practices can improve project change cost performance. A secondary aim is to explore the correlation between overall change management practice implementation and project change cost performance. Background
The Benchmarking and Metrics (BM&M) programme of the Construction Industry Institute (CIIH) has been committed to providing construction industry ‘quantitative data essential for the support of cost/benefit analyses’ (Construction Industry Institute, 2007). CII commenced the BM&M data collection in 1996, and the database currently represents over 1200 projects for which CII best practices and project performance indices have been or are being recorded. In 1990, the CII Cost/Schedule Controls Task Force published a research report about the impact of project changes on construction cost and schedule at the operational level.
Construction Management and Economics ISSN 0144-6193 print/ISSN 1466-433X online # 2008 Taylor & Francis http://www. tandf. co. uk/journals DOI: 10. 1080/01446190801918714 388 Sanders (2000) also explored the impacts of change management practices on certain projects. CII started to include change management practice in its database in 1997, which included 14 elements. This study is aimed at investigating the relationship between usage of these 14 change management practice elements and project change cost performance.
The BM&M database is the source of data analysis, and the data analysed in this research can be basically classified into three categories of metrics: project characteristics, project performance and change management practices use. Table 1 illustrates the demographic composition of the BM&M database based on respondent type, project nature and industrial group. As can be seen in Table 1, most of the projects are in the heavy industry sector. Such an extremely uneven sample population distribution poses difficulty in the subsequent data analyses, and it is therefore one of the primary considerations when the analysis techniques are selected.
Appendix II presents the 14 questions used in this research. Zou and Lee of projects (Construction Industry Institute, 2003). With all of these considerations, this research only focuses on the impacts of change management practice on project change cost performance. Research objectives It is worth noting that this research is to be explanatory instead of confirmative or predictive. In other words, the purpose of this research is to reveal potential correlations1 among project characteristics, change management practice and project change cost performance.
The two main objectives of this research are (1) to investigate the effectiveness of individual change management practice elements in terms of improving project change cost performance—e. g. for a particular change management practice element, could the construction project using it have a high probability of achieving better and more predictable change cost performance than with other elements; and (2) to explore the correlation between overall change management practice implementation and project change cost performance while controlling for project characteristics variables.
Answers to the first question could highlight which change management practice elements can singly influence project change cost performance significantly and thus deserve particular attention. By exploring the second relationship, the effectiveness of change management practice in controlling project characteristics can be validated. Research scope Cost and schedule are frequently the biggest concerns in a project, and they are also the project performance facets most sensitive to project changes.
However, the impact of project changes on project schedule is far less significant than the effect on cost, and the reason is that cost is additive, while schedule is not. A project’s duration is determined by its critical paths, and all noncritical paths contain floats (slacks) of one size or another. If a change consumes some of the floats on non-critical paths but not so much that it changes the critical path, the total duration of the project will not be changed, and thus project schedule performance is not negatively affected by the change.
In addition, change management practices have been found to be the most influential element affecting cost savings in the majority Table 1 The compositions of CII BM&M projects Project nature A few more words about change First, it is necessary to clarify the definition of ‘change’. In this research, ‘change’ refers to project changes that have been mutually agreed upon by both the owner and contractor. There are two types of project Respondent type Industrial group Heavy industrial Light industrial Buildings 4 24 6 34 22 90 45 157 191 Infrastructure 17 13 14 44 13 17 23 53 97
Total Contractor Owner Add-on Grassroots Modernization Subtotal Add-on Grassroots Modernization Subtotal Total 153 157 106 416 106 84 144 334 750 20 22 11 53 42 30 58 130 183 194 216 137 547 183 221 270 674 1221 Change management changes—project-development changes (PDCs) and scope changes (SCs). PDCs ‘include those changes required to execute the original scope of work or to obtain the original process basis’; in contrast, SCs ‘include changes in the base scope of work or the process basis’ (Construction Industry Institute, 2007).
The absolute monetary value of project change is less meaningful to this research than the ratio of it to the baseline because of different project scales. There should be some baseline against which change cost can be compared, and thus project change cost performance can be assessed. Such a denominator can be either project budget (initial predicted project cost) or project actual cost (the amount a project has spent at completion). Some researchers have used the former (Hsieh et al. 2004), but here the second metric is employed because the accuracy of the initial predicted project budget cannot be guaranteed and there is no information from the database to evaluate the accuracy of the project initial estimate. 389 the relationship between two interval variables (change management practice index and project change cost factor) within each category of categorical variables (project characteristics). ANOVA is intrinsically ideal for investigating the first type of relation, so multiple one-way ANOVA tests are conducted.
For the second correlation, linear regression is conducted because there is no theoretical or empirical support for a non-linear (curvilinear) correlation, and because the primary purpose of this research is to evaluate the effect rather than predicting. Data analyses Measuring project change cost performance: change cost factor As discussed earlier, project change cost performance cannot be measured by the absolute value of changes but rather is measured by the ratio relative to project actual total cost.
Therefore, change cost factor is used in this research to measure the performance of project change cost. This metric measures the total cost of changes as a fraction of actual total project cost (Construction Industry Institute, 1998). For industrial sector owner projects, actual total project cost includes total installation cost at turnover but excludes land costs; for building sector owner projects, it is the total cost of design and construction to prepare the facility for occupancy; and for contractor projects, it is the total cost of the final scope of work.
The impact of individual change management practice element implementation on project change cost performance There is no theoretical support for hypothesizing the existence of interaction effects of change management practice elements on change cost factor, nor are there any sound practical or empirical indications. From the perspective of the practitioner, whether owner or contractor, any of these change management practice elements can be implemented independently, although it may be convenient to use some of them in combination with others.
For this reason, multiple one-way ANOVA rather than k-way ANOVA is conducted to examine the effect of each single change management practice element on change cost factor. According to Stevens (1996), influential points rather than outliers should be of greatest concern because their involvement impacts significantly on the statistical result. A Cook’s distance greater than 1 usually flags an extremely influential point,4 and so cases with Cook’s distance over 1 are precluded from
Research limitations As with any study employing statistical data analysis techniques and tools, the reliability of the raw data is crucial. Considerable opinion-type data in this research are collected based on the Likert scale. Therefore, the data are influenced by respondents’ biases. Some preparation of data has been done prior to the data analysis process (e. g. transforming the original 0–10 project complexity measure in the BM&M database into only three levels—low, medium and high). In this way, some of the original data are truncated and become fuzzier, which means that some bias can be eliminated to alleviate the subjectivity of the data. It is necessary to point out that because this research is basically carried out by statistical means, the research processes and results are inherently vulnerable to the statistical limitations of the selected data analysis techniques and the available data. This paper relies on statistical analysis, and thus further research is suggested, including qualitative analysis of the implied relations.
Research method The two objectives of this research can be interpreted as finding two types of statistical correlations: (1) the relationship between each of a group of bivariant indicator variables3 (change management practice element question with answers Yes or No) and a single interval variable (project change cost factor); and (2) 390 further analysis here. The remaining cases are subjected to a Kolmogorov–Smirnov test to check data normality, and the hypothesis of data normality is rejected by the result.
Meanwhile, a basic descriptive statistic demonstrates that cell sizes of projects using or not using a certain change management practice element are extremely uneven. For some change management practice elements, such as 1 and 2, less than 10% of all cases answered ‘No’. In view of these violations to assumptions of ordinary ANOVA, Brown and Forsythe’s F-test of equality of means is performed as a substitute for an ordinary F-test because it is robust against non-normality, unequal group sizes and heterogeneity of variances (Garson, 2006).
In addition to the ANOVA test for the equality of means, Levene’s test is used to test the equality of the variances (in Yes and No groups). The incentive for performing this test is that change management practice elements are able not only to reduce the average level of change cost but also to control the variation range of change cost, thus making the project’s change cost performance more predictable. Table 2 shows the significant results of the two tests on change management practice elements.
For change management practice elements 4, 5, 6 and 10, the Brown–Forsythe F-test statistic is significant. In addition, the observed mean change cost factor values of the No group are higher than those for the Yes group. This indicates that the projects using each of these change management practice elements achieved significantly better change cost performance than the projects that did not use the elements.
Although it cannot be certain that these elements are the sole reason for better change cost performance, the result statistically ascertains that there is an undeniable correlation between using these change management practice elements and better change cost performance. Further, because the Levene statistic is significant and the observed standard deviation of the No group is higher than that of the Yes group, the probability of suffering an outrageously high change cost is Table 2 Zou and Lee ignificantly lower for projects following change management practice elements 4, 6 and 10 than for others. The impact of overall change management practice implementation on project change cost performance Project characteristic variable selection The efficacy of overall change management practice in different types of projects can vary widely depending on project nature, industrial type, project complexity, project size, contract methods and the level of experience of project participants.
In order to separate out the effects of project characteristics and focus on the partial effect of change management practice on project change cost factor, it is necessary to categorize and group projects based on different characteristic factors and then investigate the correlation between overall change management practice and change cost factor. Theoretical and empirical information is employed to select which project characteristic variables should be considered.
Although the database includes a number of project characteristic metrics, only the following five variables are selected: respondent type, project nature, industrial type, transformed complexity and cost category. Table 3 shows detailed categories of the five variables. Measuring overall change management practice implementation: change management practice index The overall implementation of the change management practice elements is measured by the change management practice index, a continuous variable scored on a 0 to 10 scale, with 0 meaning no use and 10 meaning extensive use of all of the elements.
The data analysis is conducted with two continuous variables—change management practice index and change cost factor—while controlling for the five project characteristic variables. As an independent Change management practice elements influencing the project change cost factor Robust test of equality of means Brown– Forsythe statistic 13. 251 8. 026 4. 157 8. 763 df1 df2 Sig. Observed means No Yes Test of homogeneity of variances Levene df1 df2 statistic 14. 919 8. 033 9. 342 1 1 1 Sig. Observed S. D. No Yes Element No. of cases no. No Yes e4 e5 e6 e10 239 181 165 115 408 447 526 567 1 1 1 1 403. 13 287. 541 234. 129 143. 073 .000 . 005 . 043 . 004 0. 095 0. 092 0. 095 0. 107 0. 065 0. 067 0. 075 0. 073 645 0. 000 0. 109 0. 084 689 0. 005 0. 114 0. 091 680 0. 002 0. 116 0. 090 Notes: 1. A 95% significance level is used. 2. Only elements with statistical significance and variables are reported. Change management Table 3 Project characteristics variables Category Contractor Owner Add-on Grassroots Modernization Heavy industrial Light industrial Infrastructure Buildings Description and note 391 Project characteristic metric Respondent type Project nature Industrial type Transformed complexity
Low Medium High ,55 5–15 15–50 50–100 . 100 The respondent in this case is a contractor in this project. The respondent in this case is an owner in this project. Addition to existing facilities Complete new construction project Revamp or retrofit of existing facilities Chemical, electrical generating, environmental, refining/processing, mining, petrochemical, oil & gas, pulp & paper, etc. Auto mfg. , consumer prod. mfg. , electronics mfg. , foods, pharmaceutical mfg. , etc. Airport, electrical distr. , flood contr. , hwy, marine facilities, navigation, rail, tunnelling, pipeline, gas distr. telecom/network, water and waste, etc. Comm. centre, dorm/hotel, office, hosp. , lab, maintenance facilities, parking/garage, phys. fitness ctr. , restaurant/club, mall, school, warehouse, residential, prison, theatre, etc. Complexity1 of 0 to 3 as the respondents’ original ratings Complexity of 4 to 7 as the respondents’ original ratings Complexity of 8 to 10 as the respondents’ original ratings Capital project or total cost of contract Cost category2 Notes: 1. Project complexity: 0 stands for the lowest complexity; 10 represents extremely complex project. 2. In millions of US dollars. ariable, the change management practice index value is not spread from 0 to 10 because this index is calculated by the implementation of individual change management practice elements, which are measured as categorical variables. This makes many data points cluster on several change management practice index levels. However, since there are more than 15 change management practice index levels, it can be deemed to be a quasi-interval variable. Therefore, linear regression is used instead of ANOVA, and in order to retain the pristine quantitative information in project change costs, non-parametric or ranking regression methods are not used.
The correlation between change management practice index and change cost factor in different project categories Before fitting the model, influential points are detected (Cook’s distance D. 1) and deleted. Normality of error terms is not a serious assumption for a bivariate linear regression (Wesolowsky, 1976). According to Wesolowsky, formal tests of normality are not necessary with large sample sizes (in this case, n. 50 for each project characteristic category) because ‘in large samples lack of normality has no important consequences’ (Wesolowsky, 1976). Meanwhile, linear regression is robust in the face of small or medium iolations against the homoscedasticity of variances assumption. Table 4 shows the significant findings of the bivariate linear regressions in each project characteristic category. In the table, for both contractor and owner projects all of the significant beta coefficients are negative for add-on projects, heavy industrial projects, medium- and high-complexity projects, and projects with budgets between US$15m and $50m. This indicates that higher change management practice index values are associated with lower change cost factor values in the corresponding project characteristic categories.
These results are acceptable for such an exploratory non-physical-science study because even the smallest sample size of these regressions is over 50 and the power of the test5 is almost over 0. 50, which is acceptable (Stevens, 1996). Conclusions Although there has been a consensus in both academia and industry that project change management practices can improve project change cost performance, it is shown that individual change management practice elements are not equally effective. Generally, for those projects in which a ‘contingency plan for changesusceptible areas in the early phases has been prepared, 92 Table 4 Zou and Lee Impact of overall change management practice implementation in different project characteristic categories Level N Std. coefficients (beta) 20. 114 20. 117 20. 237 20. 157 20. 126 20. 162 20. 149 t Sig. 95% C. I. for B Lower bound 22. 119 22. 273 23. 626 23. 447 22. 463 22. 565 22. 044 0. 035 0. 024 0. 000 0. 001 0. 014 0. 011 0. 042 20. 020 20. 014 20. 027 20. 020 20. 018 20. 021 20. 15 Upper bound 20. 001 20. 001 20. 008 20. 005 20. 002 20. 003 . 000 0. 010 0. 011 0. 052 0. 023 0. 013 0. 022 0. 017 Adj. R2 Power of test
Project category Respondent type Respondent type Project nature Industrial type Transformed complexity Transformed complexity Cost category Contractor Owner Add-on Heavy industrial Medium High 15–50 343 377 223 471 381 245 186 0. 46 0. 50 0. 90 0. 60 0. 50 0. 70 0. 50 all changes are required to go through a formal change justification procedure, and the contract specifies how to manage changes’, the possibility of incurring an extremely high project change cost compared with actual project cost is significantly lower than for other projects.
In addition, those projects in which ‘changes are evaluated against project business drivers and success criteria’ perform better on average than other projects in terms of project change cost performance. Therefore, these four change management practice elements are highly recommended for construction projects. The impact of project ownership on change management practice implementation is not noticeable because for both contractor and owner projects a high overall change management practice implementation score is associated with better project change cost performance.
However, this relationship can be further fortified in some specific project types. The analysis shows that add-on projects have better correlations between overall change management practice implementation and change cost performance than do grassroots and modernization projects. The results also indicate that heavy industrial, highly complex, and US$15–50m projects have better correlations than the other categories. necessarily independent of one another in many cases; the same argument exists with regard to the use of response variable instead of dependent variable.
In this paper, these are not strictly distinguished. Also, there are some other terms that should be deemed as synonyms in this article, such as continuous variable and interval variable. 4. Some scholars recommend a threshold such as 4/n or 4/ (n-k-1), where n is sample size and k is the number of independent variables. 5. It is the power of significance test of r at a50. 05 (twotailed). References Construction Industry Institute (1998) Benchmarking and Metrics Summary for 1997, Construction Industry Institute, University of Texas at Austin, Austin, TX.
Construction Industry Institute (2003) Benchmarking and Metrics Value of Best Practices Report, BMM2003-4, Construction Industry Institute, The University of Texas at Austin, Austin, TX. Construction Industry Institute (2007) CII Benchmarking and Metrics Home Page, available at http://cii-benchmarking. org/ main_index. cfm (accessed 22 January 2007). Eden, C. , Williams, T. and Ackermann, F. (2005) Analyzing project cost overruns: comparing the measured mile analysis and system dynamics modeling. International Journal of Project Management, 23(2), 135–9.
Garson, G. D. (2006) Weekly Topics for PA765: Quantitative Research in Public Administration, available at www2. chass. ncsu. edu/garson/PA765/weekly. htm (accessed 13 December 2006). Hsieh, T. -Y. , Lu, S. -T. and Wu, C. -H. (2004) Statistical analysis of causes for change orders in metropolitan public works. International Journal of Project Management, 22, 679–86. Lee, S. , Thomas, S. R. and Tucker, R. L. (2004) Effective practice utilization using performance prediction software. Journal of Construction Engineering and Management, 130(4), 576–85. Sanders, S. T. 2000) Change management best practice use in NAVFAC and other public projects, Masters thesis, University of Texas at Austin, Austin, TX. Notes 1. Statistically, the word correlation refers to the association between two variables. Here, it only indicates the statistical association or relation without emphasizing any logic and/or causal relationship. More explanation can be found in the Appendices. 2. This pre-treatment is performed only if the statistical method chosen requires a parsimonious data category number. In some cases, retaining as much of the information from the data entering a model is of the utmost priority. . Some statisticians advocate using indicator variable instead of independent variable because these variables are not Change management Stevens, J. (1996) Applied Multivariate Statistics for the Social Sciences, 3rd edn, Lawrence Erlbaum Associates, Mahwah, NJ. Wesolowsky, G. O. (1976) Multiple Regression and Analysis of Variance, John Wiley & Sons, New York. Williams, T. (2000) Safety regulation changes during projects: the use of system dynamics to quantify the effects of change. International Journal of Project Management, 18(1), 23–31. Zou, Y. and Lee, S. 2006) Proceedings from AACE ’06: AACE 50th Anniversary Meeting, Las Vegas, NV. 393 Power of significance test The capability of rejecting the null hypothesis when it is false, which equals 1-b, where b is the corresponding Type-II error. A common way to promote test power is to use a more stringent a-level. Appendix II Appendix I CII benchmarking and metrics questionnaire: change management practice No. Change management practice elements Was a formal, documented change management process that was familiar to the principal project participants used to actively manage changes on this project?
Was a baseline project scope established early in the project and frozen, with changes managed against this base? Were design ‘freezes’ established and communicated once designs were complete? Were areas susceptible to change identified and evaluated for risk during review of the project design basis? Were changes on this project evaluated against the business drivers and success criteria for the project? Were all changes required to go through a formal change justification procedure? Was authorization for change mandatory before implementation?
Was a system in place to ensure timely communication of change information to the proper disciplines and project participants? Did project personnel take proactive measures to promptly settle, authorize and execute change orders on this project? Did the project contract address criteria for classifying changes, personnel authorized to request and approve changes, and the basis for adjusting the contract? Was a tolerance level for changes established and communicated to all project participants? Were all changes processed through one owner representative?
At project close-out, was an evaluation made of changes and their impact on the project cost and schedule performance for future use as ‘lessons learned’? Was the project organized in a Work Breakdown Structure (WBS) format and with quantities assigned to each WBS for control purposes prior to total project budget authorization? Glossary of terms Brown-Forsythe’s test A modification of the ordinary F-statistic used in ANOVA. It is preferred to ordinary one-way ANOVA when cell sizes are unequal because it uses the deviations from medians instead of means for analysis. Cook’s distance, D A measure of the influence of a case.
Specifically it measures the effect of deleting a given observation. Observations with larger D values than the rest of the data are those that have unusual leverage. Homoscedasticity The dependent variables have similar variances in all groups formed by the independents (all cells in the factor design matrix). Kolmogorov–Smirnov test A non-parametric test for any distribution of data. It is usually used to test the normality of data. It is also relatively conservative, and therefore it can occasionally be replaced by the Anderson–Darling test because of the improvement in the tails of a distribution.
Levene’s test A test to check the assumption of homogeneity of variance. It is robust against nonnormality. Likert scale A type of psychometric response scale often used in questionnaires, and the most widely used scale in survey research. When responding to a Likert questionnaire item, respondents specify their level of agreement with a statement. The scale is named after Rensis Likert, who published a report describing its use (Likert, 1932; Wikipedia). The most commonly used forms are five-point and ten-point scales, although other types such as three-point are also not unusual. 1 2 3 4 5 6 7 8 9 10 11 12 13 14