Audit of the Evergreen Line Rapid Transit Project
- Ministry of Transportation and Infrastructure
- Ministry of Finance
- Partnerships BC
Audit Scope and Objectives
- To determine whether agencies provided sufficient and rigorous information to recommend the project option that is most likely to cost-effectively meet government’s objectives.
- To determine whether agencies adequately demonstrated that the recommended P3 arrangement represents the best procurement solution taking full account of the expected costs, benefits and risks across the project’s life-cycle.
- An appropriate strategic analysis, governance and management framework exists to guide and oversee agencies’ work.
- Investment objectives and intended outcomes have been clearly defined and aligned with government’s policy objectives.
- The costs, benefits and risks of the project options that could feasibly deliver on investment objectives have been rigorously assessed, verified and clearly communicated to decision makers.
- An appropriate procurement assessment, governance and management framework exists to guide and oversee agencies’ work.
- Project recommendations have been based on the analysis of costs, benefits and risks while taking account of stakeholder consultations.
- Agencies had prepared the management framework needed to successfully procure the project.
Main Audit Findings
- The agencies had not fully informed government’s scope decision in the material provided to Treasury Board. The 2008 and 2010 business cases summarised an extensive body of work, but fell short of meeting the Capital Asset Management Framework (CAMF) guidelines because they did not:
- assess the risks of the alternative scope options or clearly explain the difference in their costs before recommending the SkyTrain;
- explain that ridership forecasts were at the upper end of the estimated range;
- describe the risks from changes in complementary and competing transit services and how these would be monitored and managed; and
- include a framework for measuring performance.
- In addition, the audit found that agencies had not adequately documented their reviews of the material presented to Treasury Board and verified its accuracy.
- Despite these gaps and weaknesses, the audit also concluded that the preferred SkyTrain option is likely the best one to meet government’s objectives. However, this conclusion relied on information that was not presented or adequately explained in the submission to Treasury Board.
- Getting this right despite the information shortfalls is not a cause for complacency. Relying on the same approach in future capital asset projects puts government at risk of making decisions that would have been modified had government understood the full costs, benefits and risks. In the case of the Evergreen Line, the audit found that neither business case informed government decision-makers about the ridership risks or how they would be managed.
- The audit team concluded that MOTI and Partnerships British Columbia demonstrated that a short-term P3 arrangement, covering the designing, building and financing of the Evergreen Line, best meets government’s policy objectives.
- The Ministry of Finance should implement a project plan, describing the scope, required resources, timelines and deliverables, for updating the CAMF to provide comprehensive guidance for public sector agencies on:
- the information required to underpin capital project planning and how this should be documented; and
- the type of oversight that should be applied to verify the information presented to government.
- The Ministry of Finance, the Ministry of Transportation and Infrastructure, and Partnerships British Columbia should document project reviews so that the scope of these reviews, and the analysis underpinning decisions, are clearly described in written records.
- The Ministry of Transportation and Infrastructure should provide more detailed guidance on its requirements for estimating ridership and the economic benefits of transit projects.
- The Ministry of Transportation and Infrastructure should provide more detailed guidance on performance measurement so that business cases include appropriate detail on performance indicators, targets and how these indicators will be measured.
- The Ministry of Transportation and Infrastructure should develop and apply a detailed framework for measuring, managing and reporting on the performance of the Evergreen Line. The framework should describe:
- evaluation objectives and specific performance measures;
- methods for collecting reliable, meaningful information;
- how agencies will measure and manage performance and provide the necessary resources to do this work; and
- how the outcomes will be shared across government and the wider community.
- The Ministry of Finance, Ministry of Transportation and Infrastructure, and Partnerships British Columbia should improve how they assess and report on whether strategic options assessments and business cases have followed the CAMF guidelines.
- The Ministry of Transportation and Infrastructure should update its guidelines to make relevant comparisons with observed data central to justifying and explaining traffic and ridership forecasts.