Measuring What Matters: Effectiveness in Transport Assessment
New Zealand’s transport investment framework asks three questions of every proposal: is it strategically aligned, will it be effective, and is it efficient? The middle question is the hardest to answer rigorously, and the gap between what “effective” means in policy language and what it means in practice shapes which projects get funded and which do not. Understanding how effectiveness is assessed — and where that assessment falls short — is essential reading for anyone working in the transport sector or trying to make sense of why the roads they rely on keep missing the investment queue.
How the Framework Defines Effectiveness

The Investment Logic Model
Every funded transport proposal in New Zealand begins with the same question: what problem are we solving, and how confident are we that this intervention will solve it? The Investment Logic Model (ILM) is the tool that forces that question to be answered before money enters the picture.
The ILM is a structured workshop process facilitated by the Treasury-developed Better Business Cases framework and adapted by NZTA for transport contexts. Stakeholders map the causal chain from the identified problem — say, high crash rates on a rural state highway — through proposed interventions to expected benefits. The chain is explicit and testable: if the problem is correctly identified and the intervention is appropriately matched, the benefit should follow. Effectiveness, in this logic, is not a scoring dimension applied later. It is embedded in the causal structure of the proposal from the outset.
This matters because it shifts the burden of proof. A proposal that cannot demonstrate a clear causal link between intervention and benefit will struggle regardless of how shovel-ready the project is or how politically desirable the outcome. The ILM has been criticised for adding process weight to relatively straightforward proposals — a bridge replacement does not need a theory of change — but its defenders argue that the discipline it introduces has reduced the number of projects that were funded on instinct rather than evidence.
Effectiveness Criteria in the GPS and NLTP
The Government Policy Statement on Land Transport (GPS) sets the strategic direction for transport investment across a three-year funding cycle. It names priorities — economic growth, safety, climate, resilience — and signals how funding should be distributed across activity classes. What it does not do is define effectiveness in operational terms. That translation happens one level down.
The National Land Transport Programme (NLTP) is where the GPS priorities become funding allocations and where proposals are tested against effectiveness criteria. NZTA’s investment assessment framework asks three questions: is the proposal strategically aligned, is it likely to be effective at delivering the intended outcomes, and is it efficient in resource use? Effectiveness sits in the middle of that trinity, but it is not independent of the other two. A proposal that is perfectly aligned with GPS priorities but cannot demonstrate it will deliver them scores poorly on effectiveness.
The 2024-27 GPS, published by the Ministry of Transport, shifted emphasis toward economic growth and freight efficiency — a change that recalibrated which effectiveness arguments carried weight. Proposals framed around productivity and supply chain resilience moved up the queue; those framed around emissions reduction or mode shift faced a higher bar for effectiveness evidence.
How NZTA Scores Proposals
NZTA’s investment assessment process uses a structured scoring methodology with three primary dimensions: strategic fit, effectiveness, and efficiency. Each dimension is assessed separately before being combined into an overall investment profile. The relative weight of each dimension varies by activity class and by the scale of the proposal, but effectiveness consistently carries significant weight — particularly for new capital works.
Under the effectiveness dimension, assessors are looking for evidence that the proposed intervention will actually deliver the identified benefits. This is not a binary pass/fail. Proposals are scored on a scale from low to high effectiveness, with the score reflecting both the strength of the causal logic and the quality of the evidence base supporting it. A proposal supported by strong before-and-after data from comparable projects will outscore a proposal based on engineering assumptions alone, even if both have the same theoretical benefit-to-cost ratio.
The practical consequence is that effectiveness scoring rewards proposals with good precedent data. Projects on well-studied corridors — urban motorways, major interchanges — can draw on decades of monitoring evidence to support their effectiveness case. Projects in novel territory, such as rural demand-responsive transport or active travel infrastructure in low-density areas, face a thinner evidence base and often score conservatively as a result. This is not a deliberate bias in the scoring criteria, but it is a systematic effect of how effectiveness evidence accumulates over time.
The Role of the Business Case Approach
The Better Business Cases (BBC) framework, adapted from the UK Treasury model and administered by the New Zealand Treasury across central government, provides the scaffolding within which transport effectiveness arguments are made. The framework requires five cases: strategic, economic, commercial, financial, and management. For effectiveness purposes, the first two are the ones that count.
The strategic case establishes why investment is needed — the evidence base for the problem, the investment objectives, and the case for intervention over the do-nothing option. The economic case then tests whether the proposed solution is the most effective way to achieve those objectives, comparing options on benefit-to-cost ratio and on their ability to deliver the stated investment objectives. The economic case is where effectiveness and efficiency become entangled: a cheaper option that delivers 80 percent of the benefits of the preferred option may outscore it on efficiency but underscore it on effectiveness, and the framework requires assessors to make a reasoned judgement about that trade-off.
The BBC framework is scaled by investment size. Projects below a certain threshold can use a single-stage business case; larger projects move through programme business case, indicative business case, and detailed business case stages. At each stage, the effectiveness argument is expected to become more specific and more evidence-based. By the time a major project reaches detailed business case, its effectiveness case should rest on traffic modelling, confirmed design, and quantified benefit estimates — not on qualitative assertions about regional significance.
The Measurement Problem

What Gets Counted and What Does Not
New Zealand’s transport cost-benefit analysis methodology is codified in the Economic Evaluation Manual (EEM), maintained by NZTA and published on its planning and investment pages. The EEM specifies which benefit categories can be monetised and at what unit rates. The list is comprehensive by the standards of infrastructure economics, covering travel time savings, vehicle operating costs (fuel, tyres, maintenance), crash costs (by severity), emissions (CO2 and local air pollutants), noise, and road roughness effects.
Travel time savings typically dominate the benefit calculation for most road projects. NZTA applies different time values for different journey purposes — commuting, business, freight — derived from periodic willingness-to-pay surveys. The aggregation is straightforward: multiply trips by average time saving by value of time, and you have a monetised benefit. This simplicity is also the methodology’s central weakness. It assumes that all time saved is equally useful, that people value time saved in two-minute increments the same as they value an hour, and that the distribution of time savings across users does not matter — only the total.
Outside the EEM’s monetised categories, benefits exist but go uncounted in the formal CBA. Wider economic benefits — agglomeration, labour market access, business investment decisions — are acknowledged as potentially significant but are treated as sensitivity analysis rather than core benefit. For many rural and provincial projects, these wider economic benefits are arguably larger than the direct transport benefits, but they do not appear in the headline BCR.
Quantifying Benefits That Resist Numbers
Resilience is the clearest example of a transport benefit that the standard CBA framework handles poorly. A road that is upgraded to withstand a one-in-100-year flood event provides a benefit that only materialises when the flood happens. In expected-value terms, the annual benefit is small — the probability of the event multiplied by the avoided cost. But the actual benefit, when the event occurs and the road remains open while communities on alternative routes are cut off, is enormous and difficult to fully monetise. The standard EEM approach tends to understate resilience benefits, which matters most in exactly the regions where resilience is the primary investment argument.
Social connectivity presents a different challenge. Transport access to employment, healthcare, and education generates benefits that are partially captured in time savings but not fully. A rural community gaining reliable access to a regional hospital does not just save travel time — it changes health-seeking behaviour, early intervention rates, and health outcomes. These downstream effects are real but sit outside the transport model entirely. NZTA’s guidance acknowledges them under the heading of “transport disadvantage” but does not provide a systematic monetisation approach.
Environmental outcomes beyond emissions — habitat connectivity, sediment runoff reduction from road seal improvements, noise amenity — are similarly acknowledged but rarely quantified. In practice, assessors use a qualitative scoring system for non-monetised benefits, rating them on a scale and attaching a narrative description. These qualitative scores can influence the investment decision at the margin, but they carry less weight than the BCR in practice, and their assessment is more susceptible to subjective judgement.
Post-Implementation Monitoring in Practice
The logical complement to ex-ante effectiveness assessment is ex-post evaluation: did the project deliver what it promised? In New Zealand, this loop closes slowly and incompletely. NZTA requires post-implementation reviews for projects above a cost threshold, but the reviews vary considerably in rigour, and the findings are not systematically fed back into assessment methodology.
The evidence that does exist is mixed. Travel time savings on new urban capacity projects frequently underperform their forecasts within five years as induced demand fills the new capacity — a well-documented phenomenon that NZTA’s traffic models account for in theory but often underestimate in practice. Safety outcomes are more reliably delivered, partly because the causal mechanisms are better understood and partly because the KiwiRAP star-rating system provides a clear before-and-after metric for road safety quality. Emissions forecasts have been consistently optimistic, as vehicle kilometres travelled have grown faster than efficiency gains.
The gap between forecast and outturn is not a minor technical issue. It is the central challenge of effectiveness assessment: if the predictions that underpin effectiveness scoring are systematically biased in particular directions, the framework is selecting projects on flawed grounds. The New Zealand Infrastructure Commission has noted this gap in its work on infrastructure investment, and there is a credible case for a more rigorous ex-post evaluation regime that would allow assessment methodology to be recalibrated over time.
Why Effectiveness Assessment Favours Urban Scale

Volume-Weighted Metrics and Their Consequences
The mathematics of benefit-to-cost ratio (BCR) calculation is neutral with respect to geography, but its inputs are not. Travel time savings are calculated by multiplying three variables: the number of trips affected, the time saving per trip, and the value of that time. On a Wellington motorway carrying 80,000 vehicles a day, even a 30-second average time saving generates a large annual benefit when aggregated across the user base. On a Northland state highway carrying 800 vehicles a day, a two-minute time saving for every single user produces a number that barely registers in the BCR.
The consequence is not that rural projects never get funded — the NLTP includes a dedicated activity class for state highway maintenance and a separate one for local road improvements — but that they rarely score highly on the effectiveness dimension of investment assessment. A rural bypass that would transform the safety and productivity of a farming district cannot compete on effectiveness grounds with an urban project at ten times the traffic volume. The bypass might still be funded through the state highway activity class, but it enters the queue with a structural disadvantage in its assessed effectiveness.
This is not a design flaw in the sense of being unintended. The framework is designed to allocate funding where it generates the most benefit across the most users. But it produces a distributional outcome that is not always aligned with either the GPS’s stated commitment to regional development or with the lived experience of communities that depend on roads for daily functioning rather than for commute optimisation.
Rural Maintenance and the Effectiveness Gap
Maintenance investment occupies an awkward position in the effectiveness framework. Unlike new capital works, which can be assessed against a counterfactual of doing nothing, maintenance investment is assessed against a counterfactual of progressive asset deterioration — a harder comparison to make and one that tends to understate the benefit of maintaining infrastructure in good condition. A road that remains in excellent condition because it was maintained well generates no measurable improvement; it simply avoids a worsening that would otherwise have occurred.
This asymmetry hits rural roads hardest. Rural and provincial state highways carry lower traffic volumes, which means the avoided-deterioration benefit is smaller in absolute terms even when it is just as large in proportional terms. Local road networks in rural areas are maintained by district councils receiving funding from the NLTP, and those councils have consistently reported that the funding formula undervalues the cost of maintaining low-volume roads with poor natural surfaces and high climate exposure.
The deterioration of rural roads is not just an asset management issue. It has direct safety consequences — surface defects contribute to run-off-road crashes at a higher rate on rural roads than urban ones — and indirect economic consequences through increased vehicle operating costs for freight and agricultural users. The effectiveness argument for rural maintenance investment is strong in real-world terms; it is weak in the metric terms that the assessment framework uses to rank proposals against each other.
Reform Proposals and What Has Actually Changed
The structural bias in effectiveness assessment toward high-volume urban projects has been acknowledged by policymakers, academics, and the transport sector for years. The reforms proposed in response fall into two categories: changes to the assessment methodology itself, and changes to the funding structure that sit alongside assessment rather than within it.
On methodology, the main reform push has been toward better incorporation of wider economic benefits and social value into the effectiveness case. The New Zealand Infrastructure Commission has advocated for a more sophisticated approach to distributional impacts — asking not just how large the total benefit is, but who receives it and whether the distribution aligns with broader social objectives. NZTA has updated its economic evaluation guidance to allow wider economic benefits to be included in the core BCR rather than treated as a sensitivity, but uptake has been uneven.
On funding structure, the 2024-27 GPS introduced a dedicated funding boost for regional road maintenance, acknowledging in the policy document what the assessment framework struggles to capture in numbers. This is a pragmatic workaround: if the assessment framework cannot adequately value rural maintenance, ringfence some funding for it and reduce the competitive pressure on those proposals. It solves the immediate allocation problem without fixing the underlying measurement problem — which means the same debate will resurface at the next GPS revision cycle, as it has at every one before it.
Effectiveness assessment in New Zealand transport investment is a genuinely difficult problem dressed up as a technical one. The framework is coherent, the methodology is defensible, and the people applying it are mostly trying to make good decisions. The structural problems — the volume weighting, the thin evidence base for novel projects, the incomplete post-implementation feedback loop — are not the result of bad faith. They are the result of measuring hard things with imperfect tools and then treating the measurements as if they were more reliable than they are. The 2024-27 GPS has made partial adjustments, but the underlying tension between what the framework can measure and what investment effectiveness actually means for communities will not be resolved by a policy cycle or two.
3 Comments
Good breakdown of the ILM and how it front-loads the effectiveness argument. In practice, the quality of ILM workshops varies enormously depending on who facilitates them. I’ve been in sessions where the causal chain was stress-tested properly and others where it was basically reverse-engineered from a preferred solution. The framework is sound but the output is only as good as the people in the room.
The point about post-implementation monitoring is the one that needs more attention. We keep funding projects on the basis of forecasts, but the forecast methodology gets updated very slowly because the feedback loop between outturn data and the EEM is so weak. Induced demand is the obvious example — it’s been a known issue for decades and the models still underestimate it. At some point that’s a policy choice, not a technical limitation.
The GPS ringfencing workaround for rural maintenance is pragmatic but it does create a two-tier system. Ringfenced funding reduces competitive pressure, which sounds good for rural roads, but it also means those projects get less scrutiny on effectiveness grounds. You end up with rural maintenance funded outside the main framework, which makes it easier to cut when the next government wants to reprioritise. A fix to the measurement problem would be more durable than a funding carve-out.