Performance Measures, Evaluation, and Prioritization

Originally published in the April 2021 edition of Streetsmart News (V. 34)

Increasingly, agencies and organizations are calling for the use of performance measures in transportation plans and projects, particularly since the adoption of Moving Ahead for Progress in the 21st Century Act (MAP-21) in 2012 (for context and more information, see this FHWA guidebook and this work by Transportation for America). The underlying hope is that the process of measuring and tracking performance measures can help turn policy goals into practice. Indeed, the potential of performance measures to lend greater transparency and accountability to transportation decision-making partly inspired the creation of Streetsmart. The idea being: which strategies are most likely to help agencies meet their goals, as measured by performance measures?
 
Do we have the data to understand which strategies are most effective? The relationship between the built environment (i.e., land use and transportation) and travel behavior is among the most studied in transportation and planning research. However, this research doesn’t always translate to a clear relationship between interventions and outcomes. Thanks to the systematic reviews undertaken by groups such as the Community Preventive Services Task Force and County Health Rankings & Roadmaps, there is unequivocal evidence for the effectiveness of some strategies, such as pedestrian infrastructure for increasing walking and physical activity. Systematic reviews examine individual studies and describe the efficacy of the intervention as a whole. Evaluations are the bricks in the systematic review wall. 1

The above organizations operate in the field of public health, which has a tradition of project and program evaluation (as do other fields like social work and public policy, see the Campbell Collaboration). Transportation does not have a similar tradition of evaluation (that is, before-after studies), although many have called for it. If evaluation was consistently undertaken as part of transportation projects, decision-makers would know if they were achieving the goals they established. Furthermore, they would be building the evidence base for what works, and if those results collected in a publicly accessible location (ahem, Streetsmart), they would benefit other decision-makers. Alongside input from community members, decision-makers could select projects on the basis of how likely they meet adopted goals in contrast to the often-murky process used in many places.

However, agencies scramble to obtain enough funding to plan, design, and build their projects—funding for before-after studies isn’t often accommodated. A dedicated funding stream for evaluation research would help. Universities, and specifically University Transportation Centers, could be useful partners.

To summarize how performance measures, evaluation, and prioritization can work in concert:

• Full spectrum performance measures. Adopt performance measures that acknowledge transportation as a determinant of health and its impacts on climate and equity. Track the health and well being of communities, not just measures of auto-mobility.
• Before-after studies. Undertake rigorous before-after evaluations (with controls) to understand if what you are building is working. Include qualitative data of resident perceptions, needs, and barriers to travel as part of the evaluation. Better still, engage community members in data collection!
• Centralized location for evaluation and systematic reviews. Streetsmart aims to collect evaluations and systematic reviews for a library of “what works.” Streetsmart’s long-term goal is to incorporate context-based knowledge, following the realist maxim: what works, for whom, and in what circumstances?
• Evidence-informed project prioritization. A “what works” library enables agencies to prioritize projects for funding, with even greater confidence if that data is context-sensitive. However…
• Listen to the community. Always engage with the community to understand their perspectives, gain contextual knowledge, and modify designs to reflect their culture and context. 

1. Metaphor from Mark Lipsey (1997): “What can you build with thousands of bricks?”