Originally published in the January 2024 issue (v.44) of Streetsmart News

In this newsletter, we'll examine the indirect influence indicators have within a planning process, as compared to the direct and instrumental use and influence of indicators, as discussed in my previous newsletters. Drawing from my dissertation, The Use and Influence of Health Indicators in Municipal Transportation Plans, this article discusses the importance of how indicators are developed such that they influence the thinking of policy actors and can help set an agenda for policy change.

Let's begin with a couple key points about how policy learning and change happen. My research drew from Communicative Planning Theory as well as two policy process theories from political science, the Advocacy Coalition Framework (ACF) and the Multiple Streams Framework (MSF). Learning is defined as a change in beliefs about policy goals or solutions, with the assumption that beliefs are the "glue" of politics. (1) Communicative Planning Theory, and the ACF to a degree, assert that learning occurs through dialogue; that is, people learn when they are engaged in meaningful conversation in a lower-conflict, consensus-oriented discussion forum. Policy learning is a confirmed path to policy change. (2)

In all the cities under study, indicators had been developed in the context of community engagement processes, in which the public defined the key issues, goals, and objectives in the transportation plans. From there, city staff and consultants developed indicators to match the goals and objectives articulated by the public, typically with feedback from standing or plan-related advisory committees. As such, indicators were perceived as legitimate by policy actors. This differs from situations where the validity of the data is debated within contentious policy processes and/or processes where outside experts create the indicators.

My dissertation confirmed MSF research that demonstrates that indicators have the ability to draw attention to key issues and to frame them as problems. Policy actors frame issues as problems to justify public intervention. In my cases, the primary issue framed as a problem in many plans was about equity: equitable access to employment, transportation affordability, disparities in health outcomes, etc. Based on informant responses, this was the first time these problems had been so clearly defined within the city's transportation plans. Indicators helped people see disparities more clearly, such as this housing and transportation affordability indicator:

"A lot of the shock came from the expense of transportation in the Indianapolis region, because . . . there's a huge gap in regional job accessibility and it makes the combined housing and transportation costs the third highest in the country among our peer metros. And that usually takes people aback” (Informant 34). 

Informants noted that indicator selection, being so clearly connected to community values and goals, helps frame problems that policy actors want to highlight:

“The data that you show pushes your agenda . . . . The fact that we are calling out and even researching how expensive it is to travel in Indianapolis and the disparity . . . I mean, that conveys a value” (Informant 31).

Dialogue helped not only to raise awareness of the issues but create a shared understanding and changes in policy beliefs (i.e., learning), such as the dialogue in this equity workshop:

“It was the dialogue, quite frankly . . . you're sitting in a small group around a table with people who live in this community going, ‘Whoa, wait a minute here,’ and it's like, . . . ‘Oh, I had not thought about that’ . . . . I think there are a lot of people, including many of the staff members, who are like, . . . ‘This is a lot bigger conversation than we intended’” (Informant 21).

Note that these awareness and learning opportunities only happened within the earliest parts of the transportation planning process. Once the planning processes move from setting goals to the later stages of analysis and project selection, the role of indicators changes from sparking "ideas" to providing "data" and ammunition for "argument." (3)

While indicators can help illuminate a problem, informants emphasized that it was the combination of data and dialogue that produces shared understanding. In fact, it was difficult to determine if the discussion about the indicator itself or policy goals more generally generated this shared understanding because the two were so intertwined. The combination of data and argument can also set the policy agenda and help advocate for policy change:

It's "not just the numbers . . . but the point, the narrative, the messaging around that, when you can use that with a broader audience, that includes not just the public, but civic leadership, community stakeholders, or . . . organizational type partners . . . I do think that can push leadership” (Informant 35, original emphasis). 

In sum, make sure you develop indicators inclusively so that the users buy into them and they reflect community values. If indicators are developed by independent experts without community input, they run the risk of not being useful or perceived as illegitimate. Most importantly, the opportunity for learning and shared understanding is lost. The greatest opportunity for policy learning is within the earliest stages of a planning process: indicator development and the existing conditions analysis of a plan. Use these opportunities to raise awareness of problems and get them on the agenda for change.

1. Paul Sabatier (1987). Knowledge, policy-oriented learning and policy change: An advocacy coalition framework. Knowledge: Creation, Diffusion, Utilization, 8(4), 649-692.
2. Christopher Weibe (2018). Instrument constituencies and the advocacy coalition framework: An essay on the comparisons, opportunities, and intersections. Policy and Society, 37(1), 59-73.
3. Carol H. Weiss (1991). Policy research: Data, ideas, or arguments? In P. Wagner (Ed.) Social sciences and modern states: National experiences and theoretical crossroads (pp. 307-332). Cambridge University Press.

Originally published in the November 2023 issue (v.43) of Streetsmart News

This newsletter focuses on another component of my dissertation, The Use and Influence of Health Indicators in Municipal Transportation Plans. In my last newsletter, I discussed the key indicator characteristics that made them for useful for transportation decision-making. I'll continue in this newsletter discussing the instrumental use of indicators--the presumption that indicators inform decisions--but focus on the organizational aspects of developing and using indicators.

First of all, none of the people I interviewed (city and transportation planners, advocates, public health practitioners, etc.) felt that data was the problem, despite whatever challenges they may have had with spatial scale, measurability, or the like. One informant was clear:

“I don't think it's because the data isn't usable or available. It's because of the systems. We do not have the systems in place to use the data in the way they're intended to be used” (Informant 35).

In some cases, different departments, such as public health and city planning, collaborated to develop indicators. The development process helped "force different departments to talk to each other" (Informant 12) or otherwise generate some policy alignment through indicator alignment. While helpful, this was not a critical factor in terms of having an influence on administrative decision-making. The three things that mattered most were:

1. Elected and/or departmental leadership had adopted a "data-driven" governance paradigm
2. Leadership secured sufficient funding for building the data infrastructure to monitor indicators over time and ensured that staff had the technical capacity to manage indicators (or a contract in place for consultant support)
3. Leadership had changed organizational procedures with a clear use case for the indicators

​
The data-driven paradigm, as compared to one based on territoriality or a "cash grab" (Informant 36), was one that valued transparency, accountability, and fairness in decision-making. In two cases, elected leadership drove this approach, which in turn became infused within the culture of the departments. Non-profit organizations helped municipalities be accountable for the goals set in their plans--sometimes they even created their own progress reports.

Having sufficient funding may seem obvious, but some municipalities did not fully appreciate the degree of infrastructure required to monitor indicators over time. Indicator development was no problem--many cities had funding to develop the indicators as part of transportation plan development--but tracking indicators over time required new structures and funding.

Finally, having a clear procedural connection for the indicators was important. In one case, the city had data-driven leadership, sufficient funding, and was tracking indicators over time but they had not created a clear enough link from indicator reporting to decision-making. Specifically, they were reporting on systems monitoring indicators during budgeting decisions. As one informant reported:

It was “a good half-hour of city council just free-associating on their fears and anxieties. It had nothing to do with, like, ‘Are we getting where [we want to be] going?’” (Informant 25)

When reporting on indicators, you need an action proposal. Otherwise, you use indicators as an analytical input to a decision. The most successful cases used evaluation criteria as part of capital improvement programming; that is, a clear nexus between the data and the decision.

In short, to influence administrative decision-making: Ensure the organization has sufficient financial and technical capacity to create data infrastructure. Create accountability mechanisms such as required internally-produced progress reports; if the city doesn't, then advocates can step in. Make sure that indicators are used for a specific routine or analysis in order for them to be influential.