Measuring your impact is critical to understanding whether you are creating the change you want to see in your community. Continuous learning is one of our six upward mobility principles as it will allow you to learn from your work and adjust your strategies as needed. A continuous learning strategy is also a key component of sustaining your work because you need to be able to communicate your impact to get buy-in from key stakeholders and maintain accountability with your community partners.

When measuring your upward mobility initiative’s impact, your goal should be to understand how your initiative is creating systems change. This means that beyond just measuring the results of any single program, you should seek to understand how your work is changing the ways decisions are made in your community, the collaboration and power dynamics between actors, and the narratives and mental models (see chapter 5 for more information).

Keep reading to learn how to develop a measurement plan, measure systems change, identify appropriate measures and data sources, communicate your impact, understand how to implement continuous learning principles, and determine next steps.

Creating a measurement plan that builds on your theory of change and logic model can be a helpful tool to track progress toward your goals. We suggest developing a measurement plan alongside your logic model to build in your measurement activities from the beginning of your implementation work.

Our Developing a Measurement Plan for Your Coalition’s Work worksheet provides space for you to identify relevant measures for each strategic action in your logic model. Table 6-1 contains examples of measures you might identify for a hypothetical strategic action. You should ensure your measurement plan aligns with community priorities for measurement (see Urban’s report on Engaging Communities in Measurement and Data-Driven Decisionmaking for how to do this).

TABLE 6-1: Example  of a Measurement Plan

Note: Outputs are short-term observable results of actions that could lead to desired outcomes, but may not get you all the way there.

Input Measures

Inputs are necessary to implement your strategies and can include things like funding, staffing, partnerships, political will, technology, evaluation capacity, data, and regulatory approval. Input measures may reflect the amount or quality of the resources, human labor, relationships, technology, or investments being applied to implement the strategic actions. Examples of input measures include:

• dollars spent
• number of staff
• technology needed
• partnerships

Outputs

Outputs are the short-term observable results of actions that could lead to desired outcomes, but may not get you all the way there. They are likely to rely on administrative data that are collected locally and should show how much of something was done or how well it was done. Performance measures (or data that indicate how well programs, service systems, or organizations are performing) produce one type of output. Examples of outputs include:

• number of workshops facilitated
• satisfaction ratings of programs
• number of people served

Outcome Measures

Outcomes (or goals) are typically long-term, aspirational changes. They may include changes for the entire community or for specific groups, such as changes in conditions, capabilities, or relationships, and changes in systems, including policies, practices, norms, and narratives. Outcome measures should capture the consequences of outputs.

The measures you choose to track progress on outcomes should help reveal whether the intended people are being helped or the intended conditions are being changed. Outcome measures that can help you track your ability to change systems include increases in staffing or other resource capacities, changes to the structure or hierarchy of decisionmaking, or changes in prevailing narratives or norms.

You may choose to use the Mobility Metrics as outcome measures, or you may select other measures that might be more likely to show change in the medium term. Examples of outcome measures include:

• median household income
• affordable housing availability
• air quality

To adequately reflect systems-level change, the measures you choose should be specific, supported by data and evidence, easy to use, and capture the results of systems change, not just a particular program. You should identify measures that capture the characteristics of the systems that influence your desired long-term outcome. Table 6-2 provides examples of measures corresponding to the six conditions of FSG’s Water of Systems Change framework for a hypothetical long-term desired outcome.

TABLE 6-2: Example of Systems Change Measures

When thinking about your impact, remember not to overlook the work you’ve done to build a successful cross-sector coalition. By intentionally bringing partners together, you’ve already begun working toward changing relationships and connections, resource flows, and power dynamics. Examples of output and medium-term outcome measures that could reflect the success of your coalition include the following:

• number of cross-sector partners engaged
• collaboration with champions in and from the community
• coalition members agree on an effective structure for collaborative governance
• partners decrease their silos and collaborate more*
• partners implement the same race equity framework*
• coalition members strengthen their relationships with community-based organizations*
• organizations across sectors create new positions that have adequate power to advance their diversity, equity, and inclusion goals*

All items with * are adapted from Joel Gutierrez, Sarah Slachowiak, and June Han, I2L2: Impact, Influence, Leverage, and Learning (Seattle: ORS Impact, 2023).

For more information on the types of measures that best capture systems change, consult our report on Boosting Mobility and Advancing Equity through Systems Change.

To identify appropriate measures that are relevant to your coalition’s goals, revisit the data sources you used to understand upward mobility in your community in chapter 4. Data that inspired a strategic action may be a good contender for tracking progress over time. Valid data for measuring progress can come from a variety of potential sources, including the following:

• publicly available data, such as Census data or the Mobility Metrics
• local administrative data (review our resource on Obtaining More Local Data for specific recommendations on local data sources)
• qualitative evaluations such as survey or poll data
• direct observation, focus groups, and community reporting
• data documenting notable actions, policy reforms, and/or implementation efforts
• data that your coalition collects on costs, staffing, and/or participation in programming, community events, and more

When selecting a data source, it’s also important to consider how long you will need to track the information, how feasible it will be to continue collecting it throughout the lifetime of your upward mobility initiative, and what steps your coalition will need to take to gather data from that source. The latter may include finalizing data-sharing agreements, considering available funding and sustainability, projecting needed staff capacity, or planning to survey community members. The measures you ultimately select should rely on consistently collected and calculated data to enable evaluation and comparison over time. If any of your measures are ratios, you should ensure the data source and collection for both the numerator and the denominator remain consistent.

Even after you have chosen a set of measures and corresponding data sources, keep in mind that this process is flexible and iterative. As your coalition’s work evolves, you may discover that your strategies have shifted course significantly enough to require new measures, or that different measures better reflect your goals or are simply more feasible to collect and track. If so, you should adapt your approach accordingly. If there is a lot of ambiguity around what measures of success would be best for a given strategic action, consider starting the process with a wider selection of measures, with a plan to revise or narrow down your scope once you have more information to assess how well each measure is helping you track progress toward your goals.

Sharing the impact of your work with others is important to demonstrate accountability to your partners and community. It also provides an opportunity to share your successes, build momentum, and recruit new champions for your work. Make sure you consider ways to communicate your impact through both formal reporting mechanisms and storytelling. Your communications should also be tailored to specific goals and audiences. Consult this Urban report on Framing Communications to Drive Social Change for further guidance.

Reporting on Your Impact

Consider the following questions to decide what, how, and to whom you will report your progress:

• Will you report on the measures you identified to all coalition members at regular intervals?
• Which measures could demonstrate the need for a pivot if the intervention doesn’t appear to produce the impact you’re hoping for?
• What accountability do you owe your community? At what point will successes or failures be communicated to the public?
• Who will make these decisions, and how can your team be proactive and thoughtful about the language used in your communications?

Be timely and transparent with updates on the progress of your strategic actions because these are the key insights on which your continuous learning will be based. For many efforts, especially in the early years of implementation, it may be hard to determine whether you are witnessing success or failure because these terms are subject to various descriptions and perspectives. Nevertheless, general trends are important to track, and regular updates on the progress of your efforts will help you build trust with community members and create robust feedback loops to learn what’s working and what’s not.

You should also consider what dissemination methods will be most likely to reach and be understood by your target audiences and the community at large. While any plan for reporting on your progress will be contingent on the timeline and frequency of reported results (based on your data collection and evaluation), we suggest considering the following reporting frequencies:

• Biannually: Every six to eight months, report on the inputs for each strategic action. Make sure to answer the following questions:
  • Are we doing what we said we were going to do?
  • Have we held ourselves accountable for shared responsibilities?
  • (Within your coalition) Are there any adjustments that need to be considered or made to budgets, scheduling, staffing, or planned measurement?
• Annually: Approximately every year, report on the outputs for each strategic action. Make sure to answer the following questions:
  • What outputs and early outcomes are we seeing?
  • Have we addressed what is not working?
  • Do we need to modify any inputs or actions?
• Every three to five years: Report on your progress toward outcomes. Create a public progress report detailing any achievements that deserve celebration and any inputs or actions that require adaptation. Make sure to answer the following questions:
  • What positive changes are we seeing among the community or groups we hope to impact?
  • What persistent challenges may require a new approach or different ways of thinking?
  • In what ways have we changed our local upward mobility ecosystem for the better?

Storytelling

The ability to use what you have learned through your evaluation efforts to tell a compelling story about your work is a skill that will help your upward mobility initiative expand its audience, influence changemakers, keep community members informed about your work, and even change narratives and mental models.

Consider a variety of communication methods to tell the story of your work, including written publications, video, photo, multimedia projects, creative publications (e.g., a “zine”), media coverage, social media, and in-person or virtual presentations. Your storytelling should center the voices of those most impacted in your community when appropriate, including by giving community members agency to tell their own stories and having them guide the selection of platforms and venues. Some resources to support your storytelling efforts include:

• FrameWorks Institute's The Storytelling Power of Numbers
• FrameWorks Institute's Framing Success Stories
• The Opportunity Agenda's Vision, Values, and Voice: A Communications Toolkit

Selecting Headline Indicators

Headline indicators are a subset of your outcome measures that can serve as the focal points for your work and communicate your coalition’s key priorities. While your measurement plan will likely include many outcome measures to track, selecting a smaller number of headline indicators can help simplify your message in both storytelling and formal reporting efforts about your work.

If you decide to select a few headline indicators from the many measures your coalition is tracking, be sure to provide sufficient context when communicating about them, since a single indicator is unlikely to represent the full picture of upward mobility conditions in your community. You should also ensure the headline indicators you select reflect your community’s priorities.

Choosing headline indicators may be challenging for your coalition, as each partner may bring different priority measures based on their focus areas. You may wish to consider the following four factors, adapted from the Results-Based Accountability framework above, to identify headline indicators:

1. Communication power: Does the indicator communicate to a broad range of audiences? Could you stand in a public square and explain this indicator to your neighbors?
2. Proxy power: Does the indicator say something of central importance about one of your coalition’s desired outcomes? Can this measure stand as a proxy for a statement of well-being?
3. Data power: Is high-quality and timely data available for this indicator?
4. Equity: Does this indicator account for disparities or inequities in outcomes across different demographic groups or neighborhoods?

See our Selecting Headline Indicators that Communicate Your Priorities and Impact worksheet for a table you might use to prioritize different indicators.

Measuring the impact of your coalition’s work is not a discrete process contingent on specific intervals of reporting; it should be ongoing once your strategic actions are underway. By continuously tracking progress, you will be able to adapt to any signals that your work is not having the intended impact. Failures to achieve an intended goal or benchmark are critical opportunities for learning, as they are valuable touchpoints for considering whether you are falling short because of external factors outside of your control, ineffective implementation, or unrealistic or misguided expectations about how change occurs (as described in your theory of change).

Additionally, continuous learning is important because it helps you track not only whether things are succeeding or failing but also why they are succeeding or failing. By asking those responsible for implementing strategic actions why they think things are working or not working, you will gain critical insights about the actions and conditions needed to boost upward mobility and advance racial equity. Table 6-3 below describes a framework for thinking about continuous learning and suggests questions you can use to reflect on how well your coalition’s work is going.

TABLE 6-3: Framework for Continuous Learning

Source: Adapted from Mark Cabaj, “Evaluating Systems Change Results: An Inquiry Framework” (Ontario, CAN: Tamarack Institute, 2018), https://www.tamarackcommunity.ca/library/paper-evaluating-systems-change-results-an-inquiry-framework.

Understanding whether your coalition’s work is having its intended impact is critical to determining what comes next. If you find your coalition is falling short of its goals, you should course-correct as needed, whether by revisiting your logic model and theory of change to identify new strategies, recruiting new members to the coalition that can help you achieve your desired impact, or rethinking how your coalition works together and is governed and managed.

On the other hand, if you find your coalition is moving in the right direction—although slowly, as systems change takes time—you may wish to review chapter 7 to understand how to sustain your work over the long term.