If You Don’t Know Who You’re Impacting, How Do You Know You’re Making an Impact?


When I’m not trying to get philanthropy to be more proactive around embracing diversity, equity, and inclusion, I’m working to get my Ph.D. at the University of Chicago. Though the emphasis of my dissertation is on philanthropy and social movement dynamics, there are moments when my day job comes up particularly with respect to how foundations facilitate change and the kind of data and analysis they use to support their strategies and approaches.

The University of Chicago prides itself on being an institution that values understanding, with as much rigor and clarity as possible, the nature of the variables that might have an impact on the change we are trying to assess. Its pedagogy pushes investigators to embrace this rigor and become comfortable with bringing sometimes uncomfortable and messy interactions to light. So I often encounter surprise from my academic colleagues about the uneven degree to which foundations collect and use relevant data to inform their decisions about how they will invest significant resources—often quite passionately—to address big, complex social issues.

We know that philanthropy is as much art as science and that not all insights are drawn from purely empirical data. Implicitly, however, a clear-eyed assessment of the impact that various factors will have on any outcome, whether it is educational achievement, voting patterns, or behavioral changes, is at the root of almost every philanthropic investment. The degree to which some variables matter—their R2s so to speak—is easier to assess than others, but few if any serious social scientist would neglect to include key demographic variables such as race, ethnicity, gender, and socio-economic status in an analysis of the impact of an intervention. Decades of social science research have underscored the relevance of these aspects of identity on how individuals are likely to engage their environment, their community, and their world. These factors are not deterministic, nor static—they often interact in complex ways with many other factors—but neither are they trivial or irrelevant. So the analysis of the effect of any intervention on any community cannot fully be assessed if we do not collect and assess data along these parameters and use it to inform our efforts.

A hypothetical collective impact approach, for example, aimed at increasing the rate of high school to college connection rates for a community with strained public resources might develop an intervention that brings together municipal social service providers, the local community college, and the public school system. The approach might develop college prep and counseling services that engage content and counseling instructors from a nearby community college to strengthen students’ pathways to college and to offer supplemental life skills training through a regional workplace development agency. It might feature all of the components of a collective impact approach, and pre- and post- program analysis might show a meaningful uptick with respect to college attendance overall, perhaps leading the collective impact partners to conclude that the intervention as designed is working.

But without data on the demographic variables among the population, the strategy might miss the fact that most gains in graduation were accruing to males even though there were an equal number of girls and boys participating in the program. Only with this data could one surface the insight that the range of prep courses and the timing of the afterschool services conflicted with afterschool jobs that a disproportionate number of girls worked because a home health care agency actively recruited them for positions at a newly opened nursing home. Without the ability to analyze demographic variables, not only would the intervention miss this finding, but it would lack insight into ways to improve the design to promote more equitable outcomes for young men and women and thus overall success. It would miss an opportunity to gain insight into a labor market dynamic that might be entrenching larger gender-based systemic inequities over time.

As has been said, “you can’t fatten a pig by weighing it.” The only way to ‘fatten the pig,’ or to make the impact of any kind of intervention as robust as possible, is to pay attention to the variables that are known to be factors in impacting change. But while including demographic data may seem obvious to many, it’s amazing to me how rarely they are explicitly addressed within the context of assessing philanthropic effectiveness. And in many cases, there is active resistance—yes, resistance—to accepting the degree to which variables relating to identity are relevant for understanding change and impact. Yet there is too much evidence to refute the importance of this kind of data for assessing effectiveness and for understanding how to optimize our philanthropic investments.

For a long time, we lacked the tools to effectively and systematically collect and use demographic data to assess philanthropic impact. But now, in 2015, we all have the means, mechanisms, and technology to access and utilize all of the data that is necessary to understand how best to advance the common good. The demographic profile of organizations in the GuideStar Profile, the ‘Get on the Map’ effort, a collaboration between the Foundation Center and Regional Associations of Grantmakers, and DataArts (formerly the Cultural Data Project) offer new and robust tools that provide streamlined and coordinated ways for the social sector to collect, share, and use demographic data to understand their partners and communities. These systems will only have value, however, if data is collected and shared, so we urge foundations to use these tools and partner proactively with nonprofits to ensure the data is sound and the analysis appropriate and constructive.

Those of us working in philanthropy and nonprofits are committed to advancing the common good. We are in these fields because we are passionate and concerned about our communities, and our constituencies and audiences deserve the highest level of rigor and insight we can offer. The tools and data we employ to achieve this level of rigor must be commensurate with our level of passion and concern for making equitable, meaningful and sustainable change.

Read the Report: Collaborating to See All Constituents Reach Their Full Potential

More Voices from the Field

Contributors to this research scan share more recommendations on how organizations can add an equity lens to their work to help better serve their communities.

3 Levels of Racial Equity Work within Collective Impact by Juan Sebastian Arias and Jeff Raderstrong (Living Cities)

Pitfalls to Avoid When Pursuing Equity by Sandra Witt (The California Endowment)


Sign Up to Download

You will also receive email updates on new ideas and resources from Collective Impact Forum.

"*" indicates required fields

This field is for validation purposes and should be left unchanged.

Confirm Your Registration

You will also receive email updates on new ideas and resources from Collective Impact Forum.

"*" indicates required fields

This field is for validation purposes and should be left unchanged.