Using shared measures to track progress toward goals and understand where partnerships are making progress and where improvement is needed is increasingly emphasized as essential for addressing complex issues and improving our communities. However, our understanding of exactly how to harness the power of data is limited. One thing, though, is abundantly clear: Making good on the commitment to use data is hard—and the challenges aren’t just about technology.
There’s much that collective-impact efforts can learn about using data from The Wallace Foundation’s Next Generation Afterschool System Building Initiative. This multi-year initiative involves support for nine cities where city agencies, schools, and nonprofit organizations are working to better coordinate access to high-quality afterschool opportunities for children, along with independent research to learn from their experiences. These place-based collaborative efforts in Baltimore, Denver, Fort Worth, Grand Rapids, Jacksonville, Louisville, Nashville, Philadelphia, and Saint Paul share features with collective impact initiatives as they work across sectorial silos to leverage resources to affect children’s lives, use data to diagnose needs, engage and sustain partner engagement, and improve the quality of services.
A new research report on their efforts, the first of two planned volumes from Chapin Hall at the University of Chicago commissioned by Wallace, offers insights into what it takes to put data systems in place and use them to reach common goals. Key conclusion: While we typically focus on addressing the technology needs of data use, in reality, two other components—people and processes — are just as crucial, as shown in the diagram below. These findings echo and expand upon the research conducted by the RAND Corporation and published in Hours of Opportunity Volume II, which found based on a study of eight afterschool systems that using MIS data can help improve access and services but that it requires careful planning.
The new Chapin Hall study finds that often, as these collaborative efforts begin, the focus is on technology, but that leaders are caught short by the challenges posed by the people and the processes that are required to transform the data into useful information. But as the triangle suggests, the components are inter-related and each is equally necessary.
As the researchers write: ……as important as technology is, most of the factors that appear to facilitate or inhibit data use in city afterschool systems—norms and routines, partner relationships, leadership and coordination, and technical knowledge—have to do with the people and process aspects of a data system.
Not surprisingly, that’s also one of the findings from researchers at Teachers College, Columbia University, who are studying collective impact communities participating in the Ford Foundation’s Corridors to College Success initiative. In this recent brief, the researchers described “myriad challenges associated with their organizations’ capacity for data collection, data-sharing agreements, third-party data warehousing or merging, data privacy and storage, and staff capacity for meeting technical data management and analytic needs.”
The Three Legs of the Triangle
The nine cities in Wallace’s effort that are using data in their afterschool systems are finding paths that are deepening their capacity for data use. Their experiences as captured by Chapin Hall, we believe, can help people involved in other collective-impact and system building efforts.
- Start small to learn what works. A number of cities intentionally started with a limited set of measures for data collection and use, and/or a limited set of providers piloting a new data system, with plans to scale up gradually. For a collective impact effort, that might mean starting the data work with one working group, learning how to enable the people, processes, and technology work together successfully before scaling across all strategic areas of focus.
- Leverage existing data expertise. Expertise came from within as well as outside the organization coordinating the initiative. Some cities are working with a research partner who participates in all phases of the development of their data systems, providing ongoing support. Others leveraged the relationship primarily for access to data, analysis, and reporting of data collected by providers. Still others did not engage an external research partner, but identified internal staff who are capable analysts who can provide these supports to the system. Many collective-impact efforts might tap research partners, or communities have research institutions, committed to developing knowledge to positively affect lives.
- Provide ongoing training. System stakeholders learned that they needed to provide ongoing introductory trainings in using both the management information systems and the data to enable use.
Taken together, these lessons suggest that cities should acknowledge upfront and plan for the challenges of data use and that there are steps they can take toward success, namely prepare and provide ongoing support for the multifaceted interplay of people, processes, and technology.
What do you think? Share your comments and questions below:
