New Publications Examine Member Programs' Outcomes, Strategies

April 18, 2017

By Bill DeBaun, Director of Data and Evaluation 

The college access and success field in recent years has set its sights firmly on demonstrating outcomes for the students it serves. NCAN members are no different, and they have invested time, energy, and capital into practices and platforms that collect, manage, and analyze data to demonstrate program outcomes, improve program performance, and scale up program capacity. 

From the Common Measures to the Data and Evaluation Toolkit to the Benchmarking Project, NCAN has worked hard to provide members with resources that make their data and evaluation work easier. Recent publications in two new series add to the growing library of tools and insight into program improvement and new approaches. 

Last summer, through the Common Measures Learning Community project, NCAN held a series of Idea Incubators with member programs around the country. These gatherings of five to eight members each focused on a specific topic in the college access and success field and provided members with minimally structured time to brainstorm new approaches, identify program needs, and share tips and tricks. Each of these Incubators generated significant insights into their respective topics, and the first brief in a series was published in April. “Incorporating Fit and Match Into College Advising and Program Data” was based on an Incubator held at Degrees of Change in Tacoma, WA. Guiding questions at the event included: 

  • How do you recognize and capture data relevant to fit and match, including student-level data on institutional experiences?
  • How is this data incorporated into existing systems (or put into new systems) to be used in a meaningful way?
  • How can advisors best steer students to institutions that are good fits or matches? 

Each Idea Incubator brief serves as a hybrid of a white paper, a resource guide, and meeting minutes on its topic. Upcoming briefs will focus on the other Idea Incubator topics: “Managing the Database Lifecycle and Ensuring Quality Data,” “Leveraging Data and Analysis for Program Improvement, Inquiry, and Growth,” and “Collecting and Reporting Metrics across Organizations, Networks, and Collaboratives.” 

Other recent publications stem from NCAN’s Benchmarking Project. This effort, which NCAN undertakes in collaboration with our members and the National Student Clearinghouse Research Center, aggregates programs’ student-level data to produce enrollment and completion outcomes that we can compare against national benchmarks. This project has demonstrated that students served by NCAN members enroll at rates far greater than their peers do and complete at rates that, if not matching the national average, do exceed the best available completion benchmarks for low-income students. 

The Benchmarking Project also allows NCAN to identify programs whose students are enrolling and completing at high levels and whose practices may be replicable for other programs. NCAN identified four of these programs and worked with Ann Coles, Senior Fellow at uAspire, to complete case studies of them. These case studies provide an organizational overview – including documenting in-depth the services that students receive – examine staffing and organizational culture, detail policies and practices contributing to student success, and finally, assess the organization’s potential for replication. Two of these case studies have been published to-date. The first profiled Philadelphia Futures, and the second looked at I Know I Can, in Columbus, OH. 

Philadelphia Futures is an NCAN member serving 600 high school and college students directly on an annual basis and another 40,000 through outreach efforts. Coles describes Philadelphia Futures’ outcomes: “Over 98 percent of the students enroll in college right after high school. While only 10 percent of Philadelphia high school graduates earn college degrees, Philadelphia Futures’ overall college graduation rate is 60 percent. The graduation rate has risen dramatically over the past decade, resulting in a projected graduation rate of 70 percent for the high school classes of 2005-2012.” Philadelphia Futures’ data-driven decision-making, along with its peer-to-peer support, college admissions marathon, collegiate partnerships, and scholarship provision are all highlighted by this case study. 

I Know I Can “provides college awareness activities, advising, and application assistance to over 25,000 middle and high school students (11,000 6th-8th graders and 14,000 9th-12th graders) who attend Columbus City Schools’ 20 high schools and 28 middle schools. In addition, more than 1,000 CCS graduates receive postsecondary retention support annually.” Their work at the elementary, middle, and high school levels has contributed to outcomes for the students they serve that are much higher than those of their peers in the same school system. 

Two additional case studies on Degrees of Change and Bottom Line are forthcoming, and NCAN will continue to highlight members whose work other programs can learn from and whose practices lead to student success. 

Overall, these two new series, the Idea Incubator briefs and Benchmarking Case Studies, continue to point to the thoughtful, outcomes-focused manner necessary to drive the college access and success field forward.

Back to Blog

Leave a Reply:

Creative Commons License
This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivs 3.0 Unported License