Bill DeBaun, Program Analyst
Welcome to the second half of the inaugural edition of the Data Resource Roundup! In this series we will periodically share resources, including blogs, courses, white papers, and other tools, that cover various aspects of data. Whether it’s better managing and tracking of data or getting your organization to become more data-driven, it will all be here in the Roundup. This edition is particularly full (see the first part of it here) because it contains resources I’ve collected over the past year. Have your own resources that should be featured here? Be sure to let me know about them at firstname.lastname@example.org or by putting them in the comments.
Performance Indicators and Measurements
- We would be remiss if we did not start off this section with NCAN’s own Common Measures. This list of access and success indicators was developed in collaboration with members and in deference to research that exists on what moves the needle on college enrollment and completion. If you’re a college access program looking for a sense of what you should be tracking in terms of outcome measures, start here.
- Keeping in line with the “predictors of postsecondary success” theme, the American Institutes for Research offer up a denser (but more comprehensive) set of Predictors of Postsecondary Success culled from the literature. Their research is broken down into early childhood, elementary, middle, high school, postsecondary, and adult age groups and includes indicators (“measures with established thresholds...[at which students] are more likely to be prepared for their college and career pursuits”), predictors (“measures that are strongly correlated with improved postsecondary outcomes but for which a numeric threshold has not been established”) and other potential factors.
- If you work for a school district or your program works closely with a school district and you wish you could get a better system of metrics established to see how students, schools, and the district overall are performing, take a look at the College Readiness Indicator Systems (CRIS) Resource Series. This will take a little time to read through, but it’s worth it. Launched by the Bill & Melinda Gates Foundation, CRIS worked with research and school partners across the country to develop indicator systems around three dimensions of college readiness: academic preparedness, academic tenacity, and college knowledge. The resource series has six parts, but the chart in the link above describes what it included in each part and who is most likely to benefit from reading it. Notably useful for most members are likely the following:
- Menu of College Readiness Indicators and Supports – “A list of research-based indicators and supports to choose from in building an indicator system, organized across the three dimensions and three levels.”
- Selecting Effective Indicators – “A guide for determining what indicators to include in district data reporting systems, in light of a district’s priorities and capacity to provide interventions and support.”
- District Self-Assessment Tool – “A tool that supports a district’s effort to assess and strengthen its organizational capacity to plan and implement a college readiness indicator system.”
- Essential Elements in Implementation – “A report of promising implementation strategies with concrete case examples, drawing on CRIS implementation in partner sites.”
- If your organization works with middle school students and you find yourself wishing you had a better set of indicators that might give you an early sense of their college and career readiness, the University of Chicago Consortium on Chicago School Research has a report that you should definitely read. In “Looking Forward to High School and College: Middle Grade Indicators of Readiness in Chicago Public Schools,” researchers find that students’ grades and attendance patterns are better predictors of their high school and college success then their test scores. Included at this link are key findings for both middle and high school teachers (and they’re likely to have some application for advisors serving these populations). Keep in mind that the external validity here (how widely these findings can be applied) is somewhat compromised because the research is Chicago-centric, but given the dearth of research on indicators in the middle grades, this is a promising start.
Grab Bag Lightning Round!
How to…pick a data platform!
- How does your program manage the data it collects? How do other programs? What platforms are out there in the market that could transform the way to collect, manage, analyze, and report on the data that you have? NCAN, in “Data Usage and Platforms for College Access and Success: Insight from the Field,” has attempted to answer these questions. This white paper considers the strengths and weaknesses of five platforms: Office 365, Salesforce, Naviance, College Greenlight, and Beyond 12. There are also testimonials from members using these platforms. Be on the lookout for a second edition of this report later this year when we consider a new set of platforms!
How to…write a data sharing agreement!
- “Data sharing agreement” is a phrase that is likely to cause a wide range of emotions in program staff; apprehension, bewilderment, and anxiousness are just some of them. The process of setting up an agreement between two parties, be they schools, districts, or non-profit entities is one that can be confusing and complicated. Fortunately, the William T. Grant Foundation’s Research-Practice Partnerships series has an entire module devoted to Developing Data Sharing Agreements, which includes guiding questions, work samples, and other related documents. The rest of the series is also worth a look, but this is the module with the most utility for NCAN members.
- StriveTogether recently released a very handy Data Sharing Playbook that dovetails nicely with the above resource. The playbook is intended to “help community organizations effectively partner with schools on data-driven ways to improve education outcomes. This resource includes seven principles about how to begin and grow a data-driven initiative, as well as practical resources to help communities implement complex data partnerships with schools and other community partners.”
- Similarly, StriveTogether has a very handy guide to student privacy titled "Student Data Privacy Best Practices: Five Ways Community Organizations Can Ensure Effective and Responsible Data Use." If your organization uses student data, this is an important refresher in maintaining that data's security.
How to…start developing a dashboard!
- Many organizations are developing dashboards to quickly and efficiently visualize their progress against outcome targets. We are still on the lookout for a clear and concise resource on how to develop dashboards, but this guide from Salesforce is a good first step toward that end. Actually, it’s a good…five…steps. “5 Simple Steps to Reports and Dashboards.” The principles shared here do not require that you be using Salesforce for your data management.
How to…better understand the National Student Clearinghouse!
- The National Student Clearinghouse is a great source of student postsecondary data, especially when organizations use their StudentTracker service to look up enrollments and completions. Unfortunately, programs often report trouble matching their students with the Clearinghouse’s records. There are many reasons why this might happen, and “The Missing Manual: Using National Student Clearinghouse Data” by Dr. Susan Dynarski and her colleagues attempts to clear some of these up. This paper is a very useful NSC resource and includes history, institutional coverage, issues around suppressed records because of privacy laws, typographic errors, and finally “a discussion of practical issues for program evaluators using NSC data.”