Research and Evaluation
“[Year Up is] one of the few work-based training programs that proves there is a social return on investing in older youth”
– [Name], 2011 Clinton Global Initiative
About Research & Evaluation at Year Up
Year Up strives to learn and is accountable to stakeholders through its improve and prove Research & Evaluation (R&E) strategy. We have a dedicated team of internal evaluators and data scientists who conduct rigorous studies of our program to strengthen our services to students, academic and corporate partners, and demonstrate our long-term impact on young adults’ earnings and education. Our work comes to life through:
- Partnerships with dozens of leading research firms and universities across the country
- Evaluations using gold-standard randomized controlled trials
- Surveys of student, partner, and alumni experiences
- Scholarship through analysis of Year Up program and public data sources (e.g., American Community Survey, National Student Clearinghouse)
Year Up is honored to work with leading scholars, sponsors, and practitioners who support our commitment to improving and proving Year Up. We give special thanks to our partners on our recent R&E projects, including:
- Abt Associates
- The Education Innovation Laboratory at Harvard University
- Administration for Children and Families
- Tipping Point
- New Profit
- Institute of Educations Sciences
- Venture Philanthropy Partners
- University of Pennsylvania Graduate School of Education
- GreenLight Fund
- Center for Youth and Communities at Brandeis University
- Social Innovation Fund
For decades, federal resources for workforce programs that supported disconnected young adults were cut or diverted to early-childhood intervention initiatives because of a lack of solid, high-quality research to show that programs like Year Up work. Since 2007, Year Up has conducted a series of gold-standard randomized controlled trials to fairly and objectively study the impact of our program on our students’ long-term success. We have seen amazing results that demonstrate the lasting positive effects of our program.
EMC Evaluation (2007-2009)
In partnership with the Economic Mobility Corporation (EMC), Year Up conduct an impact evaluation using a randomized controlled trial with 195 young adults at three program sites in Boston, Providence, and New York. EMC tracked 135 Year Up students and 60 control group members for two years to measure and compare employment outcomes and enrollment in post-secondary education. During the second year of the study, Year Up participants earned 30% (or $3,500) more than young adults in the control group. While Year Up participants and control group members were equally likely to be enrolled in college, EMC’s president Mark Elliott called this “the most exciting evaluation results we’ve seen in youth employment in 20 or 30 years – and the first to show a really substantial earnings gain.”
Links to EMC Reports
- A Promising Start: Year Up’s Initial Impacts on Low-Income Young Adults’ Careers (Roder & Elliott, 2011)
- Sustained Gains: Year Up’s Continued Impact on Young Adults’ Earnings (Roder & Elliott, 2014)
PACE Evaluation (2011-Present)
With federal sponsorship from the Administration for Children and Families and the Social Innovation Fund and in partnership with Abt Associates, Year Up is one of nine leading programs selected to participate in the Pathways for Advancing Careers and Education (PACE) Evaluation. PACE is a randomized controlled trial impact evaluation of next-generation strategies for increasing economic self-sufficiency. In 2013 and 2014, nearly 2,500 participants from nine different Year Up program sites (Atlanta, Boston, Chicago, National Capital Region, New York, Puget Sound, Providence, San Francisco, Silicon Valley) were randomized into treatment and control groups. The Abt evaluation team is tracking these participants’ short- and long-term employment and education outcomes to determine Year Up’s impact.
The PACE evaluation will produce a series of three impact reports on Year Up. Currently, the first PACE report, which examines outcomes in the first 18 months after graduation from our program, is under review between our evaluators and our federal sponsors and will be released in 2018. Two later reports will track participants’ outcomes out to two and then five years after graduation, respectively.
While we eagerly wait for the publication of our first PACE report early this year, we invite you to read a Year Up case study, which examines how we have successfully and rapidly grown our program during the PACE evaluation. Additionally, please check out Learning Together: Building Stronger Practitioner-Researcher Partnerships (Fein, 2016), for insight on Year Up and Abt’s longstanding collaboration.
Links to Other PACE Reports
- Risk Factors for College Success: Insights from Adults in Nine Career Pathways Programs (Fein, 2016)
- Programmatic and Other Supports Accessed by Career Pathways Participants (Seefeldt, Engstrom, & Gardiner, 2016)
- Finances, Family, Materials, and Time: Career Pathways Participants’ Perceived Challenges (Seefeldt, Engstrom, & Gardiner, 2016)
- Nothing Can Stop Me: Career Pathways Participants’ Motivations and Thoughts on Success (Seefeldt, Engstrom, & Gardiner, 2016)
IES Evaluation (2015-Present)
With support from a federal Development and Innovation grant from the Institute of Education Sciences (IES), Year Up is continuing its partnership with Abt Associates as well as researchers from the University of Pennsylvania on a four-year evaluation of Year Up’s Professional Training Corps (PTC) model. The IES evaluation is being conducted in two phases: 1) three “mini-studies” of selected aspects of program implementation, and 2) a rigorous assessment of education and employment impacts using a randomized controlled trial at three Year Up PTC locations.
Phase one’s mini-studies are small, quick-turnaround research projects on key program design and implementation challenges. They assess experience with an existing practice or promising alternative approach to particular challenges to generate evidence informing program refinement and design decisions. Year Up’s three mini-studies that started in 2016, include:
- A review of educational outcomes for all Year Up students since 2010, with a focus on higher education continuation and completion among PTC students
- An in-depth analysis of interns’ experiences at their internships and support services for internship managers
- A randomized design test of modified academic monitoring and support services at three PTC sites aiming to improve grades, college completion, credit accumulation, and overall retention
Year Up began the phase two randomized controlled trial in early 2017, which will continue until the release of the first IES evaluation impact report in late 2019.
Feedback is a top priority at Year Up, which is why we dedicate significant time and resources to listen and respond to our students’, partners’, and graduates’ needs through national survey research. We administer a series of five surveys during our yearlong program to our students, interns, and corporate partners to capture ongoing service delivery needs and overall satisfaction with Year Up. We strive for a 90% response rate, and have consistently earned a 90% satisfaction rating from our stakeholders.
Our survey outreach continue after graduation, as well. In 2016, Year Up partnered with ICF International and invited all 9,479 alumni who had graduated between July 2002 and July 2016 to respond to a web-based survey to track progress on employment and education outcomes and foster alumni leadership and advancement. We are excited that nearly 30% of our alumni responded, reported they are experiencing long-term success, and identified several key areas where Year Up will offer additional support.
Overall, our alumni fare well compared to the U.S. population and families earning less than $40k annually:
- $22/hour – average wage for employed alumni
- 75% of alumni employed in full-time jobs
- 70% of alumni are pursuing or have pursued higher education
- 65% of alumni have referred a student to Year Up
- 93% of alumni are satisfied with Year Up
Please check out the executive summary of findings from our 2016 Alumni Survey to learn more about our methodology and results.
With extensive analysis of Year Up program and public data sources, we are dedicated to advancing our program and the wider workforce development and education fields through scholarship and publication. Our R&E team regularly presents at research and evaluation conferences across the country and works with others to write about promising practices and challenges in performance management, internal evaluation, data science, and workforce development and education programming.
Recent Scholarship Featuring Year Up
- National Bureau of Economic Research (NBER) working paper, Complementary Bias: A Model of Two-Sided Statistical Discrimination (Craig & Fryer, 2017)
- Centre for Cities article, Delivering change: Making apprenticeships work in cities (Thomas, Williams, & Serwicka, 2016)
- Chronicle of Philanthropy article, Data to the Rescue: Smart Ways of Doing Good (Wallace, 2016)
Selected Year Up R&E Team Conference Presentations
Maynard, R., Fein, D., Baelen, R., Warfield, G. (2017). To Improve and Prove: A Practitioner’s Perspective on Learning from an Evaluation Partnership. SREE Spring Research Conference, Washington, D.C.
Warfield, G. (2017). To Improve and Prove: A Practitioner’s Perspective on Learning from an Evaluation Partnership. APPAM 39th Annual Fall Research Conference, Chicago, IL.
Britt, J.B., Tsin, D., Leopold, D., Gajdowski, M.R., & Warfield, G. (2017). How Internal Evaluators Help Programs Make Sense of Data and Drive Decisions. American Evaluation Association Annual Conference, Washington, D.C.
Britt, J.B., Gajdowski, M.R., & Warfield, G. (2017). We Reinvented the Wheel and Made it Square – Internal Evaluation Mistakes and What we Learned From Them. American Evaluation Association Annual Conference, Washington, D.C.
Britt, J.B., Gajdowski, M.R., & Warfield, G. (2016). Improving Job-Training Course Design, Content and Delivery through Mixed Research Methods. American Evaluation Association Annual Conference, Washington, D.C.
If you have questions about Research & Evaluation at Year Up, please contact our Director of Evaluation, Garrett Warfield, Ph.D. at gwarfield.@yearup.org.